Covid-19 Detection App with Tensorflow: Transfer Learning
Estimated reading time: 6 minutes
In this second part of the series of article we using Transfer Learning to build a production-ready Covid-19 detection system using Tensorflow. In this article, we will incorporate transfer learning for our project.
Introduction
In the last part of the series, we trained our model using custom CNN based architecture. Now it’s time to go further one step ahead and try transfer learning on our problem domain.
We are performing the same steps for data collection and processing. In this part, we start by looking into MobileNetV2, which is our baseline model used for transfer learning. Further, we fine-tune the model for our use case.
What is MobileNetV2?
This is a second-generation model of famous MobileNets. The MobileNets are state of art for many vision problems, and very efficient for resource-constrained systems. MobileNets are used for many tasks like object detection, image classification, face attributes.
MobileNetV2 is based on an inverted residual structure. The main building block for this state of art architecture is lightweight depthwise convolutions. The model is available in different model size based on requirements.
- In the above diagram,
- t: expansion factor
- c: output channels
- n: repetition of layer
- s: stride
- There are two types of blocks, one residual block with stride 1 and another block with stride 2 for downsizing.
- There tradeoff between the performance and efficiency of the model, the research paper MobileNetV2: Inverted Residuals and Linear Bottlenecks explore the topic in more detail.
Transfer Learning
The intuition behind transfer learning is to use knowledge from the existing generic model in the same problem domain. In our case, we are trying to use a pre-trained model on generic data and the model is generic enough to solve other real-life problems with the existing knowledge.
We don’t need to retrain entire model, we use the baseline top model as it. Then we add our last classification layers, which we need train based on our use case. For further model tuning, we let few top layer to retrain for our use case.
The assumption to start with is that you already completed data collection and transformation steps from previous article.
#Define image shape as per last tutorial
IMG_SHAPE = (img_height, img_width) + (3,)
base_model = tf.keras.applications.MobileNetV2(input_shape=IMG_SHAPE,
include_top=False,
weights='imagenet')
base_model.trainable = False
- The above code is used MobileNetV2 which is part of Keras, and you don’t need to import anything further.
- We freeze the model by setting trainable parameter to false.
#You can check model structure
base_model.summary()
Let’s define our top layer, for our classification task.
global_average_layer = tf.keras.layers.GlobalAveragePooling2D()
feature_batch_average = global_average_layer(feature_batch)
print(feature_batch_average.shape)
prediction_layer = tf.keras.layers.Dense(1)
prediction_batch = prediction_layer(feature_batch_average)
print(prediction_batch.shape)
preprocess_input = tf.keras.applications.mobilenet_v2.preprocess_input
- The GlobalAveragePooling layer is to convert the features to a single 1280-element vector per image.
- We don’t need any activation function, as we are using it as a logit.
#Define Model
inputs = tf.keras.Input(shape=(256, 256, 3))
x = data_augmentation(inputs)
x = preprocess_input(x)
x = base_model(x, training=False)
x = global_average_layer(x)
x = tf.keras.layers.Dropout(0.2)(x)
outputs = prediction_layer(x)
model = tf.keras.Model(inputs, outputs)
#Let's compile it.
base_learning_rate = 0.0001
model.compile(optimizer=tf.keras.optimizers.Adam(lr=base_learning_rate),
loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
metrics=['accuracy'])
Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_2 (InputLayer) [(None, 256, 256, 3)] 0
_________________________________________________________________
sequential_1 (Sequential) (None, 256, 256, 3) 0
_________________________________________________________________
tf.math.truediv (TFOpLambda) (None, 256, 256, 3) 0
_________________________________________________________________
tf.math.subtract (TFOpLambda (None, 256, 256, 3) 0
_________________________________________________________________
mobilenetv2_1.00_224 (Functi (None, 8, 8, 1280) 2257984
_________________________________________________________________
global_average_pooling2d (Gl (None, 1280) 0
_________________________________________________________________
dropout (Dropout) (None, 1280) 0
_________________________________________________________________
dense (Dense) (None, 1) 1281
=================================================================
Total params: 2,259,265
Trainable params: 1,281
Non-trainable params: 2,257,984
_________________________________________________________________- We have very few trainable parameters, from the last two layers. 1 parameter from the output dense layer and 1280 from another dense layer.
- The optimizer is adam with a learning rate 0.0001.
- We have not defined activation function, that’s why we have set from logits=True in loss function.
initial_epochs = 10
loss0, accuracy0 = model.evaluate(val_ds)
print("initial loss: {:.2f}".format(loss0))
print("initial accuracy: {:.2f}".format(accuracy0))
initial loss: 0.89
initial accuracy: 0.20
#Let's train model
history = model.fit(train_ds,
epochs=initial_epochs,
validation_data=val_ds)
Epoch 1/10
346/346 [==============================] - 221s 625ms/step - loss: 0.6093 - accuracy: 0.6679 - val_loss: 0.4961 - val_accuracy: 0.7368
Epoch 2/10
346/346 [==============================] - 149s 432ms/step - loss: 0.4409 - accuracy: 0.7963 - val_loss: 0.4069 - val_accuracy: 0.8038
Epoch 3/10
346/346 [==============================] - 141s 408ms/step - loss: 0.3919 - accuracy: 0.8242 - val_loss: 0.3696 - val_accuracy: 0.8230
Epoch 4/10
346/346 [==============================] - 143s 413ms/step - loss: 0.3583 - accuracy: 0.8430 - val_loss: 0.3435 - val_accuracy: 0.8421
Epoch 5/10
346/346 [==============================] - 120s 348ms/step - loss: 0.3357 - accuracy: 0.8574 - val_loss: 0.3321 - val_accuracy: 0.8436
Epoch 6/10
346/346 [==============================] - 98s 284ms/step - loss: 0.3202 - accuracy: 0.8623 - val_loss: 0.3109 - val_accuracy: 0.8602
Epoch 7/10
346/346 [==============================] - 94s 270ms/step - loss: 0.3085 - accuracy: 0.8707 - val_loss: 0.3000 - val_accuracy: 0.8689
Epoch 8/10
346/346 [==============================] - 105s 305ms/step - loss: 0.2982 - accuracy: 0.8723 - val_loss: 0.2931 - val_accuracy: 0.8678
Epoch 9/10
346/346 [==============================] - 101s 292ms/step - loss: 0.2914 - accuracy: 0.8786 - val_loss: 0.2809 - val_accuracy: 0.8831
Epoch 10/10
346/346 [==============================] - 108s 313ms/step - loss: 0.2851 - accuracy: 0.8810 - val_loss: 0.2748 - val_accuracy: 0.8831The model did very well, let’s improve it by fine tuning it further by setting base layers trainable.
base_model.trainable = True
print("Number of layers in the base model: ", len(base_model.layers))
# > Number of layers in the base model: 154
# Fine-tune from this layer onwards
fine_tune_at = 100
# Freeze all the layers before the `fine_tune_at` layer
for layer in base_model.layers[:fine_tune_at]:
layer.trainable = False
#Let's compile again
model.compile(loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
optimizer = tf.keras.optimizers.RMSprop(lr=base_learning_rate/10),
metrics=['accuracy'])
Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_2 (InputLayer) [(None, 256, 256, 3)] 0
_________________________________________________________________
sequential_1 (Sequential) (None, 256, 256, 3) 0
_________________________________________________________________
tf.math.truediv (TFOpLambda) (None, 256, 256, 3) 0
_________________________________________________________________
tf.math.subtract (TFOpLambda (None, 256, 256, 3) 0
_________________________________________________________________
mobilenetv2_1.00_224 (Functi (None, 8, 8, 1280) 2257984
_________________________________________________________________
global_average_pooling2d (Gl (None, 1280) 0
_________________________________________________________________
dropout (Dropout) (None, 1280) 0
_________________________________________________________________
dense (Dense) (None, 1) 1281
=================================================================
Total params: 2,259,265
Trainable params: 1,862,721
Non-trainable params: 396,544
_________________________________________________________________
fine_tune_epochs = 10
total_epochs = initial_epochs + fine_tune_epochs
history_fine = model.fit(train_ds,
epochs=total_epochs,
initial_epoch=history.epoch[-1],
validation_data=val_ds)
Epoch 10/20
346/346 [==============================] - 143s 383ms/step - loss: 0.2384 - accuracy: 0.9014 - val_loss: 0.1878 - val_accuracy: 0.9062
Epoch 11/20
346/346 [==============================] - 134s 388ms/step - loss: 0.1738 - accuracy: 0.9297 - val_loss: 0.1509 - val_accuracy: 0.9453
Epoch 12/20
346/346 [==============================] - 120s 345ms/step - loss: 0.1499 - accuracy: 0.9417 - val_loss: 0.1750 - val_accuracy: 0.9381
Epoch 13/20
346/346 [==============================] - 132s 381ms/step - loss: 0.1231 - accuracy: 0.9528 - val_loss: 0.1180 - val_accuracy: 0.9450
Epoch 14/20
346/346 [==============================] - 121s 349ms/step - loss: 0.1097 - accuracy: 0.9585 - val_loss: 0.0923 - val_accuracy: 0.9623
Epoch 15/20
346/346 [==============================] - 114s 330ms/step - loss: 0.0989 - accuracy: 0.9616 - val_loss: 0.0986 - val_accuracy: 0.9569
Epoch 16/20
346/346 [==============================] - 119s 345ms/step - loss: 0.0936 - accuracy: 0.9657 - val_loss: 0.0794 - val_accuracy: 0.9736
Epoch 17/20
346/346 [==============================] - 111s 322ms/step - loss: 0.0872 - accuracy: 0.9663 - val_loss: 0.0769 - val_accuracy: 0.9718
Epoch 18/20
346/346 [==============================] - 119s 345ms/step - loss: 0.0829 - accuracy: 0.9680 - val_loss: 0.0719 - val_accuracy: 0.9772
Epoch 19/20
346/346 [==============================] - 125s 362ms/step - loss: 0.0729 - accuracy: 0.9713 - val_loss: 0.0815 - val_accuracy: 0.9638
Epoch 20/20
346/346 [==============================] - 173s 502ms/step - loss: 0.0726 - accuracy: 0.9729 - val_loss: 0.0640 - val_accuracy: 0.9776- We fine-tuned the model further to achieve ~97% accuracy on the validation set.
- The model has 1,862,721 trainable paramters.
#Let's Predict
img_path = "COVID-19_Radiography_Dataset/test/COVID/COVID-103.png"
img = keras.preprocessing.image.load_img(
img_path, target_size=(img_height, img_width)
)
img_array = keras.preprocessing.image.img_to_array(img)
img_array = tf.expand_dims(img_array, 0) # Create a batch
predictions = model.predict(img_array)
score = tf.nn.sigmoid(predictions[0])
print(
"This image is %.2f percent Covid and %.2f percent Normal."
% (100 * (1 - score), 100 * score)
)
This image is 87.85 percent Covid and 12.15 percent Normal.
#Don't Forget to save model, as we need it for next part
model.save('tl_model_tf.h5',save_format='tf')
Summary: Transfer Learning
- We successfully trained the model with transfer learning.
- In the next section, we deploy the model into production using Google Cloud Run serverless platform.
Responses