High Recall and Low Precision for Binary CNN model

Question

I was training a CNN model for binary classification. The training and validation accuracy seemed good. However, the precision is low and the recall is high (High false positive).

Recall of the model is 1.00
Precision of the model is 0.71
F1-score of the model is 0.83

The data was imbalanced, so I augmented the class with the lowest number of samples to have an equal number of samples as the other one. I've also checked that the test data is preprocessed in the same way as the training data. What could be the problem?

Here is the code:

# Structuring the model
num_classes = len(class_names)

callback = [tf.keras.callbacks.EarlyStopping(
    monitor='loss', min_delta=0.001, patience=1, verbose=1, restore_best_weights=True),
    tf.keras.callbacks.ModelCheckpoint(f"{save_dir}/pneumonia_{epochs}e.h5")]
model = keras.models.Sequential()
model.add(layers.Conv2D(32,3, activation='relu', input_shape=(img_width,img_height,1)))
model.add(layers.MaxPool2D())
model.add(layers.Conv2D(32,3,activation='relu'))
model.add(layers.MaxPool2D())
model.add(layers.Flatten())
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

model.compile(
              optimizer='adam',
              loss=tf.keras.losses.BinaryCrossentropy(),
              metrics=['accuracy'])

# Fitting the model
history = model.fit(
                    train_ds,
                    validation_data=val_ds,
                    epochs=epochs,
                    callbacks=[callback])

# Getting x and y from the test set
test_preds = []
y_test = []
for x,y in test_ds:
    test_pred = model.predict(x)
    test_preds.extend(test_pred)
    y_test.extend(y)

preds = np.array(test_preds)
y_pred = np.where(preds > 0.5, 1, 0)
y_test = np.array(y_test)

# Assessing model performance using confusion matrix
from sklearn.metrics import confusion_matrix

cm  = confusion_matrix(y_test, y_pred)
tn, fp, fn, tp = cm.ravel()

precision = tp/(tp+fp)
recall = tp/(tp+fn)
f1 = 2*((precision * recall)/(precision + recall))

output:

Epoch 1/20
219/219 [==============================] - 1491s 7s/step - loss: 0.1725 - accuracy: 0.9309 - val_loss: 0.1102 - val_accuracy: 0.9639
Epoch 2/20
219/219 [==============================] - 1434s 7s/step - loss: 0.0600 - accuracy: 0.9754 - val_loss: 0.1667 - val_accuracy: 0.9343
Epoch 3/20
219/219 [==============================] - 1452s 7s/step - loss: 0.0381 - accuracy: 0.9861 - val_loss: 0.1022 - val_accuracy: 0.9716
Epoch 4/20
219/219 [==============================] - 1416s 6s/step - loss: 0.0349 - accuracy: 0.9880 - val_loss: 0.1198 - val_accuracy: 0.9549
Epoch 5/20
219/219 [==============================] - 1445s 7s/step - loss: 0.0189 - accuracy: 0.9927 - val_loss: 0.0942 - val_accuracy: 0.9742
Epoch 6/20
219/219 [==============================] - 1428s 7s/step - loss: 0.0075 - accuracy: 0.9981 - val_loss: 0.2629 - val_accuracy: 0.9278
Epoch 7/20
219/219 [==============================] - 1422s 6s/step - loss: 0.0024 - accuracy: 0.9997 - val_loss: 0.1688 - val_accuracy: 0.9639
Epoch 8/20
219/219 [==============================] - 1411s 6s/step - loss: 0.0011 - accuracy: 1.0000 - val_loss: 0.1507 - val_accuracy: 0.9704
Epoch 9/20
219/219 [==============================] - ETA: 0s - loss: 4.6450e-04 - accuracy: 1.0000Restoring model weights from the end of the best epoch: 8.
219/219 [==============================] - 1411s 6s/step - loss: 4.6450e-04 - accuracy: 1.0000 - val_loss: 0.1839 - val_accuracy: 0.9678
Epoch 9: early stopping

Recall of the model is 1.00
Precision of the model is 0.71
F1-score of the model is 0.83

Thanks in advance! :)

High Recall and Low Precision for Binary CNN model

0 Answers0