📝 Confusion Matrix

Description

A confusion matrix counts how a classification model's predictions compare with actual labels. It shows correct predictions and the types of mistakes the model makes.
Example: a model predicts whether a user will click a recommended movie. Here, click is the positive class, and the matrix summarizes 100 impressions:

Actual \ Predicted Click No click
Click 40 — True Positive (TP) 10 — False Negative (FN)
No click 20 — False Positive (FP) 30 — True Negative (TN)

Actual \ Predicted	Click	No click
Click	40 — True Positive (TP)	10 — False Negative (FN)
No click	20 — False Positive (FP)	30 — True Negative (TN)

True positive (TP): predicted a click, and the user clicked.
False positive (FP): predicted a click, but the user did not click.
False negative (FN): predicted no click, but the user clicked.
True negative (TN): predicted no click, and the user did not click.
Naming convention: “true/false” describes whether the prediction was correct; “positive/negative” describes the predicted class. Check the axis labels because some displays reverse actual and predicted. Google's confusion matrix guide illustrates the alternative orientation.

Metric calculations: using the counts above:

Metric	Calculation	Meaning
Accuracy	$(TP + TN) / (TP + TN + FP + FN) = 70\%$	How often was the prediction correct?
Precision	$TP / (TP + FP) \approx 66.7\%$	Of predicted clicks, how many actually happened?
Recall	$TP / (TP + FN) = 80\%$	Of actual clicks, how many did the model identify?
F1 score	$2TP / (2TP + FP + FN) \approx 72.7\%$	Harmonic mean of precision and recall

Interpret metrics together: accuracy can hide poor performance on a rare class. Choose metrics based on the cost of missed positives and false alarms. If a formula's denominator is zero, its value is undefined; explicitly document how your evaluation tool handles it. See Google's classification metrics guide.

Probability threshold: convert predicted probabilities to class labels before counting outcomes. For example, predict “click” when its probability is at least 0.5. Changing the threshold can change the matrix; select the threshold on validation data and report it with the results.
Multiple classes: a single-label classifier with $K$ classes has a $K \times K$ matrix. With the same class order on both axes, correct predictions lie on the diagonal; off-diagonal cells show which classes are confused.

ROC–AUC compares true-positive and false-positive rates across thresholds.
Confidence Calibration examines whether predicted probabilities match observed frequencies.
Train, Validation, and Test Sets explains how to separate model selection from final evaluation.