You need to know about the Matthews Correlation Coefficient by David Lettier
classification-metricsimbalanced-classesmcc
Abstraction: MCC formula and edge-case behavior for imbalanced binary classification
Key points:
- MCC ranges from -1 (completely wrong) to +1 (completely correct), with 0 indicating no better than random
- Formula: (TPTN - FPFN) / sqrt((TP+FP)(FN+TN)(FP+TN)(TP+FN)) — numerator uses inner confusion matrix cells, denominator uses totals
- When a classifier predicts all negative on a 327/24 split, accuracy is 93.2% but MCC is 0 — exposes accuracy's failure on imbalanced data
- Predicting all positive yields recall of 1.0 but MCC of 0 — MCC penalizes the waste of 327 good examples
- MCC of -1 means the classifier is consistently inverted; flipping its predictions yields a perfect classifier
- MCC denominator is undefined (0/0) when all predictions are one class, conventionally treated as 0
Connections: Classification Metrics · Imbalanced Classes · Confusion Matrix
Source: https://lettier.github.io/posts/2016-08-05-matthews-correlation-coefficient.html