What you wanted to know about AUC
aucrocclassificationmodel-evaluationbinary-classification
Abstraction: AUC-ROC metric explained as threshold-invariant ranking score
Key points:
- AUC (Area Under ROC Curve) considers all possible classification thresholds, unlike accuracy which requires a fixed threshold
- Random classifier scores AUC = 0.5 (diagonal line); perfect classifier scores 1.0
- AUC is insensitive to class imbalance — a classifier always predicting 0 on 90% negative data still scores 0.5, not 0.9 accuracy
- AUC cares only about ranking, not absolute probability values; scaling predictions 700/900 vs 0.7/0.9 yields identical AUC
- For imbalanced classes, precision-recall AUC is often more informative than ROC AUC (per Gael Varoquaux)
- Available in scikit-learn as
roc_auc_score, in R via caTools, and in Matlab viaperfcurve
Connections: Scikit Learn · Model Evaluation · Classification · Roc Auc
Source: http://fastml.com/what-you-wanted-to-know-about-auc/