MACHINE LEARNING · QUESTION 55
When should you use ROC-AUC versus PR-AUC?
On this page
ROC-AUC measures how well a model ranks positive examples above negatives across thresholds, plotting true-positive rate against false-positive rate. PR-AUC summarizes precision versus recall and focuses more directly on positive predictions. With rare positives, PR curves often reveal alert quality more clearly. Neither metric selects an operating threshold or captures all business costs, so inspect threshold-specific behavior too.
LET'S MAKE IT SIMPLE
First, the idea in plain English
A classifier can output a score before anyone chooses where to draw the line between positive and negative. ROC and precision-recall curves compare what happens as that line moves. ROC considers both classes through rates; precision-recall focuses on how useful positive alerts are, which can matter more when positives are uncommon.
A USEFUL COMPARISON
Picture it this way
A security screener can be judged by how many actual risks it finds and how many safe travelers it flags. If risks are rare, the fraction of flagged travelers who are truly risky is also important. ROC and PR emphasize these aspects differently, but neither decides how many false alarms an operations team can handle.
FOLLOW THE JOURNEY
ROC and precision-recall curves emphasize different rates
Contrast true-positive versus false-positive rates with precision versus recall.
This simplified visual highlights the main relationships. In practice, the exact design depends on the requirements; Area metrics alone do not define a production decision.
ROC plots true-positive rate against false-positive rate across thresholds; the precision-recall curve plots precision against recall for the same scored classifier.
How to read this visual
This simplified visual highlights the main relationships. In practice, the exact design depends on the requirements; Area metrics alone do not define a production decision.
UNDER THE HOOD
How it works, step by step
Score examples
Obtain model scores for positive and negative labeled cases.
Why it matters: Curves assess ranking as the threshold varies.
Compute ROC rates
Plot true-positive rate against false-positive rate.
Why it matters: This shows detection versus negative-class false alarms.
Compute PR rates
Plot precision against recall.
Why it matters: This shows alert correctness versus positive-case coverage.
Choose an operating point
Select a threshold using costs and capacity constraints.
Why it matters: Area metrics alone do not define a production decision.
OUTSIDE THE TEXTBOOK
A practical example
A fraud model ranks payments for review, but fraud is rare. ROC-AUC may appear strong while most high-scoring alerts are still legitimate. The team inspects precision-recall at the number of cases investigators can review and compares expected costs at candidate thresholds.
GO A LITTLE DEEPER
Technical details that matter
- ROC-AUC can be interpreted as a ranking probability under common assumptions.
- The PR baseline depends on positive prevalence, so comparisons across populations need care.
- Different libraries may use interpolation conventions for PR-AUC; state the metric implementation when precision matters.
MAKE A GOOD CHOICE
When is this useful?
- ROC-AUC for broad ranking comparison across class thresholds.
- PR-AUC for rare-positive tasks where alert precision and coverage matter.
- Threshold selection using operational constraints and error costs.
Use calibration metrics when probability accuracy matters, and expected utility or cost when consequences are known.
WATCH OUT FOR
Common misunderstandings
- Treating either AUC as a deployed threshold or an accuracy score.
- Ignoring prevalence when interpreting PR-AUC.
- Comparing scores across different datasets without considering population shift.
REAL-WORLD TRADE-OFFS
Limitations and failure cases
- AUC summarizes many thresholds that may never be used.
- PR and ROC can hide poor performance in important subgroups.
- No curve alone represents the full cost of a decision.
WHAT TO REMEMBER
Key takeaways
- 1ROC-AUC evaluates ranking through true- and false-positive rates.
- 2PR-AUC focuses on precision and recall, useful for rare positives.
- 3Select and report an operational threshold separately.
PRACTICE THE FOLLOW-UP
What might an interviewer ask next?
Why can ROC-AUC look good for an imbalanced task?
The false-positive rate denominator includes many negatives, so a seemingly modest rate can still create many false alerts when positives are rare.
Does AUC tell you the best threshold?
No. Choose a threshold based on validation data, error costs, and operational capacity.
SAY IT OUT LOUD
How to explain this in an interview
“ROC-AUC summarizes ranking using true-positive and false-positive rates across thresholds. PR-AUC summarizes precision and recall and can be more informative when positives are rare because it focuses on the quality and coverage of positive alerts. Neither gives the production threshold or captures every cost, so I would inspect an operating point that matches the real decision.”
Connect this question to the roadmap topic it supports.