A committee asks a model vendor for evidence, and back comes a slide with one number: 94% accuracy. It looks like a verdict. It is not. A single aggregate score cannot tell you what happens when the model is wrong, or to whom — and that is precisely the question that decides whether a model is safe to deploy.

The naive move is to pick one headline metric, usually accuracy or ROC-AUC, report an impressive figure, and treat it as proof of quality. The expert move starts somewhere else entirely: from the decision the model supports and the class balance in the data. Only after those two things are fixed does it make sense to choose metrics — and the metrics you choose are the ones that expose the failure modes that actually matter. This is where evaluation stops being a scoreboard and becomes evidence.

Why one headline number hides the decision that matters

Accuracy is the fraction of predictions that are correct. On balanced data with symmetric error costs, it is a fine summary. The problem is that almost no real decision has those properties.

Consider a classifier flagging fraudulent transactions where roughly one in a thousand is fraud. A model that predicts “not fraud” every single time scores 99.9% accuracy — and catches nothing. The number is high and the model is useless. Accuracy collapses here because it averages over a distribution dominated by the easy negative class, drowning out performance on the rare positives that are the whole point of the exercise.

ROC-AUC is the usual next reach, and it is genuinely more informative: it measures ranking quality across all thresholds, independent of any single operating point. But ROC-AUC is computed from true-positive and false-positive rates, and the false-positive rate has a large negative class in its denominator. On severe imbalance, a model can post an impressive ROC-AUC while still producing a swamp of false positives in absolute terms, because each false positive barely moves a rate divided by a huge base. The curve looks good; the alert queue is unworkable. This is the same failure of aggregation that LynxBenchAI names in a different domain — a headline number that hides what is actually being measured rather than exposing it. A single figure earns trust it has not established.

When should you report PR-AUC instead of ROC-AUC?

The short answer: when the positive class is rare and finding it is the job.

The precision–recall curve plots precision (of the cases the model flagged, how many were real) against recall (of the real cases, how many the model caught). Neither axis has the large negative class in its denominator, so PR-AUC does not get flattered by an abundance of easy negatives. When positives are scarce, PR-AUC moves when the model’s behaviour on those positives changes — which is exactly the sensitivity you want.

The rule of thumb we apply in practice: report ROC-AUC when classes are roughly balanced and both error types cost similarly; report PR-AUC when the positive class is a small minority and the cost of missing it (or of false alarms about it) is what the decision turns on. Reporting both is defensible. Reporting only ROC-AUC on 1000-to-1 imbalance is how a weak model passes review.

A quick decision table for metric choice

Situation	Report this	Why
Balanced classes, symmetric error cost	Accuracy + ROC-AUC	Aggregate is honest; ranking quality visible
Rare positive class (e.g. fraud, defect, abuse)	PR-AUC + confusion matrix at chosen threshold	Negative class does not flatter the score
False positives are expensive (e.g. auto-blocking)	Precision at fixed recall	Names the cost you are trying to control
False negatives are expensive (e.g. safety screening)	Recall at fixed precision	Names the miss you cannot tolerate
Cost of the two errors is asymmetric and known	Confusion matrix + cost-weighted total	Converts errors into money or risk, not counts

(Evidence class: observed-pattern — these are the planning heuristics we bring into procurement-grade evaluations, not a benchmarked threshold that transfers unchanged to your data.)

Reading a confusion matrix to find the costly error

The confusion matrix is the least glamorous artifact in evaluation and the most decision-relevant. It is a 2×2 table (for binary classification) counting true positives, false positives, false negatives, and true negatives. Everything else — precision, recall, accuracy — is a ratio derived from these four cells. Reporting the derived ratios without the matrix is like reporting a mean without the sample size.

What the matrix answers that a scalar cannot: which error the model makes, and how often, in absolute terms. A content-moderation model that wrongly blocks benign posts (false positives) has a very different operational cost than one that lets harmful content through (false negatives) — and the same aggregate accuracy can sit on top of wildly different splits between the two. The committee’s real question, “what happens when the model is wrong, and to whom?”, is answered in these cells and nowhere else. Our sibling article on reading recall in an LLM evaluation pack works one of these cells in detail; here the point is that the matrix comes first and the ratios come from it.

How precision and recall trade off, and why the threshold is a decision

Most classifiers output a score, not a label. A threshold turns the score into a decision, and moving the threshold trades precision against recall along a single continuous dial. Lower the threshold and you catch more positives (recall up) but flag more junk (precision down). Raise it and you flag less junk (precision up) but miss more real cases (recall down). There is no free lunch on this curve; there is only the operating point that best matches the cost of the two errors.

This is why “the model scored 0.91” is an incomplete statement. Scored at which threshold? Chosen how? The defensible artifact is not a single number but a stated operating point with its justification: we set the threshold at 0.62 to hold recall at 95% on the safety-critical positive class, accepting a precision of 0.40 and the alert volume that implies, because a missed case costs more than a reviewer’s time. That sentence is auditable. “94% accuracy” is not.

Where do accuracy metrics belong in a procurement-grade eval pack?

They belong in the section that operational benchmarks explicitly cannot fill. When you evaluate a locally served model, throughput and latency numbers — the kind covered in our work on what Ollama benchmark numbers mean for a procurement evidence pack — tell you whether the model is feasible to run. They say nothing about whether it is right. Tokens per second cannot distinguish a model that answers correctly from one that answers fluently and wrongly.

Task-accuracy metrics and failure-mode analysis supply the other half. The two are complementary, not competing: one section of the evidence pack establishes operational feasibility, the other establishes task quality and error cost. A pack that reports latency without accuracy — or accuracy without the confusion matrix behind it — leaves a hole an auditor will find. This complementarity is also why leaderboard rank is a poor proxy for task quality; a model can top a general ranking and still miss on your specific classification task, a gap we examine in when leaderboard rank doesn’t predict task accuracy.

What role does explainability play, and what does an audit-ready artifact look like?

Metrics tell a committee how often a model was right. Explainability tells them why a given decision was made. Both are needed, and neither substitutes for the other. A model can be accurate on aggregate and still make individual decisions no one can justify — and in a regulated setting, an unjustifiable decision is a liability regardless of the aggregate score.

An audit-ready explainability artifact is not a heatmap screenshot. It is a documented, reproducible account of which features (or input tokens) drove a decision, produced by a named method — SHAP values, integrated gradients, attention attribution — with the method, its version, and its known limitations recorded alongside the output. The test is whether a model-risk reviewer, months later, can regenerate the explanation and reach the same conclusion. We treat this the way auditors and model-risk reviewers actually need explainability to be documented, and it is the same evidence-for-governance case that LynxBenchAI makes for benchmarks in procurement, governance, and risk management — an artifact built to survive audit, not to impress a slide.

Where explainability metrics and accuracy metrics converge is the ML explainability section of an LLM procurement evidence pack: the accuracy numbers say the model is right often enough, and the explainability artifact says each decision can be defended. Together they populate the sections of the pack that raw benchmarks leave empty — the accuracy and failure-mode evidence a committee needs to make a defensible model choice. This sits at the centre of our approach to AI governance and trust, where evaluation evidence is designed for scrutiny rather than persuasion.

Representing metrics so models compare on error cost

The final discipline is presentation. If Model A is reported at 94% accuracy and Model B at 0.88 PR-AUC, the committee cannot compare them — the numbers answer different questions on different scales. Like-for-like comparison requires that every candidate be scored on the same metrics, under the buyer’s actual class balance, at operating points chosen for the same error-cost logic.

The representation that makes this possible is a per-model row carrying: the confusion matrix at a stated threshold, precision and recall at that point, PR-AUC over the buyer’s positive-class prevalence, and the cost-weighted error total when the two error costs are known. That last column is the one that matters most, because it converts the whole comparison from “whose number is bigger” into “whose errors cost less.” A committee comparing cost-weighted error totals is comparing the thing they actually care about; a committee comparing headline accuracy is comparing marketing.

FAQ

Which model evaluation metrics actually matter, and how do precision, recall, ROC-AUC, and explainability fit together?

The metrics that matter are the ones that expose the failure mode relevant to your decision. Accuracy and ROC-AUC summarise aggregate and ranking quality; precision and recall name the two error types and their trade-off; the confusion matrix shows them in absolute terms. Explainability sits alongside these, answering why a decision was made rather than how often the model is right. No single metric is sufficient — the set is chosen to match the decision and the class balance.

When should you report PR-AUC instead of ROC-AUC, and why does class imbalance change the choice?

Report PR-AUC when the positive class is a small minority and finding it is the job. ROC-AUC is computed from rates whose denominators include the large negative class, so it can look impressive even when a model produces many false positives in absolute terms. PR-AUC’s axes — precision and recall — do not have the negative class inflating them, so the score moves with performance on the rare positives you care about.

How do you read a confusion matrix to decide whether false positives or false negatives are the costly error?

The confusion matrix counts true positives, false positives, false negatives, and true negatives. It shows which error the model makes and how often, in absolute terms, which the derived ratios hide. You map each cell to its real-world cost — a false positive that wrongly blocks a benign case versus a false negative that lets a harmful one through — and the larger cost identifies the error you must control.

How do precision and recall trade off, and how do you pick an operating threshold rather than reporting a single aggregate score?

Most classifiers output a score, and a threshold turns it into a decision; moving that threshold trades precision against recall along one dial. There is no operating point that maximises both, so you choose the one matching your error costs — for example, hold recall high on a safety-critical class and accept lower precision. The defensible artifact is a stated threshold with its justification, not a single aggregate number.

Where do accuracy metrics belong in a procurement-grade eval pack, and how do they complement operational benchmarks like tokens/sec and latency?

Operational benchmarks establish whether a model is feasible to run; accuracy metrics establish whether it is right. Tokens per second and latency cannot distinguish a correct answer from a fluent wrong one, so task-accuracy metrics and failure-mode analysis fill the accuracy and failure-mode sections of the pack that benchmarks leave empty. The two sections are complementary — a complete pack carries both.

What role does explainability play alongside accuracy metrics, and what does an audit-ready explainability artifact look like?

Explainability tells a committee why a given decision was made, where accuracy metrics only say how often the model was right; both are needed because an accurate model can still make individually indefensible decisions. An audit-ready artifact is a documented, reproducible account of the features or tokens that drove a decision, produced by a named method (such as SHAP or integrated gradients) with its version and limitations recorded. The test is whether a reviewer can regenerate it later and reach the same conclusion.

How should evaluation metrics be represented so competing models compare on error cost on a like-for-like basis?

Every candidate must be scored on the same metrics, under the buyer’s actual class balance, at operating points chosen with the same error-cost logic. The representation that enables this is a per-model row carrying the confusion matrix at a stated threshold, precision and recall at that point, PR-AUC over the buyer’s positive-class prevalence, and — where the two error costs are known — a cost-weighted error total. That final column converts the comparison from “whose number is bigger” into “whose errors cost less.”

The uncertainty that remains is whether your reported class balance actually matches production. Metrics chosen against a training-set prevalence can mislead when deployment shifts the base rate — which is why the operating point, not just the score, belongs in the evidence pack, and why the threshold you defend today should be revisited the moment the incoming distribution moves.