A vendor puts one number on the slide — “98% accuracy” — and the room nods. That single figure is the most dangerous thing in a computer vision inspection proposal, because it tells you almost nothing about how the system will behave on your line. Object detection systems are not scored by one number, and the numbers that actually govern production behaviour — precision, recall, IoU, and mAP — pull in different directions. Read them correctly and an accuracy claim becomes auditable. Accept the headline figure and you have bought a demo, not a validated system. The gap matters because each metric maps onto a distinct production consequence. Precision governs how many of the parts the system flags are genuinely defective — the driver of rework, wasted good product, and eroded operator trust. Recall governs how many real defects the system catches — the driver of escape defects that reach the customer. IoU governs how tightly a detection box lands on the defect, which decides whether a downstream measurement or robotic action is even usable. And mAP rolls all of this into an aggregate that is convenient for comparing models but easy to misread. A system can score well on the aggregate and still be wrong in exactly the place your quality tolerance cannot absorb. What do object detection metrics actually measure? Start with the two counts everything else is built on. When a detector runs on an image, each prediction is either a true positive (it correctly found a real defect), a false positive (it flagged something that is not a defect), or the defect was there and the model said nothing — a false negative, the miss. Precision and recall are just two ratios over those counts: Precision = true positives / (true positives + false positives). Of everything the system flagged, what fraction was real? Low precision means the line is drowning in false alarms. Recall = true positives / (true positives + false negatives). Of every real defect present, what fraction did the system catch? Low recall means defects are slipping through. These two move against each other. Loosen the model so it flags more aggressively and recall climbs while precision falls; tighten it and the reverse happens. There is no configuration that maximises both at once, which is why quoting one without the other is meaningless. In our experience reviewing vendor accuracy claims, the single most common omission is a recall figure paired with the precision headline — because recall is usually the weaker of the two, and it is the one that produces escape defects. The difference between precision, recall, and mAP — and which matters most Mean Average Precision (mAP) is the aggregate metric most detector benchmarks report, and it is where most misreadings begin. To compute it, you sweep the model’s confidence threshold across its full range, trace the precision-recall curve that results, take the area under that curve (the average precision for one class), and then average across all defect classes. The “mean” is the average over classes; the “average” is the area under one class’s precision-recall curve. Frameworks like the Ultralytics YOLO training loop and the COCO evaluation toolkit report mAP as the default headline, which is exactly why it leaks into procurement conversations as a stand-in for “accuracy.” We unpack how the aggregate is scored in detail in how mean average precision scores object detectors. The problem is what averaging hides. Consider a system inspecting for scratches, dents, and hairline cracks, where cracks are the safety-critical class and are also the rarest. Defect class Frequency Per-class AP Contribution to mAP Scratch Common 0.95 High weight, easy class Dent Common 0.93 High weight, easy class Hairline crack Rare 0.58 Same weight, hardest class Averaged naively, this reports an mAP around 0.82 — a number that looks perfectly acceptable on a slide (observed pattern, illustrative worked example; not a benchmarked device). But the crack class, the only one that can send a failed part to a customer, is scoring 0.58. The aggregate absorbed the weakness. This is the mechanism behind the rule every inspection buyer should internalise: a strong mAP can coexist with unacceptable recall on the one class that actually matters. The aggregate is a comparison tool for models on a fixed benchmark. It is not a substitute for reading the per-class recall on your critical defect. What does IoU measure, and why does localisation matter on a line? Intersection over Union (IoU) measures how tightly a predicted box overlaps the ground-truth box: the area of their intersection divided by the area of their union. An IoU of 1.0 is a perfect box; 0.5 means the prediction and truth overlap by half. Detection metrics use an IoU threshold to decide whether a prediction even counts as a true positive — a detection below the cutoff is scored as a miss no matter how confident the model was. This is why the same model reports different mAP figures depending on the IoU threshold used. [email protected] accepts any detection overlapping the truth by at least 50%; the stricter [email protected]:0.95 averages performance across cutoffs from 0.5 up to 0.95, rewarding tight localisation. A model can look strong at the loose threshold and collapse at the strict one — the distinction that we cover in reading detection metrics for localisation quality and, for the loose-threshold case specifically, in what [email protected] actually measures. Localisation is not academic on a production line. If a robot cell uses the detection box to position a gripper, or if a downstream stage crops the defect for a dimensional measurement, a box that is merely near the defect is a functional failure even though it counted as a hit at IoU 0.5. When localisation feeds an action, the operating IoU threshold is part of the spec, not a footnote. How the confidence threshold moves the precision/recall trade-off Every detection carries a confidence score — the model’s own estimate that a box is a real defect. The system does not act on every raw prediction; it acts on those above a chosen threshold. That threshold is the single lever an integrator uses to set the operating point. Raise the threshold and the system only acts when very sure: precision rises, false alarms drop, but marginal true defects fall below the cutoff and recall drops. Lower it and recall rises while operators get buried in false positives. The mAP figure is threshold-independent by construction — it integrates over the whole curve — which is precisely why it cannot tell you how the deployed system will behave. Deployment behaviour is defined by the one threshold you pick, and that choice is where the confidence score does its real work; we go deeper on that in what a confidence score means and how to use it. This is also the sharpest contrast with classical rule-based machine vision. On a rule-based system, these trade-offs are set explicitly — a threshold on a measured dimension, a tolerance band written into the recipe. On a learned detector they emerge from the training data and the chosen confidence and IoU cutoffs, and they must be read out deliberately before any validation claim holds. The number is only meaningful once the operating point is named. How metrics translate into rework rate and escape-defect rate Here is where the abstraction connects to production economics. Take the worked example that keeps surfacing in vendor demos: a system quoted at 99% precision but 80% recall. 80% recall means 1 in every 5 real defects goes uncaught. On a line producing defective parts at any meaningful rate, that is a direct escape-defect stream reaching the customer — the most expensive failure class in most quality systems. 99% precision looks reassuring, but its practical meaning is only that few flags are false. It says nothing about the misses. A system tuned this way passes a clean demo because demos rarely stress the miss rate. It fails in production because escape defects are the outcome that matters (observed pattern, drawn from inspection engagements; not a published benchmark). The correct reading is: false-positive rate — governed by precision — drives rework and the operator trust that keeps the system switched on; missed-detection rate — governed by recall — drives escapes. You cannot manage what you have not measured, and a single accuracy figure measures neither. This is exactly the language the A2 assessment of a chosen CV approach speaks: object detection metrics are how that assessment expresses whether recall, precision, and localisation clear the line’s defect-tolerance thresholds, rather than deferring to a marketing number. Which metrics to demand from a vision-inspection vendor Before validating any accuracy claim, ask for the numbers that make it auditable. Use this as a checklist when a vendor puts a headline figure on the table: Per-class recall on the critical defect class — not aggregate mAP. This is the escape-defect number. Precision at the same operating point — so you can size rework and false-alarm load. The confidence threshold the reported figures assume — without it, precision and recall are unanchored. The IoU threshold used to score true positives — and whether it matches how tightly your downstream stage needs the box. [email protected] vs [email protected]:0.95 — the gap between them reveals localisation quality. The test-set composition — class balance and whether the critical rare class is represented, since averaging can bury a rare-class weakness. A vendor who can answer all six is quoting engineering. One who can only repeat the headline percentage is quoting marketing. FAQ How does object detection metrics actually work? Object detection metrics score a detector by comparing its predicted boxes against ground-truth boxes, counting true positives, false positives, and misses. In practice they translate directly into production behaviour: precision governs false-alarm and rework rate, recall governs escape defects, and IoU governs whether a detection box is tight enough for a downstream action to use. What is the difference between precision, recall, and mAP, and which matters most for defect inspection? Precision is the fraction of flagged defects that are real; recall is the fraction of real defects caught; mAP is an aggregate that averages precision across confidence thresholds and defect classes. For defect inspection, per-class recall on the critical defect usually matters most, because it governs escapes reaching the customer — the metric mAP is most likely to hide. What does IoU measure, and why does localisation accuracy matter on a production line? IoU (Intersection over Union) measures how tightly a predicted box overlaps the true defect location. Localisation matters because when a detection feeds a robotic action or a dimensional measurement, a box that is only loosely on the defect is a functional failure even if it counted as a hit at a loose IoU threshold like 0.5. How does the confidence threshold change the precision/recall trade-off for an inspection system? The confidence threshold sets which predictions the system acts on. Raising it increases precision and cuts false alarms but drops recall as marginal true defects fall below the cutoff; lowering it raises recall but floods operators with false positives. This one chosen threshold defines deployed behaviour, which mAP — averaged over all thresholds — cannot express. Why can a high mAP still hide unacceptable recall on the defect class that matters? Because mAP averages performance across all defect classes with equal weight, a common easy class scoring near-perfect can mask a rare, safety-critical class scoring poorly. The aggregate looks acceptable while the one class capable of producing an escape defect is failing. How do object detection metrics translate into rework rate and escape-defect rate on the line? False-positive rate — governed by precision — drives rework and false alarms that erode operator trust. Missed-detection rate — the complement of recall — drives escape defects reaching the customer. A system at 99% precision but 80% recall passes a demo yet lets 1 in 5 real defects through. Which metrics should I demand from a vision-inspection vendor before validating their accuracy claim? Demand per-class recall on your critical defect, precision at the same operating point, the confidence threshold and IoU threshold the figures assume, both [email protected] and [email protected]:0.95, and the test-set composition. Together these make an accuracy claim auditable rather than a marketing headline. Metrics tolerances are not universal — they are set by what the line can absorb. Pharmaceutical visual inspection carries far stricter miss-rate tolerances than most industrial contexts, which is where computer vision that replaces manual visual inspection in pharmaceutical quality control makes the same metrics decisive under regulatory scrutiny. The metric vocabulary is constant; the threshold you can live with is where the real engineering decision sits. Before you accept any accuracy claim, name the operating point — because a percentage without a threshold is not a measurement, it is a slogan.
