Intersection over Union is one division: the area where a predicted box and a ground-truth box overlap, divided by the total area the two of them cover together. A perfect match scores 1.0. No overlap scores 0. That is the whole formula, and it takes about ten seconds to explain.
The part that takes longer — and the part that decides whether a reported accuracy figure means anything — is the threshold. IoU by itself is a similarity score on a single box pair. It becomes an accuracy metric only once someone declares how much overlap counts as a correct detection. That declaration is usually made silently, inherited from a benchmark convention, and then never stated again in the numbers that reach the people making a purchasing or deployment decision.
What is IoU, and how is it used to evaluate object detection?
Write the intersection area as I and the two box areas as A and B. The union is A + B − I, because the overlap would otherwise be counted twice. So IoU = I / (A + B − I).
A worked case: ground truth is a 100 × 100 px box on a scratch, so B = 10,000 px². The detector predicts a box that is shifted 30 px right and 20 px down, same size, so A = 10,000 px². The overlap region is 70 × 80 = 5,600 px². The union is 10,000 + 10,000 − 5,600 = 14,400 px². IoU = 5,600 / 14,400 ≈ 0.39.
That prediction is a correct detection at no common threshold. Shift the box only 20 px right instead of 30 and the overlap becomes 80 × 80 = 6,400, union 13,600, IoU ≈ 0.47 — still short of 0.5. Small pixel offsets move IoU a lot, and they move it non-linearly. This is the first thing worth internalising: IoU is not a gentle, forgiving measure of “roughly right”. It punishes displacement quickly, especially on small objects.
Detection evaluation then uses IoU as a matching rule. Each prediction is compared against the ground-truth boxes of its class; if the best available IoU clears the threshold and that ground truth has not already been claimed by a higher-confidence prediction, the prediction is a true positive. Otherwise it is a false positive. Ground-truth boxes left unmatched are false negatives.
From that matching, the two familiar formulas follow directly: precision = TP / (TP + FP), recall = TP / (TP + FN). Both are functions of the threshold, not fixed properties of the model. Precision and recall are not model properties; they are readings taken at a threshold you chose.
Why 0.5 and 0.75 disagree about the same model
Consider our shifted box at IoU 0.39 against a real scratch. At a 0.3 threshold it is a hit. At 0.5 it is a miss — and worse, it counts twice against the model: the ground truth becomes a false negative (recall drops) and the prediction becomes a false positive (precision drops). One prediction, two penalties. That asymmetry is why tightening the threshold degrades reported accuracy faster than people expect.
Which threshold is correct depends entirely on what happens downstream of the detection:
| Downstream action | Localisation tolerance | Reasonable threshold |
|---|---|---|
| Alert a human to inspect the part | Coarse — the operator finds the defect | 0.5 |
| Route the part to a rework station | Coarse — whole-part decision | 0.5 |
| Crop the region for a second classifier | Tight — a bad crop starves the classifier | 0.7 |
| Drive a robot end-effector or laser to the location | Very tight — position is the output | 0.75+ |
| Measure defect size against a tolerance spec | Very tight — the box is the measurement | 0.75+, plus box-dimension error |
In industrial inspection work, moving an acceptance threshold from IoU 0.5 to 0.7 typically exposes localisation error that the looser threshold had been counting as success — recall falls on the same held-out set with no change to the model at all (observed across TechnoLynx inspection engagements; not a published benchmark). Nothing broke. The measurement simply started asking a harder question.
What mAP hides
Mean Average Precision compresses the precision–recall curve into a single number per class, then averages across classes. Two conventions dominate. COCO-style mAP@[.5:.95] averages AP over ten IoU thresholds from 0.50 to 0.95 in 0.05 steps, which rewards tight localisation. Pascal VOC-style [email protected] uses a single loose threshold. The same weights can score, say, 0.89 at [email protected] and roughly half that at mAP@[.5:.95] — both figures are honest, and only one of them is usually quoted.
Three things a single mAP figure conceals:
- Class imbalance. mAP averages classes equally. A model that is excellent on the common defect and near-useless on the rare, expensive one can still post a respectable mean.
- The operating point. AP integrates over all confidence thresholds. Your deployment runs at one. The precision and recall you actually get at your chosen confidence cut-off are not visible in mAP.
- The IoU convention itself. If it is unstated, the number is not comparable between vendors or between two versions of your own model.
Anyone comparing detector scores across model cards without checking the IoU convention is comparing different measurements that happen to share a name. This is the same structural problem we describe in our account of why off-the-shelf computer vision models fail in production — a benchmark number produced under conditions the deployment will never see. Our broader approach to production computer vision engineering starts from the measurement definition rather than the model choice for exactly this reason.
Where IoU stops being a good metric
IoU is area-based, and area behaves badly for some object geometries.
Small objects suffer most. A 12 × 12 px particle has 144 px² of ground truth; a two-pixel offset in each axis drops IoU to roughly 0.51. The same two-pixel offset on a 200 × 200 px box is invisible. So a model with uniform pixel-level localisation error looks far worse on small objects than large ones — an artefact of the metric, not of the detector.
Thin, elongated defects are the second failure case. A hairline crack that is 300 px long and 4 px wide has an axis-aligned bounding box dominated by background. Two boxes can both contain the crack and still score a low IoU if their long axes are slightly offset. For this geometry, IoU on axis-aligned boxes measures box agreement, not crack agreement — which is one reason inspection teams move to oriented boxes or to segmentation masks, where per-pixel overlap is at least measuring the thing of interest.
Heavy occlusion breaks the matching rule rather than the ratio. When one object is 70% hidden, annotators disagree about whether the ground-truth box covers the visible fragment or the inferred full extent. Those two conventions differ by more than 0.5 IoU on the same object, so annotation policy — not model quality — decides whether the prediction is a hit. Variants such as GIoU and DIoU improve gradient behaviour during training for non-overlapping and misaligned boxes, but they do not resolve the labelling ambiguity.
Writing the threshold into an acceptance test
The reason to understand IoU precisely is to convert a vague accuracy claim into something testable before a model ships. In practice that means fixing five things in writing:
- The IoU threshold, chosen from the downstream action, not from convention.
- The confidence operating point the system will actually run at.
- Per-class precision and recall floors at that threshold and that operating point — a single aggregate number is not an acceptance criterion.
- The evaluation set: production-representative imagery, held out, with the annotation policy for occlusion and defect extent written down.
- The re-measurement trigger — which changes to lighting, optics, product mix, or model weights require the test to be re-run.
Fix those first and the accuracy figure in a report means the same thing to the vendor, the engineering lead, and the plant floor. Fix them after the first re-inspection cycle and the argument becomes a negotiation over what the original number meant. We put this sequencing at the front of a production CV readiness assessment because it is cheap to agree before deployment and expensive to litigate afterwards.
The uncomfortable follow-on question is what your current threshold was actually chosen for. If it is 0.5 because the framework defaulted to 0.5, and the detections feed a robot or a size-tolerance check, the reported accuracy is answering a question nobody in the deployment asked.
Frequently Asked Questions
What is Intersection over Union (IoU), and how is it used to evaluate object-detection models?
IoU is the overlap area between a predicted box and a ground-truth box divided by the total area the two cover together, giving a score from 0 to 1. Detection evaluation uses it as a matching rule: a prediction whose best IoU with an unclaimed ground-truth box clears a stated threshold counts as a true positive, and everything else falls into false positives or false negatives.
How is IoU calculated from intersection and union areas, and what does the formula look like with a worked example?
IoU = I / (A + B − I), where I is the overlap and A, B are the two box areas; the subtraction stops the overlap being counted twice. For two 100 × 100 px boxes offset by 30 px horizontally and 20 px vertically, the overlap is 5,600 px², the union is 14,400 px², and IoU ≈ 0.39.
What do precision and recall mean in object detection, and how do their formulas depend on the chosen IoU threshold?
Precision is TP / (TP + FP) — the share of detections that were real — and recall is TP / (TP + FN), the share of real objects found. Both depend on the threshold because the threshold decides which predictions become true positives, and a prediction that fails it is penalised twice: once as a false positive and once as a missed ground truth.
Why do IoU 0.5 and IoU 0.75 produce different accuracy figures for the same model, and which threshold should a deployment use?
At 0.5 a loosely placed box that covers roughly half the object still counts as correct; at 0.75 the same box is a miss, so tightening the threshold reveals localisation error that was previously scored as success. The right threshold follows the downstream action — 0.5 is defensible when a human inspects the flagged part, while robot guidance or size measurement needs 0.75 or tighter.
How do mAP, [email protected] and mAP@[.5:.95] relate to IoU, and what do they hide?
mAP averages per-class Average Precision, and the suffix names the IoU convention: [email protected] uses one loose threshold, while mAP@[.5:.95] averages ten thresholds from 0.50 to 0.95 and therefore rewards tight localisation. Both hide class imbalance, hide the confidence operating point the system will actually run at, and become non-comparable across sources when the convention is unstated.
What are the limits of IoU as a metric — where does it fail for small objects, thin defects, or heavily occluded subjects?
Because IoU is area-based, a two-pixel offset that is negligible on a 200 × 200 px box can push a 12 × 12 px object below 0.5, so uniform localisation error looks class-dependent. Thin elongated defects fill little of their axis-aligned box, making IoU measure box agreement rather than defect agreement, and heavy occlusion shifts the problem to annotation policy — whether the label covers the visible fragment or the inferred extent.
How should an IoU threshold and precision/recall floor be written into an expected-performance contract before a CV model ships?
State the IoU threshold chosen from the downstream action, the confidence operating point, per-class precision and recall floors at that pair, and the held-out production-representative set together with its annotation policy for occlusion and defect extent. Add the conditions that trigger re-measurement, such as changes to lighting, optics, product mix, or model weights, so the figure keeps its meaning after the first process change.
Why IoU Belongs in Every Detection Pipeline
Treat 0.5 as a reasonable default threshold for most bounding-box tasks, but always validate that choice against your own precision-recall curve before declaring a model production-ready. Everything else is detail.
