A YOLO detector reports 0.9 mAP on your held-out set and the pilot gets a green light. Six weeks later the line misses hairline cracks the benchmark said it would catch. The number was not wrong. It was answering a different question than the one you needed answered. This is the recurring pattern we see when a team evaluates a YOLO detector for defect inspection: a single headline number, mean average precision, gets treated as proof the system will catch defects on the line. It won’t tell you that. mAP is a composite of precision and recall measured across confidence thresholds and IoU cutoffs — a summary of how well the model ranks what it can see. It says nothing about whether the defect was ever rendered visible to the sensor in the first place. That distinction is not pedantic. It is the single most common reason a 0.9-mAP pilot quietly fails at line rate. How should you think about mAP in YOLO in practice? To read mAP you have to unpack what it is averaging. Start with a single class of defect — say, a surface scratch. For every detection the model outputs, it attaches a confidence score. You sweep a threshold across those confidences: at each threshold you get a precision (of the boxes I flagged, how many were real scratches) and a recall (of the real scratches present, how many did I flag). Plot precision against recall across the whole sweep and you get a precision-recall curve. The area under that curve is the average precision for that class. Do this for every defect class, average across classes, and you have mean average precision — mAP. YOLO’s training loop reports it because it is a stable, single-number proxy for detection quality that behaves well as an optimization target. The catch is what that single number hides. Averaging over the entire precision-recall curve rewards a model for ranking its true positives ahead of its false positives across all operating points. But you do not deploy a model across all operating points. You deploy it at one confidence threshold — the one your line’s tolerance for false alarms and missed defects dictates. mAP measures the whole curve; production lives at one point on it. Understanding how a detector produces those boxes and scores in the first place is worth grounding — our walkthrough of how an image detection model works in industrial inspection covers the mechanics that mAP then summarizes. What are precision, recall, and IoU, and how do they combine? Three ingredients go into the number, and each carries a different failure risk for inspection. Precision is the false-alarm axis. Low precision means the line stops for parts that were fine — a throughput and operator-trust cost, but rarely a safety or escape cost. Recall is the missed-defect axis. Low recall means defective parts pass. On most inspection lines this is the expensive failure: an escape reaches a customer. Recall, not mAP, is usually the number that predicts missed defects at your operating point (this is an observed pattern across the inspection engagements we run, not a universal law — the balance depends on what a false negative costs you). IoU — intersection over union — governs whether a detection counts as correct. A predicted box has to overlap the ground-truth box by some minimum fraction. At IoU 0.5, a box that half-covers the defect passes. At IoU 0.9, only tightly localized boxes count. This is where the two common mAP variants diverge, and it matters enormously for small defects. These three combine into average precision, then mean average precision, through the curve-and-average procedure above. The combination is mathematically clean and operationally lossy: by the time you reach a single mAP figure, you can no longer see which axis is carrying the score. How do [email protected] and [email protected]:0.95 differ, and which matters for small-defect inspection? This is the distinction that most often gets misread in feasibility reviews. [email protected] requires only 50% overlap between the predicted box and the ground truth. A model can score high here while localizing loosely. [email protected]:0.95 averages the score across ten IoU thresholds from 0.5 up to 0.95 in steps of 0.05. It penalizes sloppy localization — a detector that finds the defect but boxes it imprecisely loses points at the higher thresholds. For a large, obvious defect, the gap between the two is small; the box is easy to place tightly. For a small defect — a few pixels of pore, a hairline crack — the picture inverts. A model can reliably find the defect (strong [email protected]) but struggle to localize it to within 90% IoU because the object is only a handful of pixels wide, where a one- or two-pixel offset already blows the overlap. A high [email protected] paired with a much lower [email protected]:0.95 is a signature of exactly this: detection is happening, tight localization is not. For small-defect inspection, reading both numbers together tells you more than either alone. We unpack the two-number read in detail in mAP50 vs mAP50-95: reading CV inspection metrics on the line, and the single-threshold view in what mAP@50 means for defect detection. Quick-answer block: which number for which question? You want to know… Read this Not this Will defective parts escape at my operating point? Recall at the deployed confidence threshold Peak mAP Is the detector finding the defect at all? [email protected] [email protected]:0.95 alone Is localization tight enough for measurement / grading? [email protected]:0.95 [email protected] How often will the line false-stop? Precision at the deployed threshold mAP Will benchmark performance transfer to my line? Image-formation feasibility (see below) Any mAP number All of these are observed-pattern framings drawn from feasibility work, not fixed thresholds; the right operating point is set by what a false negative versus a false stop costs on your specific line. Why can a high benchmark mAP fail to predict defect detection on a real line? Here is the divergence that undoes most misread pilots: the image. A 0.9 mAP measured on images where the defect is clearly formed does not transfer to a line where lighting and optics leave that defect below the sensor’s reach. mAP is computed against a labeled dataset. If your dataset was captured under favorable lighting, with the defect crisply rendered, then the number reflects the model’s skill at ranking well-formed defects. On the line, a scratch that was obvious under a raking light in the lab may be invisible under the diffuse dome lighting that fixturing forced you into. The model cannot detect what the pipeline never rendered. This is the failure class worth naming plainly: a benchmark mAP earned on visible defects, misread as a guarantee for a line whose image formation may never surface the defect at all. The metric is measuring the model. The problem is upstream of the model — in the optics, lighting, and fixturing that decide what the sensor even sees. We separate these two questions deliberately. Whether the pipeline renders the defect is an image-formation question. Whether the model ranks rendered defects well is what mAP answers. A 0.9 mAP with a defect that image formation cannot surface is a 0.9 that will collapse the moment real lighting and fixturing constraints apply — and fine-tuning the detector will not fix it. Our note on fine-tuning YOLO for manufacturing-line defect detection is explicit about what tuning can and cannot repair; image-formation gaps sit firmly in the “cannot” column. What is the difference between peak mAP and recall at the confidence threshold you deploy at? Peak mAP is the best the model can do somewhere on its precision-recall curve. Recall at your deployed threshold is what it will actually do at the false-alarm rate your line tolerates. These can be far apart. A model can post an impressive mAP by ranking true positives ahead of false positives across the curve, yet still have modest recall at the specific high-confidence threshold you must run at to keep false stops manageable. If the number that predicts escapes is recall-at-operating-threshold, then peak mAP is answering a question your line never asks. The practical move is to stop reporting a single mAP and instead report the operating point: at the confidence threshold we will deploy, what is recall (escape risk) and what is the false-positive rate at line throughput? That is a decision-grade statement. “0.9 mAP” is not. A diagnostic checklist before you trust a YOLO mAP number Run this before a mAP figure feeds a feasibility go/no-go: What IoU is this mAP computed at? If only [email protected] is reported and your defects are small, ask for [email protected]:0.95 as well. What confidence threshold will you deploy at, and what is recall there? Not peak recall — recall at the operating point. What is the false-positive rate at that threshold, at line throughput? A tolerable precision on a static test set can become an intolerable stop rate at speed. Were the evaluation images captured under the line’s actual lighting and optics? If not, the mAP describes a different imaging condition than the one you will run. Is every defect class you care about actually present, and in realistic proportion, in the evaluation set? Averaging hides a class the model never learned to catch. Does image formation render the smallest defect you must catch? If the defect is below the sensor’s effective resolution, no mAP number rescues it. Steps 4 and 6 are where feasibility is won or lost, and they are outside the metric entirely. FAQ How does mAP in YOLO actually work? mAP is built by sweeping a confidence threshold to trace a precision-recall curve for each defect class, taking the area under that curve as average precision, then averaging across classes. In practice it is a stable single-number proxy for detection quality — but it summarizes the whole curve, while your line runs at one operating point on it. What are precision, recall, and IoU, and how do they combine into mean average precision? Precision is the false-alarm axis (flagged parts that were fine), recall is the missed-defect axis (real defects the model caught), and IoU sets how much a predicted box must overlap ground truth to count as correct. They combine into a precision-recall curve per class, whose area is average precision; averaging that across classes gives mean average precision. Why can a high mAP on a benchmark fail to predict defect detection on a real inspection line? Because mAP is computed against a labeled dataset, often captured under favorable lighting where the defect is crisply rendered. On the line, real lighting, optics, and fixturing may leave the defect below the sensor’s reach — and the model cannot detect what the pipeline never rendered. The number measures the model; the failure is upstream of it. What is the difference between peak mAP and recall at the confidence threshold you actually deploy at? Peak mAP is the best the model achieves somewhere on its precision-recall curve; recall at your deployed threshold is what it delivers at the false-alarm rate your line tolerates. Since recall-at-operating-threshold is usually what predicts escapes, peak mAP can be answering a question your line never asks. How do [email protected] and [email protected]:0.95 differ, and which matters for small-defect inspection? [email protected] requires only 50% box overlap; [email protected]:0.95 averages across ten IoU thresholds up to 0.95 and penalizes loose localization. For small defects, a high [email protected] with a much lower [email protected]:0.95 signals the model finds the defect but cannot box it tightly — so reading both together matters more than either alone. How should you read a YOLO mAP number before trusting it in a feasibility decision? Ask which IoU it was computed at, what recall and false-positive rate you get at the deployed confidence threshold and line throughput, and whether the evaluation images used the line’s real lighting and optics. Treat mAP as one input, not a verdict. What does mAP not tell you about whether the lighting and optics render the defect at all? Nothing. mAP evaluates the model’s ranking of already-rendered defects. Whether the smallest defect you must catch is even surfaced by the sensor under real lighting, optics, and fixturing is an image-formation question that lives entirely outside the metric. Where reading the number honestly leads The useful reframe is not “mAP is a bad metric.” It is a good metric for what it measures. The trap is asking it to certify something it never touched: whether your line’s image formation renders the defect at all. Once a detector reaches the line, the lab mAP has to be re-instrumented against operating-threshold recall — the reliability question the industrial-CV reliability work exists to monitor. Before a mAP figure decides a pilot, the question worth asking is not “is this number high?” but “was this number earned on images that look like my line?” That is a computer vision feasibility question, and it is answerable — an image-formation catalogue scoped to vision-pipeline feasibility separates a mAP earned on visible defects from one that will collapse under real lighting and fixturing. When a pilot’s headline mAP and its line performance disagree, the gap is almost always sitting in that catalogue, not in the model weights.
