A YOLO model card reports a single number and calls it accuracy. Someone reads [email protected] = 0.89, decides the detector is good enough, and ships it into a robot arm’s grasp pipeline. The boxes are loose, the grasp misses, and nobody can explain why the benchmark looked so clean. The metric was never wrong — it was answering a question nobody asked. mAP — mean average precision — is not one number. It is a family of numbers, and the difference between them is exactly the difference between “the detector found the object” and “the detector knows where the object is to within a few pixels.” For a lot of applications those two things collapse into one. For robotics, autonomous driving, and anything with a physical actuator downstream, they do not. How does mAP in YOLO work? Start from the two things a detector has to get right: is there an object of class C here (classification) and where exactly is it (localisation). mAP scores both at once, and the way it fuses them is where the subtlety lives. For a single class, you sweep the confidence threshold from high to low. At each threshold you compute precision (of the boxes you called positive, how many were real) and recall (of the real objects, how many you caught). Plot precision against recall and you get a curve; the area under that curve is average precision (AP) for that class. Average AP across all classes and you get mAP. But precision and recall themselves depend on a second decision: when does a predicted box count as matching a ground-truth box? That’s the Intersection over Union (IoU) threshold. A predicted box and a true box overlap by some ratio of intersection area to union area. If that ratio clears the IoU threshold, it’s a true positive; if not, it’s both a false positive (a bad box) and a false negative (a missed object). So every mAP figure is implicitly parameterised by an IoU threshold, and that is the parameter most people ignore. There’s a fuller treatment of how precision, recall, IoU, and AP compose in our walkthrough of object detection metrics for inspection pipelines. The short version: mAP is a two-axis score reported as one number, and reading it correctly means knowing which axis the number was tuned for. What is the difference between [email protected] and [email protected]:0.95, and when does each matter? This is the divergence point, and it’s worth being precise about. [email protected] uses a single IoU threshold of 0.5. A predicted box counts as correct if it overlaps the true box by at least 50%. That’s a forgiving bar. A box that’s roughly in the right place, a bit too big, slightly offset, still passes. [email protected] is essentially asking did you find the thing, more or less where it is. We cover what this number does and does not tell you in [email protected] explained. [email protected]:0.95 — the primary COCO metric — averages AP across ten IoU thresholds from 0.50 to 0.95 in steps of 0.05. As the threshold climbs toward 0.95, only boxes that hug the object tightly survive. This number rewards localisation precision, not just detection. A detector that finds every object but draws sloppy boxes will post a strong [email protected] and a much weaker [email protected]:0.95. The gap between the two is diagnostic. A large gap means the model localises loosely; a small gap means its boxes are tight wherever it detects at all. The side-by-side reading of mAP@50 vs mAP@50-95 treats that gap as a signal rather than noise. Which one matters depends entirely on what consumes the boxes: If the downstream consumer is… Localisation tightness needed Metric to gate on A counting or presence check (is the shelf empty?) Low — you need detection, not pixels [email protected] A tracker associating detections over frames Moderate — loose boxes cause ID switches [email protected]:0.95, watch the gap A 6DoF pose estimator or grasp planner High — box error propagates to the actuator [email protected]:0.95 + per-class floor A SLAM / localisation front-end High — loose boxes drift the map [email protected]:0.95 A regulated diagnostic overlay High and auditable per-class AP + fixed operating point For a retail out-of-stock detector, [email protected] is often the honest metric — you want to know a gap exists, not measure it to the millimetre. For a robot arm, the same score is a trap. Why can a model with high [email protected] still fail on tight localisation tasks like grasping or 6DoF pose? Because a 50% IoU box is enormously loose in physical space. Picture a 20 cm part on a conveyor. A box that clears IoU 0.5 can be off by a couple of centimetres in every direction and still count as a perfect detection under [email protected]. Feed that box’s centroid to a grasp planner and the gripper closes on the edge of the part, or on air. The failure is invisible in the benchmark because the benchmark, at IoU 0.5, has no vocabulary for “close but not close enough.” The information exists — it’s in [email protected]:0.95, where those loose boxes bleed AP as the threshold tightens — but if the acceptance test only ever read the headline number, the signal was thrown away before anyone looked. In our experience scoping robotics perception subsystems, this is the single most common way a detector “passes evaluation” and then fails integration. The model was validated against a metric that measures detection; the system needed a metric that measures pose-grade localisation. That’s an observed pattern across engagements, not a benchmarked failure rate — but it is consistent enough that we now insist on seeing the IoU sweep, not just the top-line figure, before signing off an accuracy criterion. The downstream tracking and pose stages are where loose boxes actually cost something. How should you choose an operating confidence threshold separately from the mAP you report? Here’s a distinction that trips up a lot of teams: the confidence threshold you deploy at is not the confidence threshold mAP is computed over. mAP integrates over all confidence thresholds — that’s the whole point of the precision-recall curve. It’s a summary of the detector’s ranking quality across the entire operating range. It tells you nothing about which single threshold you should run in production. That threshold is a separate decision, made against your cost of a false positive versus a missed detection. A safety-critical perception stack may run a low threshold to minimise misses, accepting more false alarms that a downstream tracker filters out. A high-precision inspection gate may run a high threshold to avoid false rejects. The confidence score’s role and how to set it is a decision you make from the P-R curve, not from the mAP scalar. So a reproducible acceptance test carries both: a reported mAP (typically [email protected]:0.95 on the deployment distribution) and a fixed operating point (confidence threshold, IoU threshold, per-class AP floor). Reporting mAP without the operating point is like quoting a car’s top speed as its commute time. How do per-class AP and dataset distribution change how you interpret a single mAP figure? mAP is a mean. A mean hides its own variance. A detector can post [email protected]:0.95 = 0.62 while scoring 0.85 on the three common classes that dominate the validation set and 0.20 on the rare safety-critical class that appears in 2% of frames. The headline looks respectable; the class you actually care about is failing. This matters twice over. First, the dataset distribution weights the mean — if your validation set over-represents easy classes, mAP flatters the model. Second, the class you build the system for is often the rare one: the pedestrian in an odd pose, the defective part, the anomalous obstacle. Averaging buries it. The fix is to read per-class AP and set a floor on the classes that carry safety or business risk, independent of the aggregate. A useful acceptance rubric looks like this: [email protected]:0.95 ≥ target on the deployment distribution (not the training distribution). Per-class AP ≥ floor for every safety-critical or business-critical class, no exceptions hidden by the mean. Fixed operating point — confidence and IoU thresholds recorded, not left to whatever the eval script defaulted to. Evaluated on data that matches deployment — lighting, camera, scene, class balance. A number from COCO tells you the architecture works; it does not tell you the model works on your floor. That last point is why we treat the deployment distribution as part of the metric, not an afterthought — a detector’s mAP is only meaningful relative to the data it was scored on. Why is mAP only half the acceptance criterion for a robotics perception subsystem under a latency contract? Because a detector runs inside a control loop with a deadline. A perception subsystem for a mobile robot or an autonomous vehicle has a frame-time budget — the box has to be produced before the controller needs it. A model that posts a beautiful [email protected]:0.95 but blows the frame-time budget is not a better detector for that system. It’s a worse one, because it produces the right answer too late to use. This is the pairing that defines a real acceptance test: accuracy and latency, together, as a single gate. mAP is the accuracy half. The perception latency contract is the other half — the frame-time budget the detector has to hit while it delivers the target mAP. We develop the latency side in our work on the perception pipeline under a latency contract, where the frame-time deadline is treated as a hard constraint the accuracy metric has to live inside. Practically, this reframes the whole exercise. You don’t pick the highest-mAP model and hope it’s fast enough. You fix the latency budget first — say, the detector gets N milliseconds per frame on the target hardware — then choose the highest-mAP model that fits inside it, and only then check its per-class AP floor. Chasing a leaderboard mAP without that budget produces a detector that wins the benchmark and loses the deployment. FAQ What matters most about mAP in YOLO in practice? mAP scores classification and localisation together. For each class you sweep the confidence threshold, compute precision and recall at each point, and take the area under the precision-recall curve as average precision (AP); mAP is the mean of AP across classes. Every mAP figure is also parameterised by an IoU threshold that decides when a predicted box counts as matching a ground-truth box, so a single mAP number is really a two-axis score reported as one. What is the difference between [email protected] and [email protected]:0.95, and when does each matter? [email protected] uses a single IoU threshold of 0.5 — a forgiving bar that asks whether you found the object roughly where it is. [email protected]:0.95 averages AP across ten IoU thresholds from 0.50 to 0.95, rewarding tight localisation, not just detection. Use [email protected] for presence or counting tasks; use [email protected]:0.95 whenever box precision propagates downstream, as in pose estimation or grasping. How do IoU thresholds, precision, and recall combine into an average precision score? The IoU threshold decides whether a predicted box matches a true box; from those matches you get true positives, false positives, and false negatives, which give precision and recall at each confidence level. Plotting precision against recall across all confidence thresholds produces a curve, and the area under it is average precision for that class. mAP is the mean of that AP across classes, so both the IoU threshold and the confidence sweep are baked into every number. Why can a model with high [email protected] still fail on tight localisation tasks like grasping or 6DoF pose? A box that clears IoU 0.5 can be off by a couple of centimetres in physical space and still count as a perfect detection. [email protected] has no vocabulary for “close but not close enough,” so loose boxes look fine. Feed that box to a grasp planner or pose estimator and the actuator misses — the localisation error was always there in [email protected]:0.95, but the headline metric threw it away. How should you choose an operating confidence threshold separately from the mAP you report? mAP integrates over all confidence thresholds, so it tells you the detector’s ranking quality across the whole operating range, not which single threshold to deploy at. The operating threshold is a separate decision made from the precision-recall curve against your cost of a false positive versus a missed detection. A reproducible acceptance test records both the reported mAP and the fixed operating point. How do per-class AP and dataset distribution change how you interpret a single mAP figure? mAP is a mean, and a mean hides variance: a detector can score well overall while failing a rare safety-critical class that the aggregate buries. The dataset distribution also weights the mean, so an easy-class-heavy validation set flatters the model. Read per-class AP and set a floor on the classes that carry risk, and evaluate on data that matches your deployment rather than a generic benchmark. Why is mAP only half the acceptance criterion for a robotics perception subsystem under a latency contract? A detector runs inside a control loop with a frame-time deadline, so a model with excellent mAP that misses the deadline is not an improvement. A real acceptance gate pairs accuracy with latency: fix the frame-time budget on the target hardware first, then choose the highest-mAP model that fits inside it, then check the per-class AP floor. Chasing a leaderboard mAP without the budget wins the benchmark and loses the deployment. Where this leaves the acceptance test If there’s one thing to carry away, it’s that a headline mAP is a summary, not an acceptance criterion. The acceptance criterion for a perception subsystem is a tuple: a localisation-grade mAP on the deployment distribution, a per-class floor for the classes that carry risk, a fixed operating point, and a frame-time budget the whole thing has to fit inside. Reading mAP correctly is reading it as one term in that tuple. The open question for any specific robotics stack is where the actuator’s tolerance actually sits — how tight a box the grasp or pose stage genuinely needs — because that number, not the leaderboard, is what sets your IoU threshold and your per-class floor. Getting that tolerance wrong is the failure class we scope for when we set detector acceptance criteria for a robotics CV subsystem: the metric passed, the actuator didn’t.
