A detector reported as “97% accurate” almost always means one thing under the hood: mean Average Precision measured at a single, lenient IoU threshold of 0.5. For a benchmark leaderboard, that number is fine. For an automotive augmented-reality overlay that has to lock a highlight box or a lane cue to the actual vehicle in the road frame, it is close to useless — because at IoU 0.5 a box that only half-overlaps the real object still counts as a correct detection. The overlay drifts off the target, and the driver is looking at a highlight floating next to the car instead of on it. The metric that tells you whether an overlay will stay pinned is [email protected]:0.95 — often written map50-95 in tooling and papers. It averages Average Precision across ten IoU thresholds from 0.50 to 0.95 in steps of 0.05, so it rewards boxes that hug their objects tightly, not boxes that are merely in the right neighbourhood. If you select a perception model for a heads-up display on [email protected], you are optimising for the wrong thing. The divergence point is localization strictness, and that is exactly the axis a windshield overlay cares about. What does IoU actually measure, and why is 0.5 lenient? Intersection over Union (IoU) is the ratio of the overlap area between a predicted box and the ground-truth box to the area of their union. An IoU of 1.0 means the boxes are identical; 0.5 means the overlap is half the combined footprint. That sounds strict until you draw it: a predicted box can be noticeably offset, or too large, or too small, and still clear an IoU of 0.5. For a leaderboard, “did the detector find the object at all” is the interesting question, and 0.5 answers it well. For an AR HUD, the interesting question is “how precisely does the box match where the object really is,” because that box position is what the rendering stack anchors the overlay to. A detection that passes at IoU 0.5 but would fail at IoU 0.8 is a box that will visibly misregister on the glass. This is the gap map50-95 is designed to expose: it only gives full credit to predictions that survive the stricter thresholds too. How Average Precision is computed across IoU thresholds Average Precision starts from a precision-recall curve. For a given class and a given IoU threshold, you sort every predicted box by confidence, walk down the list, and at each step mark the prediction as a true positive (it matches an unclaimed ground-truth box at or above that IoU) or a false positive. As you descend, recall rises and precision fluctuates; the area under the resulting precision-recall curve is the Average Precision at that threshold. [email protected] computes that area at a single IoU threshold of 0.50 and then averages across classes — the “mean” in mean Average Precision. map50-95 repeats the whole computation at each of the ten thresholds 0.50, 0.55, … 0.95, and averages those ten AP values (then across classes). The COCO evaluation protocol popularised this, and it is the primary metric reported for COCO object detection. Because the higher thresholds demand tight localization, a model can score respectably at IoU 0.5 and then collapse at IoU 0.85 — and map50-95 folds that collapse into the headline number rather than hiding it. The practical consequence: [email protected]:0.95 is always lower than [email protected] for the same model, and the size of the gap tells you how loose the boxes are. A detector with [email protected] of 0.90 and [email protected]:0.95 of 0.55 is finding objects reliably but placing boxes imprecisely — a warning sign for anything that anchors graphics to those boxes. Comparing the two metrics for AR overlay registration Question [email protected] answers [email protected]:0.95 answers Did the model find the object? Yes — its core job Yes, but weighted by placement quality Is the box tight enough to anchor an overlay? Not measured Directly measured (via IoU 0.75–0.95 terms) Will a highlight sit on the vehicle or beside it? Cannot tell Correlates with staying on it Good for a discovery/recall screen Strong signal Overkill Good for HUD model selection Misleading in isolation The metric to track Typical relationship Always ≥ the strict metric Always ≤ [email protected] The table is the short version of the whole argument. When we look at a detector destined for a driver-facing overlay, the [email protected] figure tells us the model works as a detector; the map50-95 figure tells us whether the model works as a localiser. Only the second property survives contact with a windshield. The same discipline shows up in adjacent perception problems — keypoint estimation for driver-facing AR faces the identical “close enough on paper, off on the glass” trap with its own OKS-based scoring. Why a high [email protected] model can still be too loose for a HUD Consider two detectors evaluated on the same road-scene validation set. Model A scores 0.92 [email protected] and 0.61 [email protected]:0.95. Model B scores 0.89 [email protected] and 0.71 [email protected]:0.95 (illustrative figures, not a benchmark). By the lenient metric, Model A looks better and a team optimising for the leaderboard ships it. On the windshield, Model B wins — its boxes are consistently tighter, so the AR stack has less registration error to correct, and the highlight stays glued to the vehicle ahead through a lane change. The mechanism is straightforward once you see it. The overlay renderer places graphics using the detected box’s coordinates. If those coordinates are systematically a few pixels off — which is exactly what a large [email protected]@0.5:0.95 gap encodes — every frame inherits that offset. At highway speed, a few pixels of box error at the input translates into a visibly floating highlight after projection through the HUD optics. The model that “wins” on the loose metric is the model that misregisters. This is also why raw detection quality is only half the story. Even a well-localised box needs to be tracked consistently across frames so the overlay does not flicker between IDs, and the detector has to run inside a real-time budget — how YOLO inference behaves on GPU under batching determines whether you can afford the stricter, heavier models at frame rate at all. Localization fidelity, temporal stability, and latency are three separate constraints; map50-95 measures only the first, but it is the one that most directly maps to overlay drift. How localization quality feeds the pose and registration budget An automotive AR HUD carries a fixed error budget: the total angular misalignment between where a graphic is drawn and where the real object sits, as seen through the combiner, must stay under the tolerance a safety review will accept. That budget is spent by several sources — camera-to-display calibration, head/eye-position estimation, latency-induced lag, and detector localization error. Detection error is the one term map50-95 quantifies. Tighter localization (higher AP at the IoU 0.75–0.95 thresholds) means the detector consumes less of the budget, leaving more headroom for the pose and calibration terms the AR stack cannot easily reduce. In our experience with real-time perception stacks, treating detector localization as a budget line item — rather than a leaderboard bragging right — is what keeps the overlay inside tolerance once every other error source is stacked on top. This is a planning heuristic, not a benchmarked rate: the exact split depends on your optics, your tracking, and your latency profile. Getting the compute path fast enough to run a tight detector is its own problem, and it starts on the GPU — see how we approach fusing GPU passes for frame-locked overlays when the rendering pipeline has to keep up. What [email protected]:0.95 targets are realistic at real-time latency? There is no universal target — it depends on the detector family, the class set, and the resolution. What matters more than an absolute number is the trade against latency. Pushing map50-95 higher usually means a larger backbone, higher input resolution, or a heavier head, all of which cost inference time. For an AR HUD you cannot spend that time freely: the perception-to-render loop has to close inside a sub-frame window, or the overlay lags the world and the latency term eats the budget you saved on localization. The realistic move is to find the smallest, fastest model whose [email protected]:0.95 clears your registration budget with margin, not the model with the highest strict-metric score in absolute terms. A slightly lower map50-95 that runs comfortably at frame rate often beats a higher one that blows the latency budget — because a late-but-tight box still misregisters. This is the same “peak number versus operationally useful number” tension that shows up across GPU workloads, and reasoning about the broader perception, latency, and safety constraints of automotive AI is the right frame for it. FAQ What matters most about map50-95 in practice? map50-95 computes Average Precision at ten IoU thresholds from 0.50 to 0.95 in 0.05 steps and averages them. Because the higher thresholds only reward tightly-placed boxes, the metric measures localization fidelity, not just whether an object was found. In practice it is the number that correlates with an AR overlay staying pinned to the object rather than drifting off it. What is the difference between [email protected] and [email protected]:0.95, and why does the stricter metric matter for AR overlay registration? [email protected] scores a detection as correct if the predicted box overlaps the ground truth by at least half; [email protected]:0.95 averages across ten stricter thresholds up to 0.95. The stricter metric penalises loose boxes that pass at 0.5 but would fail at 0.8. For AR registration, the overlay is anchored to the box coordinates, so loose boxes mean visible drift on the glass — which only the stricter metric exposes. How is Average Precision computed from precision-recall curves across IoU thresholds? For a class and an IoU threshold, predictions are sorted by confidence and each is marked a true or false positive based on whether it matches an unclaimed ground-truth box at that IoU. Walking down the list traces a precision-recall curve, and its area is the Average Precision at that threshold. [email protected]:0.95 repeats this at all ten thresholds and averages, then averages across classes. Why can a model with high [email protected] still produce boxes too loose to anchor a driver-facing overlay? A high [email protected] only confirms the model finds objects; it says nothing about placement precision. A large gap between [email protected] and [email protected]:0.95 encodes systematically offset boxes. The overlay renderer inherits that offset every frame, so at speed the highlight floats beside the vehicle instead of on it — the model “wins” on the loose metric and misregisters on the windshield. How does localization quality measured by mAP50-95 feed into the pose/registration budget of an automotive AR HUD? A HUD has a fixed total-misalignment budget spent across calibration, head/eye tracking, latency lag, and detector localization error. map50-95 quantifies the detector term. Higher AP at the strict IoU thresholds means the detector consumes less of the budget, leaving headroom for the pose and calibration terms the AR stack cannot easily reduce. What mAP50-95 targets are realistic for real-time automotive detectors, and how does that trade against latency? There is no universal target — it depends on the detector family, class set, and input resolution. Raising map50-95 typically costs inference time via a larger backbone or higher resolution, which an AR HUD cannot always afford. The realistic goal is the smallest, fastest model whose strict-metric score clears your registration budget with margin, since a late-but-tight box still misregisters. How should a team use mAP50-95 alongside safety-review criteria when selecting a perception model for a HUD? Treat map50-95 as a concrete input to the perception-quality section of a safety review, not as the whole story. Pair it with latency measurement and temporal stability, and check that the resulting localization error fits inside the registration budget the review demands. Model selection on the strict metric is one line item in the audit that gates HUD deployment. The question worth carrying out of this piece is not “which detector has the highest score” but “which detector places boxes tightly enough, fast enough, to leave my AR stack a registration budget it can actually live inside.” That is a localization question, and [email protected]:0.95 is the number that answers it — long before the misregistration shows up on someone’s windshield.
