“map 50” is [email protected] — mean average precision at an IoU threshold of 0.5 — and it is the headline number a lot of medical computer-vision teams cite when they say the model is ready. It is a genuinely useful engineering signal. It is not, on its own, the evidence an FDA submission needs, and treating the two as interchangeable is where clinical CV programmes quietly lose months.

The gap is easy to miss because the number looks authoritative. A CADe lesion detector reports [email protected] of 0.91 on a held-out set, the leaderboard agrees, and the model gets frozen for submission. Then the regulatory reviewer asks for sensitivity and specificity at a fixed operating point, stratified by lesion size and patient subgroup, tied to a reader study. Suddenly the single averaged number that drove the whole project does not answer a single one of those questions directly. The work to answer them was never done, and it has to be retrofitted onto a model that may or may not survive the scrutiny.

What [email protected] actually measures — and what it quietly averages away

Start with the mechanics, because the failure mode lives in them. For object detection, a predicted box is counted as a true positive when its Intersection-over-Union (IoU) with a ground-truth box meets or exceeds a threshold — here, 0.5. You sweep the detector’s confidence threshold from high to low, and at each step you record precision and recall. That traces out a precision-recall curve. Average precision (AP) is the area under that curve for one class; mean average precision is the mean of AP across all classes. If you want the mechanics at the level of the PR curve and the matching rule spelled out step by step, our walkthrough of what mAP50 measures and how to read it in practice covers exactly that.

Two properties of that construction matter enormously for medical work.

First, [email protected] integrates over the entire confidence range. It rewards a detector whose PR curve is broadly good, but it does not commit to any single point on that curve. A clinical device does not run at “the whole curve” — it runs at one confidence threshold, which fixes one sensitivity and one specificity. The averaged metric deliberately abstracts away the exact quantity a regulator cares most about.

Second, the “mean” in mAP hides distribution. A [email protected] of 0.91 could be one strong class dragging up two weak ones, or uniform performance across all three. Average away class, and you also tend to average away the subgroup structure a submission is judged on: performance by lesion size, by scanner vendor, by patient demographic. As a headline engineering metric [email protected] is doing its job — it compresses. As validation evidence, that same compression is the problem.

How is [email protected] computed from precision-recall curves and the IoU=0.5 criterion?

Concretely: match each prediction to ground truth using IoU ≥ 0.5, label matches as true positives and unmatched predictions as false positives; rank all predictions by confidence; walk down that ranked list computing running precision and recall; integrate precision over recall to get AP per class; average AP over classes to get [email protected]. The two knobs baked in are the IoU threshold (0.5) and the fact that you integrated rather than picked a point. Both are choices you inherited from benchmark convention, and both need revisiting for clinical use. This is the same PR-curve machinery behind the broader family of object detection metrics — precision, recall, mAP, and IoU; the medical context just raises the stakes on how you read them.

Why an IoU threshold of 0.5 can be too lenient for detection tasks that matter clinically

A 0.5 overlap means the predicted box and the true box share half their combined area. For a car in a street scene, that is plenty — you know where the car is. For a small pulmonary nodule or a subtle microcalcification cluster, a box that overlaps the ground truth by 50% can still miss the margin that determines whether a radiologist would act on it. The benchmark says “detected.” The clinical question — did the system localise the finding well enough to be useful and safe? — is not answered by a 0.5 threshold.

This is precisely why the localisation-quality distinction matters, and why the stricter averaged metric exists. The comparison between mAP@50 and mAP@50-95 is not academic pedantry: mAP@50-95 averages precision across IoU thresholds from 0.5 to 0.95, so it penalises loose localisation that [email protected] forgives. For a device where the boundary carries diagnostic weight, reporting only [email protected] can flatter a model that localises poorly. Our companion piece on the detection metric behind medical-device CV validation goes deeper on when the stricter metric earns its place in a validation package.

None of this means [email protected] is wrong. It means it answers “is the detector broadly finding things” and not “is the localisation clinically adequate.” Those are different questions, and a submission is built on the second.

From an internal signal to admissible evidence: the translation table

The core move — the one that separates a leaderboard exercise from a regulatory programme — is translating an averaged detection metric into a fixed, stratified, clinically-anchored evidence set. Here is the shape of that translation, laid out so it is usable without the surrounding prose.

Engineering artifact	Regulatory evidence it must become	What the translation requires
[email protected] (single averaged number)	Sensitivity + specificity at a locked operating point	Choose one confidence threshold; freeze it; report the sens/spec pair it produces
PR curve (whole curve)	ROC/operating characteristic at the chosen point, with confidence intervals	Bootstrap CIs on the frozen model; the point, not the curve, is the claim
Class-averaged AP	Per-class and per-subgroup performance	Stratify by lesion type, size, scanner, and patient population; report each, not the mean
IoU ≥ 0.5 matching	Clinically justified localisation criterion	Decide with clinicians whether 0.5 is adequate or a stricter IoU is required; document the rationale
Held-out test mAP	Reader study comparing readers with vs. without the device	Design the study on the frozen model; benchmark numbers do not substitute for reader performance

The principle behind every row is the same: the evidence has to trace back to one frozen model at one operating point. When teams report an averaged benchmark and plan to “re-derive thresholds at submission,” they break that traceability — and re-derivation on a model that has since changed means re-validation. Locking the operating point early is not bureaucratic caution; it is what keeps the sensitivity/specificity pairs, the per-class breakdowns, and the IoU criteria all pointing at the same artifact. In our experience across medical-CV programmes, this discipline is part of what lets teams reach cleared-device status roughly 6–12 months faster than those who retrofit it (observed pattern across engagements; not a benchmarked rate).

How do you turn [email protected] into the operating point a submission needs?

Pick the confidence threshold that gives the sensitivity/specificity balance the clinical use case demands — a CADe triage tool weights sensitivity heavily; a CADx characterisation tool balances differently — then freeze it. Everything downstream references that frozen threshold on that frozen model: the confidence intervals, the subgroup tables, the reader study. The averaged [email protected] informed which model to freeze; it does not itself appear as the operating-point claim. This is where the statistical rigour of the validation package lives, and it connects directly to A/B testing statistics for clinical CV models — the evidence has to hold up to FDA review, not just look good on a dashboard.

The failure class, named

The failure is optimising for leaderboard [email protected] without a locked operating point and stratified evidence. It shows up late, at submission, when the averaged number cannot answer the reviewer’s questions and the model has to be re-frozen and re-validated. The early warning signs are recognisable: a team that can quote its [email protected] to three decimals but cannot state its sensitivity at a fixed threshold; validation plans that say “operating point TBD”; test sets reported only in aggregate with no subgroup breakdown. Any of those, this early, is a sign the engineering metric and the regulatory evidence have not been connected.

Modern detector stacks make the aggregate number cheap to produce — a YOLO or RT-DETR training run in PyTorch prints [email protected] every epoch, and TensorRT-optimised inference will happily report it in production too. Cheapness is the trap. The number that is easiest to generate is the one least sufficient as evidence. The work that matters — freezing the operating point, stratifying the population, aligning the IoU criterion with clinical judgement, designing the reader study — none of it is printed automatically by the training loop.

FAQ

What should you know about map 50 ([email protected]) in practice?

[email protected] is the mean, across detection classes, of average precision computed under an IoU ≥ 0.5 matching rule. In practice it summarises how well a detector finds and roughly localises objects across its whole confidence range, compressed into one number — a strong engineering signal for comparing models, but an averaged one that does not commit to any single operating point.

How is [email protected] computed from precision-recall curves and the IoU=0.5 matching criterion?

Predictions are matched to ground truth when their IoU is at least 0.5; matches are true positives and unmatched predictions are false positives. Ranking predictions by confidence and walking down the list traces a precision-recall curve per class; the area under that curve is average precision, and the mean across classes is [email protected].

Why does the IoU threshold of 0.5 matter for medical detection tasks, and when is it too lenient?

A 0.5 threshold only requires half-area overlap between predicted and true boxes. For small or subtle findings — pulmonary nodules, microcalcification clusters — that overlap can still miss the margin a radiologist needs, so 0.5 becomes too lenient. Where the boundary carries diagnostic weight, a stricter IoU (or reporting mAP@50-95) is more honest about localisation quality.

How does an averaged [email protected] hide per-class and per-population performance that FDA validation requires?

The “mean” in mAP averages over classes, and aggregate test reporting averages over subgroups. A headline 0.91 can conceal one strong class carrying weak ones, or strong overall performance masking poor results for a specific lesion size, scanner vendor, or patient demographic — exactly the stratified breakdowns a submission is judged on.

How do you translate [email protected] into the sensitivity/specificity operating point a regulatory submission needs?

Choose a confidence threshold that gives the clinically appropriate sensitivity/specificity balance, freeze it on a frozen model, and report the sens/spec pair with confidence intervals, stratified by subgroup, at that single point. The averaged [email protected] informs which model to freeze; the locked operating point — not the curve — becomes the actual claim.

What are the limits of [email protected] as evidence for CADe/CADx devices, and what should complement it?

[email protected] answers “does the detector broadly find things,” not “is localisation clinically adequate” or “does the device improve reader performance.” It should be complemented by a locked operating point with sensitivity/specificity and confidence intervals, per-class and per-population stratification, a clinically justified IoU criterion, and a reader study on the frozen model.

Where this sits in a regulatory programme

The honest one-line summary is that [email protected] is where the regulatory pathway assessment separates an engineering metric from admissible validation evidence — everything upstream of the metric is model development, and everything downstream is the evidence package. If your team can produce the number but not yet the operating point behind it, that gap is worth surfacing early rather than at submission. Our work on the medical-device regulatory pathway within our computer vision practice is built around making that translation explicit before a model is frozen, not after. The remaining uncertainty in most programmes is not whether the detector is good — it is whether the evidence proving so points at the same frozen model the reviewer will actually see.