A perception model that scores 0.9 mAP on a held-out set has not earned a release signature. It has earned a benchmark number. Those are different things, and the gap between them is where autonomous-car programs quietly accumulate the failures that later force a fleet-wide rollback. The naive picture of autonomous cars deep learning is a single trained network that “sees the road” — feed it camera frames, out come bounding boxes, and if the aggregate accuracy is high the model works. That picture is comforting because it collapses a hard question (is this safe to deploy?) into an easy one (does the number look good?). It is also wrong in a way that matters, because the risk that actually causes incidents does not live in the aggregate. It lives in the long tail: the occluded pedestrian, the overturned truck the training set never saw, the camera fogged by low sun, the intersection layout that exists in exactly one city. The reframe we want to land is simple. A perception model reaches a release-signable state not when its benchmark improves, but when an engineering reviewer has evidence — scenario-sliced, drift-monitored, regression-guarded — that they can sign against for a specific operational design domain. Everything below is about what that evidence looks like and why the benchmark number alone can’t produce it. How does autonomous cars deep learning actually work? An autonomous vehicle does not run one model. It runs a perception stack — a pipeline of models that each do a narrow job and hand their output to the next stage. Understanding the release problem requires seeing the stack, not the network. At the front is detection: convolutional or transformer-based detectors (YOLO-family models, DETR variants, and their multi-camera successors) that turn raw sensor frames into object proposals — cars, pedestrians, cyclists, signs, lane markings. Detection is per-frame and stateless; it has no memory of what it saw a moment ago. Tracking adds that memory. It associates detections across frames so a pedestrian entering the scene remains the same tracked object as they cross, giving the planner a velocity and a trajectory instead of a flickering box. Tracking is where identity, occlusion recovery, and temporal consistency get resolved — and where a strong per-frame detector can still produce a dangerous track if association breaks under occlusion. Sensor fusion combines the camera view with lidar, radar, and sometimes ultrasonic returns into a single coherent scene estimate. Fusion is what lets the system trust a radar range measurement when the camera is blinded by glare, or resolve a lidar point cloud into the object the camera classified. The failure surfaces here are subtle: a fusion model that over-weights one modality inherits that modality’s blind spots. These three stages compose. A detection miss becomes a tracking gap becomes a fusion inconsistency becomes a planning error. A benchmark that scores the detector in isolation tells you nothing about how an occlusion propagates through tracking into fusion — and that propagation is where real-world perception fails. We walk through how these stages are trained and validated in more depth in Autonomous Cars Machine Learning: how perception models are built and validated; this article is about what happens after they score well, when someone has to decide whether to ship. Why is benchmark accuracy an insufficient basis for release? Aggregate accuracy is a weighted average, and averages hide their tails by construction. This is the structural problem, not a tuning problem. Consider what a single mAP figure actually measures: detection quality averaged over a test distribution that, almost by definition, resembles the training distribution. Rare and dangerous cases are rare in the test set too — so a model can drop every occluded child in the dataset and barely move the aggregate metric, because occluded children are a fraction of a percent of the frames. The number stays green while the specific failure that ends a program stays invisible. Three things follow from this that a benchmark cannot tell you: Whether the model is safe on a specific operational design domain. A model validated on California highway footage carries no evidence about a European roundabout in snow. The benchmark is silent on domain fit because the domain was never a variable in the score. How failures distribute across scenario slices. “94% recall” could mean uniform 94% everywhere, or 99% on clear daytime and 60% at dusk. Those are radically different safety cases and identical numbers. Whether the model will still behave this way next month. A benchmark is a photograph. Deployed perception operates in a distribution that drifts as seasons, construction, and sensor wear change what the cameras see. The observed pattern across the perception-validation work we have been involved in is consistent: teams that ship on aggregate metrics discover their tail-risk failure modes in the field, where the cost of discovery is an incident report rather than a test log. (This is an engagement-level observation, not a published benchmarked failure rate.) Closing that gap is the entire job of the monitoring harness, and it is the same release-readiness gate described from the infrastructure side in our production AI reliability work — applied here specifically to sensor and scene distribution. What tail-risk failure modes must the eval evidence cover? If the benchmark hides the tail, the eval harness has to expose it deliberately. That means enumerating the failure classes and building a scenario-sliced eval set that has enough examples of each to produce a per-slice number, not just a global one. The four failure classes that dominate automotive perception tail-risk: Failure mode What it looks like What the eval evidence must show Occlusion Pedestrian behind a parked car, truck partially hidden by a merging vehicle Recall on partially and heavily occluded objects, tracked through re-emergence, reported as its own slice Rare objects Debris, animals, overturned vehicles, unusual construction equipment Detection behaviour on out-of-catalog classes; how the stack treats “unknown but present” Sensor degradation Sun glare, rain on the lens, lidar attenuation in fog, a partially failed camera Graceful degradation and fusion fallback when one modality is unreliable Out-of-distribution scenes Road layouts, weather, or regions absent from training data Whether confidence and behaviour stay calibrated when the scene is unfamiliar The point of the table is not the list — it is that each row demands its own eval slice with its own pass/fail threshold. A release reviewer signs against per-slice evidence, not a global average. When a slice is under-populated (too few occluded-pedestrian frames to be statistically meaningful), that is itself a finding: the evidence is insufficient, and the honest answer to the release question is “not yet.” Curating those slices correctly depends on annotation discipline — the difference between a usable eval set and a misleading one often comes down to label quality, which we cover in COCO Labels Explained. What does the monitoring harness look like for an automotive workload? The monitoring harness is the artifact that turns “the model scores well” into “here is the evidence a reviewer can sign against.” For a perception workload it has two load-bearing sections. The eval harness. This is the scenario-sliced test infrastructure plus a regression suite. The eval harness holds the per-slice test sets described above and produces a coverage report — how many examples per slice, what the per-slice metrics are, and where coverage is thin. The regression suite is the guard against silent backsliding: every candidate model runs against a frozen set of known-hard cases, and any model that loses ground on a previously-passing slice is blocked regardless of how much its aggregate metric improved. This is where the discipline of an end-to-end ML pipeline with reliability gates at each stage becomes concrete for perception: the release gate is a checklist of slice results, not a single threshold. Drift telemetry. Once deployed, the harness monitors the input distribution, not just model outputs. It tracks sensor characteristics (exposure histograms, lidar return density, per-camera health) and scene distribution (object-class frequencies, geographic and weather mix) against the distribution the model was validated on. When live sensor or scene statistics diverge from the validated envelope, the telemetry flags it — because a divergence means the model is now operating outside the domain its evidence covers, even if no output metric has moved yet. The techniques for capturing this telemetry cleanly are the same ones we describe for line-side vision in ML in Automotive: how perception models earn a production monitoring harness. Together these sections do something a benchmark cannot: they make tail-risk quantifiable and auditable. The eval harness quantifies it before release; drift telemetry watches for it after. How do you detect and respond to distribution drift after deployment? Detection is a comparison problem. You need a validated reference distribution — the sensor and scene statistics the release evidence was built on — and a live stream to compare against it. Practically, this means logging summary statistics (not raw frames, for both privacy and storage reasons) per operational region and computing distance metrics against the reference. Common approaches use population-stability indices or divergence measures on class-frequency and sensor-health distributions; the exact metric matters less than choosing thresholds before you look at the live data, so the trigger is principled rather than post-hoc. Response is a graded escalation, not a binary alarm: Within envelope — live distribution matches the validated reference. No action; log the confirmation. Drift detected, model still in-domain — a new object frequency or a seasonal shift appears, but the model has validated evidence for it. Note it, watch the trend. Drift beyond validated envelope — the live distribution now includes conditions the eval evidence never covered. The model is operating on unsigned territory. This triggers a targeted data-collection and re-validation cycle for that slice. Drift plus output regression — divergence and a measurable behaviour change. This is the pre-incident signal that justifies restricting the operational design domain until re-validation closes the gap. The reason the response is graded is that most drift is benign and constant re-training on noise degrades a stable model. The harness exists to tell the difference — and to make the difference documentable. FAQ What’s worth understanding about autonomous cars deep learning first? In practice it is a perception stack, not a single network: detection models turn sensor frames into object proposals, tracking associates them across frames, and sensor fusion combines camera, lidar, and radar into one scene estimate. These stages compose, so a failure in one propagates through the others. “It works” means the composed stack behaves safely on a defined operational design domain — not that one network scored well on a benchmark. What are the main perception models — detection, tracking, and sensor fusion — and how do they fit together? Detection (YOLO-family or DETR-style networks) is per-frame and stateless. Tracking adds temporal memory, associating detections into persistent objects with velocity and trajectory, and resolving occlusion recovery. Sensor fusion merges modalities so the system can trust radar when the camera is blinded or confirm a camera class with lidar geometry. They fit together as a pipeline: detection feeds tracking feeds fusion feeds planning. Why is benchmark accuracy an insufficient basis for releasing a perception model? Aggregate accuracy is an average, and averages hide their tails. A model can miss nearly every occluded child while barely moving its mAP, because such cases are rare in the test set too. The benchmark is also silent on operational-design-domain fit and cannot predict post-deployment drift. Release decisions need per-slice evidence, not a global number. What tail-risk failure modes must the eval evidence cover? Four dominate: occlusion (partially or heavily hidden objects), rare objects (debris, animals, out-of-catalog classes), sensor degradation (glare, rain, fog, partial camera failure), and out-of-distribution scenes (unfamiliar layouts, weather, or regions). Each needs its own scenario-sliced eval set with its own pass/fail threshold; a thinly-populated slice is itself a finding that the evidence is insufficient. What does the monitoring harness’s eval-harness and drift-telemetry section look like for an automotive workload? The eval harness holds scenario-sliced test sets plus a regression suite that blocks any model losing ground on a previously-passing slice, and produces a per-slice coverage report. Drift telemetry monitors the live input distribution — sensor characteristics and scene mix — against the validated reference, flagging when the model operates outside the domain its evidence covers. Together they make tail-risk quantifiable before release and auditable after. How do you detect and respond to distribution drift in sensor and scene data after deployment? Log summary statistics per region, compare them to the validated reference distribution using divergence or stability metrics, and set thresholds before looking at live data. Respond on a graded scale: within envelope (log only), drift but in-domain (watch), drift beyond the validated envelope (trigger targeted re-validation), and drift plus output regression (restrict the operational design domain until the gap closes). Grading avoids re-training on benign noise. Who signs the release-readiness review for a perception model, and against what evidence? An engineering reviewer with domain authority signs the go/no-go decision, and they sign against the harness evidence — per-slice eval results, regression-suite pass status, and drift-telemetry baselines for the target operational design domain — not against a single aggregate metric. When a required slice is under-covered, the signable answer is “not yet.” This is the same release-readiness gate the AI-infrastructure release decision describes, applied to sensor and scene distribution. The question that separates a demo from a deployment The honest test for any autonomous-car perception model is not “how high is the score” but “can someone sign their name against its behaviour on this road, in this weather, with this sensor set — and defend that signature later?” A benchmark cannot answer that question because it never asked about the domain. A monitoring harness can, because it was built to slice the tail, guard against regression, and watch the distribution for the day it drifts past what the evidence covers. That is the difference between shipping on a number and shipping on evidence — and it is exactly the release-readiness artefact a perception program has to produce before the first vehicle carries the model onto a public road.
