Ask an engineering lead how ADAS works and you usually get a tour of the block diagram: camera, radar, fusion, planner, actuator. The boxes connect, the arrows point forward, and nobody can act on any of it. A more useful answer walks the same stages in reverse — starting from the driving scenarios that carry the risk, and asking which stage is the binding constraint for each one. That reading produces a measurement plan instead of a diagram.
The difference matters because the two readings end in different artifacts. Reading the pipeline top to bottom ends when the interfaces line up. Reading it backwards from scenario classes ends with a table: each row a scenario class, each row owning one pipeline stage, each row carrying the evidence needed before the behaviour can be trusted. In our experience, the largest number in the first version of that table is the count of scenario classes with no evidence at all — and it is almost never the number anyone came to the review to discuss.
What does “how does ADAS work” mean in practice?
Structurally, an advanced driver-assistance system is four stages plus an actuator. Sensing acquires the raw signal (camera frames, radar returns, sometimes lidar point clouds). Perception converts that signal into objects, lanes, and free space, usually with convolutional or transformer detectors exported through ONNX and served on an automotive SoC via TensorRT. Fusion arbitrates between modalities that disagree. Decision turns the fused world model into a brake, steer, or throttle command.
Read as an explanation, that is fine. Read as an engineering answer, it is missing the only thing that varies: which stage runs out of headroom first depends entirely on the driving scenario, not on the architecture. Night-time pedestrian crossing, low-sun glare on a wet road, a motorcycle lane-splitting at closing speed, roadworks with temporary markings — each of those stresses a different stage. The pipeline is constant; the binding constraint is not.
So the practical answer to “how does ADAS work” is: it works differently per scenario class, and you do not know how it works until you have named the classes and attributed each one to a stage. The parent article develops the underlying anatomy and why the long tail of rare scenarios dominates production risk — we work through that in our ADAS perception pipeline breakdown. What follows is the method for turning it into work a team can run this quarter.
Step 1 — Enumerate scenario classes from the feature list, not from imagination
Do not start with a taxonomy. Start with the features the vehicle actually ships: automatic emergency braking, lane-keeping assist, adaptive cruise control, blind-spot monitoring. Each feature implies a small set of situations in which it must act and a smaller set in which it must not. AEB, for example, decomposes into at least: stationary lead vehicle, decelerating lead vehicle, crossing pedestrian in daylight, crossing pedestrian at night, cyclist from the near side, and false-positive suppression under overhead gantries.
Six to twelve classes per feature is a workable first pass. The purpose is not completeness. It is to break the single aggregate accuracy metric into rows that can each fail independently, so that each feature carries its own evidence rows rather than sharing one number with every other feature.
Step 2 — Attribute each class to its binding stage
Ask what would have to improve for the failure to disappear. That question, asked honestly, assigns the stage.
| Symptom in the scenario class | Binding stage | Discriminating test |
|---|---|---|
| Target absent from the raw signal (below sensor range, occluded, saturated by glare) | Sensing | Replay the raw frames — if a human annotator cannot label the target, no model will detect it |
| Target present in the signal but not detected or misclassified | Perception | Same clip, offline model at full precision — if it detects, the deployed quantisation or resolution is the cause |
| Modalities disagree and the wrong one wins | Fusion | Inspect per-modality tracks pre-arbitration; camera says pedestrian, radar says nothing, output says nothing |
| Object correctly tracked, response late or absent | Decision | Check the planner’s log against a correct world model — the input was right, the policy was wrong |
| Systematic spatial offset across many classes at once | Sensing (calibration) | Failures cluster by image region or range band rather than by object class |
The last row is the one teams get wrong most often. Calibration drift and perception weakness produce the same top-line symptom — missed or mislocated objects — but they have different signatures in the table. A calibration problem degrades many scenario classes together and correlates with position in the frame or distance from the vehicle; a model problem concentrates in specific classes and correlates with object appearance. Misattributing one for the other costs a development cycle: a team retrains a detector for six weeks when the fix was a re-extrinsics procedure after a windscreen replacement.t.
Step 3 — Record the evidence, including the absence of it
Each row then needs three numbers, and honesty about which are missing:
- Long-tail failure rate by class — measured on clips drawn from that class specifically, not on the pooled validation set.
- Benchmark-to-production delta — the accuracy gap between the curated benchmark set and a sample of the actual production driving distribution for that class. If the benchmark contains 40 night pedestrian clips and the fleet encounters that geometry hundreds of times a week, the benchmark number is not a prediction.
- Evidence status —
measured,proxy only, ornone. Proxy metrics are the quiet failure of this method; mean average precision on a public dataset is not evidence about night-time pedestrian braking on your vehicle.
A first pass typically returns a table where a minority of rows are measured and the rest are proxy only or none. That is not a failure of the method, it is the method working — the unmeasured classes were always unmeasured, and now they are visible in a review instead of hidden inside one aggregate figure. Sustained per-class monitoring after deployment is what keeps the table from decaying; the Production AI Monitoring Harness is where those per-class counters live once the vehicle is in the field.
Keeping the table alive across releases
The table is only worth building if it survives the next model update and the next sensor-configuration change. Two rules keep the maintenance cost bounded. First, version the attribution, not just the numbers: a stage attribution changes rarely, so a release that swaps a detector re-measures the perception-owned rows and leaves the sensing- and decision-owned rows untouched. Second, treat a sensor-configuration change — new camera, moved radar, different mounting — as invalidating every row attributed to sensing and calibration, and only those. Re-running the whole mapping each release is what causes teams to abandon it by the third sprint.
This is the same discipline we apply to reliability audits outside automotive: name the operating conditions, attribute each to the component that binds it, and refuse to accept an aggregate metric as evidence about a specific condition. The vision-engineering side of that work sits under computer vision systems, where the detector and calibration questions in the table above are the day-to-day material.
The open question is where the boundary sits between a scenario class that deserves its own row and one that is a variant of an existing row. Split too finely and no class has enough clips to measure; split too coarsely and the long tail hides again inside the aggregate. We have not found a rule that settles it independently of the feature — which is itself worth knowing before the first review.
Frequently Asked Questions
What does ‘how does ADAS work’ mean, and what does it mean in practice?
As an explanation, it means four stages plus an actuator: sensing, perception, fusion, decision. In practice it means something narrower and more useful — the system behaves differently per driving scenario, and the stage that runs out of headroom first changes with the scenario. You have not answered the question in an engineering sense until you can say which stage binds which scenario class.
How do you map each ADAS pipeline stage to the driving scenario classes it constrains?
Work backwards. List the scenario classes implied by the shipped features, then for each class ask what would have to improve for the failure to disappear. If a human annotator cannot label the target in the raw frames, the class is sensing-bound; if an offline full-precision model detects it, the class is perception- or deployment-bound.
How do you decide whether a given failure belongs to sensing, perception, fusion, or decision — and why does misattributing it waste a development cycle?
Use discriminating tests rather than intuition: raw-signal replay separates sensing from perception, pre-arbitration per-modality tracks expose fusion errors, and planner logs against a known-correct world model isolate decision faults. Misattribution wastes a cycle because each stage has a different remediation team and a different lead time — a retraining sprint cannot fix an extrinsics error, and a recalibration cannot fix a policy that brakes late.
How do you build a first scenario-class list for a specific ADAS feature without boiling the ocean?
Take one feature and enumerate six to twelve situations in which it must act plus the main situations in which it must not. For AEB that is stationary and decelerating lead vehicles, day and night crossing pedestrians, near-side cyclists, and false-positive suppression cases. Completeness is not the goal; breaking one aggregate metric into independently failing rows is.
What measurements should a team collect per scenario class, and what counts as sufficient evidence before shipping?
Three per row: failure rate measured on clips from that class specifically, the delta between benchmark performance and a sample of the production driving distribution, and an explicit evidence status of measured, proxy-only, or none. Sufficient evidence means the class was measured on its own data at a volume proportional to how often the fleet encounters it — a public-dataset mAP score is a proxy, not evidence.
Scenario Classes: A Structured ADAS Testing Approach
Coverage becomes measurable only when you partition the operational design domain into testable, mutually exclusive scenario families. Revisit it when your workload shifts.
