A perception team has run the test campaign. The metrics look good. They drop the results into a slide deck, attach the benchmark scores, and send the pack to review expecting a sign-off. Three weeks and four clarification rounds later, the release window has slipped — and not one of the clarification questions was about whether the model was accurate. That gap is the whole problem. A perception validation evidence package does not clear review because the model is good. It clears review because the reviewer can answer their own approval questions from the contents of the pack without having to ask the team a single follow-up. The most common reason a release stalls is not a weak model — it is a pack structured around the team’s test backlog instead of around the reviewer’s decision. What Does a Perception Validation Evidence Package Contain? The naive answer is “the test results.” That answer is why packs get bounced. A reviewer signing off on a perception release is not grading a homework assignment; they are accepting risk on behalf of an organization, and they have a fixed set of questions they need answered before they will put their name to it. In our experience working with automotive perception teams, those questions fall into four families, and a pack that answers all four in the reviewer’s own order tends to clear on the first pass. The questions are about production behaviour, drift posture, ownership, and the rollback path. The test results are not the package — they are the evidence base the package draws on to answer each question. A workable package contains, at minimum: A production-behaviour section — how the model behaves under the operational conditions it will actually meet, not the curated test set. This is where the robustness audit results become evidence rather than a spreadsheet. A drift-posture section — what is monitored in production, what thresholds trigger an alert, and who sees that alert. An ownership section — who owns the model post-release, who owns the monitoring, and who has authority to pull a build. A rollback section — the concrete, tested path back to a known-good state, with the trigger conditions named. A traceability index — every claim in the four sections above linked to the specific test, dataset, or monitor that produced it. The traceability index is the part teams skip and reviewers fixate on. A claim with no traceable origin is a claim the reviewer cannot verify, and an unverifiable claim is a clarification round waiting to happen. How Do We Link Test Results to Production Behaviour? This is the hinge of the whole exercise. A test result measures behaviour under test conditions. A reviewer cares about behaviour under production conditions. The package has to make that bridge explicit, because if it does not, the reviewer will — and they will do it by asking. The mechanism is to anchor each production-behaviour claim to the test that supports it, and to state the conditions under which that test was run. “The detector maintains recall above the release threshold across the night-driving and adverse-weather slices” is a claim. The evidence is the slice-level results, the dataset provenance, and the operational-design-domain mapping that says those slices represent real driving. When this link is missing, the gap between test recall and production recall becomes an open question the pack cannot close. We see one pattern repeatedly: teams report a single aggregate metric — say, mAP across the full test set — and a reviewer immediately asks for the breakdown by operating condition. The aggregate hides exactly the failure modes the reviewer is paid to worry about. A pack that leads with the slice breakdown, attributes each slice to an operational condition, and links each number to its test run pre-empts that round entirely. None of this requires new tooling beyond what a standard PyTorch or ONNX Runtime evaluation harness already emits — it requires structuring the output around the question, not the run. The cost story underneath this matters too. The reason the slice breakdown is credible is that the measurement was taken under realistic load and conditions rather than a synthetic best case — the same reasoning that governs why peak benchmark numbers rarely predict steady-state behaviour. A reviewer who trusts the measurement method spends less time interrogating the numbers. Structure the Pack Around the Reviewer’s Question, Not Your Test Backlog Here is the reframe that separates a first-pass clearance from a clarification cycle. Most teams build the pack in the order they ran the work: here are the tests we ran, here are the scores, here is the appendix. The reviewer then has to reverse-engineer the answers to their approval questions out of that material. Every gap in that reverse-engineering becomes a question, and every question becomes a round. Invert it. Each top-level section of the pack is one reviewer approval question, stated as a heading. Underneath it sits the evidence that answers it, and only that evidence. A test that does not answer any approval question does not belong in the body of the pack — it belongs in the appendix the reviewer reads only if they want to. Evidence Surface → Reviewer Question Map Reviewer’s approval question Evidence surface that answers it Linked test / artefact Will it behave acceptably in production conditions? Slice-level performance by operational condition Robustness audit + ODD mapping Will it degrade silently after release? Drift monitor definition + alert thresholds Production monitoring harness config Who is accountable once it ships? Ownership matrix (model, monitoring, release authority) RACI + on-call rotation Can we get back to safe if it goes wrong? Rollback procedure + trigger conditions Tested rollback runbook Can I verify any claim in this pack? Traceability index Per-claim link to source test This table is the whole article in one surface. If a perception team can fill every row with a linked artefact, the pack is structured for review. If any row is empty or unlinked, that row is the clarification round the team is about to receive. Which Gaps Turn a Clearance Into a Re-Review? From what we have observed across perception validation work, three gaps account for most re-reviews. The first is the missing drift posture: the team proves the model is good today but says nothing about how anyone will know when it stops being good. A reviewer cannot accept a static snapshot of a system that will face a non-stationary world. This is the reliability-discipline point — the evidence package is itself the artefact that carries the drift commitment, not just the accuracy claim. The second gap is unowned monitoring. The pack describes a monitor but never says who watches it or who can act on it. An alert with no owner is decoration. The third is an untested rollback. Teams describe a rollback path in prose and never demonstrate it was exercised; reviewers have learned to distrust a recovery procedure that has only ever run in a document. What makes these costly is timing. A gap caught in review costs a clarification round. The same gap caught after release — a silent drift no one was watching, a rollback that did not work when it was finally needed — costs a field incident. The evidence package is where you pay the cheap version of that bill. This is the artefact-shaped expression of treating reliability as a discipline rather than a one-time score, and it is the same evidence-pack pattern that holds in regulated domains — the structure of a clinical-grade medical imaging validation engagement maps the same four question families onto a different reviewer. A Worked Example: From Audit Output to Release Pack Suppose a team has completed a robustness audit on a pedestrian-detection model and holds slice results, a set of monitors, and a release deadline. The naive move is to export the audit report and attach it. The structured move is to treat the audit as the evidence base and build the four-question pack on top of it. For example, if the audit measured recall across twelve operational slices, the production-behaviour section presents those twelve slices mapped to the operational design domain, with each number linked back to its audit run — an operational measurement, not a published benchmark, and scoped to the conditions tested. The drift section names which of those slices is monitored in production and at what threshold an alert fires. The ownership section names the team that owns each monitor. The rollback section points to the runbook and the date it was last exercised. The traceability index ties it together. What changed is not the underlying work — the audit is the same audit. What changed is that a reviewer can now answer every approval question without leaving the document. That difference is the difference between a first-pass clearance and a multi-week clarification cycle, and it is why the perception robustness audit and the evidence package are two halves of one motion: the audit produces the evidence, the package makes it answerable. Teams building this end to end for an OEM context can work from the cross-vertical perception validation package artefact that the same pattern is drawn from, and weave the whole effort into a broader computer vision engineering engagement. One boundary worth stating plainly: a well-structured evidence package is not a safety case in the regulatory sense, and a clean pack shape does not by itself confer regulatory acceptance. The package makes the engineering review efficient and the reliability commitments legible. Whether that evidence is sufficient for a formal functional-safety argument is a separate question with its own ISO 26262 framing — the pack is the input to that argument, not a substitute for it. FAQ What does a perception validation evidence package contain? At minimum: a production-behaviour section, a drift-posture section, an ownership section, a rollback section, and a traceability index linking every claim to the test that produced it. The test results are the evidence base the package draws on, not the package itself. A pack structured around the reviewer’s four question families clears review more reliably than one structured around the test backlog. How do we link test results to production behaviour? Anchor each production-behaviour claim to the test that supports it and state the conditions under which that test ran. Lead with slice-level breakdowns mapped to operational conditions rather than a single aggregate metric, because the aggregate hides the exact failure modes the reviewer is paid to worry about. Each number should trace back to its specific test run and dataset provenance. Who reviews the package — internal QA, customer engineering, regulator? The reviewer varies by context — internal QA, a customer’s engineering team, or a regulator — but the structure of their approval questions is consistent: production behaviour, drift posture, ownership, and rollback path. Note that a well-structured package is not a regulatory safety case on its own, and a clean pack shape does not confer regulatory acceptance; it is the input to a formal safety argument, not a substitute for one. How do we keep the evidence package current as the model updates? The drift-posture section is what keeps the package live: it names what is monitored in production, the thresholds that trigger an alert, and who owns the response. A package that proves the model is good today but says nothing about how anyone learns when it stops being good is a static snapshot of a non-stationary system, which is the most common gap that triggers re-review. How do we structure each evidence surface so it maps directly to the reviewer’s approval question rather than to our internal test backlog? Make each top-level section of the pack one reviewer approval question, stated as a heading, with only the evidence that answers it underneath. Tests that answer no approval question move to an appendix. This inverts the default order — work-as-run — so the reviewer never has to reverse-engineer the answers out of raw results, which is where clarification rounds originate. What turns a multi-round clarification cycle into a first-pass clearance — which gaps in the pack typically trigger re-review? Three gaps account for most re-reviews: a missing drift posture (proving the model is good today but not how degradation is caught), unowned monitoring (an alert with no one to watch or act on it), and an untested rollback (a recovery path that has only ever run in a document). Closing all three before submission is what converts a clarification cycle into a first-pass clearance. A pack that clears on the first pass is not a pack with better numbers — it is a pack that answered the reviewer’s questions before they had to ask them. The discipline is cheap to build and expensive to skip, because the gaps a reviewer would have caught are the same gaps a field incident eventually does. The open question for any team under release pressure is not whether the model is ready, but whether the evidence is answerable — and that is a property of structure, not of score.
