A perception validation pack does not have one audience. It has three, and they reject it for different reasons. Internal QA rejects on untraced test gaps. Customer engineering rejects on claims made about an operating domain the tests never covered. A regulatory-facing reviewer rejects on missing provenance — evidence that cannot be walked back to the requirement it was meant to discharge.
Most teams write one pack and optimise it for completeness: more test results, more benchmark tables, a longer appendix. Completeness is not the property being assessed. The reviewer is holding an approval question, and the pack either answers that specific question in the first few pages or it comes back with a clarification list.
Why does the same pack pass one review and fail another?
Because “the reviewer” is an abstraction that hides three different jobs.
Internal QA owns release discipline. Their question is whether the test suite actually covered what it claims to cover and whether regressions from previous builds are still being exercised. They are reading for coverage geometry — which slices ran, which were skipped, whether a skipped slice was skipped deliberately or fell out of the harness when a dataset path changed.
Customer engineering — the OEM or Tier 1 integrating your perception component — owns a release window and a downstream integration. Their question is how the model behaves inside their operating conditions, not yours. Night driving in one geography is not night driving in another. They also want to know who owns the model after handover and what the rollback path looks like when field behaviour degrades. In our experience, this is the reviewer most often surprised by a pack, because the pack was written by people who knew their own operational design domain so well they forgot to state it.
A regulatory-facing reviewer — internal homologation, a safety assessor, or a notified body’s engineer — owns traceability. Their question is not “is this model good” but “can I follow the chain.” Requirement to test to result to sign-off, with dates, versions, and named responsibility at each link. A gap in the chain is a finding regardless of how strong the underlying numbers are.
These are not three severity levels of the same review. They are three different failure surfaces.
Reviewer map: concern, rejection trigger, and what the pack must lead with
| Reviewer type | Primary concern | Rejection trigger | What the pack leads with |
|---|---|---|---|
| Internal QA | Test coverage and regression discipline | An untraced test gap — a slice that is missing with no recorded reason | Coverage matrix: slices planned vs run vs skipped, with skip rationale |
| Customer engineering | Production behaviour inside the customer’s operating envelope; ownership and rollback | A claim about their ODD that the evidence does not support | ODD boundary statement, then behaviour evidence per condition, then owner and rollback path |
| Regulatory-facing | Traceability from requirement to evidence | Missing provenance — evidence with no link back to a requirement or model build | Requirement-to-evidence trace table, with build IDs and dates |
The table is the working artefact. Before assembling, pick the row. The evidence surfaces themselves — dataset manifests, per-class recall, drift monitors, ownership records — are largely the same across all three rows. What changes is ordering, framing, and which claims you are willing to make in writing. We treat that as the first assembly decision, not a formatting pass at the end.
The claim boundary shifts per reviewer
This is where teams get into real trouble. A sentence that is fine in a QA pack becomes an overclaim in a customer pack and a liability in a regulatory-facing one.
“Detection recall is 0.94 on the validation set” is a QA-grade statement. It is traceable to a test run and makes no promise about the road.
Rewritten for customer engineering, it has to carry its boundary: recall of 0.94 on the night-urban slice of the validation set, measured on build 2026.31, at the deployed confidence threshold — with an explicit note that the slice does not include the customer’s tunnel-egress lighting transitions. That is a narrower claim and a stronger pack, because the reviewer now knows exactly what they are accepting.
For a regulatory-facing reviewer, the same figure needs a fourth thing attached: the requirement it discharges. If no requirement points at it, the number is decoration. And nothing in a validation pack should be read as a hazard analysis or a residual-risk argument — that boundary is worth stating in the pack itself, since a pack quietly promoted into a safety-case role is a recurring source of downstream disputes.
Can one pack serve all three?
Yes, with one structural condition: a shared evidence body and a reviewer-specific front section.
The evidence body stays single-sourced — one dataset manifest, one set of test results, one drift and monitoring posture, one ownership record, versioned against the model artefact that produced it. Duplicating the evidence per audience is how packs drift out of agreement with each other, and a reviewer who spots two different recall figures for the same build will stop reading the numbers and start auditing the process.
The front section is where you name the reviewer and route them. Roughly:
- Reviewer statement — who this submission is for and what approval it seeks.
- Claim boundary — the operating conditions the evidence covers, and the conditions it explicitly does not.
- Route table — which section answers which of that reviewer’s questions, in their order of priority.
- Trace index — requirement or ticket ID to evidence section, populated for regulatory-facing submissions and harmless to keep for the others.
Three front sections over one evidence body is cheap. Three packs is not, and three packs will disagree within two release cycles.
What identifying the reviewer changes in practice
The measurable effect is on review throughput, not on model quality. The metrics worth tracking are first-pass clearance rate per reviewer type, the number of clarification rounds per submission, and elapsed time from submission to sign-off. These are observed operational patterns from validation-pack work rather than a published benchmark, and they are specific to how a given OEM runs review — a team with two clarification rounds against a customer reviewer is not directly comparable to one with two against an internal QA gate.
Where the cost lands is schedule. When a customer-engineering review gates a release window, a re-review is a slip — and the slip has nothing to do with the model. The weights did not change. The pack answered a question nobody asked.
Our broader work on assembling automotive perception evidence for OEM review is covered in structuring perception evidence for automotive review, which walks the evidence surfaces section by section rather than by audience. The reviewer-audience map here is the input to that structure. For the underlying engineering — how detection, tracking, and fusion behaviour is measured in the first place — see our computer vision practice.
One thing we do not have a clean answer to yet: whether the reviewer-first framing survives a genuinely mixed review board, where an OEM sends a QA engineer, an integration lead, and a safety assessor into the same meeting. Our current practice is to write for the reviewer with the sharpest rejection trigger and keep the others’ route tables in an annex. Whether that is the right default, or just the safest one, is still an open question.
Frequently Asked Questions
What does “who reviews a perception validation pack — internal QA, customer, regulator” mean in practice?
Three distinct audiences—internal QA teams, customer engineering groups, and regulatory bodies—scrutinize perception validation packs with incompatible success criteria. It means treating the pack’s audience as a design input rather than an afterthought. In practice, the three reviewer types read for different properties — coverage discipline, production behaviour in their own operating domain, and requirement-to-evidence traceability — so the same evidence has to be ordered and framed differently depending on who opens it.
What does internal QA look for in the pack, and which gaps trigger an internal reject?
Internal QA reads for test coverage and regression discipline: which slices were planned, which actually ran, and whether prior-build regressions are still exercised. The reject trigger is an untraced gap — a slice missing from the results with no recorded reason, which QA cannot distinguish from a harness failure.
What does customer engineering ask that internal QA does not — operating conditions, ownership, rollback path?
Customer engineering asks how the model behaves inside their operating conditions, which are rarely identical to yours, and who owns the model after handover. They also ask for the rollback path when field behaviour degrades. QA rarely asks any of these, because QA is assessing the test process rather than accepting integration risk.
What can and cannot be claimed when the pack is read by a regulatory-facing reviewer?
You can claim that a named model build behaved within measured bounds on defined conditions, provided each figure traces to a requirement, a test run, and a build ID. You cannot claim hazard coverage, residual-risk adequacy, or safety-case discharge — the pack is evidence that feeds those arguments, not the argument itself.
How do we structure one pack so it serves more than one reviewer type without diluting any of them?
Keep one single-sourced evidence body — manifests, results, drift posture, ownership — versioned against the model artefact, and put a reviewer-specific front section in front of it containing a reviewer statement, claim boundary, route table, and trace index. Duplicating the evidence per audience is what causes packs to disagree with each other.
Which reviewer type should we identify first, and what changes in the pack once we know?
Identify the reviewer holding the nearest approval gate, since that gate sets the schedule risk. Once named, three things change: the order of the evidence surfaces, the explicit claim boundary you are willing to put in writing, and whether the trace index is populated or merely present.
Reviewer identity shapes validation scope
Internal QA, customer representatives, and regulators each demand different evidence structures—attempting a one-pack-fits-all strategy guarantees rework or rejection. The teams that do tend to ship the boring, correct version first.
