A wandb report is not the validation deliverable. It is the raw material behind one section of it — the coverage/test-evidence section that QA signs. The gap between those two readings is where sign-off stalls. The naive move is familiar. Training finishes, someone opens Weights & Biases, builds a report that pulls in the run history, the mAP curves, the confusion slices, and pastes the shared link into the validation package. The assumption is that the numbers speak for themselves — that a reviewer can open the dashboard, look at the runs, and attest that coverage is adequate. It rarely works that way, and the reason has almost nothing to do with the quality of the runs. What a wandb report actually is, and what it means in practice Weights & Biases logs experiments: metrics, hyperparameters, system stats, media, and artifacts, streamed from training and evaluation code via wandb.log. A report is a curated document layer on top of that data — panels, tables, and prose stitched together from one or more runs. It is genuinely useful. It turns a wall of run logs into something a human can read. But a report, by default, is a live view. It queries the underlying runs and renders whatever those runs currently contain. Add a run to the project, change a metric definition, re-tag an experiment, and the report re-renders. That is exactly the property you want during development and exactly the property that destroys its value as evidence. Evidence has to hold still. When QA signs a coverage section, they are attesting to a specific claim: these classes, these conditions, these slices were evaluated on this model against this data, and here are the results. If the report backing that claim can change after the signature, the signature means nothing. This is the single distinction that separates a dashboard from an artifact, and it is worth stating plainly. A live report is a working surface; a pinned report is an artifact. Only the pinned version can be signed. How a report becomes a package section rather than a standalone dashboard The mistake is treating the report as the section. It is not. In a structured perception validation package, each section has an owner and a claim, and the coverage/test-evidence section is one of several — it sits alongside hazard-linkage, release-acceptance, and traceability sections that other roles own. Our work on how perception models reach a release-signable state walks through that section-and-owner structure in more detail; the report populates exactly one box in it. So the report does not become the section by being pasted in. It becomes the backing evidence for the section once someone curates it: decides which runs count, which metrics constitute coverage, which slices matter for the hazard cases in scope, and then freezes that selection. The curation is the work. The report is what remains after you throw away everything that is not evidence. Concretely, that means answering three questions before the report is worth anything to a reviewer: Which runs? Not every experiment in the project. The final evaluation runs against the frozen eval set — usually a handful, sometimes one — with the exploratory sweeps excluded. Which metrics? Coverage is not “accuracy went up.” It is per-class detection performance across the conditions the release claims to handle: lighting, occlusion, distance bands, weather. If you read lift charts a reviewer can sign against, the same discipline applies — the metric has to map to a claim, not just look good. Which slices? The report has to show the hard cases, not the aggregate. An aggregate mAP hides the small-pedestrian-at-dusk slice that a hazard analysis flagged as safety-relevant. What the report must be pinned to before QA can sign Pinning is not a metaphor. It is a concrete set of bindings that make the report reproducible and stable. In practice, a coverage report that QA can credibly sign is bound to three things at once. Pinning checklist Binding What it fixes What breaks without it Model version The exact weights + config the metrics describe Numbers attributed to the wrong model after a refresh Evaluation set The frozen dataset artifact and its version Coverage claim silently shifts when data is added Metric slices The named class/condition breakdowns in scope Aggregate hides the safety-relevant weak slice Report snapshot A frozen, versioned copy of the report itself Live re-render changes what the reviewer saw The model version and evaluation set are best pinned as W&B artifacts — versioned, immutable objects with lineage back to the runs that produced them. Our teams treat W&B artifacts as the versioning layer for evidence precisely because they give you a content-addressed handle that cannot be edited in place. When the coverage report references model:v7 and eval-set:v3, the reviewer knows exactly what they are signing against, and so does anyone who audits it two years later. The fourth binding — the report snapshot — is the one teams most often forget. W&B supports saving a report at a point in time; the frozen copy is what goes into the package, not the live link. This is a benchmark-class claim in our terms: the numbers come from named, reproducible evaluation runs against a named eval set. That traceability is the whole point. Why a live report stalls sign-off — and how freezing fixes it Here is the failure mode we see most often. A reviewer opens the shared report link, reviews the numbers, and is ready to sign. Meanwhile, a data scientist adds a follow-up run to the same project to check something. The report re-renders. Now the numbers the reviewer looked at do not match the numbers in the report, and the reviewer — correctly — refuses to sign against a moving target. The review round is wasted, and it repeats. The cost is not just delay. It is a rework loop where signatures are collected against numbers that later shift, which in a safety context is worse than no signature at all. Freezing the report view breaks the loop. QA signs a frozen artifact tied to model:v7 and eval-set:v3; nobody can change what they saw after the fact. The coverage section arrives pre-routed to QA and closes in a single review round instead of chasing a dashboard that keeps moving under the reviewer. This is the same principle that governs how an experiment tracker feeds a production monitoring harness — the tracker is upstream plumbing, and the evidence surface has to be a stable derivative of it, never the live stream itself. TechnoLynx builds these validation surfaces as part of our production AI reliability work, and the freeze step is non-negotiable in every one of them. Which parts of the package a wandb report can never cover A report is coverage evidence. It is not the whole package, and pretending otherwise is how scope creeps into the wrong section. Two boundaries matter in particular. It does not cover hazard linkage. Whether a given coverage slice actually addresses a safety hazard is a claim owned by a safety engineer, grounded in the hazard analysis — not something a metrics report can assert. The report can show that the small-pedestrian slice was evaluated; it cannot decide that evaluating it is sufficient to close the hazard. That judgment lives in a different section with a different owner. It does not cover release acceptance. The decision to ship is a release-authority decision that weighs coverage evidence against many other inputs — field data, residual risk, regulatory posture. The coverage report is one input to that decision, not the decision. If a wandb report ever appears to authorize a release, the package structure has failed. Keeping the report inside QA’s coverage section is what keeps the whole package legible. Each role signs what they own. The vertical view of this — a named automotive engagement where a curated report backs the coverage section against real roles — is where the mapping becomes concrete, and it grounds the abstract structure in who actually holds the pen. Reusing the report structure across model refreshes Perception models get refreshed. New data, retrained weights, a new eval-set version. The temptation is to rebuild the coverage evidence from scratch each time, which is slow and error-prone. The better pattern is to treat the report structure — the panels, the slice definitions, the metric selections — as a reusable template, and re-point it at the new pinned artifacts. When the structure is stable, a refresh becomes a re-attestation rather than a rebuild: QA reviews the same coverage layout populated with model:v8 and eval-set:v4 metrics and signs the refreshed view. The curation work — deciding what counts as coverage — is done once and amortized across every refresh. What changes is the numbers, not the shape of the evidence, and that is exactly the property that makes a validation package maintainable over a program’s life rather than a one-time heroic effort. FAQ How does a wandb report work? Weights & Biases logs experiment data — metrics, hyperparameters, media, and artifacts — from training and evaluation code. A report is a curated document built on top of that data, combining panels, tables, and prose from one or more runs. By default it is a live view that re-renders as the underlying runs change, which is useful during development but a liability when the report is meant to be evidence. How does a wandb report become the coverage/test-evidence section of a perception validation package rather than a standalone dashboard? It becomes backing evidence only after curation: someone decides which runs count, which metrics constitute coverage, and which slices matter, then freezes that selection. The report populates one section — coverage/test-evidence — that QA owns and signs, sitting alongside hazard-linkage and release-acceptance sections owned by other roles. Pasting a live dashboard link into the package is not curation and does not make it a signable section. What must a wandb report be pinned to before QA can credibly sign the section it backs? Four bindings: the exact model version, the frozen evaluation set (both best held as versioned W&B artifacts), the named metric slices in scope, and a frozen snapshot of the report itself. Together they make the coverage claim reproducible and stable, so the reviewer knows precisely what model and data the numbers describe and nothing can change after the signature. Why does a live, mutable wandb report stall sign-off, and how does freezing a report view fix that? A live report re-renders whenever the underlying runs change, so numbers a reviewer looked at can shift before or after they sign — and a reviewer will correctly refuse to sign against a moving target. Freezing the report view ties it to specific model and eval-set versions, giving QA a stable artifact to attest to in a single review round instead of chasing a changing dashboard. Which parts of the validation package can a wandb report never cover? It can never cover hazard linkage or release acceptance. Whether a coverage slice actually closes a safety hazard is a safety engineer’s judgment grounded in the hazard analysis, and the decision to ship is a release authority’s call weighing many inputs. The report is coverage evidence and one input to those decisions — it never authorizes them. How do you reuse the same report structure across model updates so QA re-attests refreshed metrics instead of rebuilding the evidence? Treat the report structure — panels, slice definitions, metric selections — as a reusable template and re-point it at the new pinned artifacts for each refresh. QA then reviews the same coverage layout populated with the new model and eval-set versions and re-attests the refreshed numbers. The curation work is done once and amortized, so what changes is the metrics, not the shape of the evidence. The open question for most programs is not whether the report is good enough — the runs are usually fine. It is whether the team has drawn the line between the working dashboard and the frozen artifact, and whether the coverage section stays inside QA’s ownership rather than drifting into claims about hazards it was never built to make.
