A shared Weights & Biases Report link is not a validation artefact. It is a live tracking surface over mutable runs, and the moment a reviewer asks which run configuration, dataset snapshot, and code commit produced a headline metric, the link stalls. That single question — where did this number come from? — is where the naive workflow breaks down, and it breaks down at the worst possible moment: procurement review for a regulated clinical-imaging deployment. The reflex is understandable. A W&B Report renders beautifully, updates as new runs land, and packages metrics, plots, and config side by side. It feels like the finished document. But a report reflects the state of the tracking layer right now, and a validation pack entry has to be frozen, provenance-stamped, and adjudicable long after the runs that produced it have been superseded. Confusing the two is the difference between a review that clears on the first pass and one that gets re-litigated for weeks. This article explains how W&B Reports actually work, what they capture, and — the part that matters for regulated work — which of their fields become frozen pack evidence and which stay in the live tracking layer where they belong. How should you think about W&B Reports in practice? A W&B Report is a document builder that sits on top of your logged experiment runs. You log metrics, config, and artefacts during training and evaluation with wandb.log and the run API; the Report then lets you pull selected runs into a narrative surface — line plots, metric tables, scalar panels, embedded media, and prose — that you can share by URL. If you already log training telemetry through W&B, the broader pattern of feeding tracked runs into a monitoring harness is the same plumbing seen from the training side. The key property, and the one people forget, is that a Report references runs rather than snapshotting them. A panel that plots validation Dice against epoch is a live query into the run’s history. Delete the run, re-tag it, or add three newer runs to the same project and the report can move underneath you. That is exactly the behaviour you want during active experimentation — the report is a working surface, not a frozen record. The confusion starts because the rendered output looks like a deliverable. It has a title, sections, numbers, and a stable-looking link. In practice it is closer to a saved dashboard view than to a signed document. What a W&B Report captures versus a static results document The distinction that governs everything downstream is mutable reference versus frozen content. A static results document — the kind that goes into a validation pack — asserts a fixed set of numbers tied to a fixed set of inputs, and it does not change when the project changes. A W&B Report asserts the current view of selected runs, and it changes whenever those runs or the project do. Property W&B Report Frozen pack section Data source live query into run history snapshot fixed at freeze time Changes over time yes, when runs/project change no, immutable once signed Provenance implied by run linkage explicitly pinned and stated Audience the team, during experimentation reviewer / auditor, after the fact Adjudicable no — no fixed contents to sign against yes — that is its purpose Correct role tracking layer pack contents entry This is not a criticism of Reports. It is a statement of what they are for. The failure is only when a report link is handed to a reviewer as the evidence, because a reviewer’s job is to adjudicate a fixed claim, and there is no fixed claim in a surface that can shift under them. Which report fields belong in a frozen pack, and which stay live? The useful move is to treat the report as a source you extract from, not a document you submit. Some fields carry directly into the pack as frozen contents; others are scaffolding that stays in the tracking layer. Mapping this once, explicitly, is what makes the extraction repeatable at the next site instead of reconstructed from memory. Extract into the frozen pack section: The specific metric values that constitute the validation claim — for a segmentation model, that means the frozen Dice, IoU, or detection metrics read at the threshold the pack commits to, not a live-updating plot. The run configuration that produced them: hyperparameters, model version, evaluation harness version. The provenance triple — dataset snapshot identifier, code commit hash, run ID. The validation-set definition and any adjudication or reader-consensus labelling that grounds the metric. Leave in the live tracking layer: Sweep exploration across dozens of runs (the search, not the result). Interactive plots meant for the team to slice during experimentation. Draft commentary and in-progress panels that have not been frozen. The clean version of a W&B Report used as coverage evidence in a validation package works precisely because it applies this split rigorously — the report shows the exploration, the pack pins one adjudicated run. How do you pin run provenance so a reviewer can trace a metric? Pinning provenance means making the path from a headline number back to its inputs mechanical, not conversational. Three fields do most of the work, and W&B logs all three if you configure it to. Dataset snapshot. Log the exact validation set as a versioned artefact, not a folder path. W&B Artifacts give each dataset version an immutable identifier; the versioning discipline used for calibration evidence in condition monitoring is the same mechanism, applied to the clinical validation set. The pack records the artefact version, so a reviewer can confirm the metric was computed on the set the pack claims. Code commit. Log the git commit of both the model code and the evaluation harness. A metric is only reproducible against a known evaluator; recording the commit is what lets someone re-run the number rather than trust it. Run config. The full config dict — hyperparameters, seeds, precision, thresholds — captured at run time, frozen into the pack alongside the metric. The single discipline underneath all three: freeze the specific run, not the project view. A pack entry that names run ID, dataset artefact version, and commit hash answers “where did this come from?” on the first read. A report link answers it only after a round-trip through whoever remembers. Why a shared report link can’t be the audit record The core reason is legal-adjacent, not technical. An audit record has to be fixed at a point in time and attributable, because its purpose is to let a reviewer adjudicate a claim that will not move. A W&B Report is mutable by design — anyone with edit access can re-select runs, and the underlying runs can be re-tagged or deleted. A record that can silently change is not a record; it is a view. There is a workflow cost too. When a report link is treated as the evidence, every reviewer question (“which run?”, “which dataset?”, “which threshold?”) becomes a back-and-forth request loop against whoever has the tribal knowledge. Pinning the provenance triple into a frozen pack section collapses that loop — an observed pattern across the validation engagements we run, not a benchmarked figure. It also makes re-validation at a new site an extraction from a documented mapping rather than a reconstruction, which is where the repeatable-versus-reconstructed distinction pays off. Teams that keep the live tracking layer and the frozen evidence section separate avoid the multi-week re-litigation that a mutable link invites when it is mistaken for the audit trail. Getting this separation right is one facet of the broader production AI reliability posture that regulated deployments demand. Mapping report-derived metrics to the pack’s validation-set contents The pack’s structure tells you the target shape for the extraction. A clinical-imaging validation pack expects a validation-set construction section and a metrics section with defined adjudication, and the report has to be decomposed to fill exactly those. This mapping is what a clinical-grade medical imaging AI validation engagement formalises: the vertical methodology defines the pack contents, and the W&B Report is where the runs and metrics live before they are frozen into it. Concretely, the validation-set construction contents take the dataset artefact version and its labelling/adjudication provenance; the metrics contents take the frozen scalar values, the run config, and the code commit. Every headline number in the pack points to one pinned run. Nothing in the pack points to a live query. FAQ How does wandb reports actually work? A W&B Report is a document builder layered over your logged experiment runs — it pulls selected runs into shareable panels of metrics, plots, and config. Critically, it references runs rather than snapshotting them, so a report reflects the current state of the tracking layer and can change when runs or the project change. In practice it is a working surface for the team during experimentation, not a frozen deliverable. What does a W&B Report actually capture from an experiment run, and how is it different from a static results document? It captures live views of selected runs: metric histories, config, artefacts, and embedded media, all queried from the run store. A static results document asserts a fixed set of numbers tied to fixed inputs and does not move; a Report asserts the current view and shifts whenever its runs or project change. That mutable-reference-versus-frozen-content distinction is the whole reason a report link cannot stand in for a signed pack section. Which fields from a W&B Report belong in a frozen clinical validation pack section, and which stay in the live tracking layer? Extract into the frozen pack: the specific metric values that form the claim, the run configuration that produced them, and the provenance triple of dataset snapshot, code commit, and run ID. Leave in the tracking layer the sweep exploration, interactive slicing plots, and in-progress commentary. The report shows the search; the pack pins the one adjudicated result. How do you pin run provenance — dataset snapshot, code commit, config — so a reviewer can trace a metric back to its source? Log the validation set as a versioned W&B Artifact with an immutable identifier, log the git commit of both the model and the evaluation harness, and capture the full run config at run time. Freeze the specific run — named by run ID, dataset artefact version, and commit hash — into the pack, not the project view. That triple lets a reviewer trace any headline number to its inputs on the first read. Why can’t a shared W&B Report link serve as the audit record for a regulated deployment? An audit record must be fixed and attributable so a reviewer can adjudicate a claim that will not move, but a Report is mutable by design — runs can be re-selected, re-tagged, or deleted. A record that can silently change is a view, not a record. Treating the link as evidence also forces a back-and-forth request loop for every provenance question, which a frozen pack section eliminates. How should report-derived metrics map to the pack’s validation-set construction and adjudication contents? The validation-set construction contents take the dataset artefact version plus its labelling and adjudication provenance; the metrics contents take the frozen scalar values, the run config, and the code commit. Decompose the report so each pack section is filled from the corresponding run fields. Every headline number in the pack points to one pinned run, and nothing points to a live query. The open question for any team already running W&B is not whether their tracking is good — it usually is. It is whether the boundary between the live report and the frozen pack section has been drawn explicitly, because that boundary is what a reviewer tests on the first pass. Where that line sits, and which run each number is pinned to, is the failure class the validation-set construction checklist exists to close.
