A PCB inspection line does not fail loudly when a model drifts. It fails quietly: the false-call rate creeps up, an operator starts second-guessing the flagged boards, and a week later someone notices the escape rate moved too. By then the question that matters — which change did this? — is nearly impossible to answer, because the only record of the deployed model was a training dashboard someone closed months ago. That is the gap this article is about. Most teams treat Weights & Biases (W&B) logging as a training-time convenience: watch the loss curve, confirm validation accuracy climbs, then move on once the model ships to the line. That view is not wrong so much as it stops one step too early. On a reliability-critical AOI line, the logged run is not a dashboard — it is the durable provenance record that a drift incident is measured against, and the clean rollback target the recovery runbook points to. What does wandb logging actually do in practice? At its mechanical core, wandb.log() streams metrics, media, and metadata from a running process to a hosted (or self-hosted) tracking server, grouped into a run. A run captures the code state, the config dictionary, the environment, logged scalars over steps, and — through the Artifacts API — versioned files: datasets, model checkpoints, configs. Each artifact gets a content-addressed version, so model:v7 is an immutable reference you can pull back later. That last part is the part most training tutorials skip. Logging a scalar named val/accuracy gives you a chart. Logging your trained model, your defect-taxonomy config, and your evaluation baselines as versioned artifacts linked to that run gives you something categorically different: a reproducible reference point. The distinction is not cosmetic. It is the difference between “the model looked good in July” and “the model that shipped on 12 July was aoi-detector:v7, trained on pcb-dataset:v4 with taxonomy config defects:v3, and here are the exact false-call and escape numbers it passed validation against.” For the mechanics of streaming metrics from a running model — including the line-side telemetry side of the story — our walkthrough of using wandb.log for production vision covers the call-level detail this hub deliberately keeps at arm’s length. The divergence point: the first production change staged validation never saw Here is where the naive and expert approaches split. In staged validation, everything is controlled: fixed lighting, a curated defect set, a frozen board revision. The model passes. It ships. Then production does what production always does — the reflow profile gets tweaked, a component vendor changes package tolerance, the line camera’s illumination ages, or a board revision lands with a new silkscreen. None of these were in the validation set. When the false-call rate starts drifting after one of those changes, the team needs to answer a chain of questions fast: Is this the model, or the line? Which model version is actually running? What did that version score at validation? Is there a known-good version to roll back to? Without the training run captured as a versioned artifact, none of those questions has a clean answer. You are reverse-engineering the incident from partial memory and Slack history. This is why we treat the W&B run as the anchor of the production AI reliability posture rather than a development artifact. The run is where the pinned model version, the defect taxonomy, and the false-call/escape baselines live as a single traceable unit. Line-side drift telemetry is then measured against that baseline — not against a number someone remembers. What should a PCB AOI training run log? The instinct is to log everything. That is almost as unhelpful as logging nothing, because a run cluttered with hundreds of unlabelled scalars is as opaque as a run with none. The useful discipline is to log the things a rollback runbook and a re-validation reviewer will actually need. The table below separates the durable provenance layer (what makes the run a baseline) from the training-diagnostic layer (what helps you debug the model during development). What to log in a W&B AOI training run Item Log as Provenance-critical? Why it matters later Trained detector weights Versioned artifact (aoi-detector:vN) Yes The exact rollback target; the thing that “was running” Defect-taxonomy config Versioned artifact + config Yes Defines what a “call” even means; re-validation depends on it Training dataset snapshot Versioned artifact (pcb-dataset:vN) Yes Establishes what the model has and has not seen False-call rate at validation Summary metric on the run Yes The baseline drift telemetry is compared against Defect-escape rate at validation Summary metric on the run Yes The other half of the operator-trust equation Board revision / reflow context Run config / tags Yes Scopes which production conditions the baseline covers Loss and mAP curves Step-logged scalars No (diagnostic) Development debugging; not a rollback dependency Sample predictions / confusion media Logged images / tables No (diagnostic) Useful for review, not required for provenance The provenance-critical rows are the ones that turn a run into a baseline. If a row in that top group is missing, the run is a nice dashboard and nothing more — you have visualisation without an audit trail. If you are logging through a training loop, the pattern in PyTorch Lightning with W&B for capturing training telemetry that feeds drift monitoring shows how to wire the callback so the artifact linkage happens automatically rather than as an afterthought someone forgets. How run artifacts become the baseline drift telemetry is measured against Once the model version, taxonomy, and validation baselines are pinned as a versioned run, the line-side story becomes tractable. Production telemetry — the running false-call rate, the escape rate sampled against a re-inspection audit, the confidence-score distribution — is logged separately and compared to the run’s recorded baselines. Drift is then a delta against a named reference, not a vibe. The reason this works is that the reference is immutable and attributable. When aoi-detector:v7 is the deployed artifact and its validation summary recorded a false-call rate in a specific range, a production false-call rate drifting well outside that range is a measurable, defensible signal — not an argument. That is an observed-pattern claim, drawn from how these incidents play out across the industrial-CV lines we have worked with; it is not a benchmarked universal rate, and the acceptable delta is always line-specific. For teams building the monitoring side of this loop, how to use Weights & Biases to feed a production AI monitoring harness covers how the training tracker connects to the live telemetry surface — the piece that turns pinned baselines into a running comparison. How does versioned logging support an hours-not-days rollback? Consider the difference concretely. An AOI update ships on a Tuesday. By Thursday the escape rate has moved and an operator has escalated. Two teams handle it very differently. The team without versioned run logging spends the first day establishing basic facts: which model is running, when it was deployed, what it scored before. They are reconstructing provenance during the incident, which is the worst possible time to do it. Triage stretches across days, and every hour the model keeps making calls the operators no longer trust. The team with versioned W&B runs does something narrower. They pull the deployed artifact’s version, compare the live escape rate to that version’s recorded baseline, confirm the regression is real, and — critically — they already know the last known-good version because it is a named artifact with its own recorded baseline. The rollback target is not a search; it is a lookup. Recovery lands in hours because the provenance work was done at training time, not incident time. That compression from days to hours is the ROI anchor here, and it is entirely mechanical: structured logging lets you tie a false-call or defect-escape regression back to a specific model version, config, and training run within minutes. Teams that skip it cannot reliably identify which change moved the escape rate, which is exactly the identification a rollback decision requires. The experiment tracker’s role in this — as the upstream feed for the whole monitoring loop — is developed further in how an experiment tracker feeds a production AI monitoring harness. Where does W&B logging stop being useful? This is the boundary worth stating plainly, because over-reaching here creates its own failure mode. W&B is a training and experiment tracker. It is excellent at holding the versioned baseline and the run-level provenance. It is not the place to run high-frequency, low-latency production drift detection on a fast line. Line-side drift telemetry — per-board confidence scores at inspection cadence, real-time false-call alarms, the incident logs an operator files — belongs on the line, in a system built for streaming telemetry and fast query. W&B holds the baseline; the production layer holds the stream and references the baseline. Confusing the two leads teams to either overload the tracker with production-rate data it was never meant to serve, or to lean on the tracker for alerting it cannot deliver at line cadence. Keep the roles separate: the tracker is the reference standard, the line-side system is the sensor. The compound-failure signals that actually drive PCB escapes — lighting drift stacking with a package-revision shift, for instance — are the kind of thing that only becomes traceable when the model version is pinned. That interaction between line conditions and model behaviour is where this connects to how computer vision systems handle real-world visual variation, and why the training run and the line telemetry have to reference the same versioned baseline to be jointly interpretable. How is a logged run linked to the AOI validation pack? The final link in the chain is auditability. When a board revision or reflow change forces re-validation, the reviewer needs to know exactly what was validated, against what taxonomy, at what baseline. A logged W&B run — with its pinned model version, taxonomy config, dataset snapshot, and recorded false-call/escape numbers — is precisely the evidence bundle the PCB AOI validation pack formalises. This is the anchoring relationship: the validation pack does not replace the run, it references it. The run holds the raw provenance; the validation pack turns that provenance into a signable statement that re-validation after a change stays auditable. When those two are tied, a board-revision change becomes a scoped re-validation against a known reference rather than a fresh start. The way W&B reports serve as coverage evidence inside that package is worked through in W&B reports as coverage evidence in a perception validation package. FAQ How does wandb logging actually work? wandb.log() streams metrics, media, and metadata from a running process to a tracking server, grouped into a run that captures code state, config, environment, and — through the Artifacts API — versioned files like datasets and model checkpoints. In practice, the value is not the metric charts but the content-addressed artifact versions: an immutable model:vN reference you can pull back later. That turns “the model looked good in July” into “the model that shipped was aoi-detector:v7 with these exact baselines.” What should a PCB AOI training run log in W&B so the model version and defect-taxonomy config are pinned for later comparison? Log the trained detector weights, the defect-taxonomy config, and the training dataset snapshot as versioned artifacts, plus the false-call and defect-escape rates at validation as run summary metrics, and the board-revision/reflow context as config or tags. These are the provenance-critical rows that turn a run into a baseline. Loss and mAP curves and sample predictions are useful development diagnostics but are not rollback dependencies. How do W&B run artifacts serve as the versioned baseline that line-side drift telemetry is measured against? Once the model version, taxonomy, and validation baselines are pinned as an immutable run, production telemetry — running false-call rate, sampled escape rate, confidence distribution — is compared against those recorded baselines. Drift becomes a measurable delta against a named, attributable reference rather than a subjective judgement. The reference’s immutability is what makes the comparison defensible. How does logging false-call and defect-escape baselines to W&B support an hours-not-days rollback when an AOI update regresses? With versioned runs, the deployed artifact’s version is known, its recorded baseline is a lookup, and the last known-good version is a named artifact with its own baseline — so the rollback target is found, not searched. Recovery lands in hours because the provenance work happened at training time, not during the incident. Teams without it spend the first day just reconstructing which model is running and what it scored, stretching triage across days. Where does W&B logging stop being useful, and what production drift telemetry belongs on the line rather than in a training tracker? W&B is a training and experiment tracker that excels at holding the versioned baseline; it is not built for high-frequency, low-latency production drift detection or line-cadence alerting. Per-board confidence scores at inspection cadence, real-time false-call alarms, and operator incident logs belong in a streaming telemetry system on the line. The tracker holds the baseline; the line-side system holds the stream and references it. How is a logged W&B run linked to the AOI validation pack so re-validation after a board-revision or reflow change stays auditable? The validation pack references the run rather than replacing it: the run holds the raw provenance (pinned model version, taxonomy, dataset snapshot, recorded baselines), and the pack turns that into a signable, auditable statement. When the two are tied, a board-revision or reflow change becomes a scoped re-validation against a known reference instead of a fresh start. That linkage is what keeps re-validation auditable across production changes. The habit worth building is small and boring: never let a run that ships to a line end as a chart. Pin the model, the taxonomy, and the baselines as versioned artifacts, and the day a defect escape appears, you will be asking which version — with an answer ready — instead of asking which model was even running.
