A solder-bridge mask on frame N and the “new” mask on frame N+1 after the panel index shifts are, more often than a per-frame pipeline can tell, the same defect. Treating segmentation as a per-frame problem — run the model on each captured board image, threshold the defect mask, log the call — is where automated optical inspection (AOI) telemetry starts to drift. The mask is correct. The count is wrong. Segmentation tracking is the discipline that fixes the count. It maintains mask identity and location across an inspection sequence, associating each segmented region with a specific component footprint and pad geometry, so that a defect detected under one lighting angle or panel position is recognised as the same tracked entity when those conditions change on the next frame. Get this right and your false-call and defect-escape rates stay measurable against the AOI validation baseline. Get it wrong and the numbers inflate from double-counted and fragmented masks until operators stop trusting the calls. What does segmentation tracking actually mean on a PCB line? Per-frame segmentation answers “where is the defect in this image?” Segmentation tracking answers “is this the same defect I saw before, and which component does it belong to?” Those are different questions, and the second one is the one that makes the telemetry meaningful. Concretely, a segmentation model — a fine-tuned instance-segmentation network, or a promptable backbone like the Segment Anything Model wrapped around your defect classes — produces per-pixel masks for each captured frame. On a panelised board, or a line where a single board is imaged from multiple angles or under structured-light sequences, the same physical defect appears across several frames. A per-frame pipeline treats each appearance as an independent event. Tracking ties those appearances together and, crucially, anchors each mask to a stable reference: the component footprint and pad on which the defect sits. That anchoring is what distinguishes inspection tracking from generic multi-object tracking. In a surveillance or sports context you track motion. On a PCB line the board is nominally static and indexed; what moves is the acquisition condition — panel position, lighting angle, exposure. So the association problem is less “predict where the blob went” and more “reconcile this mask against a known component reference under a changed view.” This is closer in spirit to how object trackers behave in line-side CV inspection than to a pedestrian tracker, but the reference frame is the board layout, not a Kalman-predicted trajectory. How is a defect mask tied to a specific component footprint? The stable identity you need does not come from the mask itself — masks shift shape and area under lighting drift. It comes from projecting the mask into a board coordinate frame and asking which footprint it overlaps. The mechanism, in the pipelines we have worked on, runs roughly like this: Fiducial or fiducial-free registration. The captured frame is aligned to the CAD/Gerber-derived board model, either from fiducial marks or from a learned homography against a golden reference. This gives every pixel a board-relative coordinate. Footprint assignment. Each defect mask is projected into board coordinates and matched to the component footprint and pad it overlaps. The tracked entity is now keyed on (component_ref, pad_id, defect_class) — not on a frame-local mask ID. Cross-frame association. When the same footprint is imaged again — different panel index, different angle — the new mask on that footprint is associated with the existing tracked defect rather than logged as new. The payoff of keying on component reference rather than pixel geometry is that a solder-bridge mask that grows by 30% under a glare band on the next frame is still the same tracked bridge on pad 3 of U12. Component orientation and placement themselves are separately verifiable — the pose-estimation checks that confirm component orientation in PCB AOI share the same registration substrate, which is why teams that already run pose checks find the tracking anchor cheaper to add. Why does one defect fragment into two masks — or two merge into one? This is the failure that a per-frame view cannot see, and it is the reason IoU on a staged set overstates readiness. A single solder bridge under a glare band can segment as two disconnected mask components: the model sees a bright specular region splitting the defect, and the connected-component step returns two blobs. Per-frame, that logs as two defects on one pad — an inflated false-call contribution. The inverse also happens: two adjacent tombstoned components under a low-contrast exposure blur into a single mask, and one real defect is dropped because the merged region gets classified as a different, benign class. Tracking that is anchored to footprint geometry catches both. Two masks landing on the same pad within the same footprint reconcile to one tracked defect. A mask spanning two footprints flags for split rather than being counted once. This is a concrete instance of the compound-failure pattern where lighting drift and package revisions stack into miscounted defects — no single factor is fatal, but a glare band plus a package revision that moved the pad plus a re-panelisation together produce a count that no per-frame threshold can reconcile. In our experience across inspection deployments, mask fragmentation under specular lighting is the single most common source of apparent false-call inflation in the first weeks of a segmentation-based AOI go-live — an observed pattern, not a benchmarked rate, and it is precisely the class of error that never shows up on a clean validation set shot under controlled light. Per-frame vs. tracked segmentation: when is the added complexity worth it? Not every line needs tracking. If each board is imaged exactly once, from one fixed angle, and never re-panelised, per-frame segmentation with a well-calibrated threshold is defensible. The tracking machinery earns its cost the moment any of the following are true. Condition on the line Per-frame segmentation Tracked segmentation Single fixed-angle capture, one image per board Adequate Overkill Multi-angle or structured-light capture sequence Double-counts across views Reconciles to one defect per footprint Panelised boards with variable panel index Miscounts on re-panelisation Stable per-footprint identity Frequent component-package revisions Silent count drift after ECO Drift attributable to the changed footprint Specular/glare-prone solder joints Fragments defects, inflates false-call Merges fragments on shared pad False-call/escape telemetry must track a validation baseline Baseline decouples over time Counts stay comparable to baseline The decision variable is not model accuracy — it is whether your acquisition produces multiple looks at the same defect and whether your product mix changes. Both are properties of the line, not the model, which is why a segmentation model that validated cleanly in the lab can still ship telemetry that drifts within a quarter. How does tracking keep false-call and escape counts aligned with the AOI baseline? The AOI validation baseline pins a defect taxonomy and a false-call/escape rate against a known board set. That baseline is only meaningful if the counting unit stays constant. Per-pixel masks are not a stable counting unit; tracked defect identities keyed to component footprints are. Segmentation tracking is the layer that maps per-pixel masks back onto the taxonomy the validation pack defines, so a “solder bridge on U12.3” counts once whether it was imaged from two angles or fragmented by glare. This is the same discipline that governs where reliability gates belong at each stage of an ML pipeline: the gate is only trustworthy if the quantity it measures is the quantity the baseline measured. When the counting unit drifts, the baseline silently decouples, apparent false-call rate climbs, and — an observed pattern across the deployments we have supported — operators revert to manual re-review within a quarter of go-live because they no longer trust the automated calls. Our work on production AI reliability treats this counting-unit stability as a first-class validation concern, not an afterthought: the industrial-CV validation pack pins the taxonomy and baselines, and tracking is what keeps tracked defect counts comparable to them as the line changes. What telemetry warns you before a defect escapes? Escaped defects are a lagging indicator — by the time one reaches the customer, the tracking has been degrading for a while. The leading indicators are cheaper to instrument and worth watching. New-track rate per footprint class. A sudden rise in newly-created tracked defects on a specific component class usually means a package revision moved a pad and registration is now mis-assigning masks. Fragment/merge ratio. The share of masks that reconcile to an existing track vs. spawn a new one. A climbing fragment ratio flags a lighting or exposure change before it shows as false-call inflation. Registration residual. The alignment error between captured frame and golden reference, per panel position. Rising residual on one panel index points at a fixturing or panel-layout change. Track-lifetime distribution. How many frames each defect is tracked across. Shortening lifetimes mean association is failing and defects are being logged as one-frame events. When one of these moves, the value of the anchoring shows: drift is attributable to a specific component class or panel position and can be recovered in hours rather than diagnosed blind. The telemetry itself belongs in the same line-side CV telemetry and incident store that holds the rest of the inspection record, so a false-call spike can be joined back to the frame, footprint, and registration residual that caused it. FAQ How does segmentation tracking actually work? It maintains mask identity and location across an inspection sequence rather than treating each frame independently. In practice, each per-pixel defect mask is projected into a board coordinate frame, matched to the component footprint it sits on, and associated across frames by that footprint reference — so the same physical defect stays one tracked entity as lighting angle or panel index changes. How is a segmented defect mask associated with a specific component footprint or pad so it stays the same tracked entity across frames? The captured frame is registered to the CAD/Gerber board model (via fiducials or a learned homography), giving every pixel a board-relative coordinate. Each mask is then projected into that frame and keyed on (component_ref, pad_id, defect_class) rather than a frame-local mask ID, so a mask that shifts shape under glare is still recognised as the same defect on the same pad. Why does a single defect sometimes fragment into two masks, or two defects merge into one, under lighting drift or panel-index changes? A glare band can split one defect into two disconnected blobs at the connected-component step, inflating the count; a low-contrast exposure can blur two adjacent defects into one mask, dropping a call. Footprint-anchored tracking reconciles multiple masks on the same pad to one defect and flags a mask spanning two footprints for a split, catching both directions of error. How does segmentation tracking keep false-call and defect-escape counts aligned with the AOI validation baselines? The validation baseline is only meaningful if the counting unit stays constant, and per-pixel masks are not a stable unit. Tracking maps masks back onto the taxonomy the validation pack defines, keying counts to component footprints so a given defect counts once regardless of how many frames or angles saw it, keeping tracked counts comparable to the baseline as the line changes. How does tracking behaviour change when a board revision or reflow profile alters component packages and pad geometry? A package revision moves pads, so registration can start mis-assigning masks and the new-track rate rises on the affected component class. Because identity is keyed to footprint, that drift is attributable to the specific class or panel position rather than showing up as a diffuse false-call climb, which is what makes recovery a matter of hours rather than a blind diagnosis. What telemetry indicates that segmentation tracking is degrading before it shows up as escaped defects? Watch the new-track rate per footprint class, the fragment/merge ratio, the registration residual per panel position, and the track-lifetime distribution. Rising new-track rate or registration residual points at a package or fixturing change; a climbing fragment ratio flags a lighting or exposure shift — all of them move before an escape reaches the customer. How does segmentation tracking differ from per-frame segmentation, and when is the added tracking complexity worth it on a PCB line? Per-frame segmentation answers where the defect is in one image; tracking answers whether it is the same defect and which component it belongs to. The added complexity is worth it whenever the line produces multiple looks at the same defect (multi-angle or structured-light capture, panelisation) or the product mix changes through revisions — properties of the line, not the model. Before assuming a clean IoU on a staged set means the deployment is ready, ask the question the staged set cannot answer: does tracked defect identity survive the next re-panelisation and the next package revision the line actually sees? That is the failure class the PCB AOI validation pack is built to hold to account.
