A packaging redesign lands the same week the seasonal daylight angle shifts and a replacement conveyor belt changes part presentation by two millimetres. Each change is inside tolerance on its own. Together they push the inspection model past its decision boundary, the rejection rate moves, and not one of the three single-cause monitors trips. That is a compound failure mode, and whether it is written down in the reliability pack is the difference between an hours-long recovery and a multi-day root-cause hunt.
Most CV pilots ship a failure list organised by cause: lighting drift, occlusion, motion blur, class imbalance — each with its own detection threshold and its own remediation note. That structure is fine for reasoning about the model in isolation. It is a poor match for how incidents actually arrive on a line, because line-side conditions change in bundles. Maintenance shifts, SKU introductions and line refreshes each touch several variables at once.
What does “compound CV failure modes documented inside the pack” mean in practice?
It means the pack contains a register of combinations, not just a register of causes. A compound-mode entry names a specific co-occurrence of line conditions, records the telemetry signature that combination produces, and states what to do about it — including the point at which the answer is model rollback rather than a line-side adjustment.
The practical test is simple: an on-call engineer looking at three near-threshold alerts at 02:00 should be able to match that multi-signal pattern to a named entry, rather than opening three separate investigations and hoping one of them explains the rejection rate. In our industrial CV work, the incidents that consume the most engineering time are almost never the ones with a single clear cause; they are the ones where every individual signal looks survivable (observed pattern across TechnoLynx inspection engagements, not a benchmarked incident rate).
The second thing compound-mode documentation buys is decidable rollback. You can only pin a model version with confidence when you know which combination of line conditions that version was validated against. A pinned “last known good” model with no record of the condition envelope it was good under is a guess dressed as a control.
What a compound-mode entry contains
| Field | What it records | Why it matters during an incident |
|---|---|---|
| Mode name and ID | Stable label, e.g. carton-revision × low-sun-angle | Lets telemetry, incident logs and runbooks reference the same thing |
| Contributing conditions | Each variable and its individual state (usually all in-tolerance) | Stops the engineer discarding a cause because it looks fine alone |
| Telemetry signature | The multi-signal pattern: which monitors move, in what direction, in what order | Turns pattern recognition into lookup |
| Reproduction conditions | How the combination was staged or captured, and against which golden set | Makes the mode a test, not an anecdote |
| Remediation path | The ordered actions, with owner per step | Removes the improvisation |
| Rollback trigger | The threshold at which the answer becomes version rollback | Makes the escalation decision non-negotiable |
| Evidence link | Model versions validated against this condition envelope | Tells you which pinned version is actually safe to restore |
The register lives inside the validation pack alongside the drift telemetry definitions and the rollback runbook, not as a separate document — an entry whose rollback trigger is not the same trigger the runbook names is worse than no entry at all.
Which line-side variables combine most often
On assembly and packaging lines we have worked with, the recurring combinations cluster around a small number of variables: ambient and fixture lighting, part presentation from fixturing and conveyor variance, packaging or label revisions, and operator novelty after a shift or crew change. Two patterns dominate.
The first is scheduled change plus environmental drift. A planned intervention — new SKU, fixture shim, belt replacement — lands during a period of slow environmental movement, typically seasonal light or a degrading lamp. The planned change is validated; the drift is not, because on its own it never crossed a threshold.
The second is appearance change plus geometry change. A carton redesign alters texture and contrast while a re-shimmed fixture alters the pose the camera sees. Detection confidence and localisation degrade together, which many single-cause monitors read as two unrelated mild anomalies rather than one significant one.
Neither pattern is exotic. Buyers commissioning industrial CV work usually recognise both from experience, which is why the presence of a compound-mode register is a fair proxy for pack quality.
How the telemetry has to be instrumented for this to work
A compound-mode register is only readable if the telemetry can express a joint pattern. Three instrumentation requirements follow, and all three are ordinary engineering rather than novel research:
- Shared timebase and correlation keys. Per-inspection summary records need model version, SKU/recipe, station, shift and fixture revision attached, so signals can be joined rather than eyeballed side by side.
- Sub-threshold retention. Signals must be recorded continuously, not only when they breach. A compound mode is by definition built from readings that never individually alarm; discarding them destroys the evidence.
- Composite conditions in the alerting layer. At least one rule must fire on a conjunction — two or more monitors simultaneously in their upper quartile — rather than only on single-signal breaches. Prometheus-style multi-condition rules or an equivalent in the plant monitoring stack are sufficient; the constraint is design, not tooling.
The sampling and budget mechanics — keeping this inside takt time and off the accept/reject path — are a separate discipline, and we treat drift instrumentation as an asynchronous sidecar for exactly that reason. The broader production-hardening argument, including how these artefacts fit together at release, is developed in our work on production AI reliability for line-side systems, and the release-criterion question underneath it in why pilot accuracy is the wrong release criterion.
Using and maintaining the register
During a live incident the register is read, not written: match the observed signal pattern to an entry, execute the remediation path, and escalate to rollback when the entry’s trigger is met. The measurable outcomes are the ones worth reporting to the plant — time-to-diagnosis per drift incident, the share of incidents matched to a documented mode rather than triaged from scratch, and whether the line stays on the model instead of reverting to manual inspection.
Writing happens afterwards. Every incident that did not match an existing entry becomes a candidate new mode, with its signature captured from the retained telemetry while the evidence is still there. Line refreshes and new SKU introductions trigger a review in the other direction: which existing entries describe a line that no longer exists, and which condition variables have just been added to the space. A register that is never pruned decays into folklore.
The uncertainty worth naming: the register is inherently retrospective. It documents combinations someone has seen, and the combination that finally moves your rejection rate may be its first appearance anywhere. What the pack can do is shrink the unknown set incident by incident, and make the rollback decision available while the unknown is still unknown. Which combination on your line has already happened twice without being written down?
Frequently Asked Questions
How do compound modes differ from the single-cause failure list most CV pilots ship with?
A single-cause list pairs one condition with one threshold and one remediation note. A compound-mode entry names a co-occurrence of conditions that are each individually in tolerance, and records the joint telemetry signature they produce. The practical consequence is that a compound mode is discoverable by pattern lookup, whereas a single-cause list leaves the engineer to reconcile several near-threshold alerts unaided.
How does an on-call engineer use the compound-mode register during a live incident, and when does it escalate to model rollback?
The engineer matches the observed multi-signal pattern against the register’s telemetry signatures, then executes the matched entry’s remediation path in order. Escalation is not a judgement call: each entry carries an explicit rollback trigger, and when that threshold is met the pinned model version validated against a known condition envelope is restored. Rollback stays decidable precisely because the entry records which conditions the pinned version was validated against.
How is the register updated after each incident and after line refreshes or new SKU introductions?
Any incident that did not match an existing entry becomes a candidate mode, with its signature reconstructed from retained sub-threshold telemetry before that data ages out. Line refreshes, lighting retrofits and new SKUs trigger the reverse review — retiring entries that describe conditions the line no longer has, and adding the new condition variables to the combination space. Both directions are pack changes and should enter the same change queue as a model change.
Twelve failure patterns that co-occur in real deployments
Occlusion plus motion blur, lighting shift during class imbalance, and label noise compounded by jpeg artifacts—these combinations break models far more reliably than any single stressor in isolation.
