Set automated optical inspection (AOI) thresholds per severity tier, not with one global cutoff. The critical tier gets a lower, more conservative threshold, chosen from the false-negative rate measured at each candidate threshold. The major and minor tiers run closer to the false-call load the line can absorb. Escape rate and false call rate pull against each other, and a system tuned loosely to cut false calls lets real defects through to functional test or into the field. The tiers can follow classes inspectors already use, such as the MIL-STD-1916 Critical, Major and Minor classes, where a critical defect means an immediate hold with no release authorisation.

A threshold only means something once three things underneath it are right: the model was trained at the scale it now sees, the metric actually registers thin defects, and the acquisition is not hiding defects before the classifier scores them. After that, the threshold needs records that let someone reproduce it and a rollback path for when it drifts. All of this is one part of the wider work of AI engineering for manufacturing and automotive lines, and the sections below follow that order.

Get the defect scale and domain right before changing the architecture

When recall on thin, elongated, low-contrast defects is poor, first check whether the model was trained at the scale, resolution and viewing angle it now sees in production. A mismatch there can outweigh any difference between architectures.

TechnoLynx’s Computer Vision R&D work starts at that point. TechnoLynx treats the miss as a possible domain mismatch, defines a taxonomy for the thin defects, fine-tunes a suitable segmenter on footage that matches the real acquisition, and checks the result against the current baseline on held-out data (Computer Vision R&D).

A 2026 study of high-resolution PCB inspection found that a detector trained on full boards resized to 640 pixels fell from mAP@50 of 0.70 to 0.01 when run on native-resolution tiles. The authors concluded that consistency between training scale and inference scale mattered more than the choice of architecture.

Once the domain matches, the architecture choice depends on which constraint binds. If boundary precision and recall on thin structures matter most, a U-Net-style network or Mask2Former is the starting point. If real-time inference on high-resolution frames is the hard limit, YOLOv8-seg is the pragmatic choice. One lens-inspection study picked YOLOv8 for the same reason, as a balance of accuracy, speed and computational efficiency.

Line-aware losses are worth testing, but the evidence for them is mixed. The authors of Skeleton Recall Loss (2024) report better Dice and clDice than earlier thin-structure losses on concrete crack segmentation, with slightly worse Betti numbers than clDice loss. An independent attempt to reproduce those results (2025) found that models trained with it did not differ significantly from plain models on test metrics. Run any streak-aware loss as an ablation on your own held-out data, watching false positives on surface texture as closely as recall. None of this works without a labelled set of the thin defects that is large and varied enough to fine-tune on.

Which metric suits filament corrosion better than mask IoU?

For thin, elongated defects, score detection with a centreline measure such as clDice or a boundary-focused measure such as Boundary IoU. Report recall for the thin-defect group as its own result, so a global average cannot hide it.

LynxBenchAI measures a candidate model’s throughput and latency on the target hardware.

The problem with mask IoU comes down to geometry. A small sideways offset on a defect a few pixels wide can remove most of its overlap with the ground truth, while the same offset barely changes IoU on a compact blob. clDice is computed on the intersection of the segmentation masks with their morphological skeletons, and its authors prove that it preserves topology for binary 2D and 3D segmentation. The Skeleton Recall Loss authors make a useful point about it: clDice is an inefficient training loss but an effective metric for connectivity. Boundary IoU (Cheng et al., CVPR 2021) is much more sensitive than Mask IoU to boundary errors on large objects and does not over-penalise errors on small ones. For structures that branch or have barbs, a multiscale IoU measures overlap across several spatial scales and responds to missed edges.

These metrics have a cost. Gaps in the labels distort the scores. In the lens study, small defects that were never labelled showed up as false positives and false negatives and pulled down both precision and recall.

Precision and recall for the safety-critical class, reported on its own

Compute precision and recall for the critical defect class as a separate row, and plot its own precision-recall curve. Precision, recall and accuracy averaged across classes can hide a detector that misses most real defects. The 2025 lens study shows how: overall recall was 0.624, but recall was 1.000 for lenses and only 0.249 for defects. Precision for defects was 0.431 against 0.944 for lenses.

Defect size matters too. The PCB tiling study reported small-defect recall separately, restricted to ground-truth boxes under 64 px² at the 640-pixel input scale. One more rule: when a sample contains no reference defects, recall is unavailable. It is not 100%.

Validating across camera angle and glare

Generalisation across viewpoint and lighting has to be shown on a held-out set split by those conditions, because a random split from the same pool says little about production. In one manufacturing study, a classifier scored almost perfectly on its validation set and then failed on a holdout set prepared the same way but captured on a different day. Lighting alone moves results. The KARMA paper reports U-Net mIoU falling from 0.729 at nominal illumination to 0.651 at half intensity.

The operating point moves too. In a 2026 hazelnut X-ray benchmark, validation-selected thresholds for the top method ranged from 0.42 to 0.78 across data splits. The authors advise recalibrating on independent, production-representative lots before deployment. If a condition such as steep angles or heavy glare is under-represented, expand the data for that condition before trusting any threshold. Accept, too, that rare extremes may stay unvalidated.

Is our escape rate a lighting problem or a threshold problem?

You can find out before retraining anything. Re-image the known escapes under controlled, standardised lighting and score them again with the existing classifier. If they now score correctly, the cause points to acquisition. If they still pass, the cause is the model or its threshold.

TechnoLynx defines the escape cases, triages the logged AOI scores and image metadata first, confirms with controlled re-imaging on the actual camera setup, and then implements whichever fix the evidence supports: optics reconfiguration, threshold change or retraining.

Running the diagnostic first matters because optics failures are easy to mistake for random noise. Lighting interference is the most common and least diagnosed cause of degraded AOI inspection, and it shows up as a gradual, inconsistent decline that looks like random noise. On a dented surface, strong diffuse light can make the dent almost as bright as flat copper, so the machine misses it. Turning the diffuse light down instead creates false catches on oxidation, dust and oil stains. A defect about the size of one pixel may go undetected if it straddles four pixels, each seeing only a quarter of it. In the KARMA failure analysis, illumination-related cases made up 22% of failures, and cracks were among the hardest classes because they get confused with shadows and normal structure.

Diagnostic procedure: acquisition first, threshold last

1. Pull the logged scores for confirmed escapes and see where they sit relative to the threshold. In the KARMA analysis, failure cases had much lower confidence scores (0.39 against 0.87).
2. Map where false positives and escapes occur on the part. Clusters near reflective solder joints suggest specular reflection.
3. Compare the current false-positive rate with the baseline recorded at commissioning, and confirm that the active program matches the current Gerber revision.
4. Only then adjust tolerances, one change at a time, watching the escape rate over the next runs before making the next change.

This sequence follows one AOI tuning guide’s rule: fix the physics before the software, because lighting, focus and 3D measurement settings resolve low-contrast false calls that no threshold change can. The test has two weak spots. Intermittent lighting faults may not reproduce in a single controlled capture, and when optics and threshold problems occur together, the result can come back ambiguous.

Can we give critical defects a much more conservative cutoff?

Yes. Give each severity tier its own threshold, and choose the critical tier’s cutoff from a sweep of false-negative rate against threshold on held-out critical-tier data, accepting the extra false calls that brings.

This is core work for TechnoLynx. TechnoLynx agrees the tiers and the false-negative ceiling for the critical tier, biases training toward recall on severe defects, runs the threshold sweeps, reviews the triage load the chosen cutoff creates, and deploys tier-aware thresholding with the critical tier’s false-negative rate monitored in production.

Threshold tuning can only work with the scores the detector produces. If critical defects mostly score low, no cutoff can catch them without flooding the line, so recall bias has to come first. The hazelnut benchmark tested one simple form of this: binary cross-entropy with pos_weight of 2.0, which doubles the loss contribution of defect samples to make the model more sensitive to false negatives. For scarce PCB defect images, practitioners also use augmentation, focal loss and oversampling of minority defect classes.

Threshold policy by severity tier

Tier	Threshold policy	Evidence required at sign-off
Critical	Lowest cutoff, set to meet an agreed false-negative ceiling; false calls accepted as the cost	False-negative rate with counts and an interval at each candidate threshold, on independent held-out data
Major	Cost-weighted cutoff balancing escapes against rework and review load	Escape rate and false call rate as a pair at the chosen point
Minor	Cutoff set mainly by the false-call budget the review station can absorb	False call rate trend against the commissioning baseline

What false-negative rate should we be able to show at each candidate threshold?

There is no universal number. The acceptable rate depends on what an escape causes: a missed bacterial contaminant and a cosmetic paint bubble are not the same risk. What you should be able to show is a table for the critical tier alone, giving the false-negative rate, the raw miss count and an uncertainty interval at each candidate threshold, measured on data the threshold was not tuned on. In the hazelnut benchmark, a mean defect recall of 79.4% corresponded to 18 to 24 false negatives per split on test folds of 126 to 128 kernels.

Be careful about how much a validation false-negative rate tells you. In a 2026 study of highly imbalanced industrial problems, false-negative rate generalised much worse than AUCROC, and the authors warn against judging models on validation FNR. They set targets on false-positive rate instead, because it is more reliable when faulty examples are few. Two formal tools help when the false-negative ceiling is the requirement. Conformal risk control (Angelopoulos et al., ICLR 2024) can bound the expected false-negative rate. Neyman-Pearson classification minimises one error type while holding the prioritised error under a user-set level, and its proponents see it as the more natural choice when there is no consensus on how to assign costs to errors.

Two conditions apply to every figure in that table. First, it is only as good as the critical-defect labels: in the hazelnut benchmark, expert reassessment of 15 kernels changed mean balanced accuracy by up to 3.4 percentage points. Second, it holds only for the conditions it was measured on, so a new sensor, angle or part type means sweeping again.

Should we tune to avoid missed defects even if false rejects go up?

For critical and edge-case defects, yes. Weight a false pass well above a false fail, derive the threshold from that weighting explicitly, and record the derivation so the choice can be defended in an audit.

For TechnoLynx this work is adjacent to its core delivery. TechnoLynx defines the cost asymmetry with the quality team, chooses a cost-weighted threshold method that leaves an audit trail, integrates it into the inference pipeline on the target hardware, tests it on edge-case data, and keeps the documentation current as assumptions change.

The asymmetry is real, but false calls have a cost too. A missed defect is usually the more serious error because the part may reach the customer, yet the false positive rate largely decides whether automated inspection pays for itself at all. High false call rates slow the line and wear operators down, to the point where they may start passing boards automatically and real defects escape. One AOI guide gives under 500 PPM as a world-class false call target, and notes that a rate of zero usually means sensitivity is too low and escapes are happening. A cost weighting that prices false calls at nothing will eventually come back as escapes.

Elkan’s foundation for this (IJCAI 2001) is simple: predict the class with the lowest expected cost, using the classifier’s conditional probability for each class. His recommended practice is to train the classifier on the data as given and then compute cost-optimal decisions from its probability estimates. He also shows how to avoid defining a cost matrix that is economically incoherent.

Worked example: turning a cost ratio into a cutoff

Assume the model outputs a calibrated probability p that a part is defective, and borrow the cost ratio from the scikit-learn credit example, where a missed bad credit costs five times a false alarm. Flagging the part is cheaper whenever p × 5 > (1 − p) × 1, which gives p > 1/6, a threshold of about 0.17. That is far below the default of 0.5, and the scikit-learn example finds the same pattern: the tuned threshold sits much lower than 0.5, with much higher recall and much lower precision. In that example, a credit-risk task rather than inspection, tuning the threshold improved the business gain by almost a factor of two.

The working only holds under its assumptions. The probabilities have to be calibrated, which is why the scikit-learn example tunes the model with a proper scoring rule (log loss) before touching the threshold. The threshold must never be tuned on the data the classifier was trained on, because of overfitting. In that example, a broad plateau of near-optimal gain across many thresholds was symptomatic of that overfitting. And with few minority-class examples, an offline estimate of the business metric can be unreliable and should be confirmed on live data. Cost estimates for missed defects are uncertain, so revisit the ratio whenever it changes. For when and how to recalibrate a cost-weighted cutoff over time, our page on calibrating and recalibrating cost-weighted detection thresholds goes further.

What records make a threshold choice defensible in a customer audit?

Keep enough to reproduce the decision exactly: the data, the method, the result and who approved it. TechnoLynx treats release-readiness reviews and validation packages as deliverables of their own, distinct from model accuracy. In practice that means:

• For every inspection: who performed it, when, the measured value rather than just pass or fail, and the action taken. ISO 9001 clause 8.6 requires retained evidence of conformity, traceable to the person who authorised release.
• Before release: the program version, the approved sample set, coverage exceptions and reviewer authorisation, with the recipe linked to its assembly revision and the revalidation triggers stated (library change, changed optics, software update, component substitution).
• The reference used for validation, shown to be independent of the AOI verdict, since agreeing with the machine proves nothing.
• The full threshold sweep, not just the winner. scikit-learn’s TunedThresholdClassifierCV can store every threshold and score from cross-validation when store_cv_results is enabled.
• For every threshold change: a change request, an impact assessment, validation and approval before production.

Keep the raw confidence scores, ground-truth comparisons and calibration outputs at part or lot level for as long as an audit could ask for them. Where a system falls under the EU AI Act’s high-risk category, Article 19 requires automatically generated logs to be kept for at least six months, and Article 18 requires documentation to be kept for 10 years after the system is placed on the market. Whether a given AOI line falls in that category is a legal question.

How fast can a miscalibrated threshold be caught after release?

As fast as your rollback trigger and the signal behind it allow, so define both before go-live. That means a rollback strategy, quality thresholds with a responder notified when they are breached, and drift alerts on input distributions.

Roll the change out in stages and break the metrics down by population. The Google SRE Workbook gives the arithmetic: a canary on 5% of traffic running at a 20% error rate shows up as only 1% in whole-system monitoring, but becomes obvious when the canary and the control are measured separately. Applied to AOI, a new threshold on one line or one shift should be compared against the old one directly, not lost in plant-wide yield figures.

The monitors need explicit limits. The ML Test Score rubric says quality thresholds should be set (for example, from the quality bounds at launch), a responder notified as soon as quality leaves them, and both sudden and slow-leak regressions watched. Vertex AI Model Monitoring, as one implementation, logs an alert when the statistical distance between an input feature’s distribution and its baseline exceeds a set threshold, 0.3 by default. On the line itself, run a known-defect board with at least 10 types of simulated error daily, with zero escapes allowed. Model versioning and rollback strategy belong in the same plan.

A human check should stay in place after launch. One tiered pattern sends confidence scores below a calibrated threshold to an operator and logs high-confidence passes and fails automatically. On top of that, a small random sample of auto-passed parts covers what the automated checks cannot see. Even so, a reduced sample can miss rare severe failures, and drift alerts need a stable baseline that may not exist in the first weeks. Calibration drift is slow and has no single failure event to point to, so it requires scheduled intervention, not a reactive response. Our page on revalidation, monitoring and rollback for deployed models covers when a change should trigger revalidation.

Who can help us calibrate and validate AOI thresholds for critical defects?

TechnoLynx engineers the model and evaluation side of AOI threshold work. That covers working out whether escapes come from the optics or the classifier, setting tiered, cost-weighted cutoffs with false-negative rates quantified for the critical tier, and stratified validation with audit-trail documentation. When the model itself has to change to see thin or low-contrast defects, that is the TechnoLynx Computer Vision R&D engagement. It starts by checking for a scale or domain mismatch before any retraining.

Whoever does it, ask them to show recall for the critical class on its own and false-negative intervals at each candidate threshold, not a single averaged accuracy figure.

FAQ

What does a threshold of 0.5 mean?

A threshold of 0.5 means the classifier predicts the positive class whenever its estimated probability is above 0.5. That default ignores costs. When a miss costs more than a false alarm, a tuned threshold is often much lower.

What is the Youden optimal threshold?

The Youden optimal threshold is the cutoff that maximises sensitivity plus specificity minus one, which amounts to maximising balanced accuracy. It treats both errors as equally costly, so it suits model comparison better than setting a critical-tier operating point.