An INT8 edge vision model is ready to ship when three things are written down and met. Per-class recall on the rare, critical classes stays within an agreed gap of the FP32 baseline, measured on the target device. The validation set covers the field conditions (dust, glare, night) with real footage, not only simulated degradation. And an accountable owner signs the result against thresholds fixed before any data was collected. Headline accuracy hides the risk: calibration-set composition barely moves average accuracy but shifts individual categories, and compression hits the under-represented long tail hardest. No single “good enough” number works across edge AI devices. The bar follows what a missed event costs in the deployment at hand.
Can dust, glare and night cases come from the logs we already have?
If vehicles or cameras have already been running the route, we would start with archived footage. Mine it for naturally occurring dust, glare and night segments, triage them with the current model, and book new field passes only for the conditions the archive cannot cover.
TechnoLynx builds this harvesting pipeline on existing logs for edge vision teams. The engagement agrees the edge-case taxonomy and per-condition coverage criteria first, then triages the archive and schedules only the real passes needed to close its gaps.
Triage the archive before booking trial time
Kognic, an annotation vendor, describes edge-case mining as finding rare or model-challenging scenarios in fleet data, classifying them and routing them into annotation. Its triggers include direct glare and lighting transitions such as night or tunnel entry. Production pipelines combine several automated detectors: uncertainty sampling on low-confidence outputs, behaviour-triggered mining, embedding-space anomaly detection and cross-sensor inconsistency checks. Kognic also gives a filter for what is worth labelling. A segment qualifies when the model fails or nearly fails on it, the failure repeats across conditions, and the frame can be labelled reliably. We would rank candidates the same way. A rare condition the model already handles teaches nothing, and a frame fully blanked by dust cannot be labelled at all.
Mined footage may arrive without labels for the rare condition, so budget annotation effort for it. Kognic recommends a specialist review tier for ambiguous or rare-class frames, because a labelling error on a rare scenario carries disproportionate weight in training.
How many extra real passes per condition?
Size the extra passes with statistics rather than a round number. Chen (2007) gives an exact method for the minimum sample size needed to estimate a Poisson rate (how often a condition occurs per hour of route, say) with a prescribed margin of error and confidence level. For recall, which is a binomial proportion, Chen’s companion paper warns that the usual normal approximation can leave coverage well below the stated confidence level, and that the problem is worst when the proportion’s upper bound is small.
Volume of footage is not the same as coverage. In the LiDARDustX dataset, natural dust mainly increased false positives and false negatives, while dust raised by vehicles degraded orientation accuracy and increased false positives. A dust quota that never separates dust sources can therefore pass a model that fails on the other kind.
A season-ahead plan that mixes simulation with real capture
When the collection plan has to be built before the season starts, combine scheduled real captures of dusty and backlit scenes with physics-based simulation and synthetic augmentation. Simulation supplements a validated minimum of real passes and never replaces it. Physics-based simulation is credible for some sensors. The ALDUS model for automotive LIDAR in dust clouds produced simulated output in good agreement with experiments on an Ouster sensor, and Volvo Autonomous Solutions said dust simulation would help it take mitigating actions before deploying in mines. Learned synthetic data can help as well. A GAN-generated soiling set (2019) raised soiling-segmentation accuracy on real soiled test images from 73.95% to 91.71%. Training on the artificial images alone, though, cut accuracy significantly.
The case for keeping real passes rests on three findings. The authors of Occluded nuScenes (2025) state that their synthetic occlusions approximate real sensor degradation but do not fully replicate it. An evaluation of 204 ImageNet models across 213 test conditions (2020) found little to no transfer of robustness from synthetic perturbations to natural distribution shift. In the GLARE study, traffic-sign detectors trained on LISA without sun glare averaged mAP0.5:0.95 of 19.4 on glare images, against 39.6 trained on GLARE and 42.3 trained on both. Simulation is still needed, because real driving cannot reach rare-event volumes on its own: RAND (2016) estimated that demonstrating autonomous-vehicle safety by road testing alone would take hundreds of millions of miles, and pointed to simulators and scenario testing as supplements.
Before synthetic dust or glare counts towards sign-off, validate the simulator against real sensor degradation on the same hardware. “Enough” coverage is defined by passing a cross-season, cross-weather validation harness whose condition cells were agreed with stakeholders in advance. Domain shift and out-of-distribution evaluation are covered in catching domain shift before it hurts.
What INT8 recall gap against FP32 should fail a release?
Write the threshold down before testing, per class. For example: INT8 recall on each rare class must stay within 2 percentage points of the FP32 baseline, enforced by an automated accuracy-regression gate. A larger gap blocks the release until calibration, retraining or a fallback to FP16 or mixed precision closes it.
TechnoLynx implements that accuracy-regression gate. TechnoLynx identifies the rare classes, builds the INT8 pipeline against the FP32 baseline, measures per-class recall on the target device and reports the outcome to the person who signs. Choosing the device is a separate, hardware question. LynxBenchAI is the tool for it: its published results measure candidate AI hardware, not your model’s recall on your data.
The 2-point figure is a choice, not a law. Qualcomm’s 2021 quantisation white paper reports W8A8 within 0.7% of floating point for all models it tested. NVIDIA’s 2020 workflow kept 8-bit accuracy within 1% of the baseline on every network studied. MLPerf Inference rules (MLCommons) gate several benchmarks at 99% of FP32 accuracy, and automotive 3D object detection at 99.9%. Detectors in published tests, measured in mAP rather than classification accuracy, lost more. Static INT8 TensorRT engines for YOLO12 lost roughly 3 to 7% absolute mAP50-95 on clean COCO data. A YOLOv8-L teacher lost 23% mAP under INT8 post-training quantisation on BDD100K (2026), measured on a desktop GPU rather than automotive edge hardware. For a cost and latency view of the same INT8 decision, see accuracy, latency and cost evidence for INT8 and FP16 deployments.
Rare classes make the recall estimate noisy
A test set with 50 positives of a rare fault gives recall in steps of 2 percentage points (1/50 = 0.02), so a 2-point bar on that set is a one-event bar. Calibration adds its own variance. A 2023 benchmark of post-training quantisation reliability that quantised each model 50 times with different random calibration samples found per-class accuracy far more sensitive than the average. In its classification experiments, growing the calibration set beyond 32 samples did little to reduce that per-class variance. Size the rare-class test set with an exact binomial sample-size method for the margin of error the bar requires. Set the bar per class where the consequences differ.
Measure on the target device, not in simulation
Quantisation simulation on training hardware is the usual way to explore options, because it is far easier than running real quantised kernels. Sign-off cannot rest on it. A 2026 cross-platform study found FP32 predictions bit-identical across every hardware pair, while INT8 predictions agreed on only 958 to 965 of 1,000 inputs whenever two targets used different integer kernels. Top-1 accuracy was preserved, so a standard accuracy report showed nothing. The same study found that Qualcomm’s Hexagon HTP silently ignored externally supplied quantisation scales: on-device accuracy fell from 0.75 to 0.005 while the model compiled and ran without error. Its recommendation is to re-validate INT8 per target and treat vendor-native quantisation as mandatory.
Fix the evaluation setup too. In that study, a 1,000-image subset overstated top-1 by 9.77 percentage points on average. A change of resize and crop moved top-1 by 1.07 points, about nine times the 0.12-point cost of quantisation itself. Run the gate under sustained load, not just a short bench pass.
The release gate in order
- Fix the rare classes, the per-class recall bars and the condition cells before collecting validation data; EASA’s concept paper on Level 1 and 2 machine learning applications (Issue 02) asks for thresholds to be specified early, tied to safety objectives.
- Run an FP32 sanity check: with quantisation bypassed, the pipeline should reproduce FP32 accuracy.
- Freeze the field validation set and fix the preprocessing for both the FP32 and INT8 runs.
- Measure per-class recall per condition cell on the actual target device, through the vendor’s own quantisation path.
- Compare each gap and its confidence interval with the agreed bar; any breach blocks the release.
- Hand the report to the accountable owner for a recorded decision.
Who signs off a quantised model trained on under 1% rare events?
The accountable safety or compliance lead signs, not the engineer who built the model. That person accepts the gate result against thresholds agreed before testing. EASA’s guidance on machine learning applications sets out logic we would carry over. Performance metrics should conservatively estimate the probability of the model’s failure modes. The generalisation gap should be covered by explicit margins on performance requirements. That release-readiness evidence is part of production AI reliability work; it supports the sign-off but does not replace safety certification or regulatory approval.
Who can build the validation harness and release gate for our edge models?
TechnoLynx builds the engineering for on-device validation: the field-data harvesting and coverage harness, the INT8 accuracy-regression gate against FP32, and the per-device measurement the sign-off rests on. An engagement starts by framing the rare classes, condition cells and thresholds. It then covers the gate implementation and the report that owner signs. For choosing the hardware itself, LynxBenchAI publishes measured results for AI hardware.
Should we re-tune confidence thresholds, fine-tune on site, or hold the release?
There is no vendor-neutral number that settles this. Make the call case by case: weigh the measured gap against how many errors the deployment can tolerate, against what a local fine-tune would gain and cost in time, and against the risk of moving a threshold instead of fixing the model. Getting close to 100% accuracy is rarely plausible, so the workflow needs fault tolerance and room for a human in the loop.
Start by splitting the gap into two kinds of error. Some misses sit just under the operating point and can be recovered by moving the confidence threshold. That trade is explicit in rare-event work: a 2024 safety-critical study set its threshold so that most positive samples were kept while easy negatives were filtered out. Other misses come from a distribution mismatch (a lighting regime or dust type the model has never seen), and no threshold fixes those without paying for them in false alarms. Set the acceptable error level from the cost of a false positive and a false negative at this site.
Decision table: ship, re-tune, fine-tune or delay
| Condition after measuring the gap | Choice | Risk to check |
|---|---|---|
| Misses cluster just below the threshold, and moving it keeps false alarms within the agreed cost | Ship with the re-tuned threshold, recorded in the release report | Threshold changes trade false positives against false negatives and leave distribution mismatch untouched |
| Both FP32 and INT8 miss on local conditions, a quick pilot shows a small local fine-tune closing most of the gap, and enough local frames exist to hold some out | Fine-tune, then re-run the full gate | A handful of images can overfit and give a false sense of improvement |
| Neither route closes the gap within the timeline | Delay the release | A demo-only pass can hide a production failure |
The gain from a local fine-tune is bounded by how many local images there are and how well they represent the site. Ten frames from one afternoon say little about the night shift.
Is there a quick fix short of full retraining before the deadline?
Sometimes, but the evidence for lightweight on-site fixes in field deployments is thin. What follows is our judgement from adjacent research, not a settled method. The fastest legitimate route depends on whether the shortfall comes from quantisation or from the domain.
If the gap is quantisation-induced, post-training repairs come first because they need little or no labelled data. Per-channel quantisation brought EfficientNet lite to within 1.4% of full precision in Qualcomm’s experiments. Cross-layer equalisation combined with bias correction reached near-FP32 performance without any data. AdaRound needs only a small set of unlabelled samples and no end-to-end fine-tuning. On a vendor NPU, those levers may only be available through the vendor’s own toolchain. Calibrating on deliberately degraded images is not a reliable shortcut: a 50/50 clean-and-degraded calibration set for YOLO12 performed much like clean calibration in most scenarios (2025). Quantisation-aware training closes more of the gap, but it brings the usual training costs: longer runs, labelled data and hyperparameter search.
If both precisions fail on local conditions, two lighter adaptations exist. Replacing batch-normalisation statistics with those of the shifted images cut ResNet-50’s ImageNet-C error from 76.7% to 62.2% mCE, and even one sample helped (2020). In a deployed INT8 engine, batch normalisation can be folded into the weight tensor. So in practice this means adapting the float model, re-quantising and re-running the gate, which is our reading of how the technique transfers to edge engines. Fine-tuning only the last layer of a detector on rare classes beat meta-learning methods by roughly 2 to 20 points on few-shot benchmarks. Those are benchmark results, not field data.
Any overnight fix still needs re-validation: rebuilding the engine on the device and scoring it on held-out local frames that played no part in the tuning. A fix that skips that step has not been tested.
FAQ
Can GANs be used for data augmentation?
Yes. A 2019 study generated realistic camera-lens soiling with a GAN and raised soiling-segmentation accuracy on real soiled images by about 18 percentage points (73.95% to 91.71%). Models trained on the synthetic images alone performed significantly worse, so this kind of data works as a supplement to real data.
Does calibration affect accuracy?
For post-training quantisation, it does. The composition of the calibration set barely moves average accuracy but can shift accuracy on individual classes. With per-tensor MinMax calibration, one outlier image can stretch the range and quantise every other activation more coarsely.
