A quantised model is ready for production once three things are true. It has been run on the exact accelerator it will ship on. It has been compared output by output against the FP32 reference. And it has passed pass/fail thresholds that the business and quality owners agreed before testing began. A build that passed on a different device does not carry over, because INT8 predictions agree across targets only when they share an integer kernel. After the build passes on target, roll it out in stages: one site or a few tenants first, with a tested fallback to the higher-precision model for any site that fails. The discipline is the same whether the model cleans broadcast video, runs automotive perception, watches a bedside ECG or serves a contact-centre voice stack.

Why “quantise once, deploy anywhere” breaks on embedded hardware

The usual assumption is a fair one to hold. INT8 post-training quantisation is the default recipe for edge deployment, and most people believe it makes inference faster at a small, predictable accuracy cost, and that a model quantised once can be carried to any target. A 2026 cross-platform measurement study of INT8 inference on embedded and automotive accelerators found that this portability fails in three ways.

The first is speed. Across the four CPUs the study tested, the sign of the INT8 speedup depended on whether the CPU has a dot-product instruction (ARM dotprod/SDOT, x86 VNNI). Cores that have one ran up to 2.1× faster. Cores without one ran 1.7× slower, with the identical model and runtime. The second is numerics. FP32 predictions were bit-identical on every platform pair (1000/1000 inputs). INT8 predictions agreed 1000/1000 only when two targets shared an integer kernel, and 958 to 965 out of 1000 when they did not, even with identical quantisation scales. Top-1 accuracy stayed the same because the flips cancelled out, so a standard accuracy report would never show the divergence.

The third failure is the most dangerous. Vendor NPUs own quantisation. An externally quantised ONNX graph submitted to a Qualcomm Hexagon NPU compiled, profiled and ran with no error, yet top-1 accuracy collapsed from 0.75 to 0.005 because the external scales were ignored. The DEEPX compiler rejected the same kind of graph with a hard error, which is the safer way to fail because nothing broken can ship. In both cases, only the vendor’s native quantisation path produced a correct engine.

A power-of-two scale constraint did not fix this either. Forcing every scale to a power of two moved an x86-to-Cortex-A76 pair further from agreement (958/1000 fell to 869/1000), and the study’s authors concluded that re-validating on each target is the dependable path. Their recommendations are to re-validate INT8 per target, to treat vendor-native quantisation as mandatory, and not to assume that two different compute units running the same INT8 model agree on every input. That last point matters for redundant automotive compute. Separate 2026 work on quantised language models reaches the same conclusion: behavioural validation has to run on the configuration you actually deploy, not on the source-precision checkpoint.

Testing on real hardware means running the INT8 build on the production accelerator class itself, with per-layer outputs instrumented and compared against the FP32 run on identical inputs. If an automotive-grade embedded GPU ships, test on one. If a Jetson Orin-class module is the target, test on that. Simulators and convenient substitute boards are fine for early iteration. They are not a basis for freezing the model, because on-device quantised behaviour depends on the backend, and desktop INT8 evaluation is not a reliable proxy for it.

What numerical differences from the FP32 model should we expect on the device?

Do not expect bit-exact agreement with the FP32 model. NVIDIA’s TensorRT documentation says bit-level accuracy is rarely possible with Q/DQ graphs, because valid optimisations such as reordering floating-point operations change the results. The quantiser itself trades two errors against each other. A wider range reduces clipping error but increases rounding error, since rounding error grows with the scale factor. Hardware support differs too. Per-channel weight quantisation is increasingly common, but not all commercial hardware supports it, so check your target device.

The runtime adds its own changes. With implicit quantisation, TensorRT may force some layers near the network output to run in FP32 to smooth quantisation noise, so the precision layout on the device can differ from the one you designed. An INT8 calibration cache is guaranteed bitwise identical only when generated on the same device with the same batch size and the same calibration inputs. It can be carried between devices only if calibration happens before layer fusion, because fusions are not guaranteed to be the same across platforms.

Microcontrollers show how far apart two backends can be. A 2026 study of an activity-recognition model measured 85.7% agreement with the FP32 PyTorch model on a Raspberry Pi Pico 2 (Cortex-M33, CMSIS-NN kernels), against 97.9% on an ESP32 running esp-nn kernels. “Parity” in that study means the share of test samples where the on-device prediction matches the original FP32 model’s prediction. The same model’s FP32 path on the Pico 2 reached 100% parity, showing the INT8 loss came from quantisation, not the architecture. For two other models, keeping weights in INT8 but moving activations to INT16 recovered parity that full INT8 export had lost.

Preprocessing can matter more than quantisation. In the 2026 portability study, the choice of resize and crop moved top-1 accuracy by −1.07 points, roughly nine times the −0.12-point cost of quantisation itself. A quantisation delta reported without fixed, documented preprocessing tells you very little.

On-device parity checklist

• The reference and the device use the same fixed, reported preprocessing.
• Layer outputs are dumped on the device and compared with golden values from a reference runtime such as ONNX Runtime, using a tool like Polygraphy, which also flags NaNs and Infs.
• Parity is reported per input (agreement count), not only as aggregate accuracy, because aggregate accuracy can hide prediction flips.
• The quantised engine was built through the target vendor’s native quantisation path.
• Repeated runs of an engine built for a specific GPU give bit-accurate outputs, as TensorRT documents.
• The runtime version on the target is recorded, since versions often cannot be made equal across platforms.

Agreeing pass/fail thresholds before the first on-device run

Write the thresholds down before anyone sees on-device results. Clinical trial methodology offers a useful model. The European Medicines Agency’s draft guideline on equivalence comparisons says the clinical justification of a margin matters more than the statistical one and usually produces a stricter margin. It also says the chosen margin drives the required sample size. The same logic carries over to model sign-off: define what difference from the FP32 baseline would actually harm the use case, then size the test set to resolve it.

Published results give a sense of what is achievable, but they are not your threshold. Qualcomm’s AIMET white paper (2022) reports 8-bit post-training quantisation within 1% of FP32 accuracy for several classification and segmentation networks. It also shows an ADAS object detector that needed AdaRound before INT8 came within 1% of FP32. Your margin should come from what the application can tolerate, measured both on the full set and on the input groups most sensitive to quantisation noise. Averages can hide regressions in those groups, which is why per-bucket gates belong in the sign-off. See our piece on per-bucket accuracy gates and why average metrics hide regressions.

Systems that feed their own outputs back in need a further check. In a 2026 FPGA deployment of a quantised recurrent plasma-prediction model, the 16-bit model landed within +2.8% MSE and −4.8% explained variance of the FP32 reference on held-out data. Rolled out autoregressively, though, the quantised ensemble trailed the baseline by 0.51 explained-variance points after 100 steps, a closed-loop amplification that single-step metrics could not show. If your model drives a control loop or a tracker, the threshold has to apply to the closed-loop output.

For a medical device that cannot be updated in the field, the goal is equivalence to the validated FP32 model. Run the original clinical reference dataset through the quantised build on the target accelerator and report per-case metrics (Dice or IoU for segmentation) against the FP32 baseline, inside a margin the clinical and regulatory owners justified beforehand. Report the subgroups most sensitive to quantisation noise separately. That validation report is engineering evidence. It does not replace the manufacturer’s regulatory judgement.

Accuracy per watt: the go/no-go between a quantised Jetson model and a bigger one

Edge modules have set power budgets and throttling limits, so measure power at the operating point you will ship. NVIDIA’s Jetson Linux developer guide lists power budgets such as 10 W, 15 W and 30 W for Orin modules. For the Orin NX 16GB it gives a 25 W average and a 30 W instantaneous input-power limit, with throttling levels of 50% and 87.5% on CPU and GPU when those limits trip. A 2025 study of quantised small language models on edge boards measured power at 100 Hz with an Otii Ace Pro and reported accuracy per watt alongside throughput and energy per inference. On that LLM workload, the GPU path on a Jetson Orin Nano beat the CPU-based setups on energy efficiency. It is a different workload from perception, but the method transfers well: measure power on the device under the real duty cycle, then normalise accuracy by it.

Suppose the choice is between a quantised model on Jetson and a larger non-quantised model that needs a different enclosure or more power. A defensible go/no-go measures accuracy per watt and accuracy per unit cost for both candidates under the same duty cycle, then checks each against the compliance accuracy threshold. If the quantised model clears the threshold, the cheaper, lower-power deployment wins. If it does not, weigh a change of hardware path (enclosure, power supply or accelerator) with its full costs.

For the hardware half of that decision, LynxBenchAI’s published results show how candidate devices run the public PyTorch stack your own models are built on. That makes them broadly indicative for custom PyTorch models, but they do not measure your specific model or application.

Proving a compressed model holds frame rate and picture quality on broadcast appliances

Before a national rollout, prove a compressed video model on representative transmitter appliances, not on a development workstation. Time every frame of real broadcast test content against the real-time deadline on the appliance itself. Inspect the output for tone-mapping artefacts under standardised viewing conditions. Keep a pre-validated fallback ready for any site that fails. TechnoLynx does this broadcast and media AI engineering work through the AI Porting & Deployment Pack. The pack covers deploying the quantised model on a sample of your appliances, building the hardware-in-the-loop harness and running the frame-by-frame and artefact tests. It starts by agreeing the frame deadline and the visual acceptance criteria with your engineering and QC owners.

The real-time proof has to cover the worst frame, not the average one. A model that meets its deadline on average can still miss it on individual frames. In the 2026 portability study, edge-NPU latency was governed by data movement rather than compute. On one PCIe-attached NPU, a lighter-compute YOLOv5s detector ran at 41 fps against 1079 fps for ResNet-50 on the same device, because the detector had a far larger output to move across the link. So measure on the appliance with its real I/O path, run the test long enough to reach thermal steady state, and log the slowest frames.

For tone-mapping artefacts, choose test content that stresses the tone curve hard. View the quantised output side by side with the FP32 output on calibrated reference monitors in controlled ambient light, so that two reviewers would reach the same verdict. Broadcaster delivery rules set the bar. SVT’s quality-control requirements say there must be no visible contouring or artefacts from digital processing, that quantisation noise must not be apparent, and that eyeball QC is mandatory.

A subset of sites failing should not block the whole rollout. Those sites get a less aggressive quantisation variant or the original model, and that fallback has to have passed the same tests already, or the failing sites have nothing proven to fall back to. A sample of appliances cannot guarantee every hardware revision in the fleet, so record which revisions were tested and treat untested ones as pilot sites.

How do we set objective quality thresholds and still get an editor’s sign-off?

Quality sign-off for a quantised broadcast model runs in three gates. First, objective metrics across representative content, with pass/fail thresholds agreed by editorial and QC. Second, a structured human review inside the live production workflow. Third, a single-site pilot before network-wide deployment. Neither the metric nor the reviewer is enough alone, because objective measures and human judgement each catch problems the other misses.

LynxBenchAI publishes results showing how candidate broadcast appliances run the public PyTorch stack, which is broadly indicative for your models though it does not measure your own model, and the first step is to read its published results for your appliance class.

The first step is to read its published results for your appliance class.

Signing off on-air footage with an editor’s eye

A benchmark number alone should not approve noise reduction for on-air news. A person has to look at it, and broadcast delivery specifications already require that look: SVT makes eyeball quality control mandatory to confirm that video and audio quality are consistent throughout. A workable process gives the editor or QC reviewer blind A/B pairs of quantised and FP32 output inside the edit tool they actually use, on footage that covers low light, fast motion and compression-heavy sources. The reviewer records a signed approval against the specific build. If the reviewer can reliably pick the quantised clip as worse, the build does not pass, whatever the metric says.

Testing for audible artefacts with PEAQ and listening tests

PEAQ, defined in Recommendation ITU-R BS.1387, compares a reference signal with the signal under test and outputs an Objective Difference Grade (ODG) meant to correspond to the subjective difference grade a listening panel would give. The method is validated against subjective listening tests, not directly, and its ODG is resolved to one decimal place. According to the recommendation, two ODG values that differ by a tenth should be treated as equal quality. A gate tighter than 0.1 ODG against the FP32 output therefore asks for more precision than the method has. Test items typically run 10 to 20 seconds, and the reference and test signals must be time-aligned to within 24 samples. So check alignment again after any change to the pipeline.

PEAQ has documented limits. One published analysis found PEAQ’s results depended significantly on the codec, and noted that the model was never trained on HE-AAC impairments. A 2022 performance analysis found that PEAQ’s disturbance-loudness measure describes degradation well for speech but much less well for music. A 2021 review of objective measures warns more broadly that these methods are built for one application domain and are often used outside it without evidence that they still hold. So use the objective gate to filter out obviously bad builds, and let a listening test make the final call. For speech, that test is a panel rating on the ITU-T P.800 scale, and full-reference algorithms such as PESQ and POLQA serve as the lab-side approximation. Express the threshold as a delta against the FP32 model’s output on the same content, never as an absolute score.

Should we pilot on one site or a few tenants before switching everyone over?

Yes, once both the objective and human gates have passed. A single regional site or one or two real tenants will surface conditions that a test set did not cover. The pilot has to run long enough to catch intermittent artefacts, which means covering a full programming cycle or the tenant’s real traffic peaks, not one afternoon. Pick the pilot with care, because a quiet site or a forgiving tenant can hide problems that only appear at full scale. If either gate fails before the pilot, stop and rework the quantisation approach rather than piloting a marginal build.

What goes in the go/no-go evidence package

The go/no-go package works as a release-readiness review. It is the artefact that catches regressions before customers do, and it is distinct from a headline accuracy figure. Make it reproducible and tie it to the exact build. A 2026 paper on certified no-regression audits for model updates, including quantisation, has each audit produce a machine-checkable evidence record suitable for EU AI Act post-market monitoring.

Evidence	What it shows	Measured on
Per-input parity against FP32	Prediction agreement, not just aggregate accuracy	Target device, fixed preprocessing
Per-bucket and worst-subgroup accuracy	No hidden regression in sensitive inputs	Target device
Worst-frame and p99 latency	Real-time deadline holds	Target device, real I/O path
Sustained power and thermal behaviour	Budget holds at the shipping power mode	Target device, real duty cycle
Objective plus human perceptual result	Output quality acceptable to the people who approve it	Representative and edge-case content
Throughput and cost per tenant	Margin target met under real load	Production-like multi-tenant load
Fallback test result	The rollback path works and is itself validated	Target device
Build identity	Exact artefact, runtime version, calibration set	Recorded with the package

For a multi-tenant quantised Whisper deployment, the package that convinces both engineering and finance puts measured throughput, latency and cost per tenant from the optimised stack next to a definition of “no audible quality loss” built from objective metrics plus a listening test. All of it should be run reproducibly under real multi-tenant load, not idealised single-tenant runs, with a real-tenant pilot whenever the margin or the quality evidence is marginal.

For contact-centre voice, the perceptual gate is a blind listening-panel MOS comparison of quantised against FP16 output. Go-live depends on the measured MOS delta sitting inside the client’s contractual threshold with margin to spare, not just inside it. Run both arms through the same telephony codec, since every codec caps the MOS a call can reach and no network can push a call above that ceiling.

Will customers hear the difference after an INT8 or TensorRT conversion?

They might. Quantisation and TensorRT conversion change the computation: lower precision and reordered floating-point operations. TensorRT notes that its default precision choice can result in poor accuracy, so audio quality needs its own check.

LynxBenchAI shows how the target accelerator runs the public PyTorch stack before anything is converted, which is broadly indicative for your voice model but does not measure it. The first step is to read its published results for that device.

Audio validation procedure after a precision pass

1. Define the objective metrics and the listening-panel protocol before converting anything.
2. Build a test set of representative and edge-case audio (quiet passages, sibilance, music beds, crosstalk).
3. Calibrate with representative data. TensorRT’s documentation says the amount needed is application-dependent and that about 500 images suffice for ImageNet classifiers, so audio calibration sets have to be sized on their own terms.
4. Run the FP32 baseline and the optimised engine on identical inputs, and compare both objective scores and panel ratings.
5. If degradation exceeds the threshold, recalibrate with more representative data or pin sensitive layers to higher precision, which TensorRT allows per layer.
6. Re-run the whole procedure after any model or pipeline change.

If quantisation cannot hit the accuracy or power target, what do we fall back to?

Work down a ladder of options, ordered from least to most expensive to try. Start by fixing the recipe. AIMET’s debugging flow first separates the impact of weight quantisation from activation quantisation, then quantises one layer at a time while keeping the rest at 32 bits. If accuracy is still short, it moves to higher bit-widths, finer granularity or quantisation-aware training.

Some networks need quantisation-aware training (QAT) to maintain accuracy. NVIDIA reports that EfficientNet-B0 drops from a 77.4% FP32 top-1 baseline to 33.9% with post-training quantisation but recovers to 76.8% with QAT, while ResNet-50 quantises fine without QAT. Its recipe fine-tunes for around 10% of the original training schedule. In the 2026 FPGA study, naive post-training quantisation destroyed the model outright, and two-stage fine-tuning recovered it. Our article on validating compressed models from PTQ to QAT on the edge goes into when that extra training pass pays off.

Mixed precision comes next. A 2025 review of quantisation for microcontrollers recommends uniform INT8 in most cases, with FP16 used selectively for sensitive boundary layers or accumulators where profiling shows a problem. A smaller backbone or pruning is the step after that. A Cortex-M deployment study kept the accuracy drop under 5% when combining aggressive structural pruning with quantisation, and QAT kept the loss close to zero. The 2026 portability study saw DETR lose 42.9% of its mAP under INT8 on a discrete GPU, with the same collapse confirmed on Jetson. When that happens, the fallback is a different architecture, or a hardware path that does not need INT8 for that model.

What you see	Next step
Small accuracy gap	Better calibration data, per-channel weights, AdaRound
Gap concentrated in a few layers	Keep those layers at FP16 or INT16 activations
Large gap from naive PTQ	Quantisation-aware training
Accuracy fine, power or memory over budget	Pruning or a smaller backbone
Architecture collapses under INT8	Different accelerator or architecture

Keep model replacement last. In production, the cost and latency levers below the model (quantisation, batching, kernels, runtime choice) can usually be tuned on their own and cost far less than a new model.

Can on-device validation run on every new quantised build, and how large a sample do we need?

Yes. Build a field validation set from real production imagery, such as in-store produce photos stratified by occlusion, lighting and SKU variety. Run it automatically as a gate whenever a new quantised model is pushed to the handhelds, and block the rollout if accuracy on that set falls below a threshold agreed relative to the lab set.

A lab set alone is not enough. Accuracy on the lab set does not establish accuracy on the shelf. Prefer an automated gate to manual spot checks once pushes are frequent, so every build faces the same test. Use manual review for borderline results or under-sampled strata. Score per stratum rather than in aggregate, for the reason set out in our piece on per-bucket gates: one good stratum can mask a failing one.

A paired design keeps labelling cheap. Run the FP32 and INT8 models on the same field images. Wherever they agree, that image contributes nothing to the risk difference between them, so labels are only needed where they disagree. The 2026 certified-audit work found models agreeing on 80% to 99.9% of traffic under realistic updates. In its pilot at a 2% tolerance, 98.2% of benign update pairs earned a zero-label certificate.

Worked example: sizing one stratum

Assume you want to estimate a stratum’s accuracy to within ±3 percentage points at 95% confidence, and that you expect accuracy around 90%. The standard sample size for a proportion is n = z²·p·(1−p) / E². With z = 1.96, p = 0.9 and E = 0.03, that gives n = 3.8416 × 0.09 / 0.0009 ≈ 384.2, so 385 images for that stratum. Twelve strata at that precision need about 4,620 images in total, and halving the margin to ±1.5 points quadruples the count. That arithmetic is why “how many per store” depends on how many strata you define, not on a fixed figure.

The gate only catches failures that appear in the sampled imagery. New SKUs, seasonal produce or a store with a different lighting rig can fall outside every stratum, so review the strata on a schedule.

Edge arrhythmia detection: matching the cloud model and failing safe at the bedside

Edge-deployed arrhythmia detection is ready for clinical use only under two conditions. The quantised edge model must match the full cloud model within a pre-agreed tolerance on accuracy and, above all, on false-negative rate, measured on your own patient population. And a fail-safe behaviour for loss of edge compute or connectivity must have been specified and tested. Until both hold, the cloud path stays primary. TechnoLynx handles the engineering side: accuracy-preserving quantisation on the bedside hardware, the one-shot release-gate comparison of edge against cloud outputs, and fail-safe tests under simulated hardware and network failure. The work starts by fixing the tolerances against the cloud baseline with your clinical team.

Published results show the engineering is feasible, but they are not evidence for your device. A 2026 Bioengineering paper reports a quantised three-class arrhythmia classifier reaching 97.6% accuracy on the MIT-BIH database under benchmark conditions, with a memory footprint below 24 KB and 200 ms per heartbeat on an ultra-low-cost microcontroller. A reduced three-class benchmark on the public MIT-BIH database is not your patient population. The validation evidence needs a clinically justified tolerance and a sample sized to resolve it, as the EMA guidance describes for equivalence margins.

Accuracy is the wrong headline here. A missed ventricular event costs far more than a false alarm, so report false-negative rate per arrhythmia class and treat any regression there as a hard stop. The fail-safe design (raise an alarm, escalate to the central station, or hold the last state) involves clinical risk trade-offs that engineering alone cannot settle. What engineering owns is proving that the chosen behaviour actually fires when the edge module or the network fails.

Swapping the FP32 reference for a quantised model on a shared test bench

Replacing the FP32 perception model on a bench that also exercises sensor fusion and planning is safe only if the model sits behind a versioned interface. That interface fixes the output schema and timing contract, so the FP32 reference can be reselected per test without touching fusion or planning code. Rollback speed is then however long it takes to flip that selector and rerun the affected regression subset. TechnoLynx designs that model-selection layer. The work covers an FP32/INT8 switch with matching output schema and timing, per-scenario routing so a single edge case can fall back to FP32 without a global rollback, and CI hooks that rerun the fusion and planning suite under both paths. It starts by mapping where the perception-to-fusion contract is implicit today.

LynxBenchAI publishes results showing how candidate perception hardware runs the public PyTorch stack, which is broadly indicative before you choose the bench’s INT8 target though it does not run your model, and the first step is to review its published results for that device.

The first step is to review its published results for that device.

A downstream regression suite that runs in minutes makes per-test rollback cheap. A slow or manual suite turns every rollback into a maintenance window. The integration risk lies in hidden coupling, such as hard-coded latency assumptions or output format differences between the two models, which can break fusion while the interface still looks stable. INT8 targets will not agree on every input: the 2026 portability study found disagreement whenever two targets did not share an integer kernel. So fusion tests that compare against frozen perception outputs need tolerances, not exact matches. Fallback on individual edge cases is useful but can hide a systematic quantisation problem, so count every per-case fallback and review the count as a metric.

Monitoring in production so a regression rolls back the affected site or tenant

A rollback has to be built into the serving path before go-live. Keep the full-precision model deployable, score the quantised model’s quality continuously, and route an affected site or tenant back to FP16 or FP32 automatically when a trigger fires. A 2026 paper on certified quantised attention keeps FP16 originals in system RAM behind a multi-stage fallback ladder. With the fallback disabled, the same INT8/INT4 configuration dropped retrieval accuracy to 5 to 10%, against 39 to 66% for the dense baseline. The fallback has a cost: that system ran at 2.7 to 4.8 times the latency of dense attention. Budget the fallback path’s latency and capacity before you need it.

Shadow comparison is a label-free trigger. Score a sample of live traffic with both the quantised and reference models and watch the disagreement rate. The 2026 certified-audit protocol does this with an anytime-valid confidence sequence that stays valid however often the operator checks it, and labels only sampled disagreements. Tune the per-tenant trigger against normal quality variance so it does not roll back on noise. On edge devices, read it alongside thermal and clock telemetry, since Jetson modules throttle CPU and GPU when power limits trip. TechnoLynx’s Production AI Monitoring Harness instruments this per-tenant quality scoring and rollback in the production pipeline.

Who can validate a quantised model on our target hardware and run the rollout?

TechnoLynx validates quantised and compiled models through its AI Porting & Deployment Pack on the accelerator you will actually ship, against your FP32 reference, and turns the results into a go/no-go package that your engineering, quality and finance owners can sign. The work starts by fixing thresholds with those owners. It then deploys the build on target hardware with a hardware-in-the-loop harness, measures it against the FP32 reference, and ends with a reproducible runbook. When the open question is which accelerator to buy, LynxBenchAI is the measuring tool. Its published results show how devices run the public PyTorch stack, which is broadly indicative for your models, though it does not run your workload.

Keeping the work in-house makes sense when your team already owns the target toolchain (TensorRT or the vendor’s NPU compiler) and has a rack of production units to test on. A specialist earns their place when the target is new to you, when the vendor’s quantisation path is unfamiliar, or when the evidence has to convince an outside reviewer. Whoever does the work, ask for per-input parity reports from the real device and a fallback that has itself been tested. Those two artefacts separate real validation from a lab number.

Sign-off has to be repeated when the toolchain changes

A validated build is validated only for its target, so re-validate INT8 per target. The 2026 portability study ran four CPUs on three different ONNX Runtime versions because the versions could not be made equal across targets. TensorRT calibration caches are not guaranteed identical across devices or batch sizes. Treat every compiler, runtime or firmware update on the target as a new build that passes back through the same gate.

FAQ

What is a quantised model?

A quantised model stores its weights and activations at lower precision than the 16 or 32 bits it was trained in, usually 8-bit integers. Moving from 32-bit to 8-bit cuts tensor storage by a factor of four.

What is 4-bit and 8-bit quantisation?

The figure is the bit width of each stored value. TensorRT, for example, supports INT8, weight-only INT4, FP8 and FP4 types. INT8 remains the standard for microcontroller-class deployment because mainstream frameworks have limited support below 8 bits.

Sources

1. Is INT8 Portable? A Cross-Platform Measurement Study of Quantized Inference on Embedded and Automotive Accelerators, arxiv.org
2. Quantization-Triggered Backdoors in Language Models: Cross-Quantizer Transferability and the Validation--Deployment Gap, arxiv.org
3. MicroBi-ConvLSTM: An Ultra-Lightweight Efficient Model for Human Activity Recognition on Resource Constrained Devices, arxiv.org
4. Working with Quantized Types — NVIDIA TensorRT, docs.nvidia.com
5. Neural Network Quantization with AI Model Efficiency Toolkit (AIMET), arxiv.org
6. Improving Model Accuracy — NVIDIA TensorRT, docs.nvidia.com
7. Europa, ema.europa.eu
8. Real-Time Plasma State Prediction via FPGA-Accelerated Quantized Recurrent Probabilistic Neural Networks, arxiv.org
9. Jetson Orin Nano Series, Jetson Orin NX Series and Jetson AGX Orin Series — NVIDIA Jetson Linux Developer Guide 1 documentation, docs.nvidia.com
10. Characterizing and Understanding Energy Footprint and Efficiency of Small Language Model on Edges, arxiv.org
11. Quality Control (QC) - SVT B2B, b2b.svt.se
12. Recommendation ITU-R BS.1387-2: Method for objective measurements of perceived audio quality (PEAQ), itu.int
13. Perceived Audio Quality Analysis in Digital ..., radioeng.cz
14. Can we still use PEAQ? A Performance Analysis of the ITU ..., arxiv.org
15. Objective Measures of Perceptual Audio Quality Reviewed, arxiv.org
16. Mean Opinion Score (MOS): What It Is and How the SBC Maintains Voice Quality - TelcoBridges, telcobridges.com
17. Pay Only for Disagreement: Certified No-Regression Verdicts for Model Updates with Matching Label-Complexity Bounds, arxiv.org
18. Achieving FP32 Accuracy for INT8 Inference Using Quantization Aware Training with NVIDIA TensorRT, developer.nvidia.com
19. Quantized Neural Networks for Microcontrollers: A Comprehensive Review of Methods, Platforms, and Applications, arxiv.org
20. Energy-efficient Deployment of Deep Learning Applications on Cortex-M based Microcontrollers using Deep Compression, arxiv.org
21. Real-Time Cardiac Arrhythmia Classification Using TinyML on Ultra-Low-Cost Microcontrollers (Bioengineering 2026), doi.org
22. Runtime-Certified Bounded-Error Quantized Attention, arxiv.org