After a hardware, toolchain or model change, INT8 quantization validation for a parenteral inspection model means requalifying the engine on the exact target device, per defect class, against a frozen FP32 baseline, and recording the result under change control. A quantised INT8 model is not a portable artefact. One portability measurement study ran the same ResNet-50 classifier artefact with the same quantisation scales on different targets. Predictions matched on 1000 of 1000 inputs only when two targets shared an integer kernel, and on 958 to 965 of 1000 when they did not, while top-1 accuracy stayed unchanged.

TechnoLynx does this requalification work for regulated inspection lines: it rebuilds and recalibrates the engine on the target hardware, compares it class by class with the FP32 reference, and delivers the test evidence a quality team needs for its change-control decision, as part of its AI engineering for life sciences.

An unchanged accuracy figure does not clear a new INT8 engine

The roughly 4% of inputs that change answer between targets are invisible to a standard accuracy report, because the flips cancel out: some wrong answers become right and an equal number of right answers become wrong. The same study found FP32 predictions bit-identical across every platform pair it measured, so the divergence follows the integer kernel, not the hardware boundary. What breaks agreement is a different re-quantisation epilogue in the integer kernel.

For an accept/reject classifier on a fill-finish line, a 4% flip rate is a different machine, even if the headline accuracy is identical. Work on compressed language models found the same pattern: aggregate accuracy within 2% of the baseline while 5% or more of individual answers changed. The domain differs, but the lesson transfers: a quantised model is meant to be a drop-in replacement, so it should be judged by its distance from the baseline, not by its own score.

Averages also hide per-class damage. A 2023 benchmark of post-training quantisation found that average accuracy held up as the calibration set changed, while the accuracy of individual categories moved with noise and intra-class bias in that set. Hooker and colleagues showed that compression hits the under-represented long tail of the data distribution hardest, including atypical and noisy images. Rare defect classes (a hairline crack, a particle at the vial shoulder) are exactly that long tail.

The requalification sequence, in order

The steps below run in this order, because each one fixes a variable the next one depends on.

1. Freeze the reference. Lock the FP32 model, the preprocessing and a full validation set. In the portability study, a 1000-image subset overstated top-1 by 9.77 percentage points on average and flipped the sign of three quantisation deltas. Changing the resize and crop moved top-1 by 1.07 points, about nine times the 0.12-point cost of quantisation itself.
2. Prove the graph runs where you think it runs. Compile the exact model on the target and read the operator placement and fallback report (next section).
3. Calibrate on the target with representative data. Ultralytics’ TensorRT documentation says to export INT8 on the same device that will run the engine, because calibration results vary across devices.
4. Compare against FP32 per class and per output. Report per-class sensitivity, specificity and confusion matrices, plus agreement or flip counts against the FP32 reference.
5. Escalate only on failure. Partial or mixed precision first, then quantisation-aware training (QAT), if a class misses its criterion.
6. Re-measure latency on the target and freeze the artefact. Record the engine, toolchain and firmware versions, and file the evidence under change control.

Does the target runtime run every operator in our inspection model, or will we rewrite parts of it?

You can only know by compiling and running that exact model, custom layers included, through the target SDK and reading the operator-support and fallback report it produces; a datasheet’s operator list or a generic benchmark cannot answer it.

TechnoLynx starts this step by cataloguing every layer and custom op in the model’s graph, compiling the full graph on each candidate board’s runtime, and turning the support report into a fix plan: a custom kernel or plugin for the target, or a rewrite of that subgraph in supported primitives.

The reports look different on each platform. On a Jetson DLA, layers TensorRT cannot place on the DLA fall back to the GPU, and NVIDIA’s tutorial notes that a fully supported model appears as a single “ForeignNode” block. The Coral Edge TPU compiler partitions the graph at the first unsupported operation and runs everything after it on the CPU. Google warns that even a small CPU fraction can slow inference by an order of magnitude. TensorRT rejects pre-quantised ONNX models that use QLinearConv, QLinearMatMul, ConvInteger or MatMulInteger, with an import error.

Some fallbacks are silent. The portability study describes a TensorRT execution provider listed as available that ran on the CPU because a library was off the loader path (11.83 ms per inference against 0.41 ms once fixed), and an opset down-conversion that exited cleanly while producing an invalid graph. A vendor compiler can also auto-partition a model, as one did with a DETR detector, leaving most of it on the host CPU in FP32.

The decision follows from the report. If the full graph compiles natively, the board is viable as it is. If ops fall back or fail, the next step is to cost the custom kernels or rewrites and compare that work and its latency impact across the candidate boards before choosing; if no board runs the graph without major rewrites, the model architecture or the shortlist needs another look. A proof-of-concept run on an evaluation kit for each candidate is the cheapest way to settle it before the hardware is locked. LynxBenchAI’s published results can narrow a hardware shortlist on standard workloads, but they cannot tell you whether your graph compiles on a given runtime.

Three caveats belong in the pass criterion. The result is tied to the SDK and firmware version tested. A CPU fallback can let the model “run” at a latency the line cannot accept, so pass or fail needs a latency threshold, not just functional completion. Custom-op support proven for one graph says nothing about the next model version.

A repeatable PyTorch-to-INT8 pipeline

A reliable route from a trained PyTorch checkpoint to a validated INT8 engine is a fixed sequence that can be rerun: export, calibration set curation, post-training quantisation (PTQ) or QAT, engine compilation on the target, then per-layer and per-class comparison with FP32. The toolchain is chosen by the accelerator, not by preference.

TechnoLynx builds that pipeline through its GPU Performance Engineering work: it sets the accuracy budget and the target accelerators with the client, chooses PTQ or QAT and the matching toolchain, then tunes the calibration set and validates on the target before rollout.

A pipeline that does not need hand-debugging per layer

NVIDIA’s 2020 integer quantisation paper sets out a workflow that kept every network it studied within 1% of its floating-point baseline once partial quantisation or QAT was added where needed: quantise the compute-heavy layers, try max, entropy and 99.99% or 99.999% percentile calibration, and escalate only if none reaches the target. It recommends per-channel weights and per-tensor activations, the configuration TensorRT’s documentation also names as giving the best accuracy. In PyTorch, Torch-TensorRT builds an INT8 calibrator from a standard DataLoader over target-domain data and can write a calibration cache for reuse. TensorRT Model Optimizer can calibrate INT8, run PTQ or QAT and export to ONNX.

Two NVIDIA documents need reading together. TensorRT’s documentation says a calibration cache is portable across devices when calibration runs before layer fusion, while a built engine is tuned for its build GPU and runtime and is not portable. Ultralytics advises calibrating on the deployment device. Both can be true. Even where a cache ports, the engine is rebuilt per target, and the portability study recommends re-validating INT8 per target rather than once.

How large and how varied the calibration set should be

TensorRT’s documentation reports that about 500 images were enough for ImageNet classification networks, while noting that the amount is application-dependent. Google’s LiteRT guidance suggests a subset of roughly 100 to 500 training or validation samples. More is not always better: for the PointPillars LiDAR detector, four calibration frames beat 4,096 by 23.82% mAP.

Composition matters for individual classes, not just the average. In the 2023 PTQ reliability benchmark, growing the calibration set from 1 to 32 samples sharply reduced the variance of accuracy across runs. The same benchmark found that individual classes reacted to imbalance and noise in the calibration set even when the average did not. For an inspection model, TechnoLynx tunes the composition of the calibration set within the pipeline and runs per-class accuracy regression on the target hardware. Calibrating on degraded images is no shortcut: in a YOLO TensorRT study, a 50/50 clean and degraded calibration mix gave no consistent robustness gain except on the largest model under noise.

Checking the INT8 engine against the FP32 model before release

Validation of a quantised model should compare the INT8 engine with the FP32 model on the same full validation set and preprocessing, judging its distance from the baseline as well as its accuracy. ONNX Runtime calls the tensor-level version of this quantisation debugging: compare FP32 and quantised weights and activations, find where they differ most, and leave those tensors unquantised or change the calibration method. LiteRT recommends converting a float model first, as an operator-compatibility check and a baseline for debugging quantisation error. INT8 can shift confidence values, so Ultralytics advises choosing the operating threshold from the INT8 model’s own F1 curve; for a reject decision, that threshold is part of what you validate.

Release-gate validation like this is a reliability discipline separate from model accuracy: its products are regression suites and validation packages that catch silent regressions before they reach a production line.

Which quantisation toolchain fits a given embedded accelerator?

Start from the accelerator vendor’s own path. The portability study found that vendor NPUs own quantisation: Qualcomm’s Hexagon HTP ignored external QDQ scales and dropped top-1 from 0.75 to 0.005 while compiling and running without error, and the DEEPX compiler refused the same kind of graph outright.

Target accelerator	Toolchain to start from	What to check first
NVIDIA Jetson GPU and DLA	TensorRT, via ONNX or Torch-TensorRT	DLA supports only INT8 and FP16; incompatible layers fall back to the GPU
Google Coral Edge TPU	LiteRT/TFLite full-integer model, then the Edge TPU Compiler	Requires fully 8-bit quantised models; compilation stops at the first unsupported op
General ARM or x86 CPU	ONNX Runtime or LiteRT/TFLite	INT8 speedup ranged from 2.1× faster to 1.7× slower depending on the CPU’s dot-product instructions
Qualcomm Hexagon NPU	Qualcomm’s native quantisation flow (AI Hub), fed FP32	External QDQ scales silently ignored
Other vendor NPUs	The vendor compiler’s own PTQ from FP32	Rejected graphs or auto-partitioning to host CPU

In ONNX Runtime (the CPU row), S8S8 in QDQ format is the default and first choice, its GPU path supports only S8S8, and static quantisation is recommended for CNNs.

Timeline and cost depend on three things the first two steps reveal: how much of the graph falls back, whether PTQ passes per class, and how many hardware revisions and firmware variants the rollout spans. A fleet of inspection stations is validated across every hardware revision in service, not on one reference unit, and PTQ results do not carry automatically from one model version or site to the next.

Why do rare defect classes drop more than overall accuracy after quantisation?

Work on compressed language models attributes flips to quantisation noise perturbing output probabilities, which makes small-margin answers the likeliest to change. Compression also hurts the under-represented long tail, including atypical and noisy images, disproportionately. Lower bit-width also cuts dynamic range, and the 2023 PTQ benchmark observed some classes losing accuracy as a result while others did not.

TechnoLynx’s Inference Cost-Cut Pack covers this rescue: it measures the rare-class drop per layer and per class, chooses between rebalancing, mixed precision and QAT from that sensitivity analysis, and re-measures rare-class accuracy, overall accuracy, latency and memory after each change.

Is INT8 post-training quantisation enough for a parenteral inspection model, or does it need QAT?

For many CNNs, 8-bit PTQ is enough: Qualcomm’s 2021 white paper (Nagel et al.) states that in most cases PTQ reaches 8-bit with close to floating-point accuracy. The exceptions are real. NVIDIA’s paper cites work showing MobileNets do not reach baseline with INT8 PTQ even per-channel and need QAT. A YOLO study measured a 3 to 7% absolute mAP50-95 loss for static INT8 in TensorRT, larger on smaller models. Static INT8 was also more sensitive to noise than FP32 or FP16 in that study.

The decision is per class, not per model. If every defect class meets its acceptance criterion under PTQ with a well-composed calibration set, stop there; if a rare class misses, first try rebalancing the data before quantisation and keeping sensitive layers in higher precision, and invest in QAT only if that still falls short. QAT inserts quantisation operations before fine-tuning so the network adapts to quantised weights and activations. NVIDIA suggests fine-tuning for around 10% of the original schedule, starting from the best calibrated model. The cost is data: QAT needs access to the training data. A deeper treatment of when compression needs QAT rather than a cheap PTQ pass covers the training side.

Can some layers stay at higher precision and keep rare-class accuracy?

Yes: NVIDIA’s workflow places partial quantisation between PTQ and QAT. NVIDIA’s paper observes that a few quantised layers often cause most of the accuracy loss, and that leaving them in floating point trades some speed for accuracy. In the PointPillars work, moving layers to FP16 brought PTQ to 63.84% mAP against 64.47% with QAT, which sat 0.17 points below FP32. That study found sensitive layers by quantising one layer at a time and measuring end-to-end mAP. End-to-end measurement matters here, because the portability study found per-layer SQNR did not predict the top-1 change (Spearman ρ of −0.030 over 21 layers).

On TensorRT, layer precision is set with ILayer::setPrecision and setOutputType. Those constraints are hints by default: if another implementation is faster, TensorRT uses it and issues a warning. Keeping layers in float does not work on the Edge TPU, which supports only fully 8-bit quantised models. A rare-class fix can also cost overall accuracy, latency or memory, which is why each intervention is re-measured on all of them.

Who can requalify our quantised inspection model on new edge hardware?

TechnoLynx requalifies quantised vision models when the edge hardware, toolchain or model changes: it confirms the graph runs natively on the target, rebuilds and calibrates the INT8 engine there, validates it per defect class against the client’s FP32 reference, and fixes any class that fails with mixed precision or QAT. Whether to do this in-house depends on whether your team already owns the target vendor’s toolchain and has a per-class regression harness; if both exist, the work is mostly execution. When choosing a partner, ask for acceptance criteria agreed before work starts (per-class metrics, latency, agreement with FP32) so that handover can be checked rather than trusted. The INT8 pipeline engineering runs through TechnoLynx’s GPU Performance Engineering.

The engagement delivers the engineering evidence, not GMP sign-off, regulatory authority or your quality unit’s review; your QA team owns the validation decision. The hardware vendor’s field application engineers are a realistic route for toolchain questions on their own silicon, and a computerised-system validation consultancy is the right owner for the protocol and documentation itself.

What the requalification file has to show under Annex 22

The draft EU GMP Annex 22, published for stakeholder consultation, covers AI models in critical manufacturing applications that affect patient safety, product quality or data integrity, such as classifying product. It applies to static models that give identical outputs for identical inputs. Any change to the model, the system it runs in or the process, including physical objects the model uses as input, is to be documented and evaluated to decide whether the model needs retesting. A new accelerator, TensorRT release or camera is such a change. The consultation also foresees ongoing oversight through change control, performance monitoring and human review.

The test evidence follows from that text. Suitable metrics for an accept/reject model include a confusion matrix, sensitivity, specificity, accuracy, precision and F1. Acceptance criteria should be at least as high as the performance of the process the model replaces. Test data should be stratified and cover all subgroups and rare variations within the intended use. That last requirement is the regulatory form of per-class validation, and a rare-defect class cannot be waved through on an average.

The qualification frame many sites already use maps onto this: IQ for correct installation against specification, OQ for function under defined conditions, PQ for performance in routine use, with software updates listed among requalification triggers. Field checks that the edge model’s detection rate matches the lab baseline under real production conditions, and a standing change-triggered procedure, sit alongside this file rather than inside it.

One point catches teams out on determinism. TensorRT calibration is deterministic for the same inputs, order, batch size and device, but separate builder runs typically do not produce bit-identical engines, because tactic selection can depend on timing noise. An editable timing cache records and reuses tactic choices for reproducible builds. The practical rule: version the built engine file, toolchain and firmware together, and treat a rebuild on the “same” hardware as a change that needs its own comparison with the FP32 reference. The engine you validated has to be the engine the line runs.