Edge GPU model qualification for a Jetson inspection stack is a sequence of linked checks, with the engine build and every measurement done on the Jetson itself. You rebuild the TensorRT engine on the deployment device, prove its outputs agree with the source checkpoint within stated tolerances, and measure cycle time at each candidate precision under fixed clocks. Then you freeze the stack and re-verify it against that qualified baseline after every rebuild or patch. The rebuild comes first because, by default, a serialized TensorRT engine only works with the TensorRT version and the exact GPU model it was built on. TechnoLynx can run this qualification, and the first step is a target-side FP16 baseline build and an equivalence check against the source checkpoint.

The procedures below are written for camera inspection on production lines, which is the setting of our AI engineering for manufacturing and automotive. One idea runs through all of them: matching version numbers do not prove matching behaviour. Two engines built from the same network can give numerically different outputs, because the builder picks kernels by timing and timing is noisy.

How do we check, layer by layer, that TensorRT conversion has not changed the model’s outputs?

Run identical inputs through the exported ONNX graph in ONNX Runtime and through the TensorRT engine. Compare the final outputs with explicit absolute and relative tolerances. Where they disagree, compare per-layer outputs to find the layer where the divergence starts. The acceptance test is whether defect classifications stay the same on a representative validation set. Per-layer comparison tells you where a failure comes from. It is not the sign-off.

TechnoLynx leaves this conversion-validation pipeline as a versioned artefact that the validation package references and that is re-run for every later model update.

The conversion has two hops, and each needs its own check. The first is PyTorch to ONNX. Validate the ONNX outputs against the source checkpoint with the same letterboxing, RGB channel order, scaling, tensor layout and output decoder. Do this before TensorRT enters the picture, so that a preprocessing mismatch is never blamed on the engine.

The second hop is ONNX to TensorRT, and Polygraphy is the tool for it. polygraphy run model.onnx --trt --onnxrt runs both backends on the same inputs and compares their outputs. Adding --fp16 or --int8 builds the engine at reduced precision for the same comparison. Polygraphy’s default tolerances are usually right for FP32 but may not be for reduced precision, so set them deliberately with --atol and --rtol and record the values in the protocol. When outputs do not match, --trt-outputs and --onnx-outputs expose per-layer results so you can see where the error enters. NVIDIA’s accuracy guidance also uses Polygraphy’s --validate option to catch NaNs and Infs, and compares layer outputs against golden values from ONNX Runtime.

Two cautions shape how you read the results. First, do not expect bitwise identity. Small differences should not shift a metric such as mAP, and a significant drop may point to a genuine TensorRT bug. Second, marking intermediate tensors as outputs can block optimisations and change the results. The engine you inspect layer by layer is therefore not quite the engine you ship, so run the end-to-end agreement test on the unmodified engine.

When the divergence is in a specific place, the fixes are specific too. You can then force a layer’s input and output types with setPrecision and setOutputType, or use NVIDIA’s experimental debug precision tool to find the layers that need higher precision. Layers prone to FP16 overflow, such as Reduce and Element-Wise Power, are candidates to pin to FP32. Any of these changes produces a new engine, so the whole pipeline runs again before sign-off. Choosing the tolerance band itself, and defending it to a reviewer, is covered in depth in our guide to proving kernel equivalence after porting or precision changes.

FP16 or INT8: which precision leaves defect classification unchanged?

Move to INT8 only if FP16 misses the latency or power budget, and accept INT8 only after it passes the same classification-agreement test on representative defect data. The published measurements below support that order. They do not support treating any precision as safe in general.

In a 2026 study on a Jetson Orin Nano, an FP16 EfficientNet-B0 engine matched FP32 with 99.95% argmax agreement. Quantisation-aware INT8 reached 0.8895 accuracy, still short of FP16, and ran slower than FP16 at 5.969 ms, which the authors attribute to insufficient Q/DQ fusion in TensorRT 8.5.2. Their recommendation is to confirm first whether FP16 fits the target latency and, if it does, to make FP16 the first choice.

In the 2026 YOLO26s study on a Jetson Orin Nano Super, INT8 reduced mAP50-95 by 0.0213 to 0.0393 against FP32, while its inference latency of 6.073 to 6.254 ms was almost the same as FP16’s 6.157 to 6.181 ms. A 2025 study of YOLO12 models found that static INT8 TensorRT engines gave roughly 1.5 to 3.3 times speedup over FP32 for an absolute mAP50-95 drop of about 3 to 7%, with smaller models hit harder. The same study found static INT8 consistently more sensitive to image noise.

Sensitivity is not uniform across a network either. Per-tensor quantisation is known to lose accuracy in layers with wide activation ranges, and the 2026 Orin Nano study saw a large drop when it applied post-training quantisation to an EfficientNet-B0 containing Squeeze-Excite blocks and SiLU activations. A Jetson obstacle-detection guide warns that quantisation can reduce accuracy on small or unfamiliar obstacles. That is why we would set the INT8 acceptance threshold per class rather than globally.

Post-training calibration measures activation distributions on representative input data. The 2026 YOLO26s study drew calibration images only from the training partition and never from validation or test images. Calibration data should also cover production lighting, object sizes and clutter, with preprocessing that matches production exactly.

The sources differ on where calibration should run. NVIDIA’s TensorRT 8.6 documentation says the calibration cache is portable across devices when calibration happens before layer fusion, although the cache data is not guaranteed to be bitwise identical across devices. Ultralytics tells users to export INT8 on the same device that will run the engine, because calibration results vary across devices. For a qualified build we would calibrate on the target, because the artefact you qualify should be the artefact you tested. Calibration tables are also not portable across TensorRT versions, so a TensorRT change means recalibrating. Taking an existing engine from calibration through to a validated INT8 build is covered in requalifying the edge runtime from trained model to validated INT8 engine.

Edge GPU model qualification: how do we prove a Jetson build meets the cycle time before sign-off?

Build the engine on the deployment Jetson from the frozen ONNX export. Choose precision and batch by sweeping latency against accuracy at the batch the line actually uses. Measure cycle time under the power mode and clocks the line will run, and gate sign-off on a written checklist whose pass criteria were fixed before testing began.

TechnoLynx’s AI Porting & Deployment Pack turns your cycle-time budget and accuracy requirement into qualification criteria, then rebuilds the engine on your Jetson. LynxBenchAI’s published results show how candidate devices run the public PyTorch stack; they do not measure your workload.

A model that performs on the training GPU is a different artefact from the engine that performs on the Jetson. Closing that gap is engineering work, not a hand-off. In the 2026 YOLO26s study, engines were built directly on the hardware they were evaluated on and never moved between the reference GPU and the Jetson. TensorRT’s hardware compatibility mode, which relaxes the device match on other platforms, is not supported on JetPack at all.

Worked example: a published FP16 build on an Orin Nano

The 2026 Orin Nano study exported its EfficientNet-B0 classifier to ONNX at opset 17 with a fixed batch of 1 and no dynamic axes, then built the FP16 engine with trtexec --fp16 on the device itself. The device ran JetPack 5.1.2 and TensorRT 8.5.2 in the 15 W MAXN power mode. Latency was measured with no other processes running, using --warmUp=1000 --iterations=1000 --avgRuns=100 --useSpinWait.

The results came out as follows:

• mean latency of 3.192 ms per image including host-to-device and device-to-host transfer
• 3.139 ms of GPU compute alone
• p99 latency of 3.204 ms including transfer
• about 9.4 times headroom against the study’s 30 ms target
• 99.95% argmax agreement, checked by comparing the real engine’s predictions with FP32 predictions from ONNX Runtime

The example’s stated limits matter just as much. The authors did not measure the latency, power and thermals of their full ensemble running continuously with a camera-fed detection and tracking pipeline, and they list that as the next stage of work. Engine latency feeds into a cycle-time proof. On its own it does not prove the line’s cycle time.

Latency and throughput answer different cycle-time questions

A pipelined vision system can sustain a frame rate whose period is much shorter than the time any single frame spends in the system. In the 2026 YOLO26s study, the DeepStream pipeline processed more than 30 frames per second, yet each frame took 121.0 to 156.1 ms to travel from decoder to tracker, because decoding, inference and tracking run concurrently on different frames. The authors measured maximum-throughput and timestamp-synchronised modes separately, so that computational capacity could be told apart from the ability to keep up with the configured video rate.

For an inspection line, write down which budget you are proving: a per-part deadline (the reject actuator must fire before the part passes) or a sustained rate. Each needs a different measurement. Measure capture-to-detection latency as well as kernel time, and report median and tail percentiles.

Timing with trtexec under locked clocks

Defensible numbers come from fixed conditions. The 2026 YOLO26s study set the power mode with nvpmodel, locked CPU, GPU and memory-controller clocks with jetson_clocks, enabled active cooling, and kept that configuration for every run. NVIDIA notes that locked clocks give more deterministic tactic selection and more stable measurements, while average performance is lower than with floating clocks. In trtexec, --useSpinWait gives more stable latency at the cost of CPU load, and --noDataTransfers isolates GPU execution from host-device copies. For a cycle-time claim, the transfers belong inside the number, so use the isolated figure for diagnosis only.

Nominally identical boards are not identical. A 2021 study of 13 identical Jetson AGX Xavier boards running the same workloads under the same settings found intrinsic variability in both runtime and power consumption, large enough to matter at application level. NVIDIA’s TensorRT Edge-LLM tutorial notes that local Jetson results vary with Jetson model, power mode, memory pressure, thermal state and CUDA or TensorRT version. We would qualify timing on more than one production unit, set the pass band with margin for unit-to-unit spread, and treat thermal behaviour in an enclosed line cabinet as a separate characterisation. Field validation of detection rates under real lighting and fixture wear is also its own protocol, run after bench qualification passes.

The qualification sequence, in order

1. Freeze one validation-selected checkpoint and use it for every precision mode, so differences come from the backend and not from retraining.
2. Export to ONNX with a fixed batch and no dynamic axes, then validate the ONNX outputs against the checkpoint with identical preprocessing.
3. Build an FP16 engine on the deployment Jetson and measure its accuracy before introducing INT8.
4. Compare TensorRT against ONNX Runtime with stated tolerances, and go to per-layer comparison only on disagreement.
5. Re-evaluate accuracy on the unchanged test set after engine construction instead of assuming equivalence.
6. Measure latency under a fixed power mode and locked clocks, with warm-up runs and a sustained timed run.
7. Try INT8 if FP16 misses the latency or power budget.
8. Record the model, preprocessing, calibration data, TensorRT version, CUDA version and target hardware in an engine manifest.
9. Execute the protocol against acceptance criteria written before testing began, and sign it.

Once the sequence is stable, an MLOps CI/CD pipeline for model deployment can automate it. Each validated checkpoint becomes a registered artefact that triggers export, an engine build on a Jetson runner (engines must be built on the device that runs them), the equivalence and latency checks, and a staged push to a canary group. If canary accuracy or confidence metrics regress, the pipeline rolls back to the last good model. TechnoLynx builds that export, validate and stage pipeline on top of an existing training workstation.

Who can qualify our Jetson inspection model before line sign-off?

TechnoLynx qualifies your Jetson inspection model before line sign-off: it rebuilds the engine on target, runs on-device cycle-time and accuracy tests, and leaves a qualification checklist for sign-off. On building in-house versus bringing in a specialist: a team already running Polygraphy comparisons and on-device timing has most of the mechanics, and outside help adds most in the acceptance criteria and the re-verification tooling. When choosing a partner, ask how they define equivalence for your defect classes and what evidence they hand over, not only what latency they hit. The engagement is the AI Porting & Deployment Pack, and it starts with a target-side FP16 baseline build and equivalence check.

Some teams do not need it. Teams with time and Jetson experience can run NVIDIA’s own tools in-house, and carrier-board vendors publish their own flashing, package-hold and upgrade guidance for their images.

Did the patch move L4T, TensorRT or the engine off the qualified versions?

Take a fresh snapshot of the device and diff it against the manifest recorded at qualification. If anything has drifted, restore the pinned versions or re-flash the qualified image. Then confirm behaviour with an output-equivalence run on reference inputs before declaring the station restored.

Most of the snapshot comes from what the platform already exposes. cat /etc/nv_tegra_release reports the L4T release and revision. The same file lists checksums for NVIDIA’s driver files, and sha1sum -c /etc/nv_tegra_release verifies them. dpkg-query --show nvidia-l4t-core reports the installed core package version from dpkg’s own database. Record the L4T r-number rather than only the JetPack label, because the L4T number is what driver and BSP compatibility depends on.

Record the TensorRT shared library versions too, not just the Python package. An engine plan stores the major, minor, patch and build versions of the library that created it, and fails to deserialise if the runtime does not match. In a December 2025 NVIDIA forum case, builder and runner containers on Orin NX both reported TensorRT 10.3.0 in Python. Inference still failed with a serialization version-tag mismatch. Installing libnvinfer 10.7.0 fixed it. Add hashes of the engine file, calibration cache, timing cache and model artefacts, and keep the manifest under version control. We would build this as a manifest script assembled from those checks, not wait for a single product that covers the whole driver, runtime and engine stack.

Restoring depends on what moved. L4T point releases arrive through apt upgrade and can bring a new kernel. A JetPack major-version change has no in-place path back: the kernel, bootloader and root filesystem all change, and the device has to be reflashed. Know the limits of the diff, too. It only catches drift in tracked artefacts, and re-pinning restores versions without re-verifying behaviour. If the baseline manifest was incomplete in the first place, a clean diff proves less than it appears to.

How do we pin L4T, driver, firmware and TensorRT so the qualified state stays frozen?

Hold the kernel and firmware packages, cut off the update source so nothing new can arrive unannounced, keep the qualified image as the restore point, and log every deliberate change through change control.

On the Jetson itself, the mechanisms are apt tooling. ARK Electronics’ Jetson documentation holds the kernel and firmware packages with sudo apt-mark hold linux-firmware nvidia-l4t-kernel nvidia-l4t-kernel-dtbs nvidia-l4t-firmware nvidia-l4t-kernel-headers nvidia-l4t-kernel-oot-headers wireless-regdb, and lists the holds with apt-mark showhold. On the NVIDIA developer forum in March 2025, the advice for disabling OTA upgrades was to comment out /etc/apt/sources.list.d/nvidia-l4t-apt-source.list, so that apt update and apt upgrade stop pulling them. NVIDIA’s own packaging adds a safeguard: nvidia-l4t-core blocks a partial upgrade in which one Jetson Linux package moves to a new major release while its dependants stay behind. Point releases still arrive through apt upgrade, so the holds are still needed.

Agree the lock with your hardware vendor before qualification.

What lock evidence will an auditor accept?

General computer-system-validation guidance expects protocols that define what will be tested and what counts as a pass before testing starts. It expects executed OQ and PQ reports, deviation records showing failures were investigated and re-tested, and a signed validation summary. Every post-validation change needs a change-control record assessed for re-validation impact before it is applied. For a Jetson station, we would attach the apt-mark showhold output, the disabled apt source, the checksum-verified manifest and the engine hash as objective evidence that the qualified state is frozen. Agree the evidence set with your quality function before qualification, not during the audit.

How often should a qualified edge inspection model be re-verified?

Re-verify on a defined cadence written into the validation plan, and also on every change event, because a validated state has to be re-confirmed on schedule rather than assumed. We would re-verify before each PQ cycle and immediately after any patch, flash or engine rebuild. Escalate to full requalification only when the check finds drift. The right cadence still depends on your regime and on how often the station is touched. A watchdog that checks the manifest between cycles can catch unplanned drift earlier, but it needs validating in its own right so it does not give false confidence.

Each re-verification needs three pieces of evidence: a clean manifest diff, output agreement on a frozen reference set within the qualified tolerance, and latency inside the qualified band. The 2026 Orin Nano study repeated its evaluation over five seeds and reported 0.9348 ± 0.0042 macro recall to show the result was not a single-seed artefact. Re-verification should also run the frozen configuration on held-out data without retuning, and report what it finds. In the 2026 YOLO26s study, no tracker or counting parameter was adapted to the test sequences, and the frozen pipeline reported an event recall of 0.629. The same discipline applies to upgrades: the Orin Nano authors kept their verified teacher model rather than replace it on the sole basis of a newer generation.

After a patch, how do we prove the stack still matches the qualified baseline, beyond version strings?

Fingerprint the whole stack (versions plus file hashes for the libraries, engine and calibration artefacts), and pair that fingerprint with an output-equivalence run on frozen reference inputs compared against the qualified outputs. Version strings are not evidence of equivalence; outputs on reference inputs are.

TechnoLynx implements fingerprinting across driver, L4T, TensorRT runtime and compiled engine, and wires the equivalence test into a CI check that runs on every firmware or engine rebuild. The AI Readiness Scorecard is the catalogue route for this assurance work.

Does re-flashing a Jetson or rebuilding the TensorRT engine change inspection outputs, even when the version numbers match?

Rebuilding the engine can change outputs with every version string unchanged. Running the same engine binary in the same runtime environment gives the same outputs. TensorRT’s builder times candidate kernels, timing noise can lead successive builds to choose different implementations, and different implementations order floating-point operations differently. NVIDIA’s guidance is to validate accuracy after each rebuild. NVIDIA also states that different builder invocations typically do not produce engines with bit-identical results. A study on Jetson Xavier NX and AGX measured this directly. Engines built repeatedly from the same model disagreed on 0.1 to 0.8% of predictions, and run times differed between engines built on the same board because the mapping to CUDA kernels changed from build to build.

Once built, an engine is deterministic. Given the same input in the same runtime environment, a built engine produces the same output, apart from documented exceptions such as IFillLayer and IScatterLayer. NVIDIA expects bit-accurate outputs across runs of an engine on the GPU it was built for. The practical rule follows: keep the qualified engine binary in a registry and redeploy that file after a re-flash, rather than rebuilding. The IISWC authors suggest the same approach across units, building one engine and deploying that exact binary everywhere so outputs and latencies stay consistent; NVIDIA’s guidance limits that reuse to units of the same device model. A move to JetPack 7.2 removes the choice, because serialized TensorRT engines have to be rebuilt on it.

When a rebuild is unavoidable, make it as repeatable as possible. Locking GPU clocks before the build can improve the determinism of tactic selection, and more timing iterations per tactic help at the cost of longer builds. Reusing the same timing cache with the same network and builder configuration makes TensorRT select an identical set of tactics, provided the hardware and software configuration matches. An algorithm selector can replay recorded choices from an earlier build. In TensorRT 10, the editable timing cache lets you assign tactics per layer. Even with all of these levers, the rebuilt engine goes through the output-equivalence run before it returns to the line.

What a change invalidates, and what to re-run

Change	Why it matters	Re-run
New model weights or preprocessing	Changes what the engine computes; the manifest must track model and preprocessing	Full conversion check, accuracy, timing
TensorRT version	Engines and calibration tables are not portable across versions by default	Rebuild, recalibrate INT8, full equivalence and timing
JetPack major upgrade	Requires a reflash, with kernel, bootloader and root filesystem all changed	Rebuild engines on the new release, full requalification
L4T point release through apt	Can bring a new kernel, and out-of-tree modules must be rebuilt for it	Manifest diff, driver rebuilds, equivalence and timing
Engine rebuild, same versions	Tactic selection can differ, changing outputs	Equivalence on reference set, timing
Re-flash to qualified image, same engine binary restored	A built engine is deterministic in the same runtime environment	Manifest diff, short equivalence run
Same binary on another unit of the same model	Identical boards vary in runtime and power	Timing check on that unit

Separating timing and trigger faults from model drift

On a line, a sudden change in detections can come from the camera trigger, timestamping or synchronisation rather than the model. That kind of timing audit belongs in its own standing control, with a written procedure checked whenever cameras, triggers or firmware change. During diagnosis, replaying frozen reference images through the qualified engine shows whether the engine itself has changed. If the replay still matches the baseline, the engine has not changed and the cause lies in what reaches it. Genuine data drift shows up differently, for example as a shift in the model’s confidence score distribution against a reference captured at validation time.

Who should own the ongoing check versus a one-off audit?

The one-off qualification audit sits with whoever owns validation. The recurring re-verification needs a named owner in operations or controls engineering, plus an automated job, because unowned recurring checks lapse quietly.

Can we reconstruct model lineage from Jira, email and a partial experiment tracker?

Usually, yes, as long as the result is presented as a reconstruction with stated confidence for each segment, not as restored certainty. Model lineage reconstruction correlates timestamps and IDs across Jira, email and tracker exports to infer which data, configuration and code produced the deployed model. Every segment carries a confidence tier and a gap flag.

Start with the segments an auditor is most likely to question, and include the low-confidence ones in the package rather than leaving them out. A disclosed gap is defensible, and an inferred-away gap is not. How fast this goes depends mostly on how quickly the exports arrive, and the confidence levels can only be as good as the secondary sources behind them. Going forward, a model registry that stores the training dataset version, training configuration, evaluation metrics and compatible runtime versions with every deployed artefact gives a rollback its full provenance chain.

FAQ

Is ONNX faster than TensorRT?

ONNX is a model format, and the usual comparison is between ONNX Runtime and TensorRT. A 2026 benchmark of industrial machine-vision inference at FP32 found TensorRT gave the lowest inference time on every NVIDIA GPU tested, including Jetson AGX Orin, while ONNX Runtime was fastest on the Jetson’s CPU.