We would not choose the edge module for a field robot, a drone or a regulated imaging workflow from a datasheet or a lab benchmark alone. Boards sold as the same model differ measurably in runtime and power, and synthetic stress tests misjudge the thermal and power demands of a real inference workload. A benchmark result becomes decision evidence only once it is mapped to the latency or turnaround limit the deployment has to meet, and recorded so a reviewer can follow it.
TechnoLynx measures candidate edge modules for robots, drones and autonomous machines. It starts by fixing the requirement the module must hold, then measures the candidate modules under deployment conditions and writes the results up as a validation record. The work follows a set order. Measure variability across several units, test thermal and decode behaviour where the hardware will actually run, run a benchmark built on standard scenarios, and record all of it in a form an inspector can read.
Why aren’t vendor datasheets and lab benchmarks enough to choose an edge module?
A study of nominally identical NVIDIA Jetson AGX boards measured this directly. Its authors ran several ML inference workloads on 13 NVIDIA Jetson AGX boards of the same model, with identical hardware, software, inputs and frequency settings. Pairwise tests rejected the hypothesis that the boards behave alike, at the 1% significance level, for about 77% of sampled configurations on runtime and about 99% on power. The differences were large as well as detectable: the authors rated them large for 49% of board pairs and configurations on runtime and 90% on power, and still 27% and 82% under a more conservative threshold.
Runtime differences mostly stayed within ±5%. For ResNet, MobileNetV2 and ShuffleNetV2, though, runtime ranged from −10% to +20%, and power differences for GoogleNet, MNASNet and ShuffleNetV2 reached −30% to +20%. The authors’ conclusion is the one that matters for procurement. A performance or power model built on one board predicts poorly across nominally identical boards, so samples have to come from several boards.
These boards were also kept cool on purpose. The fan ran at maximum and temperatures stayed below 50 °C throughout. That variability exists before the field adds heat, dust or a sealed enclosure, which is why we would never sign off a fleet on the strength of one lab unit.
Thermal behaviour can mislead in the other direction too. A 2026 study in Electronics (MDPI) placed a Jetson AGX Orin in an outdoor enclosure with power supplies, an LTE module and a thermostat-controlled fan, and tested it across ambient temperatures from −20 °C to +40 °C. At +40 °C the synthetic stress test triggered throttling at GPU and CPU temperatures of 95.6 °C and 99.0 °C. Under the same conditions a YOLOv8s workload sustained 108.8 FPS at 19.1 W, roughly half the 36.5 W of the stress test, and stayed well below the throttling threshold. A stress test that throttles does not rule a module out. The workload’s own duty cycle is the test.
NVIDIA’s own documentation makes a related point about power modes. On Orin, MAXN is not the maximum-performance mode, because hardware throttling engages once total module power exceeds the TDP budget. On AGX Orin an over-current limit on the summation channel triggers hardware clock throttling of the GPU and CPU. Orin NX and Orin Nano modules enforce average and instantaneous power limits on VDD_IN in the same way. NVIDIA tells users to test their own use case to settle core counts and CPU, GPU and memory frequencies.
How do you design a field pilot to benchmark edge modules under real dust, vibration and thermal conditions?
Design a field pilot around the acceptance criterion first and the hardware second. The order below matters, because each step depends on the one before it.
- Write the acceptance criterion. State the latency percentile, throughput, power budget and duty cycle the module must hold, in the terms the robot, the drone or the clinical workflow actually imposes.
- Get several units of each candidate. The Jetson variability study found that runtime and power models need samples from several boards, not one.
- Freeze and record the software stack. For ‘Available’ submissions, MLPerf’s rules require, for reproducibility, the software version of every component, the hardware configuration, the software stacks, the Docker containers and all settings.
- Choose the power mode on purpose. Test it with the real workload rather than assuming MAXN is fastest.
- Instrument the board. Orin thermal zones report temperature in millidegrees Celsius. The integrated INA3221 power monitor can be read through sysfs, and sysfs nodes count under-voltage, average over-current and instantaneous over-current events. The Jetson Power GUI can log power data to a file over a set duration.
- Discard warm-up and run long enough. The variability study dropped the first five of 105 timed iterations. MLPerf requires every run to last at least 60 seconds so that power management and DVFS reach equilibrium. On top of both, run the real duty cycle across the ambient range the site will see.
- Run in the enclosure, at the site. Log throttling events, over-current counts, failures and resets alongside the inference metrics, so each anomaly can be tied to a condition.
- Compare units, then compare against the criterion. Report the spread across units as well as the mean, and judge the worst unit against the requirement.
Dust and vibration need a plain caveat. Formal test levels for those conditions belong to the product’s environmental qualification. We do not quote them here, and a field pilot complements that qualification without replacing it. What the pilot adds is evidence of how the compute behaves while those conditions are present, which a qualification certificate does not record.
Measurement rules worth borrowing from MLPerf
The MLPerf Inference benchmark (Reddi et al.) was developed with input from more than 30 organisations. It defines scenarios that mimic mobile, autonomous-vehicle, robotics and cloud workloads. We would carry three of its rules over to an edge pilot.
First, report tail latency, as MLPerf does. The single-stream scenario measures 90th-percentile latency, and the multistream scenario measures the 99th percentile.
Second, run enough queries to support the percentile you claim. The original paper requires 1,024 queries for single-stream, while a 99th-percentile guarantee for vision tasks in the multistream and server scenarios needs about 270K queries. The current rules allow early stopping on shorter runs, at the cost of a slightly higher effective percentile that offsets the extra variance of a small sample.
Third, time the whole pipeline, as MLPerf does. Pre-processing and post-processing outside a short approved list count towards wall-clock time, and non-maximal suppression in object detection is timed because production systems need it. MLPerf also requires almost all implementations to stay within 1% of the FP32 reference model’s accuracy. That makes it a sensible bar for checking a quantised or converted model before anyone trusts its speed. Throughput under a latency constraint is lower than offline throughput for every system, because the constraint forces less efficient batching. A headline offline figure therefore overstates what a latency-bound robot will get.
To write all of this down so another engineer can rerun the pilot, we recommend a reproducibility checklist for fair benchmarks. Before units are bought, LynxBenchAI is the measuring tool for the hardware itself. Its published results help narrow a shortlist, and the field pilot then measures your workload on the units you will deploy.
How do you confirm a TensorFlow Lite or ONNX Runtime model really runs on the module’s accelerator?
We do not treat “the model loads and returns results” as proof that it runs on the accelerator. Check where every operator executes, on the module itself, inside the application.
ONNX Runtime splits a model graph into subgraphs, one for each available execution provider, by asking each provider through the GetCapability() API which nodes it can execute. Any operator that a specialised provider cannot take falls back to the default CPU execution provider. A model can therefore run correctly while part of its graph sits on the CPU. The check is to inspect which nodes each provider claimed and to time the session with that exact split. If several execution providers are built into one ONNX Runtime package, all of their dependent libraries must be present in the execution environment.
TensorFlow Lite (now LiteRT) benchmark tools share one measurement core across the Android and iOS apps and the command-line binaries. The published Android figures, however, were measured with CPU affinity set to the big cores, and your application may not run that way. LiteRT interpreter events can be captured with Android tracing tools alongside the app’s own events. Running more inference threads than there are available cores adds scheduling overhead that degrades performance. The questions worth answering on the real module are these: how much of the graph the delegate actually took, what the app measures under its normal thread and core settings, and whether converted-model accuracy stays within the bar set above.
Checking AV1 decode across the devices that actually receive the stream
When an edge module streams AV1 video to operator tablets, phones or ground-station clients, the decode question belongs to that client population, not to the module’s codec list. The AV1 white paper gives the core warning. In lab benchmarking, many A53-class phones appear able to decode AV1 to level 3.1, yet many with similar specifications cannot, so as a group A53-class devices cannot reliably support AV1.
Hardware AV1 decode is common on high-end mobile SoCs but limited in mid-tier and low-tier ones. Those devices depend on software decoders such as dav1d, at the cost of higher CPU use and battery drain. Meta defines “successful decoding” as decoding without frame drops or thermal events, with the battery sustaining 14 or more hours of continuous decoding on one charge. It recommends classifying each device by the maximum level it can decode in software and limiting streams to that level. That gives a pilot a success criterion to test against, where a supported-codec flag only says the codec exists.
The platform APIs help, within limits. On Android, isSoftwareOnly() flags software codecs, which make no guarantee about rendering performance. getSupportedPerformancePoints() lists the resolutions and frame rates a hardware codec can handle, but it returns null on devices upgraded to Android 10 or later without an updated vendor image. Media Performance Class 33 requires handheld devices to have an AV1 hardware decoder supporting Main 10 at Level 4.1, and six concurrent hardware decoder sessions (AVC, HEVC, VP9, AV1) at 1080p and 30 fps. In browsers, the W3C Media Capabilities API reports whether a configuration is supported, smooth and power-efficient. The same specification tells user agents not to mark hardware decoding as power-efficient by default. Treat all of these as screening signals. Then sample real devices from each tier and test them against the frame-drop, thermal and battery criterion.
What documentation shows CAP that the compute sizing supports the validated turnaround time?
We would document it as an evidence package that ties the GPU sizing decision for whole-slide-image inference to the turnaround time the laboratory has validated. It should include four things: throughput measured on the target hardware under a caseload that matches production, the turnaround that throughput implies, a comparison against the validated limit with a stated margin, and a sign-off. File it as part of the validation record, not as a capacity-planning note.
TechnoLynx builds this package. The work starts by establishing the turnaround requirement and the sizing evidence the accreditor expects. TechnoLynx then defines a WSI inference benchmark that reflects the real caseload, runs the sizing on the target hardware, confirms the margin, and writes the result into the validation record through its AI Governance and Trust work. TechnoLynx supplies the engineering evidence, and the laboratory’s medical director still owns the study design and its acceptance criteria.
The regulatory basis does not prescribe the study details. According to CAP, neither CLIA nor CAP has published specific guidelines for validating AI or image-recognition systems, so study design, specimen numbers and acceptance criteria are left to the medical director. CAP does state that the validation study should closely emulate the real-world clinical environment, cover the whole WSI system, and be repeated after any significant change. Under CLIA (42 CFR 493.1253), a laboratory that modifies an FDA-cleared test system, or introduces one not subject to clearance, must establish its performance specifications before reporting patient results and document every activity. The Digital Pathology Association’s 2011 white paper addresses the validation environment. The test environment should be the go-live environment, and turnaround time is one of the parameters to address; for frozen sections, scan and viewing time belongs in the study.
CAP accreditation already monitors turnaround. The CAP laboratory general checklist lists stat turnaround time as a quality indicator, measured as a mean or median or as the percentage of specimens within an established limit. Its evidence of compliance asks for a defined benchmark for each indicator, a monitoring frequency, and corrective action when targets are missed. Write the sizing evidence in the same terms the indicator uses.
Mapping a sizing benchmark onto the turnaround limit
| How the turnaround limit is stated | What the sizing benchmark should report | Why |
|---|---|---|
| Mean or median turnaround time | Mean and median per-case processing time at the expected caseload, on the target GPU | The statistic must be the same kind as the limit it is compared against |
| Percentage of cases within an established limit | A tail percentile of per-case time, with cases arriving at random like a live queue (MLPerf’s server scenario uses Poisson arrivals) | A mean can pass while the share of late cases fails |
| Indicator monitored at a defined frequency, as rates at least quarterly | Repeat measurement from production logs at the same frequency | Sizing that holds at validation may not hold as caseload grows |
| Records of follow-up when targets are not met | A stated margin, and a re-sizing trigger when that margin is used up | The trigger turns a missed target into a planned action rather than an incident |
Three caveats limit how far the mapping holds. The benchmark must use the slide sizes and case mix of the real diagnostic workload, or the mapping will fail in production. We would build the package to the specific accreditor’s format. The WSI sources also give case counts for validation studies. CAP’s WSI guideline update found that going beyond 60 cases did not improve mean concordance, while the earlier DPA paper suggested about 100 cases spanning routine complexity. Both concern diagnostic concordance, not compute sizing. The newer guideline rests on a systematic review, so we would treat it as the stronger reference. For sizing, the DPA’s point about reflecting the spectrum and complexity of routine cases is the part that carries over.
Capturing the benchmark as CSA validation evidence instead of a one-off IT exercise
The FDA’s final guidance on Computer Software Assurance, published in September 2025, covers software used in medical device production or quality management systems. Using its record structure for a laboratory’s sizing evidence is our recommendation. The FDA recommends that the record state the intended use, the result of the risk-based analysis, the testing performed, any issues found, and a conclusion on acceptability that includes how issues were resolved. Adding the objectives tested, and who performed the testing and when, gives the seven elements CSA practitioners work to.
Rigour scales with risk under CSA. Higher-risk functions call for more objective evidence: scripted or hybrid testing with documented rationale and traceability, while lower-risk functions can use exploratory or scenario-based testing with clear objectives. For high-risk functions the guidance’s own examples include repeatability testing across scenarios, which a multi-unit, multi-run benchmark provides. The FDA also prefers digital records such as system logs and audit trails over screenshots or paper. A benchmark harness that records queries and responses, as MLPerf’s LoadGen does, produces evidence in that preferred form.
A prospective silent trial is one way to collect turnaround evidence under real conditions before go-live. In a silent trial, a frozen AI system runs on live clinical data while its outputs stay hidden from clinicians. The STEP platform, deployed at three institutions, keeps run state and audit history that support measuring case capture, inference success, failure rates and turnaround time.
The record also has to anticipate change. CAP’s checklist guidance requires software changes to the analyser, LIS or middleware to be validated before use in patient testing. The DPA notes that no definition of a “significant” change exists, and that hardware or software changes which could affect interpretation call for revalidation. A documented impact assessment decides whether a change touches a high-risk function and needs re-testing. A driver update, a new board revision or a heavier caseload should each go through that assessment and not wait for the annual review.
Who can run an edge module pilot and turn the results into validation evidence?
TechnoLynx designs and runs field pilots for candidate edge modules, and turns the measurements into evidence a purchase decision or an accreditation review can rely on. For regulated imaging, the results go into the validation record through AI Governance and Trust, with support for re-validation as load changes. Where candidate hardware needs comparing before units are bought, LynxBenchAI is the measuring tool.
The engagement does not replace the laboratory’s own validation. Under CAP’s guidance the medical director decides the study design and acceptance criteria. The engagement is meant for teams that have a shortlist and a real workload to test. A team that only needs a first comparison of hardware classes can start from LynxBenchAI’s published results.
What the first pilot cannot tell you
A field pilot answers the question in front of it: these units, this software stack, this environment, this caseload. Its lasting value is the record. When the next JetPack release, board revision or rise in slide volume arrives, a record that lists conditions, unit spread, tail latency and margin turns the question into a repeat measurement against a known baseline, not a new procurement argument.
Some uncertainty remains in the evidence itself. The board-to-board variability study measured Jetson AGX boards kept below 50 °C, not boards exposed to dust or vibration. A pilot on Orin-class modules should treat unit spread and environmental effects as things to measure, not to assume.
