Edge AI thermal testing should run through the full duty cycle on the real SoC, inside the real enclosure, at the worst credible ambient. A peak result posted at startup does not show whether the system still meets its deadline late in a mission. Peak throughput is not the operating ceiling. A 2026 arXiv study of sustained on-device LLM inference found that the iPhone 16 Pro lost roughly 40% of its peak throughput within three iterations, while the Samsung Galaxy S24 Ultra lost about 15% over 20 iterations before settling on a plateau set by its governor. Those are phones running a language model, not a perception stack in a vehicle, but the mechanism behind the loss is the same one an embedded GPU in a sealed bay runs into: the clocks come down when the junction gets hot.
A credible test captures when throttling starts and what it does to clocks, P99 latency, dropped frames and recovery time. That evidence is what decides whether you ship, mitigate or change hardware.
Will the model hold its frame rate across a full shift, or only in a burst test?
To know whether a model holds its required frame rate, qualify it under the complete workload on the actual hardware until thermal equilibrium. That run has to last until temperatures and clocks settle, or through the complete mission duty cycle if that is longer. A short run on a cool, open bench misses the clock state the board falls into once the heat sink and a sealed enclosure have soaked up heat.
TechnoLynx does this work through the AI Porting & Deployment Pack. The engagement starts by fixing the sustained workload profile (shift length, duty cycle) and the power, thermal and accuracy pass criteria. It then profiles and tunes the model on the target hardware, using quantisation and layer-level work, until the model fits the envelope or the gap has been measured.
A worked example of how fast a burst result goes stale
One study ran continuous CNN inference on a Raspberry Pi 4B with no active cooling. The CPU reached its 80 °C limit after about 50 seconds. Throughput then fell from around 16 fps to a stable 10 fps, and it took roughly 1,000 seconds of continuous inference to settle there. 1,000 seconds is nearly 17 minutes, so a ten-minute test on that board would have ended before throughput settled. The same authors kept every test running until performance reached steady state, and their longest indoor run lasted 5,340 seconds. On a Jetson AGX Orin, a 2026 study recorded the first sustained GPU frequency reduction at 22.25 minutes and 98.56 °C, followed by further staged reductions.
The report worth having compares the first five minutes with the final thermally stable interval and states the sustained loss, not one average over the whole run. High average throughput can sit alongside a P99 latency that breaks a control period. Sustained behaviour varies with OS, software framework and model, so it has to be measured under real operating conditions.
Three caveats shape the test plan:
- A passive or sealed production enclosure changes the thermal margin an actively cooled dev kit shows, so a pass on the kit does not carry over to the product.
- Synthetic load is a poor stand-in for the real workload. A 2026 study of a Jetson AGX Orin in an outdoor enclosure found that synthetic CPU and GPU stress tests substantially overestimated the thermal and power demand of a YOLOv8s inference workload.
- Driver and runtime versions (TensorRT, JetPack) belong in the test header next to the module, carrier, model hash, precision, power mode and cooling assembly.
If sustained draw goes over budget, TechnoLynx’s next step is further quantisation and layer-level tuning on the target hardware to fit the power envelope. Where a target has no baseline yet, the first soak exists to create one, and it should not be read as a pass.
Can scheduling, core isolation or a lower frame rate keep inference below the throttle point?
On one passively cooled board scheduling did, but only within a mapped ambient range. A study on a passively cooled Raspberry Pi 5 running YOLOv8n found that an empirically calibrated DVFS scheduler removed every observed throttling event across sustained 30-minute runs. It also delivered a 6.8% higher frame rate than a reactive baseline that used temperature alone. The same study found that the passive envelope closes at ambient temperatures of 27 °C and above, where nonlinear leakage defeats DVFS-based control. Its authors state that correct scheduling made a fan unnecessary on that board only inside that mapped envelope.
TechnoLynx tests these levers through the Inference Cost-Cut Pack: it designs a test matrix across frame rate, model size and duty-cycling on the target and implements the lighter model or duty-cycling mitigation where the baseline fails, and the first step is defining the sustained-use workload profile and risk thresholds.
Before choosing a lever, find out what is eating the margin. Tegrastats reports current and average power per rail, in milliwatts, alongside processor-block temperatures. On a shared die, sustained GPU or NPU load can throttle the CPUs that run sensor drivers and control orchestration, even when the model workload itself has not changed. So a frame drop in the control path can have a thermal cause that sits outside the model.
What the attribution shows decides the lever:
- The package limit binds while inference is a modest share of the heat. Duty-cycling and scheduling are the better levers, because shrinking the model will not move the binding constraint much.
- The inference kernels dominate. Cutting model size, input resolution or frame rate comes first, before any hardware change.
- Either way, check the result against the minimum frame rate the task needs to stay useful.
On Jetson, nvpmodel power modes cap core counts and frequencies to match targets such as 10, 15 or 30 W. That is a coarse but pre-qualified way to trade throughput for heat. NVIDIA calls MAXN an experimental setting and does not recommend heavy workloads in it for long periods; it also does not guarantee the best performance, because hardware throttling engages when total module power exceeds the TDP budget. Framework settings count as well. The S24 Ultra in the 2026 phone study held a sustained plateau only with the display off, a smaller context window and a smaller prefill chunk, which spread the prefill compute out and lowered the thermal spike per iteration.
Core isolation solves a different problem. The Linux kernel documentation describes it as a trade between noise-free isolated CPUs and extra overhead on the housekeeping CPUs, and it calls full dynticks (nohz_full) the most invasive option. Isolation targets workloads sensitive to kernel noise, such as very low latency processing. Each of these results holds for the hardware and duty cycle it was measured on, and a bench figure needs checking against field ambient and enclosure airflow before anyone relies on it.
Does thermal throttling change a model’s accuracy, or only its latency?
On a fixed-precision inference path, a lower clock changes how fast the arithmetic runs, not what it computes, so throttling shows up as latency and lost throughput rather than different outputs. Accuracy can still suffer indirectly: throttling can slow packet processing and callback execution, leaving state estimates stale. Whether any of that happens has to be measured for each model and device.
A policy that starts near 20 Hz can settle at a much lower rate after throttling, so the observations behind each decision are older than the designers assumed. A 2026 survey of edge deployment notes that failures caused by timing interference, such as oscillations or delayed corrective action, are easily blamed on perception or policy quality instead. A 2026 Jetson Orin Nano study treated output stability and P99 latency as independent safety properties, and showed that an accuracy-only test would have passed a CPU path whose P99 reached 165 ms against a 100 ms cycle budget. That timing failure came under CPU, memory and I/O load, not heat, since no thermal throttling was recorded. The lesson still holds: a correct output delivered too late is a missed detection.
The method for measuring both effects together is covered in characterising accuracy and throughput under thermal throttling.
What to log in the field so throttling is caught before a missed detection
On Jetson-class hardware, tegrastats reports processor, memory, thermal and power data. It reports CPU load and frequency per core, external memory controller bandwidth use and frequency, GPU activation and GPC frequency, temperature per thermal zone, and current and average power per rail. It samples at a configurable interval and can write to a log file for scripts to consume. One gap matters. Hardware throttling events are not reported to the host OS, so the drop in clock rates only becomes visible to a tool that samples the CPU cycle counters. Logs should also keep thermal throttling (a hardware response) apart from application rate limiting (a software policy). The difference tells the team whether to improve cooling, optimise the pipeline or change the task cadence.
Align these readings with P95 and P99 latency and dropped frames on one timeline. Treat any link between thermal state and detection misses as unproven until your own fleet logs show it consistently.
How that telemetry should feed back into runtime configuration is the subject of from thermal telemetry to trustworthy edge-inference configuration.
If throttling can’t be prevented, how should a perception system fail safely?
Make degradation an explicit, tiered policy triggered by live thermal state, so the application decides what to give up before the hardware cuts clocks underneath it. Left alone, a Jetson AGX Orin applies software clock limits once a sensor crosses its threshold, and uses hardware throttling as a last resort that significantly reduces performance.
A tiered fallback policy, step by step
- Trigger before the hardware does. A 2022 study measured ResNet50 throttling a phone at about 77 °C and set its switching threshold at 73 °C. Android’s thermal API takes a predictive approach instead: getThermalHeadroom estimates how long the device can hold its current performance, and the documentation says to call it no more than once every 10 seconds.
- First tier: cut duty cycle or frame rate. Lowering request frequency is a software policy the application controls directly.
- Second tier: switch to a pre-qualified lighter model. In the 2022 study, switching between a large and a small model by CPU temperature kept latency stable and prevented throttling across an hour-long experiment. The price is real, though: the two Slimmable ResNet50 widths used carry Top-1 accuracies of 0.768 and 0.638. The fallback model therefore has to be validated in advance against the task’s safety metric (defect catch rate, obstacle recall), not just against throughput.
- Third tier: warning-only or reduced-function mode. If no variant meets the minimum detection bar, handing the task to a driver or line operator is safer than running an unvalidated model silently.
- Add hysteresis and dwell. The 2020 Raspberry Pi study turned its fan on at 75 °C and off at 65 °C to avoid throttling. The 2026 Pi 5 scheduler only stepped between states after confirmed samples and a dwell time had passed. The same principle stops a model switch from toggling back and forth.
The limit of this approach is clear in the evidence. For compute-heavy models such as BERT, even the smallest variant could not keep the phone at a suitable temperature. Designing the driver- or rider-facing warning, so that it is noticed without causing alarm fatigue, is a safety and UX review question beyond compute engineering.
When a fallback tier becomes the normal operating mode, the question becomes whether to compress further or to change the hardware or power budget. That is settled by comparing the measured power per inference, latency and accuracy of compressed variants on the deployed hardware against candidate accelerators measured on the same workload, never against their headline figures. A cost model comparing a new accelerator, a bigger power system and further engineering only becomes necessary once compression gains flatten or the accuracy loss becomes unacceptable.
An edge AI thermal testing protocol quality teams can cite and rerun
A repeatable soak protocol records the module, carrier, Jetson Linux and JetPack, CUDA, TensorRT, kernel, model hash, precision, power mode and cooling assembly in its header, then runs a representative load for a set duration. The same pinned header serves a regulated design change and a refresh-cycle comparison, and any change to a pinned item means running it again.
What a sustained-run acceptance record should contain
| Record | Why it matters |
|---|---|
| Hot P95 and P99 latency | Tail latency in the stable hot interval is what a control deadline meets or misses. |
| Deadline-miss rate and sensor-drop rate | These turn “it slowed down” into a count of missed work. |
| Minimum clocks and maximum temperature | Together they show how deep throttling went and how close the board came to its limit. |
| Mean and peak power, memory headroom | These separate thermal limits from power and memory limits. |
| Recovery or reboot events | A thermal trip resets the system, so it has to be visible in the record. |
| Clock reduction, P99 increase, dropped frames and recovery time around each throttle event | Throttling onset and its consequences are what the test exists to capture. |
For medical devices, the 2026 Jetson Orin Nano study argues that FDA-2024-D-4488 and IEC 62304 Clause 6.3.2 call for verification under foreseeable conditions of use, which accuracy-based testing alone cannot provide. Whether a particular protocol satisfies a reviewer is a regulatory judgement. The engineering contribution is a protocol specific and reproducible enough to put in front of one. The instrumentation side is covered in instrumented sustained-operation profiling for thermal and power behaviour, and carrying the same duty cycle from bench to car is covered in a repeatable bench-to-vehicle thermal and power validation protocol.
What ambient and solar load should a soak on the real enclosure simulate?
A credible soak reproduces the worst field day on the sealed chassis itself, with continuous 24-hour cycles of simulated solar radiation at realistic maximum levels, repeated until the peak temperature stops rising. Method 505.7 of MIL-STD-810H supplies the numbers. It specifies a maximum irradiance of 1,120 W/m² and at least three continuous 24-hour cycles, continuing until the peak repeats within ±2 °C or seven cycles have run. Airflow is 1.5 to 3.0 m/s, or no less than 0.25 m/s where the item is shielded from wind.
TechnoLynx handles this inside the AI Porting & Deployment Pack. The first step is translating field conditions (solar load, ambient temperature, shift duration) into a test specification. The team then builds the profiling harness on the target accelerator and reports throttling, power and throughput in a reproducible, vendor-independent format that field engineers can sign off.
The enclosure has to be part of the test. The standard says that where the item is installed inside an enclosure, that enclosure must be provided so the heating effects are characterised properly. Solar heating is directional, and how much heat is absorbed depends on surface colour and finish, so the test chassis needs production-standard surfaces. Fully exposed equipment can exceed 60 °C in ambient temperatures as low as 35 to 40 °C. The run should also include the operating modes that draw the most power and the full input-voltage range, wherever voltage affects heat dissipation.
The field evidence shows how much ambient moves throughput. In the 2020 Raspberry Pi study, the temperature inside an outdoor box ranged from 38 °C in afternoon sun to 21 °C at night over about 520 minutes, and throughput moved between 17.7 and 22.5 fps.
ASTM G155 also describes xenon-arc exposure, with 14 representative test cycles, but it is a materials-weathering standard and does not provide performance characteristics of materials. For thermal load on operating electronics, MIL-STD-810H is the more direct reference. Some residual risk remains. A chamber result still has to be reconciled with field data, since outdoor conditions rarely provide zero wind.
When is measured margin enough, and when do you need a bigger heatsink, active cooling or a different module?
Margin is enough when the model and board stay inside the module’s documented thermal envelope through the worst-case soak, with headroom left for sensor uncertainty. If they don’t, choose the cheapest change that restores margin and profile again.
LynxBenchAI publishes results showing how candidate devices run the public PyTorch stack, a broadly indicative guide for custom PyTorch models when a module change is on the table, not a run of your own workload; the first step is checking those results for the boards on your shortlist.
TechnoLynx runs this decision in four steps. It profiles the current model on the candidate board under simulated worst-case load, quantifies headroom against the board’s rated envelope, profiles each mitigation that might close a gap, and writes the sign-off report that engineering and procurement review.
NVIDIA’s thermal design guide for Jetson AGX Orin defines the recommended operating limit as the temperature below which the module runs without performance reduction. It tells designers to size their headroom to how much reduction they can tolerate, allowing for ±3 °C of sensor uncertainty. Module and heat-sink thermal resistances add up to give the total from SoC to ambient. In the guide’s worked case, 0.58 °C/W at 50 W and 50 °C ambient gives 79 °C. Read the other way, at that resistance every extra watt costs about 0.58 °C of margin.
| Soak result | Usual response |
|---|---|
| Stays below the operating limit at worst-case ambient, with margin beyond sensor uncertainty | Sign off as fit to ship and record the configuration. |
| Throttles, but scheduling removes it inside the expected ambient range | Mitigate in software, and document the ambient ceiling where the passive envelope closes. |
| Throttles, and active cooling is affordable | Add a fan with hysteresis control. In the 2020 Raspberry Pi 4B study this prevented throttling and improved throughput by up to about 90%, for a fan drawing 0.65 W (roughly 10% more system power). |
| A lower-resolution model closes the gap | Accept it only after re-validating detection performance, not just temperature. |
| No software or cooling change closes the gap | Change module or SoC, then repeat the full protocol on the new part. |
Judge the “cheapest” choice against your own bill of materials and requalification costs, not a general rule. Fanless designs bring their own budgeting problems, which are covered in on-device thermal and power budgeting for fanless embedded inference.
When a sealed box drops frames over a long soak, logs that separate thermal throttling from application rate limiting tell you whether to improve cooling, optimise the pipeline or change the task cadence.
Measure the quantised model’s draw on the actual dev board first. If it is over budget, move along the power-accuracy trade-off with more aggressive quantisation or pruning, and fall back to a smaller backbone or a different SoC only once accuracy stops being acceptable. Even then, a dev-board figure is provisional, because the production enclosure and power delivery will shift it.
How much GPU headroom is left once the SoC is held to its power envelope?
The only figure to plan capacity on is the sustained clock and memory-bandwidth headroom, measured with the real workload on the real SoC after thermal soak. Peak settings do not guarantee the best performance, because hardware throttling engages when total module power exceeds the TDP budget.
TechnoLynx’s GPU Performance Engineering work starts by defining the power envelope and frame-rate target. The team then instruments GPU occupancy and memory bandwidth under soak, establishes which of the two the frame drops follow, optimises that path on the target and re-validates under sustained load.
Telling the two causes apart starts with the counters. Tegrastats reports GPU activation time and GPC frequency separately from the memory controller’s bandwidth use and frequency. When parameter streaming and high-rate sensor ingestion share LPDDR, accelerators can stall on memory delivery rather than arithmetic. The 2026 survey cites diffusion policies falling from roughly 30 FPS in isolation to 6 to 12 FPS under realistic sensor load because of memory contention alone.
Hardware throttling engages as a last resort when software throttling cannot hold the temperature. NVIDIA’s Jetson Linux guide lists levels labelled light, medium and heavy, with clock-throttled percentages of 50, 75 and 87.5. It also documents a 65 W instantaneous-power limit on the 60 W AGX Orin 64GB, with CPU and GPU throttled to 50%.
Approve the device refresh, require a mitigation, or switch hardware?
Approve the device as it is only if sustained thermal, latency and memory behaviour stays inside a pre-set threshold with margin. If it breaches the threshold but a software mitigation (lower frame rate, lighter model, duty-cycling) closes the gap on re-test, approve it with that mitigation. Evaluate alternative hardware only when mitigations fail or cost too much accuracy.
TechnoLynx supports this decision through the Inference Cost-Cut Pack. The engagement begins by defining the sustained-use workload profile and risk thresholds, designs a test matrix across frame rate, model size, duty-cycling and case or heat sink, and implements the lighter model or duty-cycling mitigation if the baseline fails.
The 2026 study of on-device LLM inference on phones concluded that the operating ceiling for an always-on workload is the plateau throughput, not the peak. When an alternative device is needed, LynxBenchAI’s published results help shortlist candidates by showing how each runs the public PyTorch stack. The application itself still needs its own trial. Ambient conditions and duty cycles differ across sites, so a field pilot on a subset of locations is worth running before a full rollout.
For a live system with a human fallback, such as a race-strategy model on a car, the monitor should flag a leading indicator (SoC temperature, a power trend, or its rate of change) rather than output that has already degraded, so the pit wall can switch to manual calls in time. The threshold has to be tuned against false triggers, which cost strategic value, and confirmed in the car, where cooling differs from the bench.
Who can run a full-duty-cycle thermal soak on our hardware and act on the result?
TechnoLynx works out whether heat, power or the model is the binding limit, then fixes it on the target, as part of its AI engineering for manufacturing and automotive work. The engagement is the AI Porting & Deployment Pack, and its first step is agreeing the duty cycle and pass criteria.
On build versus buy: an in-house team can carry this when it already owns embedded GPU profiling and can keep a protocol pinned across releases. Without that, a specialist saves you from learning thermal behaviour on production units. When choosing a partner, look for one that hands over the protocol, raw logs and pass criteria so your team can rerun them, not only a report. The module vendor’s thermal design guide gives the resistance arithmetic for sizing a heat sink. For chamber time under MIL-STD-810H, an accredited environmental test lab is a realistic alternative route.
FAQ
At what temperature does a Jetson Orin throttle?
NVIDIA’s Jetson Linux guide lists software throttling at 99 °C and hardware throttling at 103 °C on the CPU and GPU thermal zones of the AGX Orin, Orin NX and Orin Nano. Software shutdown follows at 104.5 °C and hardware shutdown at 105 °C.
Is it safe to disable thermal throttling?
No. NVIDIA states that removing all thermal zones does not guarantee maximum performance and can cause resets or irreversible damage, and that tuning cannot remedy a faulty thermal design.
What does graceful degradation mean for an edge AI system?
It means the system gives up capability in planned, tested steps (a lower frame rate, a lighter model, a warning-only mode) instead of failing abruptly or silently when it runs out of thermal headroom.
