A Jetson-class module can run detection plus re-ID tracking inside a servo loop only when its profiled worst-case latency fits inside the loop’s deadline with margin to spare. In one published pipeline, a YOLOv7-seg detector with a ConvNeXt re-ID model took 42–73 ms per frame on a Jetson Orin Nano and 18–35 ms on a Jetson AGX Orin. In one open-source project’s 20-second runs at 1280×720 and 60 fps with default clock governors, a lean TensorRT YOLOv8n plus ByteTrack pipeline on an Orin Nano Super averaged 12.5–15 ms from capture to tracks, with p99 around 16 ms. Heavier models and tighter deadlines call for an industrial PC with a discrete GPU or a tethered workstation.

TechnoLynx validates the detection and re-ID pipeline on candidate hardware for a rig through the AI Porting & Deployment Pack. The work captures the loop’s exact latency budget and failure tolerance, builds and optimises the detection and re-ID pipeline on the candidate hardware, and checks worst-case latency and jitter against the deadline before anything is fixed into the rig. The first step is pinning down that budget.

Which GPU, Jetson-class or desktop, does the rig need for detection plus re-ID?

Size the rig to the smallest GPU whose profiled worst case, measured across the whole pipeline, stays under the loop period with margin. A spec sheet cannot give you that number. A published per-stage breakdown from a study of a foreign-object detection and tracking framework, run on both Orin modules, gives a reference point:

Stage (per frame)	Jetson Orin Nano	Jetson AGX Orin
YOLOv7-seg inference (TensorRT FP16)	26–45 ms	12–22 ms
ConvNeXt re-ID embedding, 2–3 crops (ONNX FP32)	15–25 ms	6–10 ms
Cosine similarity + Redis lookup	~1 ms per feature	<1 ms per feature
Total latency per frame	42–73 ms	18–35 ms
GPU utilisation	~70–85%	~50–65%
Memory use	~2.2 GB	~3.5–4.5 GB
Power	~9–15 W	~18–45 W

All rows come from the study’s hardware benchmarks on the two modules.

The authors describe the result as near real-time on the Orin Nano and real-time on the AGX Orin. They also note that the AGX Orin costs almost four times as much per unit and has the headroom for high-throughput or multi-camera work. They recommend hardware for the framework with sustained throughput of 20 TOPS (INT8) or beyond.

Two things in the table matter for sizing. First, re-ID accounts for 15–25 ms of the Orin Nano’s 42–73 ms total, roughly a third of the frame, and that cost was measured with two or three crops. Second, the spread within each column (26 to 45 ms for detection alone on the Nano) is the jitter a control loop has to absorb, and a range that wide makes the upper bound the figure to plan against.

For a first pass at which devices deserve a profiling run at all, LynxBenchAI publishes results showing how each device runs the public PyTorch stack your own models are built on, so they are broadly indicative for custom PyTorch models. LynxBenchAI does not run or measure your detection and tracking pipeline. That number still has to come from profiling on the device itself.

How much of the servo loop’s period is actually left for detection and re-ID?

We take the vision budget as the loop period minus everything else in the loop: camera exposure and readout, transport into memory, decode, and the time to commit the result to the controller. Whatever is left, minus a safety margin, is the most that detection, re-ID and tracking together may take in their worst case.

Where the loop sits on that scale changes the answer completely. Visual servo control can need position set points or feedback at rates from tens of milliseconds down to sub-millisecond. Low-level loops for high-speed, high-accuracy motor control cannot tolerate added jitter and latency. A loop that wants fresh vision feedback every 30 ms sits in a different hardware class from one that wants it every millisecond.

Published latency figures also need checking for where their timing boundary falls. One Orin Nano benchmark starts its clock when the application receives a complete compressed frame, so camera exposure and earlier sensor, USB and driver delays fall outside the reported number. A receipt-to-decision figure therefore understates the true sensor-to-actuator delay, and the missing part has to be measured separately on the rig.

The budget is also a policy decision about what happens on a miss. A GPU-resident design studied in a 2026 arXiv paper makes overload behaviour explicit: process the newest sample and drop older ones, or repeat the latest output when the actuator commits faster than new results arrive. Deciding in advance whether the controller can hold the last estimate for one frame, or must fault, tells you how much margin the budget needs.

Worked example: checking measured pipelines against a frame-rate deadline

Assume the loop needs one fresh tracking result per camera frame, so the deadline equals the frame interval. That is an assumption about the loop, not a measurement.

• At 60 fps the frame interval is 1000 / 60 ≈ 16.7 ms. The YOLOv8n plus ByteTrack pipeline measured p99 around 16 ms at 1280×720 and 60 fps. It fits, but with less than a millisecond of headroom, which is thin for closed-loop use.
• At 20 fps the interval is 50 ms. On an Orin Nano at about 20 fps input, a TensorRT YOLO pipeline recorded a median per-run p99 of 39.63 ms, against 98.82 ms for PyTorch FP32. The TensorRT build fits on receipt-to-decision time, which leaves out camera exposure and earlier sensor and driver delays. The PyTorch build’s p99 of 98.82 ms is nearly double the 50 ms deadline, even though it processed nearly all frames.
• Against the same 50 ms deadline, the YOLOv7-seg plus ConvNeXt pipeline’s 42–73 ms on the Orin Nano overruns at its upper end, while the AGX Orin’s 18–35 ms fits.

Counting processed frames would not have shown the PyTorch miss. The latency distribution is what shows it.

How much do model size and input resolution change latency on a Jetson?

Model size moves Jetson latency sharply. On a Jetson AGX Xavier, YOLOv7-tiny ran at 25–35 ms per frame against 60–75 ms for full YOLOv7. YOLOX-S takes around 10 ms on a V100 but around 80 ms on the AGX Xavier, roughly eight times slower.

Precision and runtime come next. Compiling YOLOv8 with TensorRT cut latency by roughly 30–56% compared with the next best framework, on both desktop NVIDIA GPUs and a Jetson AGX Orin, in an industrial machine vision benchmark. The same study found the advantage did not carry over cleanly to the transformer-based Grounding DINO, so the runtime choice depends on the model architecture as well as the hardware. On one tensor, the YOLOv8n pipeline’s TensorRT FP16 detections matched ONNX Runtime FP32 within 0.22 px for all confident candidates.

For a convolutional detector, compute grows roughly with the number of input pixels. In a tracking pipeline, every detection is another re-ID crop. Scene content matters too: YOLOv7 inference on the AGX Xavier ranged from 60 to 70 ms depending on scene complexity. In the dense MOT20 scenes, none of the existing methods, HopTrack included, achieved real-time tracking on the embedded device.

Re-ID extraction is the main tracking-side overhead

Appearance feature extraction is the main source of overhead in tracking-by-detection, according to a 2024 arXiv study of selective re-ID on a Jetson TX2. The same authors note that pipelines with a separate detector and embedding model remain the most accurate and modular option, but that the overhead makes them impractical on edge devices if every detection is embedded.

There are two levers:

• Extract selectively. Embedding only the detections that need appearance matching raised FPS by 80% on MOT17 and 52% on DanceTrack with no accuracy loss. Extracting from only 24% of detections cost nothing on DanceTrack-Val.
• Extract less often. NVIDIA’s DeepStream tracker documentation describes re-ID as computationally expensive and suggests raising reidExtractionInterval to improve performance, or disabling re-ID for motion-only re-association.

Which lever fits depends on what an ID switch costs the rig. Where losing an object’s identity for a few frames is harmless, motion-only association is the obvious saving. Where the servo is meant to stay on one specific target through occlusions, appearance features earn their cost, and the interval is the parameter to tune.

Throttling and jitter: why short benchmark runs mislead

A benchmark that runs for a few seconds on a bench tells you little about sustained latency inside a sealed enclosure. NVIDIA’s Jetson Linux documentation states that MAXN mode does not guarantee the best performance, because hardware throttling engages when total module power exceeds the TDP budget. On an Orin Nano 8GB the 15 W average power limit and the 20 W instantaneous limit each trigger clock throttling on the CPU and GPU. Thermal cooling states then cap the CPU, GPU and memory clocks at lower frequencies as temperature rises.

Light load can slow a pipeline without any throttling: on the Orin Nano Super, the default GPU governor held the clock at 306–408 MHz of its 1020 MHz maximum because the load was light, so YOLOv8n took 8–11 ms in the pipeline against 3.96 ms in trtexec at full clock. NVIDIA’s TensorRT documentation warns that two runs of the same engine on the same GPU can disagree by tens of percent if the clock floats in one run and is locked in the other, or if thermal throttling starts halfway through. The opposite outcome is also on record: a continuous 60-minute Orin Nano run held p99 between 39.60 and 39.70 ms, with GPU temperature between 60.8 and 62.5 °C. Sustained behaviour depends on the cooling and the power mode, which is why it has to be measured on the rig rather than assumed.

Jitter matters, not just the average. When the CPU sits between the sensor and the GPU (receiving the frame, submitting CUDA work, waiting, committing the output), closed-loop latency is exposed to interrupts, host scheduler decisions and CUDA API overheads. In a camera-to-display pipeline on IGX Orin with a discrete RTX 6000 Ada, a CPU-only receiver averaged 24 ms with 22 ms of jitter, while a GPU-resident CUDA Graph design brought jitter down to 0.66 ms. That result depends on locked GPU clocks, a fixed SM partition and static device memory, with no CPU interaction in steady state. These are deployment constraints to design for, not settings to switch on later.

Can tracking run fast enough on an embedded GPU, or does it need a tethered workstation?

Tracking on an embedded GPU is fast enough only when the profiled worst case, including decode, re-ID and post-processing, stays below the loop deadline with margin under sustained, throttled conditions. When it does not, the rig needs a tethered workstation or an industrial PC with a discrete GPU. On a Jetson AGX Xavier, a typical tracking frame took around 5–20 ms, against 60–70 ms for the detection network.

A tether buys compute headroom and costs a cable, a transport hop and its own latency and jitter, all of which come out of the same budget. A smaller detector, a TensorRT build and selective re-ID have each cut latency on embedded hardware. These are cheaper changes than redesigning the rig. Where the loop needs sub-millisecond feedback, a host-orchestrated pipeline exposes latency to interrupts, scheduler decisions and CUDA API overheads. For that regime, the published work points to GPU-resident execution, shown in a camera-to-display pipeline on IGX Orin with a discrete RTX 6000 Ada, and FPGA-based heterogeneous processing as ways to cut jitter.

When does an industrial PC with a discrete GPU or OpenVINO fit better than a Jetson module?

An industrial PC with a discrete GPU fits better when the profiled worst case misses on the largest Jetson module that fits the power envelope, when several cameras share one box, or when the model is large enough that desktop-class GPUs leave Jetson latency far behind. For Grounding DINO, even TensorRT on the Jetson AGX Orin took 1082.7 ms, a figure the study notes is much higher in absolute terms than on its desktop GPUs.

An OpenVINO-accelerated industrial PC suits an Intel x86 platform, since OpenVINO is primarily optimised for Intel hardware. On three Intel desktop platforms, OpenVINO gave the lowest CPU inference time for every model tested, cutting YOLOv8 inference time by 30–60%. On the Jetson CPU, by contrast, ONNX Runtime was faster, consistent with OpenVINO being optimised for Intel hardware. A trade article from automation press lists the Jetson strengths as TensorRT, CUDA and DeepStream support for GPU-heavy vision, and its drawbacks as 15–60 W power draw, supply variability and closed-source drivers.

The evidence has a gap here. The OpenVINO comparison measures network inference only, excluding pre- and post-processing. That comparison therefore gives no end-to-end detection plus re-ID tracking latency on an OpenVINO industrial PC. Choosing that route means producing the tracking measurement yourself, under the same worst-case rules as the Jetson candidate.

How should you profile a detection-plus-re-ID pipeline before committing to embedded hardware?

Profile the complete application on the candidate device under deployment conditions, and judge it on worst-case latency against the deadline. We hold that the context for any optimisation or hardware decision comes from profiling the actual workload, not from vendor figures or general best practice. The steps run in this order:

1. Fix the deadline and the statistic. Write down the vision budget from the loop analysis, and judge candidates on p99 and maximum latency, not the mean.
2. Build the engine as it will ship. Use the deployed runtime and precision. trtexec reports GPU compute time (kernel execution) separately from host latency (host-to-device copy, compute and device-to-host copy). It excludes transfer time unless you add --includeDataTransfers.
3. Time the whole application, capture to decision. In one Orin Nano pipeline, decode, preparation and post-processing took 19.12 ms, about 60% of the total, and JPEG decoding alone took about 10 ms. A 7.99 ms GPU replay median was not the latency of the complete camera application.
4. Load it like the real scene. Run whatever else shares the GPU and CPU: in HopTrack’s tests, GPU contention stretched inference from 75 ms to more than 600 ms.
5. Log the clocks while it runs. Read the GPU clock from /sys/devices/platform/bus@0/17000000.gpu/devfreq/17000000.gpu/cur_freq while the pipeline runs. On Jetson, nvidia-smi is not available for this.
6. Compare and decide. Accept the module only if p99 and maximum latency clear the deadline with the margin the miss policy requires.

For the OpenVINO candidate, OpenVINO’s benchmark_app gives the latency-oriented view when the number of streams is set to 1 and the latency performance hint is used. Its latency percentile is configurable and defaults to the median, so set it explicitly to the tail you care about. When several modules are on the shortlist, running the identical detection workload through one reproducible harness keeps the comparison about the devices rather than the setups. Our piece on profiling-based validation for GPU workloads covers how to make such measurements trustworthy. When a stage overruns and the cause is not obvious, kernel-level profiling of GPU bottlenecks is the next level down.

Decision table: Jetson module, industrial PC or workstation

What profiling shows	Typical workload	Hardware route
p99 and max well under the deadline, sustained in the enclosure	Nano-class detector, TensorRT FP16, motion-only or selective re-ID	Jetson Orin Nano class
Misses on Orin Nano, fits on AGX Orin with margin	Mid-size detector plus CNN re-ID, or several cameras	Jetson AGX Orin class
Misses on the largest Jetson the power envelope allows	Large or transformer models, dense scenes	Industrial PC with a discrete GPU, or a tethered workstation
Intel x86 platform required, CNN models	Detector and re-ID on Intel CPUs	OpenVINO industrial PC, with end-to-end tracking latency measured yourself
Deadline or jitter tolerance near sub-millisecond	High-speed, high-accuracy motion control	GPU-resident execution with locked clocks, or FPGA-based heterogeneous processing

Who can size and port our detection and tracking pipeline to the rig’s hardware?

TechnoLynx sizes the hardware for a detection and re-ID pipeline against your servo loop’s real deadline, then gets the pipeline running inside that deadline on the chosen device. The work captures the loop’s budget and failure tolerance, picks edge-suited model variants and precision, optimises the pipeline with TensorRT or OpenVINO, and validates latency and jitter under bursty load, with a tethered workstation as fallback. TechnoLynx delivers that work through the AI Porting & Deployment Pack, as part of its computer vision R&D practice.

The engagement does not suit every team. If your models already meet the deadline with margin on hardware you have chosen, and you have the profiling discipline in-house, extra porting work adds little. LynxBenchAI’s published device results are a reasonable first filter for ranking candidate devices before buying; they do not measure your own pipeline, so final sizing comes from profiling it.

What can still change the answer after the hardware is chosen

A sizing conclusion holds only for the pipeline that was profiled. A re-ID model swap is one such change: the framework above used 1024-dimensional ConvNeXt embeddings, while DeepStream’s sample re-ID configuration uses a 128-dimensional feature from a 128×64 crop. Moving between models that far apart changes the per-crop cost the budget was built on. The same goes for a new detector, a new camera resolution or a JetPack upgrade. Re-run the worst-case profile after any of them when validating changes before production, and treat the margin you kept at the start as the buffer for those changes.