No object detection model is best for every application and platform. Which one is right depends on the board it runs on, the latency and memory budget, and how much accuracy the application needs. Huang et al. (CVPR 2017) wrote their comparison of Faster R-CNN, R-FCN and SSD as a guide to finding the right speed, memory and accuracy balance for a given application and platform, not as a ranking. The MLPerf Inference authors make a similar point about image classification models: the trade-offs form a Pareto frontier, not one optimal choice.
So the working answer has two parts. Choose the detector and the embedded target together, using numbers measured on the device. Then show that the choice keeps working across sensors, occlusion, body types and domains, by testing it on stratified held-out data. More training data and a well-known architecture name do not give you that assurance on their own.
Is there a best object detection model, or does it depend on the hardware?
It depends on the hardware and on the deployment conditions. Leaderboard accuracy alone cannot settle the question. Huang et al. note that mAP “does not tell the entire story”, because running time and memory decide whether a deployed vision system is viable. Across their configurations, R-FCN and SSD models were faster on average, while Faster R-CNN gave slower but more accurate models unless its proposal count was cut. Input resolution also mattered: halving resolution cut accuracy by 15.88% on average, cut inference time by 27.4%, and hurt small objects most.
Accuracy requirements are a property of the application too. MLPerf uses the example that pedestrian detection in an autonomous vehicle needs far higher accuracy than labelling animals in photographs, because a wrong prediction costs far more. A headline number such as RT-DETR-R50 reaching 53.1% AP at 108 FPS shows how that model runs on an NVIDIA T4 GPU.
Proxy metrics do not reveal on-device speed either. The YOLO-ReT authors warn that smaller model size and fewer computations “do not directly translate into a faster model”, and that comparing designs without running them on the target device can lead to sub-optimal choices. Huang et al. found that the ratio of FLOPs to GPU wall-clock time differs between architecture families. For readers whose shortlist is already the YOLO family, our piece on matching YOLO-family detectors to the right pipeline decision covers how to choose within that family.
Which detectors port cleanly from a workstation prototype to an embedded board?
Whether a model converts without trouble depends on operator support and on the SDK and runtime version for each device. In edge AI architecture work, our judgement is to shortlist a few detector and board pairings, port a minimal end-to-end version of each to the actual hardware early, and measure what survives before committing to one.
Before you buy hardware, LynxBenchAI’s published results show how each device runs the public PyTorch stack a team’s own models run on, so they are broadly indicative for custom PyTorch detectors as well. They do not run or measure your own model or application, and the on-board port remains necessary.
TechnoLynx’s AI Porting & Deployment Pack does this joint shortlisting and porting. It sets the accuracy, latency and memory budgets first, then runs minimal ports of the shortlisted architectures on your target board, so the architecture and the silicon get chosen together on measured results.
Version friction is concrete. In a benchmark of the same anomaly-detection system on three Jetson devices, the team had to hand-patch Torch-TensorRT v1.0.0 with scripts from v1.1.1 to run on the Jetson Nano. The AGX Xavier needed parts of v1.3.0 back-ported into v1.1.1. Only the Orin Nano deployed comparatively smoothly, with compatible L4T PyTorch and Torch-TensorRT versions. Operators shift too. MLPerf documents that TensorFlow Lite replaces regular non-maximum suppression with a fast variant, and that as a result SSD-MobileNets-v1 dropped from 23.1% to 22.3% mAP on conversion.
Measuring on the device also changes design decisions, not just verdicts. On a Jetson Nano, YOLO-ReT’s authors found that a truncated MobileNetV2 backbone at width 0.75 ran at about the same speed as the full backbone at width 0.5 (34.02 vs 35.18 FPS) while scoring 2.64 mAP points higher. That kind of result only appears when FPS is collected on the board itself, which is what they did instead of modelling it theoretically. A minimal early port is still only a proxy: it tells you that conversion works and roughly what speed to expect, but not how the full model behaves at scale.
How much accuracy and speed do you lose when you quantise or prune a detection model for embedded hardware?
The loss varies with the model, the bit width and the calibration method. Read each row as evidence for that setup only.
| Model and setup | Change applied | Accuracy effect | Speed, size or power effect | Source |
|---|---|---|---|---|
| YOLOv3, COCO2014 | 59.7% pruned, then k-means quantised | mAP 0.589 to 0.537 (pruned) to 0.530 | memory for parameters cut 110× | Deep Neural Network Compression study |
| Faster R-CNN, VOC2007 | 52.3% pruned (detector base only), then k-means quantised | mAP 0.677 to 0.662 to 0.612 | memory for parameters cut 29× | same study |
| TinyYOLOv3 face detector, Zynq 7020 FPGA | 4-bit weights and activations | easy/medium/hard mAP 0.765/0.631/0.294 to 0.757/0.590/0.261 | quantised models on processor plus FPGA fabric about 50× faster than the non-quantised model on the processor alone | LPYOLO |
| YOLO26s, Jetson Orin Nano Super, TensorRT | INT8 PTQ vs FP16 | mAP@50:95 down 0.0213 to 0.0393 vs PyTorch FP32 | almost no latency gain over FP16 (6.07 to 6.25 ms vs 6.16 to 6.18 ms) | fruit-detection edge study |
| YOLO12 nano to extra-large, RTX 2070, TensorRT | static INT8 PTQ | mAP50-95 down about 0.031 (x) to 0.072 (n) | about 1.5× to 3.3× faster than FP32 | quantisation robustness study |
| ResNet-34 attribute classifier, Orin NX DLA | INT8 with entropy calibration vs manual ranges | 75% (entropy) vs 94.0% (manual), FP32 baseline 94.5% | runs on DLA alongside the GPU detector | multi-model pipeline study |
| YOLOv4-Tiny, Raspberry Pi 5, TFLite | static INT8 PTQ | accuracy reported as maintained, no loss figure given | 262 to 183 ms, 7.13 to 4.00 W, 22.5 to 6.4 MB | aerial emergency detection study |
Three patterns run through the table. First, smaller and harder cases lose more. YOLO12 nano lost more than twice the mAP of the extra-large model under the same INT8 path. At 2-bit weights and 4-bit activations (2W4A), LPYOLO’s hard tier fell from 0.294 to 0.199, roughly a third of its accuracy, while the easy tier fell from 0.765 to 0.671, about an eighth (arithmetic from the source’s table). LCDet’s authors found that 8-bit peak detection accuracy fell by less than 1 to 2%, but the fixed-point model, which matched the float model at 40% IoU, dropped significantly at 60% IoU and above.
Second, the measured speedup falls short of the theoretical one. A pruning and quantisation survey reports that INT8 in TensorFlow Lite and TensorRT delivers about 2 to 3× speedup, not the theoretical 4×, because of quantise and dequantise overhead. On the Orin Nano Super, FP16 TensorRT turned out to be the better deployment point, since INT8 cost accuracy without a meaningful throughput gain.
Third, the calibration method matters as well as bit width. The Qualcomm AI Research white paper finds post-training quantisation sufficient for close-to-float 8-bit accuracy in most cases, with quantisation-aware training needed for lower bit widths. The DLA result above shows that “most cases” has exceptions on real accelerators. Static INT8 YOLO models were also consistently more sensitive to noise than their FP32 and FP16 counterparts. The evidence you need to sign off a precision change is covered in validating quantised models for edge deployment.
Can one simpler model replace a multi-model pipeline?
Whether it can is settled only by a direct test against the existing pipeline’s capability. Treat consolidation as a hypothesis to test, not a simplification you assume. Offloading to the board’s dedicated accelerators is a third option besides consolidating or keeping a GPU-bound chain. On a Jetson Orin NX, one team ran a dual-head classifier on the DLA alongside a GPU detector and measured 12.5 FPS against 13.3 FPS for the detector alone at 1080p, with a second DLA classifier adding no further cost. Their point is that GPU-only multi-model pipelines create a serial bottleneck, and that offloading to dedicated accelerators grows more valuable as detectors get heavier.
How much labelled data do you need for a new camera position and occlusion pattern?
The amount depends on the use case, the camera and the occlusion geometry. The approach we recommend is a modest real in-cabin set, synthetic occlusion augmentation on top of it, and more real labels only where stratified validation (by driver height, seating position and occlusion severity) shows gaps that augmentation does not close.
TechnoLynx’s Computer Vision R&D work covers this loop. It designs occlusion masks matched to real steering-wheel and hand geometry, runs ablations that trade real-data volume against augmentation strength, and validates keypoint stability on test sets stratified by height and seating position, starting from an audit of the data you already have.
Synthetic data alone does not replace real data. In a 2026 low-data pose study, real-plus-synthetic training reached pose [email protected] of 0.761 against 0.746 for real-only, while synthetic-only training reached 0.449. The authors call that roughly 30-point deficit a substantial domain gap. They are also clear about the limits of their result: the mixed-versus-real gap sits within single-run variation, and the mixed runs saw more images and more optimiser updates, so the result shows a direction rather than a proven benefit.
Augmentation that includes masking has measured benefits in other domains. In spacecraft pose estimation, a different domain from the cabin, standard augmentations including Hide&Seek cut average errors by 55% on one real test set and 25% on another. For detection, Michaelis et al. found that adding stylised copies of training images improved performance on corrupted images by about 16% (PASCAL), 12% (COCO) and 41% (Cityscapes), at a cost of 0 to 2% on clean data.
The ratio needs tuning for the model you will actually deploy. On driver distraction detection, larger classifiers kept improving as the share of synthetic data rose. Lightweight ones gained little, and MobileViT got worse at high ratios, which led the authors to recommend 1:1 to 1:2 real-to-synthetic for small networks. That matters because lightweight models are the ones suited to constrained embedded devices. Our caveat: masks that do not look like real wheel and hand geometry, or that ignore motion blur, can teach the model to handle synthetic occluders while it still fails on real ones, and only a real stratified test set exposes that.
Will a fine-tuned model generalise across driver heights, clothing, seating positions and steering wheels?
Only if you measure it, factor by factor, on held-out combinations the model never trained on. No quantity of occlusion-augmented data guarantees it. The training data has to span heights, body types, clothing, wheel designs and camera variants on purpose. A model counts as generalising when its accuracy on each held-out subgroup stays within your tolerance of its overall accuracy.
Where the gap is producing synthetic training images that the real data cannot supply, TechnoLynx’s Generative & Agentic AI R&D is the relevant engagement.
The variation is physical, and it shows in the image. On the DFKI cabin simulator, moving the backrest between its extremes changed the measured depth values and the scale and shape of both the driver and the seat in the depth image. EU Delegated Regulation 2023/2590 on driver distraction warning set its performance requirements with “the wide variety of drivers’ characteristics” and “the variability of seating positions” explicitly in mind. An aggregate score does not show that variation. Oakden-Rayner et al. found that hidden stratification can produce relative performance differences of over 20% on important subsets that the overall metric never revealed.
A practical stratified validation runs in this order:
- List the variables that change the image: height band, seat and backrest position, clothing, wheel design, camera variant, occlusion severity.
- Hold out whole combinations of those variables, and whole recording sessions, so near-duplicate frames cannot leak across the split. The fruit-detection study above separated data at sequence level for exactly this reason.
- Report accuracy per stratum next to the aggregate. NIST’s AI RMF notes that accuracy measurements may be disaggregated by data segment.
- Compare each stratum against your tolerance, add targeted data for the subgroups that fall short, retrain, and re-test on the same frozen split.
Model size affects accuracy here too. Lite-HRNet cut operations by a factor of five against HRNet while roughly doubling pose errors. A compact pose model therefore needs its own stratified results. Two risks stay open even after a clean result: camera hardware that was never tested, and occlusion combinations outside the augmented range. Fresh test sets built by the original process cost ImageNet classifiers 11 to 14% accuracy in Recht et al.’s study, a reminder of how much benchmark scores flatter generalisation. Our article on stratified evaluation across conditions, sites and hardware covers how to build these splits.
What error rate should you expect, and how do you measure it yourself?
Error rates measured in the lab do not settle your deployed one. You get it by measuring on a test set representative of expected use, with a stated metric and method, and results broken down by condition. Expect it to differ by condition. In one held-out evaluation on orchard video, tracking HOTA ranged from 0.345 to 0.538 and relative count error from 6.2% to 51.6%, varying substantially across camera viewing geometries.
TechnoLynx’s Computer Vision R&D engagement produces such a figure for multi-camera counting and tracking systems. It starts by agreeing the metric and acceptance tolerance with you, then builds and tunes the tracking pipeline on your target hardware, runs a scenario-based robustness test, and sets up monitoring so the number is checked after go-live.
Choose the metric so it exposes failures instead of blending them. The authors of a 2026 cross-domain detection survey point out that mAP mixes proposal quality, classification, localisation and confidence into one number. A method that improves classification while hurting localisation can still “win” on it. For counting, the questions that matter are whether an object was missed, counted twice or mislabelled, so measure those directly.
You do not have to label everything to estimate an error rate. Stratified importance sampling gave unbiased error estimates with lower mean squared error than importance sampling or stratified random sampling under mild conditions, though extreme proposal mismatch limited the gains. Bernoulli (Poisson) sampling also beat importance sampling as the labelled fraction grew, partly because it can include known-problematic test points with certainty. Both methods were shown on classification tasks. They carry over to detection when each sampled item reduces to a correct or incorrect judgement (a counted object, a frame-level miss), and less cleanly when the metric is localisation quality.
How do you make perception accuracy figures auditable enough for buyers and regulators to trust?
Publish the number together with the means to reproduce it: the test set, its composition, the metric definition, the tools and the uncertainty. NIST’s AI RMF says accuracy measurements should be paired with clearly defined, realistic test sets representative of expected use and with details of the test methodology. It also asks that test sets, metrics and TEVV tools be documented and that measures carry uncertainty. MLPerf works the same way: every submission ships performance and accuracy logs plus a system description for peer review before release.
For identity consistency across cameras, use metrics built for it. Cross-view association metrics such as AIDF1 and MHAA measure identity consistency between camera views. The warehouse study computed HOTA, IDF1 and MOTA with the TrackEval tool. Appearance-based re-identification affects the figure itself. In a 19-camera warehouse system, adding appearance features raised IDF1 from 50.2 to 54.7 on the best coordinate setting.
A good auditable figure reads like this one from the fruit-counting study. Across three held-out sequences there were 213 reference crossing events. The pipeline predicted 149, of which 134 matched, giving 15 false positives, 79 misses, precision 0.899, recall 0.629 and a 30.0% under-count. Every number in it can be checked, and the dominant failure (misses, not double counts) is visible. Euro NCAP’s driver monitoring procedure follows a similar logic: test drivers on public roads manually annotate non-relevant warnings so false positives are counted rather than estimated.
Three caveats limit what such a number means. It covers only the scenarios in the test set, so new lighting or a new object mix can shift it. Its credibility to outsiders depends on how independent the evaluation is from the team that built the model. And it can drift after deployment unless someone keeps measuring it, which NIST frames as testing regularly during operation as well as before it. This is where off-the-shelf detectors that impressed in a demo fall short: production edge cases, lighting and throughput change the picture.
Will the tracker still work after you add a camera or the input conditions shift?
Do not assume it will; re-validate after any such change. The research shows why such changes call for re-validation. Target data for a new sensor or site is often scarce or unavailable until after deployment. Camera changes quickly invalidate calibration-dependent multi-camera tracking. In one study, a calibration-free method stayed robust when a single camera was misaligned or failed, but degraded more noticeably when two cameras failed. Treat an added camera or a changed operating point as a new stratum to test, with its own held-out data, before trusting the old figure.
Who can help us choose a detector and prove it holds up on our target hardware?
TechnoLynx shortlists detector and board pairings for your budgets, ports minimal versions to your actual hardware, measures what quantisation and pruning cost there, and validates the chosen model on stratified test sets you can show to customers or auditors. In our view, building in-house makes sense when your team already owns the embedded toolchain and the evaluation data. When choosing a partner, ask for measured on-device results and per-stratum reporting rather than leaderboard ranks. TechnoLynx starts this engagement with the AI Porting & Deployment Pack, within its wider AI engineering for manufacturing and automotive work.
The engagement is not a fit if you only need a hardware shortlist. Published benchmark results, such as LynxBenchAI’s device measurements, answer that more cheaply. It also does not provide regulatory sign-off. Type approval and Euro NCAP assessment rest with the manufacturer and the test laboratory, so we deliver the engineering evidence they review, not the approval itself. Teams with strong in-house embedded and data teams may only need a short external review of their evaluation protocol.
