Two teams benchmark DETR against YOLO on the same automotive dataset, pick the higher mAP-at-target-FPS number, and consider the decision made. Both packs clear the offline test. Only one survives the reviewer’s questions about production behaviour. The detector you pick changes the shape of the evidence you owe — and a comparison that stops at the leaderboard leaves that debt unpaid. The naive framing treats DETR-versus-YOLO as a single scalar contest: whichever architecture posts better mean average precision at the target frame rate wins. That framing is not wrong so much as incomplete. It answers “which model scores higher on held-out data?” while the reviewer is asking a different question entirely — “how will this model behave when the road stops looking like the validation set, and can I see that in the evidence?” The two questions rarely have the same answer. What actually differs between DETR and YOLO Start with the mechanism, because the failure profiles fall out of it directly. YOLO — the single-stage family running through YOLOv8 and its successors — predicts objects against a dense grid, historically anchored, and resolves overlapping predictions with non-maximum suppression (NMS). The detector emits many candidate boxes per region; NMS keeps the most confident and discards the rest by an intersection-over-union threshold. This pipeline is fast, well-understood, and heavily optimised across TensorRT and ONNX Runtime deployment paths. DETR — the detection-transformer family, including the deployment-oriented RT-DETR — replaces the grid-and-NMS pipeline with set prediction. A fixed number of learned object queries attend to image features and each query is matched to at most one ground-truth object through bipartite (Hungarian) matching during training. There is no NMS at inference. The model predicts a set directly, and the attention mechanism lets each query reason about global context rather than a local receptive field. That structural difference — dense anchors plus NMS versus learned queries plus attention — is the whole story. Everything the reviewer cares about downstream is a consequence of it. How do DETR and YOLO differ in failure profile? This is the question the leaderboard cannot answer, and it is the one that decides production behaviour. Under occlusion, YOLO’s NMS step is a known liability. When two pedestrians overlap heavily, NMS can suppress the box for the partially-hidden person because it looks like a duplicate of the visible one. The detector “sees” both but the post-processing throws one away. DETR’s set prediction has no NMS to trip over — distinct queries can hold distinct objects — but it depends on having enough queries and on attention actually separating the crowded region. Neither is immune; they fail differently, and a validation pack has to show which failure you inherited. On small objects — the distant cyclist, the fallen debris — YOLO’s multi-scale feature pyramid is generally strong, which is why slicing approaches like SAHI-assisted YOLO inference extend it so effectively. DETR historically struggled here because early attention operated on coarse feature maps; deformable and multi-scale variants close much of the gap but not for free. This is an observed pattern across perception work, not a universal ranking: the small-object story is architecture-and-configuration dependent, and the number that matters is recall on the small-object slice of your operational domain, not the aggregate mAP. Under distribution shift — dusk, rain on the lens, a geography the training set underrepresents — the divergence is subtler. YOLO’s confidence scores tend to stay high even when wrong, so the failure is often a confident false positive or a confident miss. DETR’s query-based confidence degrades differently, and its global attention can either help (more context) or hurt (spurious long-range associations). The point is not that one is safer. The point is that the shape of the degradation is different, so the drift monitoring you attach to each is different. We see this repeatedly: a detector that wins the aggregate benchmark loses on the specific slice that matters operationally, and the team discovers it only after integration. The failure modes that survive benchmarks are exactly the ones a slice-structured evidence pack is built to surface — a theme we develop in the perception failure modes that survive benchmarks. The comparison the reviewer actually reads Here is the axis matrix a perception validation reviewer works from. It is not a scorecard that declares a winner; it is a map of what evidence each choice obligates you to produce. Reviewer axis YOLO (anchor + NMS) DETR (set prediction + attention) What the pack must show Crowded / occluded scenes NMS can suppress valid overlapping boxes No NMS; depends on query capacity Recall on an occlusion-stratified slice, not aggregate mAP observed-pattern Small / distant objects Strong feature pyramid; extendable via slicing Weaker unless multi-scale/deformable Per-scale recall at operational distances observed-pattern Distribution shift Confident errors; stable but overconfident Context-sensitive; degrades differently Calibration + drift monitor matched to failure shape Post-processing determinism NMS threshold is a tunable, auditable step End-to-end; fewer hand-tuned knobs Documented threshold provenance (YOLO) / query count (DETR) Latency at deployed FPS Highly optimised export paths Improving (RT-DETR) but check your stack Latency held at deployed frame rate, not offline Confidence semantics NMS-filtered class confidence Query-matched set confidence How confidence maps to your acceptance gate The final column is the article’s point. Choosing YOLO means you owe the reviewer a documented, reproducible NMS threshold and evidence that it does not suppress safety-relevant overlaps. Choosing DETR means you owe an argument that query capacity is sufficient for your worst-case object density and that attention behaves under your shift conditions. Different detector, different evidence contract. Why the deployed frame rate — not the offline one — is the operating point A benchmark reports mAP at some FPS measured on the benchmarking rig. That number is nearly always taken at the model’s most favourable operating point: full precision or a clean INT8 calibration, a batch size that maximises throughput, no competing load on the accelerator. Production does not run there. The deployed pipeline shares the GPU with tracking, fusion, and planning; it runs at the batch size and precision the integration allows; and the latency budget is fixed by the control loop, not by the benchmark. The accuracy you can actually hold is the accuracy at that operating point. The discipline is to fix the frame rate first — the number the vehicle’s control loop demands — and then measure the accuracy each detector delivers at that latency on your hardware and software stack, in TensorRT with your real precision setting. A DETR variant that looks competitive at an offline 60 FPS may fall off a cliff when quantised and co-scheduled; a YOLO export may hold its operating point better on the same silicon. This is a benchmark-class measurement only when you run it on your own executor — the vendor’s published FPS is not your FPS. For the broader question of which metric proves what, we keep coming back to what each performance metric actually proves. When would we choose DETR over YOLO? There is no context-free answer, which is why the decision has to be documented against your operational domain rather than a leaderboard. A useful rubric: Start from the dominant failure risk, not the headline number. If your worst-case is dense, overlapping vulnerable road users, the NMS suppression risk pushes toward DETR’s set prediction — but only if you can afford its latency on your stack. If distant small objects dominate, YOLO with slicing may be the stronger and cheaper base. Check the operating point on your hardware. Run both at the deployed frame rate, quantised as you will ship them. Discard any accuracy claim measured elsewhere. Cost the evidence. A hybrid — YOLO for the fast common path, a transformer stage for hard slices — can be the right call, but it doubles the evidence surface. That cost is real and belongs in the decision record. Write down why. The decision record links the architecture choice to the specific reviewer questions it answers, so the choice is auditable later. This connects directly to how you fine-tune YOLO for automotive perception once the base detector is fixed. Whichever way the choice lands, the through-line is the same: the detector comparison is an input to the perception validation package, and it must be documented against production behaviour and drift posture — the surfaces our work on computer vision systems treats as first-class evidence, not appendices. FAQ How should you think about detr vs yolo in practice? YOLO predicts objects against a dense grid and resolves overlaps with non-maximum suppression, giving a fast, heavily-optimised single-stage pipeline. DETR replaces that with set prediction: a fixed set of learned queries attend to image features and are matched one-to-one with objects, so there is no NMS at inference. In practice the choice is not “which scores higher” but “which failure profile you inherit and which evidence you must then produce.” What are the concrete architecture differences — anchors and NMS in YOLO versus set prediction and attention in DETR? YOLO emits many candidate boxes per grid region (historically anchored) and NMS keeps the most confident by an IoU threshold. DETR uses learned object queries and Hungarian bipartite matching during training, predicting a set directly with no NMS and with attention providing global context. That single structural difference — dense-grid-plus-NMS versus queries-plus-attention — drives every downstream reliability behaviour. How do DETR and YOLO differ in failure profile under occlusion, small objects, and distribution shift? Under occlusion YOLO’s NMS can suppress a valid overlapping box, while DETR avoids NMS but depends on sufficient query capacity. On small objects YOLO’s feature pyramid is generally strong and extends well with slicing, whereas DETR needs multi-scale or deformable variants to compete. Under distribution shift YOLO tends to make confident errors while DETR’s context-sensitive confidence degrades differently — the shapes differ, so the monitoring you attach differs. How does the detector choice change what the perception validation evidence package must show a reviewer? Choosing YOLO obligates a documented, reproducible NMS threshold plus evidence it does not suppress safety-relevant overlaps. Choosing DETR obligates an argument that query capacity covers your worst-case object density and that attention behaves under your shift conditions. The detector fixes the evidence contract: same reviewer axes, different obligations per architecture. How do we compare latency and accuracy at the deployed frame rate rather than at an offline benchmark operating point? Fix the frame rate the control loop demands first, then measure the accuracy each detector holds at that latency on your own hardware and software stack — TensorRT, your real precision setting, your co-scheduled load. Published FPS is measured at the model’s most favourable point and rarely survives quantisation and shared-accelerator scheduling. It is a benchmark-class claim only when you run it on your own executor. When would we choose DETR over YOLO (or a hybrid) for an automotive perception stack, and how do we document that decision? Start from the dominant failure risk: dense occluded road users favour DETR’s set prediction; distant small objects often favour YOLO with slicing. Verify the operating point on your hardware, cost the evidence surface (a hybrid doubles it), and write a decision record linking the architecture choice to the specific reviewer questions it answers. That record is what makes the choice auditable later. The detector-choice failure profile is not a footnote to the perception validation package — it is one of its load-bearing surfaces, and it belongs in the same reliability discipline that governs the rest of the pack.
