“It runs at 60 FPS, so it’s real-time.” That sentence has quietly wrecked more deployment timelines than any model bug. The number on the model card was measured on a workstation GPU, at a resolution and batch size chosen to look good, on a class distribution that has nothing to do with the parts moving down your line. Real-time object detection is not a property you read off a benchmark. It is a contract between latency, accuracy, and the hardware you actually deploy on — and teams that quote a benchmark FPS have accepted a throughput-versus-accuracy trade they never measured. That is the gap this article is about. Detection models are easy to run and hard to run on time at the accuracy your process needs. Getting the frame rate you want almost always costs you something on the accuracy side, and the cost lands exactly where it hurts most: small, distant, or partially occluded objects. The useful question is never “is this model real-time?” It is “what latency budget does my deployment enforce, and what miss rate does hitting that budget impose?” How does real-time object detection actually work? A modern detector — a YOLO variant, RT-DETR, or an SSD-style single-shot network — takes an image and returns a set of bounding boxes, each with a class label and a confidence score. “Single-shot” means the network predicts boxes and classes in one forward pass rather than proposing regions first and classifying them second, which is what made this family fast enough to consider for streaming workloads in the first place. If you want the geometry of how those boxes and scores come out of the network, how object detection with YOLO works covers the mechanics; the confidence value each box carries is worth understanding on its own terms in what a confidence score in computer vision means. The mistake is treating that forward pass as the whole story. A frame does not arrive as a tensor. It arrives as a camera signal that has to be captured, decoded, colour-converted, resized, and normalised before the model sees it — and after the model runs, the raw predictions have to be filtered through non-maximum suppression (NMS), thresholded, and turned into an action: a reject signal, a track update, an alert. Every one of those stages consumes time, and only one of them is the inference the benchmark measured. This is why “real-time” has to be defined as an end-to-end latency budget. If a robot cell has to divert a defective part before it reaches the next station, the clock that matters starts when photons hit the sensor and stops when the divert actuator fires. The model’s forward pass might be 8 ms of that; capture, pre-processing, NMS, and the control hop can easily be another 20. A detector that benchmarks at 60 FPS in isolation can miss its real deadline by a wide margin once the full pipeline is on the clock. What is the difference between benchmark FPS and production real-time performance? Benchmark FPS is a throughput figure produced under favourable, controlled conditions — often batched, often at a resolution and precision chosen to flatter the number, always on a specific GPU. Production real-time performance is a tail-latency figure on your hardware, under your input distribution, with the whole pipeline attached. These are different quantities, and one does not predict the other. Three structural differences drive the divergence, and they compound: Batching. Benchmarks push throughput by batching frames; a live camera hands you one frame at a time. Batch-of-one latency is often far worse per frame than the batched throughput number implies, because the GPU never fills. The measured surface. A model card measures the forward pass. Your deadline measures capture-to-action. NMS in particular scales with the number of detections — a crowded scene with hundreds of candidate boxes costs more post-processing time than the sparse benchmark scene ever showed. Tail vs. average. Benchmarks report averages or medians. A control loop lives and dies on p99. The one frame in a hundred that takes twice as long is the frame that misses the divert window. The operational measures that actually matter are three numbers stated together: frames per second at a fixed resolution, p99 end-to-end latency, and the false-negative rate you accept to hit them. Quote any one alone and you have said nothing decision-grade. This is the same distinction — measured surface versus deployment reality — that SPEC benchmarks for computer vision inference run into: a benchmark measures what it measures, not what you deploy. How latency and accuracy trade off — and where the cost lands Here is the mechanism that turns a frame-rate requirement into an accuracy problem. When a detector can’t hit the required latency, engineers have two easy levers: shrink the input resolution, or swap to a smaller model. Both reduce compute per frame. Both also reduce the effective pixel budget available to represent an object — and small, distant, or occluded objects are the first casualties. A defect that occupies 12×12 pixels at 1280p may occupy 6×6 at 640p. Below a certain size, the detector’s feature maps simply can’t resolve it, and the box never fires. The frame rate went up; the miss rate on the exact objects you cared about went up with it. This is not a tuning nuisance — it is the structural failure at the heart of real-time detection, and it is why “60 FPS, therefore real-time” is a claim about the GPU, not about whether the system does its job. Detection accuracy is usually reported as mean average precision (mAP), and the resolution/model trade-off shows up directly in the mAP-versus-latency curve. If you’re reasoning about which accuracy metric to hold the system to, object detection metrics explained walks through precision, recall, mAP and IoU and what each one hides. There are also engineering escape hatches that break the naive resolution/latency trade: tiling a high-resolution frame and running inference on slices, as in SAHI + YOLO slicing inference for small-object detection, recovers small-object recall at a compute cost you can budget for — instead of silently paying it in missed detections. What does the end-to-end latency budget include? Everything between the sensor and the action. Naming each stage before deployment is what turns “real-time” from a marketing word into an agreed number. The table below is a worked decomposition for a single 1080p camera feeding an inspection cell; the millisecond figures are illustrative — a planning template, not a measured benchmark — but the structure is what every real budget looks like. Worked example: an end-to-end latency budget Assumptions: one 1080p H.264 stream, single-shot detector on an edge GPU, batch-of-one, deadline set by a mechanical divert 300 ms downstream. Stage What happens Illustrative budget Capture + decode Frame off the sensor, H.264 decode ~6 ms Pre-processing Colour convert, resize, normalise ~4 ms Inference (forward pass) The number the model card quotes ~10 ms Post-processing (NMS) Filter overlapping boxes, threshold ~3 ms (scales with detection count) Transfer + action Result to controller, divert decision ~5 ms End-to-end Capture → action ~28 ms The lesson is in the arithmetic: the forward pass is about a third of the budget here. A team that optimises only the model — quantising to FP4, fusing kernels, upgrading the GPU — improves the smallest addressable slice while capture, decode, and the control hop sit untouched. When decode dominates the budget, the codec matters as much as the model; 10-bit HEVC and what it means for video analytics pipelines is one place that shows up. This is a market-direction observation, not a benchmark: across CV deployments the pre- and post-processing overhead is routinely underestimated at scoping time because the model is the visible, benchmarkable part. How does deployment hardware change what “real-time” means? The same model is a different system on a Jetson-class edge module than on a datacentre GPU. Memory bandwidth, the presence or absence of hardware video decode, the inference runtime (TensorRT, ONNX Runtime, OpenVINO), and thermal headroom all move the latency budget — sometimes by an order of magnitude. A model that comfortably hits its deadline on an RTX-class workstation can blow past it on an edge device with a fraction of the HBM bandwidth, even after quantisation and TensorRT engine compilation. This is why the deployment hardware has to be in the loop before anyone signs off on a frame rate. “Real-time on what?” is not pedantry. An edge device with hardware H.264/HEVC decode may actually beat a workstation on end-to-end latency for a streaming workload, because it takes decode off the CPU entirely — the workstation’s raw compute advantage is irrelevant if it’s spending it decoding frames. Choosing the target device is a genuine trade-off exercise, and it’s exactly the kind of scoping what a computer vision consultant does is meant to make explicit rather than discover on the line. For the broader engineering context around deploying vision systems, our computer vision practice page is the starting point. How to measure a real-time target before shipping Measure the whole pipeline, on the deployment hardware, under the real input distribution, and report the tail — not the average. A quick diagnostic before you commit to a frame rate: State the deadline first. What downstream action does the deadline serve, and how many milliseconds does it allow, capture-to-action? If nobody can answer this, “real-time” is undefined. Measure batch-of-one, not batched. Live streams don’t batch. Benchmark the way you’ll actually run. Feed the real class distribution. Crowded and occluded scenes cost more in NMS and more in missed detections. Test on hard frames, not clean ones. Report p99, not the mean. Your control loop misses on the tail, not the average. Record the miss rate at the chosen latency. State the false-negative rate on small/occluded objects that hitting the frame rate imposes. If it’s unmeasured, it’s unmanaged. Do these five and “real-time” stops being a claim and becomes a contract: FPS at fixed resolution, p99 latency, and the accepted false-negative rate, all stated together. FAQ What’s worth understanding about real-time object detection first? A detector returns bounding boxes, class labels, and confidence scores from a single forward pass over each frame, but the frame first has to be captured, decoded, and pre-processed, and the predictions then filtered through non-maximum suppression and turned into an action. In practice “real-time” means the whole capture-to-action pipeline completes within a deadline set by the downstream action — not that the model’s forward pass alone is fast. What is the difference between benchmark FPS and real-time performance in a production deployment? Benchmark FPS is a throughput figure measured under favourable controlled conditions — often batched, at a chosen resolution and precision, on a specific GPU. Production real-time performance is a tail-latency (p99) figure on your hardware, under your input distribution, with the full pipeline attached. One does not predict the other, mainly because benchmarks batch frames and report averages while a live camera hands you one frame at a time and your control loop misses on the tail. How do latency and throughput trade off against detection accuracy on small or occluded objects? Hitting a required frame rate usually forces lower input resolution or a smaller model, both of which cut the pixel budget available to represent an object. Small, distant, and occluded objects are the first to fall below the size the detector can resolve, so the miss rate rises exactly where it matters most. The frame rate improves and the false-negative rate degrades on the same change. What does the end-to-end latency budget include beyond model inference itself? Capture and video decode, pre-processing (colour conversion, resize, normalisation), the inference forward pass, post-processing (NMS and thresholding), and the transfer of the result to whatever takes the action. The forward pass the model card quotes is often only about a third of the total, which is why optimising the model alone leaves most of the budget untouched. How does deployment hardware (edge device vs GPU workstation) change what “real-time” means? Memory bandwidth, hardware video decode, the inference runtime, and thermal headroom all shift the latency budget, sometimes by an order of magnitude. A model that meets its deadline on a workstation GPU can miss it on an edge module — yet an edge device with hardware HEVC decode can also beat a workstation on end-to-end latency for streaming, because it takes decode off the CPU. “Real-time on what hardware?” is a required part of the specification. How should a real-time throughput and latency target be measured before a CV system ships? Measure the whole pipeline on the deployment hardware, under the real input distribution, batch-of-one, and report p99 rather than the average. State the deadline the downstream action imposes first, then record the false-negative rate on small and occluded objects that hitting the frame rate costs — so the accuracy sacrificed to hit the frame rate is a known number, not a surprise on the line. When does hitting a required frame rate force a model or resolution change that raises miss rates? Whenever the detector cannot meet the deadline at the resolution and model size that resolve your target objects. At that point engineers reach for the two easy levers — lower resolution or a smaller model — and both reduce the pixels available to represent small or occluded objects, pushing some below the detectable size. That is the moment a throughput requirement quietly becomes an accuracy regression. The question worth settling before you deploy Real-time is not a model attribute you can borrow from a benchmark. It is a trade you make on your hardware, under your load, against your accuracy floor — and it exists whether or not you measured it. The teams that ship reliable systems are the ones who named the trade first: they wrote down the deadline, the p99, and the miss rate they’d accept to hit it, then checked all three on the deployment hardware before the line was live. Our Production CV Readiness Assessment characterises exactly this — real-time latency and throughput on the actual deployment hardware — so the accuracy sacrificed to hit frame rate is a decision on paper, not a discovery on the line.
