GPU-accelerated video analytics becomes necessary as stream count rises, as models per camera multiply, or with heavy models such as face recognition. At that point CPU-only scaling needs many servers to do what a few GPUs can. It does not pay for a handful of cameras running light detection at low frame rates, where a CPU instance can be the cheaper option. Between those two cases, benchmarking the pipeline on one GPU gives the answer. Cost and latency can often be cut below the model (batching, caching, quantisation, serving configuration) without replacing the model.
Here, video analytics means running detection, tracking and classification models over live or stored video in a pipeline you operate yourself. If you want the basics first, how video analysis works and why models degrade on real footage is covered on its own page. The economics below are about running the pipeline on your own compute rather than calling a hosted service. That difference is set out in pipeline engineering vs content-understanding APIs.
Where a GPU helps the analytics, and where it does not
A GPU’s advantage is far higher stream density per node than CPU-only. One card can run many concurrent streams with several models per stream through a shared inference engine, and that density is where the better streams-per-watt and streams-per-rack-unit numbers come from at scale. The opposite case is just as real. CPU-only analytics suits small camera counts running lightweight models, where simplicity and lower upfront cost matter more than density.
Kaseb et al. (2018) tested both sides of this on Amazon EC2 instance types. In one scenario, four camera streams at frame rates high enough that each CPU instance could handle only one stream fitted onto a single GPU instance, which cut hourly cost by 61%. In a second scenario, with VGG-16 at 0.2 fps and ZF at 0.5 fps, one non-GPU instance handled both streams and cost 36% less per hour than the GPU instance. Which instance type was cheaper depended on the analysis programs and frame rates. In those two scenarios, the authors’ resource manager, which considers both GPU and non-GPU instances, had the lowest cost of the strategies compared.
Work too small to saturate the GPU leaves it underused. Stone et al. found that the GPU saturated only on images larger than about 128K square pixels, while the active regions they extracted from camera frames averaged 4K to 8K square pixels. That size gap left the GPU rarely fully used under the naive scheme. Deployments that land in this trap, through tiny inputs, sparse events or a single light model, are covered under wrong workload shape.
How many streams and models before a GPU beats CPU-only on cost?
No fixed stream count per GPU tips the decision. Capacity depends on streams × resolution × frame rate × model complexity, and the GPU’s decode and inference throughput limit it. Even VMukti, a vendor arguing for GPUs, sets the threshold by workload: light, few cameras favour CPU, and dense or multi-model workloads favour GPU. Sizing therefore comes from measured throughput. Define one camera unit (resolution, fps, model), benchmark the batched pipeline on one GPU, then divide the camera count by the sustained streams per GPU and add headroom. The steps are worked through in sizing GPU instances for a video-analytics workload.
NVIDIA’s DeepStream figures show channel counts changing with the model. On a Tesla P4, an older data-centre card, a ResNet-18 detector reached 16 channels of 30 fps HD before the GPU cores saturated and the pipeline became inference-bound. With a smaller ResNet-10, the same card handled 30 channels until the hardware decoder hit 100% and the pipeline became decode-bound. A cascaded pipeline with a detector, a tracker and three secondary classifiers ran 25 720p streams at 30 fps on that card. Those figures come from a Tesla P4. What carries over is the pattern: a different model moves the bottleneck as well as the throughput.
Frame rate is the other multiplier. Kaseb et al. measured CPU and GPU requirements rising linearly with frame rate, so one test run at a known rate lets you estimate others. Analysing at 5 fps instead of 30 is a product decision, and it can matter more than the choice of card.
The cost model for GPU-accelerated video analytics at production volume
For a pay-per-minute service, per-camera cost depends on the minutes of video processed. AWS prices Amazon Rekognition streaming video events at $0.00817 per minute processed, with no upfront cost or minimum fee. AWS’s own worked example assumes 10-second clips on 7 motion events per camera per day, or 35 minutes per camera per month. At that rate, 35 × $0.00817 comes to about $0.29 per camera per month.
Continuous analysis gives a very different number. The same per-minute rate across a full day is 1,440 × $0.00817 ≈ $11.76 per camera per day, or about $353 over 30 days (arithmetic on the AWS rate). Stored-video label detection is listed at $0.10 per minute, about $6.00 per hour of stored video (arithmetic on the AWS rate). No owned-GPU cost per stream is given here. Your own benchmark supplies the other side: the streams one GPU sustains at your resolution and frame rate.
An owned pipeline also has to carry the streams across the network and size its GPUs to keep pace. The same GPU decode and encode engines serve both transcoding and deep-learning video work. The cost and quality side of that is in how video transcoding cost and quality trade-offs work, and the line between analysis and transcoding is drawn in video content analysis functions.
Decision table: which compute model fits the workload shape
| Workload shape | Likely cheapest route | Why |
|---|---|---|
| Few cameras, short event-triggered clips | Pay-per-minute API | You pay only for processed minutes, with no provisioning or minimum fee. |
| Few streams, light model, low frame rate, continuous | CPU-only instance | In one test, one CPU instance carried two low-frame-rate streams at 36% less hourly cost than a GPU instance. |
| Many continuous streams, standard detector | Owned or rented GPU with batched inference | Multi-camera batching on a shared engine is what makes fleet-scale vision economical. |
| Heavy or multi-model per stream (face recognition, multi-camera tracking) | GPU, sized per model | Heavy models cut streams per GPU, so size each model separately. |
| Mixed fleet with varied models and frame rates | GPU and CPU instances mixed by profile | In Kaseb et al.’s two test scenarios, mixed GPU and CPU allocation had the lowest cost of the strategies compared. |
Does moving video decode onto the GPU change the economics of what follows?
Yes. A single 1080p H.264 stream decoded on a CPU uses about 30-40% of a core, according to a 2026 sizing guide that cites NVIDIA DeepStream. That leaves little headroom before a second stream. NVIDIA’s NVDEC application note says that with decoding offloaded to the hardware decoder, the graphics engine and CPU are free for other work. That freed CPU is what the tracking, post-processing and business logic downstream need. NVIDIA presents hardware encode and decode as raising stream density per form factor, which lowers dollars per stream and watts per stream compared with software codecs.
Two details shape the capacity planning. Some GPUs have two or three NVDEC engines, and the driver balances load across them. The combined capacity only applies when several decode sessions run at once, because a single session cannot go faster than one engine. NVIDIA also states that on Blackwell GPUs, NVDEC doubles H.264 decoding throughput. Hardware decode can still become the limit itself. The ResNet-10 pipeline above stopped at 30 channels because the decoder saturated, not the compute. When one fleet both transcodes and analyses, NVDEC and NVENC serve both jobs. That layout is covered in combining GPU transcoding and analytics in one pipeline.
Profile the analytics pipeline before buying GPUs
A DeepStream-style pipeline is usually decode-bound, inference-bound or CPU-bound, depending on how heavy the models are. TechnoLynx profiles before optimising, so this diagnosis comes before any hardware decision. Generic advice about “what GPUs are good at” does not stand in for measuring your own mix.
Diagnostic checklist for bottleneck placement
- Measure throughput per stream end to end, from capture and decode to metadata, with rendering, on-screen display and tiling turned off. This matches how NVIDIA benchmarks DeepStream.
- Run
nvidia-smi dmonand check GPU, decoder, encoder and memory utilisation. A decoder near 100% points to a decode bottleneck, and GPU cores near 100% point to an inference bottleneck. - Run
htopand look for any core near 100%. That is the sign of a CPU-bound pipeline. - Check temperature and power draw, because NVIDIA notes that going past platform limits caps performance.
- Profile with NVIDIA Nsight Systems. DeepStream’s GStreamer plugins carry NVTX annotations, so the timeline shows the time taken per frame by each plugin.
- Turn on DeepStream’s frame and component latency measurement (
NVDS_ENABLE_LATENCY_MEASUREMENT=1,NVDS_ENABLE_COMPONENT_LATENCY_MEASUREMENT=1) to see where the latency builds up.
A full run of this method, with throughput, cost and accuracy measured together, is in the worked throughput, cost and quality example. The reasons to trust only numbers from your own workload are argued in benchmarking GPUs against your own workload. If parts of your analytics stack use OpenCL rather than CUDA on Linux hosts, the setup steps are in installing OpenCL on Ubuntu for video-analytics workloads.
What keeps returning to the CPU, even with GPUs in the rack
The first culprit is preprocessing. Kang et al. (PVLDB 14) measured ResNet-50 on an NVIDIA T4 at nine times the throughput of CPU-based image preprocessing. They found that preprocessing cost about 11 times as much as DNN execution on that AWS instance. In the same study, every model they listed was preprocessing-bound except the largest detectors (SSD ResNet at 1200 and Mask R-CNN). They also noted that optimising DNN execution alone, model compression included, will not improve end-to-end throughput once preprocessing is the bottleneck.
The second culprit is post-processing. In a case study of a multi-camera analytics pipeline, batching brought inference below 5 ms per frame, but pre- and post-processing together still took more than 15 ms per frame. The outputs had to move from GPU to CPU, where post-processing ran in Python-implemented code. Even after optimisation, steps such as connected-component analysis and bounding-box extraction stayed CPU-bound, and adding parallel jobs stopped helping once the CPU was saturated.
Kang et al.’s rule is to put as many operations as possible on the CPU when DNN execution dominates, and on the accelerator when preprocessing dominates. Where each stage should run is covered in depth in AI video analytics pipeline stages and placement. The symptom you will see first, a GPU sitting idle while the pipeline lags, is diagnosed in CPU/GPU stage mismatch and low GPU utilisation. Paying for GPU time while doing little useful work on it is the same pattern described in the hidden cost of GPU underutilisation, here inside a media pipeline.
Tuning below the model moves most of the cost
If a profiled pipeline is too slow or too expensive, the cheap fixes are usually in the serving path, not the network weights. NVIDIA’s DeepStream reference documentation lists several fixes that are config changes:
- Inference interval. Infer on every other frame and let a tracker predict object positions in between. Going from
interval=0tointerval=1can roughly double channel throughput. - Lower precision. FP16 needs no new model, only a change to
network-mode. INT8 needs a calibration cache. NVIDIA reports no significant accuracy loss from calibrated INT8 inference. - Batch matching. Set the streammux and primary-detector batch sizes to the number of input sources so the pipeline runs at full capacity.
- No unwanted scaling. Set the streammux resolution to the input resolution.
- No rendering you don’t use. Disable OSD, tiling and the output sink when only metadata leaves the box.
In one poultry-welfare monitoring case study, parallelisation, GPU-accelerated code, vectorised clustering and leaner post-processing halved end-to-end runtime. Cost per video on AWS g4dn.2xlarge instances fell from $0.691 to $0.312, a 54.8% cut, with minimal loss of accuracy and no model replacement. Stone et al. reached more than 4x inference throughput on a single GPU by batching active regions across feeds, at the cost of a 5% drop in average precision. They call that a worthwhile trade, while noting that whether it is acceptable depends on the application.
These gains do not simply add up. The same case study found that combining batched inference with multiprocessing caused GPU contention: 23 seconds with batching alone, against 25 seconds with both. Each change needs re-measuring against the profile, because two good changes can cancel each other out.
Server-side or edge analytics: how do we choose?
Server-side analytics fits when you already have cameras you want to keep, or when you need more than one or two simple detections, models you can improve over time, or accuracy a camera chip cannot reach. A central GPU can run larger models and multi-stage pipelines (detect, track, classify, correlate across cameras) that a small edge processor cannot. VIDIZMO’s summary of the trade-off: the more your cameras already exist, and the more you expect the analytics to grow, the more firmly you belong on the server side.
The edge wins on placement: bandwidth, latency and reach. The price is that each camera holds whatever models its chip can fit. You pay for a processor in every camera, busy or idle, and every model change means touching the whole fleet. On cost, a central pool lets a smaller amount of compute serve many feeds that are rarely all busy at once. The server route has costs of its own: streams have to cross the network, and the GPUs have to be sized to keep up. The network that carries those streams is part of that engineering. Cabling choices are covered in DAC vs fibre in GPU fabrics.
Hardware tiers follow from the same choice: Jetson-class devices for low stream counts on site, L4 or L40S for cost-efficient data-centre analytics, and H-class cards for the heaviest models or highest density. Before picking a tier, write down the latency the use case needs. Some monitoring tolerates minute-scale delay. The poultry-welfare pipeline in the case study above processes 5-minute segments each hour. A tight latency target pushes toward the edge, where latency is one of its strengths; a loose one leaves the server side open. To set that target properly, start from what real-time means in a video-analytics pipeline.
Who can profile our analytics pipeline before we commit to a GPU fleet?
TechnoLynx profiles a media pipeline’s analytics workload mix and says which functions justify GPU economics and which should stay on CPU, before a fleet-wide GPU rollout is approved. The engagement is a GPU Performance Audit scoped to video-analytics workloads. It starts from the deployed serving path and looks for cost and latency improvements below the model before any model replacement is recommended. A specialist is worth it when the bottleneck crosses decode, CPU post-processing and GPU scheduling at the same time, because that is where single-stage fixes stall. When choosing a partner, ask whether their recommendations come from profiles of your own streams.
The audit is not a fit for every reader. If your footage is sparse and event-triggered, Amazon Rekognition charges only for minutes processed, with no provisioning or minimum fee. If you have a few cameras running one light model, a CPU-only build may be enough, and profiling would mostly confirm that.
A broadcast workload shows where the line falls
The question this page leaves open is a specific one: which of your analytics functions survive the profile. Picture a broadcast operator running logo detection, scene classification and content moderation alongside transcoding, each function profiled on its own. The broadcast worked example follows one such workload through the same steps.
FAQ
Does hardware acceleration affect video quality?
NVIDIA reports no significant accuracy loss from calibrated INT8 inference. For encoding, NVIDIA says the UHQ mode for AV1 on Blackwell NVENC gives quality comparable to software encoding at higher frame rates.
