GPU transcoding gets bought on a frames-per-second headline and justified after the fact with a unit-cost story. The two numbers rarely agree, because the headline is measured at a concurrency and a quality target that your workload does not share. The honest version of this decision is narrower and more useful: GPU transcoding wins on cost-per-stream only above the concurrency at which the fixed-function encoder blocks saturate, and only with a ladder deep enough to amortise decoding each source once and encoding it many times. Below that line, you are paying an hourly rate for encoder capacity you cannot fill, and CPU transcoding is cheaper per stream.
That is a per-workload answer, not a per-vendor one. We have seen the same fleet come out GPU-correct for VOD backfill and CPU-correct for long-tail live channels, on the same hardware, in the same month.
What moves the crossover point?
Four variables do most of the work, and only one of them is the hardware.
Workload shape. Live and VOD behave differently under the same encoder. VOD backfill can queue work until every encode session on the accelerator is busy, which is exactly the condition GPU economics needs. A long-tail live channel arrives at whatever concurrency the schedule dictates — often one or two sessions per node — and the accelerator bills for the rest regardless.
Ladder depth. The GPU’s structural advantage in transcoding is decode-once/encode-many: pull the source into device memory once, then drive several rungs from it. A two-rung ladder barely exercises that; a seven- or eight-rung ladder with a resolution mix spanning mobile to 4K amortises the decode and the memory transfer across every output. Ladder depth is frequently the single largest swing factor in the calculation, and it is a product decision, not an infrastructure one.
How much wall-clock is actually in the encoder. This is the one teams skip. If scene-change analysis, packaging, DRM, subtitle muxing and object-store I/O consume a large share of the job’s wall-clock, accelerating the encode stage moves the total less than the encoder benchmark implies. Amdahl’s law applies to transcoding pipelines as brutally as to anything else. A job-level trace — per-stage wall-clock, GPU-seconds, queue wait — tells you whether the encoder is even the constraint. Our approach to job-level instrumentation for cost-per-minute attribution covers how to capture that before making a placement call.
Quality target. Hardware encoders (NVENC, Quick Sync, AMF) and software encoders (x264, x265, SVT-AV1 at slower presets) do not sit at the same rate-distortion point. A hardware encoder typically needs more bits to reach the same perceptual quality as a well-tuned software preset. So a comparison held at constant bitrate flatters the GPU and quietly ships a quality regression; a comparison held at constant quality-of-experience — VMAF at the rungs viewers actually consume — is the only one that survives review. Holding quality constant sometimes moves the crossover further than changing the accelerator does.
The rubric
Score your workload class — not your fleet — against each row. Every input here is something a profiling pass measures on your own content; none of it can be read off a vendor datasheet.
| Input | Measure | Favours CPU | Favours GPU |
|---|---|---|---|
| Sustained concurrent encode sessions per accelerator | Achieved sessions at your quality target, not theoretical | Well below fixed-function saturation | At or above saturation, sustained |
| Ladder depth and resolution mix | Rungs driven per decoded source | 1–2 rungs, narrow resolution range | Deep ladder, wide resolution mix |
| Encoder share of job wall-clock | Per-stage trace of a representative job | Packaging / I/O / analysis dominate | Encode stage dominates |
| Quality constraint | VMAF per rung, per device class | Tight quality floor at low bitrate | Quality floor reachable by hardware encoder within bit budget |
| Hourly rate exposure | Effective $/hour incl. idle | On-demand GPU, spiky arrival | Reserved or owned fleet, steady queue |
| Arrival profile | Concurrency over 24h, not peak | Bursty, unpredictable | Backlogged or schedulable |
Read it as a tally, not a formula. Four or more rows pointing the same way is a decision; a split tally is a mixed-fleet answer, and the mixed-fleet answer is legitimate more often than procurement expects.
The hourly rate is a variable, not a constant
The same accelerator has several prices, and the placement decision changes with each of them. An on-demand cloud GPU instance charges for idle encoder capacity at the same rate as busy capacity, which makes the utilisation requirement severe: if you cannot keep the encoder blocks fed, on-demand GPU is the most expensive way to transcode a sparse workload. A reserved instance or an owned fleet lowers the marginal cost of imperfect packing, which pulls the crossover down toward lower concurrency. An owned fleet also introduces the opposite risk — a procurement cycle for capacity that profiling would have shown sits idle, which is a cost the spreadsheet usually omits.
So the correct denominator is not the list hourly rate. It is the effective rate: hourly rate divided by achieved utilisation across the transcoding fleet over a real week. That is why GPU utilisation belongs in the secondary metric set alongside cost-per-stream. A fleet that reports 22% sustained utilisation has already answered the question.
What to measure before you decide
The rubric only scores if the inputs are measured on your workload. A profiling pass has to produce, at minimum:
- Achieved concurrent encode sessions per accelerator at your quality floor, sustained — not a burst figure from a synthetic clip.
- Per-stage wall-clock breakdown for representative jobs across your resolution mix, so the encoder’s share of the pipeline is explicit.
- VMAF (and PSNR only as a regression tripwire) per ladder rung, per device class, for both the hardware and software encoder paths at their respective operating points.
- Cost-per-stream, or cost-per-encoded-hour, at that held-constant quality, for each candidate placement.
- Concurrency over a full weekly cycle, so the arrival profile is characterised rather than assumed from peak.
- CPU-hours displaced per encoded hour, which is what makes the GPU case financially legible next to the existing fleet.
We run this as a scoped profiling sprint rather than a modelling exercise, because the numbers that matter are workload-specific: the Inference Cost-Cut Pack produces the measured throughput-per-accelerator and cost-per-stream figures this rubric is scored against. The profiling discipline behind it is the same inference-cost-audit methodology we apply to model-serving workloads — a transcoder is an inference pipeline with a different kernel in the middle.
Where the mixed fleet is the right answer
Teams resist the mixed answer because it is operationally more work: two encoder configurations, two quality-validation paths, two capacity models. That resistance is usually cheaper to overcome than the alternative. A catalogue backfill with a deep ladder and a schedulable queue is close to an ideal GPU workload. Forty low-concurrency live channels with a tight latency budget and a shallow ladder are not, and forcing them onto the accelerator to keep the architecture uniform buys uniformity with unit cost.
The practical shape we most often land on: GPU for the workload classes that can be packed to saturation, CPU for everything with an arrival profile too sparse to fill an encoder block, and a documented threshold — expressed in sustained sessions and ladder depth — that decides which bucket a new channel or catalogue drops into. The threshold is the deliverable. Without it, the next workload gets placed by whoever argued loudest last quarter.
Broadcast and streaming operators working through this alongside codec and ladder decisions will find the wider cost surface — egress, storage class, device-population decode coverage — mapped out in our media and telecom engineering work.
Frequently Asked Questions
What does ‘when GPU transcoding beats CPU transcoding on unit cost’ mean in practice, and how do we calculate the crossover for our own workload? Hardware accelerators flip the economics of video transcoding when utilization stays consistently high. It means finding the concurrency and ladder depth at which cost-per-stream on the accelerator drops below cost-per-stream on CPU, with quality-of-experience held constant. Calculate it by measuring achieved sustained encode sessions per accelerator at your quality floor, dividing the effective hourly rate (list rate ÷ achieved utilisation) by that throughput, and comparing against the same figure for your CPU fleet. It is arithmetic on measured inputs, not a published benchmark.
Which workload characteristics — live versus VOD, ladder depth, resolution mix, concurrency — move the crossover point most? Ladder depth and arrival profile usually move it most, because together they determine whether the fixed-function encoder blocks can be kept fed and whether the decode-once/encode-many pattern is exercised. VOD with a deep ladder and a schedulable queue sits well above the crossover; sparse live channels with a shallow ladder sit well below it. Resolution mix matters mainly through its effect on how much work each decoded source supports.
How do we compare GPU and CPU transcoding fairly when hardware and software encoders sit at different rate-distortion points? Hold quality-of-experience constant rather than bitrate. Fix a VMAF target per ladder rung per device class, then let each encoder use whatever bitrate it needs to reach that target, and compare cost at that operating point. A constant-bitrate comparison systematically flatters the hardware encoder and hides a quality regression that shows up in player metrics weeks later.
How many concurrent encode sessions does an accelerator actually sustain at our quality target, and how do we measure that rather than assume it? The sustained figure is typically below the burst figure in vendor material, because it is bounded by fixed-function encoder block count, memory bandwidth and your quality settings rather than by clip length. Measure it by ramping concurrency on representative content at your production encoder configuration until per-session throughput starts degrading, and hold that load long enough for thermal and driver behaviour to settle. Record the last concurrency at which every session still met its real-time or throughput obligation.
How does the GPU hourly rate (cloud instance versus owned fleet, on-demand versus reserved) change the answer? It changes the denominator. On-demand cloud instances bill idle encoder capacity at the busy rate, so they demand high utilisation to beat CPU; reserved instances and owned fleets lower the penalty for imperfect packing and pull the crossover toward lower concurrency. Always use the effective rate — hourly rate divided by achieved fleet utilisation over a real week — not the list price.
When is the correct answer a mixed fleet, with some workload classes on GPU and others left on CPU? Whenever the rubric splits, which is common in operators running both catalogue backfill and long-tail live. Place the workload classes that can be packed to encoder saturation on GPU, leave sparse or latency-constrained classes on CPU, and document the threshold — in sustained sessions and ladder depth — that assigns new workloads to a bucket.
What does a profiling pass have to measure before this decision can be made with confidence? Sustained sessions per accelerator at the quality floor, a per-stage wall-clock breakdown showing the encoder’s real share of each job, VMAF per rung per device class on both encoder paths, cost-per-stream at held-constant quality, weekly concurrency profile, and CPU-hours displaced per encoded hour. Anything missing from that list turns the decision back into an argument about datasheets.
Hardware breakeven math for transcoding workloads
Calculate utilization, amortize capital over three years, and factor cooling—only then does GPU vs. CPU resolve into a number. Revisit it when your workload shifts.
