A 4,096-stream-processor AMD card is not 33% stronger than a 3,072-CUDA-core NVIDIA card. The two numbers count lanes inside different scheduling hierarchies, so subtracting or dividing them tells you almost nothing about achievable FLOPs on your workload. Both figures are vendor marketing aggregates, and they aggregate over different things.
What is the difference between a stream processor and a CUDA core?
A CUDA core is a lane inside an NVIDIA streaming multiprocessor (SM), scheduled in warps of 32 threads. A stream processor is a lane inside an AMD compute unit, scheduled in wavefronts — historically 64 work-items, 32 on RDNA. That difference in scheduling granularity propagates into everything that determines real throughput: how many threads must be resident to hide memory latency, how much register file each resident thread can claim before occupancy drops, and how badly a divergent branch costs you.
Neither number is a lie. Both are lane counts. The error is treating them as a common unit of compute, the way one might compare two CPUs’ clock speeds — and even that comparison stopped working two decades ago.
Why lane counts don’t predict your speedup
Occupancy is the mechanism. A kernel that needs 96 registers per thread will hit a register-file ceiling and leave a large fraction of those advertised lanes idle, regardless of vendor. The ceiling arrives at a different point on a 32-wide warp machine than on a 64-wide wavefront machine, because the scheduling quantum is different. We see the same pattern in audits regularly: a card with more lanes on paper delivers lower effective occupancy on the customer’s actual kernel, and the paper 2x advantage measures out at something closer to 1.1x.
The two numbers that do carry signal are achieved FLOPs and memory bandwidth utilisation, measured on a representative kernel from your own workload. If your kernel is bandwidth-bound — and most inference and most image-processing pipelines are — lane count is close to irrelevant, and the DRAM path is the thing you are actually buying.
What to compare instead of core counts
| Instead of | Measure | Why it matters |
|---|---|---|
| CUDA core / stream processor count | Achieved FLOPs on your kernel | Captures occupancy and divergence losses that lane counts hide |
| Peak theoretical bandwidth | Measured bandwidth utilisation (%) | Bandwidth-bound kernels are decided here, not in the ALUs |
| Vendor TFLOPS headline | Cost per unit of delivered throughput | Turns a spec comparison into a procurement decision |
| Lane width assumptions | Warp (32) vs wavefront (32/64) behaviour | Determines register pressure and latency-hiding headroom |
| Generic “AI performance” | Tensor core / matrix core path utilisation | Low-precision matrix units sit alongside the lanes, not inside the count |
Tensor cores and AMD matrix cores deserve a note here, because they are a separate execution path rather than a subset of the lane count. For ML inference at FP16, BF16 or INT8, most of the useful arithmetic runs on those units — which is why an ML throughput comparison based on CUDA core counts can be wrong by an order of magnitude, in either direction.
The API decision rides on this
The scheduling model you tune for is the one your memory access patterns get baked into. Coalescing rules, shared-memory bank conflict avoidance, and unrolling factors chosen for a 32-thread warp do not automatically remain optimal on a wavefront machine, even under a translation layer such as HIP or an OpenCL/SYCL port. Functional portability and performance portability are separate problems, and the second one is where the schedule slips. We work through the trade-offs in more depth in our GPU optimisation and acceleration work, where the API choice and the hardware choice are treated as one decision rather than two.
Before signing a hardware order on a lane count, run one representative kernel on both candidates and record achieved FLOPs, bandwidth utilisation, and occupancy. If you cannot get access to the silicon to do that, the honest position is that you do not yet know which card is faster for your code — which is a better place to stand than a confident conclusion drawn from a spec sheet.
Which of your kernels is actually arithmetic-bound, and do you have the measurement to prove it?
