cuDNN and CUDA are not two options you choose between. They are stacked: CUDA is the general-purpose GPU compute platform and toolchain, and cuDNN is a closed-source library of hand-tuned deep-learning primitives — convolution, pooling, normalisation, and attention-adjacent kernels — that runs on top of it. You cannot run cuDNN without CUDA, and no team “migrates from CUDA to cuDNN”.
The comparison question that actually matters is different, and it is the one we ask first on GPU work: how much of your workload’s performance comes from vendor-tuned library kernels rather than from code your team wrote?
What does cuDNN vs CUDA mean in practice?
| CUDA | cuDNN | |
|---|---|---|
| What it is | Compute platform, driver interface, compiler toolchain, language extensions | Library of pre-tuned deep-learning kernels |
| Who writes the kernel | You (or another library) | NVIDIA, closed source |
| Typical hot-path role | Custom ops, pre/post-processing, fused glue code | Convolution, pooling, batch norm, RNN/attention-adjacent primitives |
| Needs the other to run | No | Yes — cuDNN depends on CUDA |
| Portability under OpenCL / SYCL / ROCm | Portable in principle; CUDA-specific memory patterns rarely translate for free | No direct equivalent library to port to |
In a typical PyTorch or TensorFlow training loop, most of the convolution and normalisation wall-clock time is spent inside cuDNN kernels the team never wrote and cannot read. That is usually a good thing — those kernels are hard to beat — but it changes what “we use CUDA” means.
Why the distinction changes your lock-in estimate
Teams that never separate we use CUDA from we depend on cuDNN mis-estimate their lock-in surface in both directions. A workload whose hot path is 90% cuDNN calls has a very different portability profile from one built on custom CUDA kernels: the former leans on a vendor library with no direct equivalent under OpenCL or SYCL, while the latter can at least be ported — though, as we cover in our wider treatment of GPU acceleration and API choice, porting is not the same as porting performantly, because CUDA-specific memory and launch patterns do not translate for free.
There is a second cost that shows up later: version coupling. The cuDNN release, CUDA toolkit, driver, and framework build form a version matrix that pins container images and quietly blocks upgrades. We see this pattern regularly — an otherwise routine framework bump stalls because the cuDNN/driver pairing underneath it is fixed.
The measurement that settles the argument
Read the split from profiler traces per workload: GPU wall-clock time inside cuDNN kernels versus time inside team-authored CUDA kernels. That single number bounds two things at once — the realistic scope and cost of any future API migration, and the optimisation headroom you actually own. It also surfaces the pleasant case: a hand-rolled kernel that a cuDNN call would beat, where throughput comes back at essentially zero engineering risk.
Do you know that split for your heaviest training job, or are you estimating it from the fact that the code says import torch?
