A CUDA compute capability is a two-part version number, X.Y, that every NVIDIA GPU has. It records which features the GPU supports and some of its hardware parameters. For AI work, the number decides whether a compiled kernel will load on the card, which toolkit first supports it, and which library features apply to it. It does not tell you whether you should be writing CUDA in the first place. Which is better, OpenCL or CUDA is a separate question. That one turns on which hardware vendors you need to reach and how much measured performance portability costs you.
Reading the X.Y number
The major version is the number before the dot and the minor version is the one after it, so CC 12.0 has major version 12 and minor version 0. A cubin compiled for one major version will not load on a GPU of another.
The number is more than a label. Instruction throughput per clock cycle depends on the compute capability of each NVIDIA GPU, an input to theoretical peak performance calculations. In build files, compute_XY names a PTX target and sm_XY names a cubin (binary) target.
How do I check my GPU’s compute capability?
The command-line check is nvidia-smi. This command prints each GPU’s name and compute capability:
nvidia-smi --query-gpu=name,compute_cap --format=csv
Inside a program, you can read the same values through the CUDA Runtime API (cudaDeviceGetAttribute), the Driver API (cuDeviceGetAttribute) or NVML (nvmlDeviceGetCudaComputeCapability):
#include <cuda_runtime_api.h>
int major, minor;
cudaDeviceGetAttribute(&major, cudaDevAttrComputeCapabilityMajor, device_id);
cudaDeviceGetAttribute(&minor, cudaDevAttrComputeCapabilityMinor, device_id);
Which binaries run on which GPU
A CUDA application can carry GPU code in two forms: cubin, which is finished machine code for a given compute capability, and PTX, a forward-compatible intermediate assembly. When a kernel launches, the CUDA Runtime checks the GPU’s compute capability and looks for the best match. It uses a compatible cubin as-is if one exists. If not, it JIT-compiles the PTX. If the binary contains neither, the launch fails.
Compatibility rules at a glance
| The binary contains | It runs on | Trade-off |
|---|---|---|
| cubin built for CC X.Y | GPUs with the same major version X and a minor version of Y or higher (a cubin for 8.6 runs on 8.6 and 8.9) | No JIT cost, but it is limited to one major version |
| cubin built for one major version, loaded on another | Nothing: a cubin for 8.6 will not load on a 9.0 GPU | None. The kernel simply fails to launch |
PTX for compute_XY |
Any GPU with that compute capability or higher, after JIT compilation by the driver (PTX for compute_80 can be compiled for sm_120) |
Longer application load time, but it picks up compiler improvements from newer drivers and runs on GPUs that did not exist at build time |
| cubin for CC 10.0 or later | Compatible GPUs in the same family, through driver-managed “off-target finalisation” | The driver does a just-in-time finalisation step on the target GPU |
Code built with the a suffix (e.g. sm_100a) |
Only the exact compute capability it was compiled for | Full architecture-specific feature set, no portability |
Code built with the f suffix |
GPUs with the same major version and an equal or higher minor version | Family-specific features only, a subset of the architecture-specific set |
A build that ships only cubins fails to launch on a GPU none of them is compatible with. A build that ships PTX keeps running on newer GPUs, at the cost of longer application load time. NVIDIA’s guidance resolves this: build a cubin for each architecture you know the code will run on, then embed PTX for the newest architecture available for forward compatibility. NVIDIA’s own example targets CC 8.0, 9.0 and 10.0:
nvcc -gencode arch=compute_80,code=sm_80 \
-gencode arch=compute_90,code=sm_90 \
-gencode arch=compute_100,code=sm_100 \
-gencode arch=compute_100,code=compute_100 \
-c kernels.cu
CUDA toolkit version vs compute capability: CUDA 12, CUDA 13 and your GPU
The CUDA toolkit version and the compute capability are separate numbers. The compute capability describes the GPU’s supported features and some of its hardware parameters. The toolkit version (CUDA 12.8, CUDA 13.4 and so on) covers a set of independently versioned components such as NVCC.
Three rules link the toolkit to the driver. First, each toolkit requires a minimum NVIDIA driver version. Second, the driver is backward compatible, so a newer driver runs applications built with older toolkits. Third, the driver reports the highest CUDA version it supports, and it can run applications built with toolkits up to that version. This is why the “CUDA Version” in the nvidia-smi header is a ceiling set by the driver, not the toolkit you have installed. Updates within a CUDA minor version share one driver branch.
Support for an architecture also starts with a particular toolkit. NVIDIA’s matrix lists Ampere (CC 8.0 and 8.6) as supported from CUDA 11.0 onwards and Blackwell (CC 10.0 and 12.0) from CUDA 12.8 onwards. The same document warns that toolkits and drivers may deprecate and drop GPU architectures over a toolkit’s life cycle. Before you move a fleet from a CUDA 12.x toolkit to a CUDA 13.x toolkit, check the matrix for your oldest card’s compute capability, not only the driver branch.
What compute capability do Blackwell GPUs have, and what does sm_120 mean for building AI code?
According to NVIDIA’s architecture matrix, Blackwell GPUs report compute capability 10.0 or 12.0, and the first toolkit that supports them is CUDA 12.8. The CUDA 12.8 release added compiler support for three Blackwell targets: SM_100, SM_101 and SM_120. In practice, sm_120 is a Blackwell target that the compiler supports from CUDA 12.8. nvcc from CUDA 11.8 to 12.7 generates native cubins up to Hopper (CC 9.x).
Older binaries that include PTX are not stranded. NVIDIA’s Blackwell compatibility guide states that applications built with CUDA toolkits 2.1 through 12.8 are compatible with Blackwell GPUs as long as they include PTX versions of their kernels. A further change sits under the hood. From compute_100 onwards, compilation uses a newer NVVM IR dialect based on LLVM 18.1.8, which differs from the dialect used for earlier architectures.
The a and f suffixes arrived with Hopper and Blackwell. Architecture-specific features arrived with CC 9.0 and require the a suffix. CUDA 12.9 added family-specific features for Blackwell with the f suffix, guaranteed across GPUs of the same major version and higher minor versions. NVIDIA is explicit that these suffixes are not a magic optimisation technique. You only get the features if you write PTX directly or call a library that does.
Checking a Blackwell machine before you debug the model
- Query the compute capability with
nvidia-smi --query-gpu=name,compute_capto confirm whether the card is 10.x or 12.x. - Confirm that the build toolkit is CUDA 12.8 or later if you need native Blackwell cubins.
- List what the binary carries (
cuobjdump --list-elfand--list-ptx). If it has neither ansm_120cubin nor PTX, the kernel launch will fail. - Check library release notes for your exact compute capability, because they list support and known issues by CC.
Which compute capability features matter when choosing a GPU for AI?
For AI workloads, the features that matter are the numeric formats the tensor cores accelerate and whether your libraries use them on your exact compute capability. NVIDIA lists Blackwell as supporting FP64, FP32/TF32, FP16/BF16, INT8/FP8, FP6 and FP4. NVIDIA introduced NVFP4, a 4-bit floating-point format, with the Blackwell architecture. NVIDIA states that NVFP4 cuts model memory footprint by roughly 3.5x relative to FP16 and 1.8x relative to FP8. It also reports 1% or less accuracy degradation against FP8 on DeepSeek-R1-0528 benchmarks. Both figures are NVIDIA’s own claims.
The harder part is that library support differs between compute capabilities of one architecture. Library releases are where that shows.
| Feature | Where NVIDIA’s documents place it | What to check |
|---|---|---|
| FP8 matrix multiplication | cuBLAS support for FP8 on Ada GPUs, per the CUDA 12.8 notes | Whether your framework’s kernels use it, not only whether cuBLAS exposes it |
| Micro-scaled 4-bit and 8-bit tensor-core GEMM | cuBLASLt, CC 10.0 and higher, from CUDA 12.8 | The toolkit version on your build machine |
| NVFP4 grouped GEMM | Experimental cuBLASLt API, CC 10.x and 11.0, in CUDA 13.4 Update 2 | CC 12.x is not in that list, so a Blackwell card is not automatically an NVFP4 card for every path |
| Grouped GEMM with FP8, FP16, BF16 inputs | Requires CC 10.x or 11.0 | Same caveat for 12.x GPUs |
| Known strided batched GEMM issue | CC 12.0 and 12.1, introduced in CUDA 13.0 Update 2 | Known-issues section of the toolkit you deploy |
Sustained throughput at the precision you will actually deploy tells you more than a format list, and the compute capability’s job is to tell you which kernels are even eligible to run. How to read hardware benchmark results is a measurement question in its own right. TechnoLynx’s GPU performance engineering work profiles and tunes workloads to run faster and cheaper on the target hardware.
Compute capability beyond NVIDIA: the portability routes for AI code
CUDA runs only on NVIDIA GPUs. It is not an open standard, and NVIDIA can change its capabilities and interface at its own discretion. Among the seven programming models compared in a 2024 performance-portability study (arXiv 2402.08950), CUDA was the only one that did not support both NVIDIA and AMD devices. CUDA compute capability is therefore an NVIDIA-only concept, and what makes a GPU CUDA-capable in the first place is a question about NVIDIA’s architecture.
The portability routes each fit a different starting point.
- OpenCL is the Khronos open standard for heterogeneous CPUs, GPUs and other devices. It offers a low-level hardware abstraction aimed at expert programmers who want portable but efficient code. It also has its own version number (1.0 to 3.1). That number describes the specification, not the hardware.
- HIP is AMD’s C++ runtime API and kernel language for AMD GPUs, and it closely mirrors CUDA. The HIPIFY tools translate CUDA source automatically.
hipify-clangneeds a working CUDA install and correct code, whilehipify-perluses pattern matching and needs neither. HIP replaces CUDA’s compute-capability checks (__CUDA_ARCH__) with feature-specific macros, which removes the need to know which CC supports which feature. Because AMD also supplies a HIP implementation on top of CUDA, HIP code can run on NVIDIA hardware as well. - SYCL is a Khronos open standard for heterogeneous programming in single-source ISO C++. A 2023 study (arXiv 2309.09609) names Intel oneAPI as the most mature SYCL implementation.
A comparison of the CUDA, OpenCL and SYCL compute APIs covers workload-by-workload selection, and Vulkan compute is weighed in choosing between Vulkan, OpenCL, SYCL and CUDA.
Which is better, OpenCL or CUDA, for an NVIDIA-first AI stack?
On NVIDIA hardware, CUDA is the native and more fully featured choice. OpenCL’s case rests on reaching hardware CUDA cannot reach, and even there it does not guarantee that performance carries over. Across the 2024 performance-portability study, CUDA was the best model, or within 3% of the best, in eleven of fifteen cases. CUDA and HIP expose more GPU features than OpenCL, a committee-run specification that GPU makers cannot extend on their own.
The measured gaps vary by workload. A 2010 astrophysics paper (Barsdell, Barnes and Fluke) cites one study measuring identical CUDA and OpenCL performance on an NVIDIA Tesla GPU. It cites another where the gap depended on problem size, up to 70% and falling to 10-20% as the simulation grew more complex. The same authors said they favour general, long-lived code over the last, say, ten per cent of speed. A Futhark comparison on A100 and MI100 explains where the gaps come from. OpenCL lacked the memory-model and progress guarantees needed for a single-pass scan, so it fell back to a slower two-pass version. Where OpenCL looked faster, the cause was in almost all cases an unfair comparison: OpenCL defaults to less precise single-precision division and square roots, which gave it a 1.71x lead on one MI100 benchmark until correct rounding was forced. The same author found no sign that NVIDIA sabotages OpenCL code generation. NVIDIA’s OpenCL simply stopped gaining new features.
On NVIDIA GPUs, CUDA and SYCL performed comparably in the 2023 Smith-Waterman study, and SYCL also ran on AMD and Intel GPUs with higher architectural efficiency in 3 of 4 cases. The same authors warn that SYCL is still immature and their results do not generalise beyond the application and compilers they tested. The 2024 study found that SYCL achieved performance portability most consistently of the models tested.
| Your situation | Reasonable choice |
|---|---|
| Existing CUDA codebase, NVIDIA-only deployment | Stay on CUDA. Native vendor models were usually the most performant on their own hardware |
| Existing CUDA codebase, AMD targets added | HIP via HIPIFY, keeping one API close to CUDA |
| New code that must run on NVIDIA, AMD and Intel | SYCL, with performance validated per target |
| Hardware where OpenCL is the main API, such as Intel Xe GPUs | OpenCL |
| Expectation that portable code will perform equally everywhere | No model guarantees that. OpenCL programs may need significant changes to perform on each platform |
Portability work across these toolchains pays off most when scoped against a measurable trigger: a performance ceiling, a deployment-target constraint, or a lock-in cost.
The compute capability still decides one thing even in that case. If NVIDIA stays in the fleet, HIP code still runs on those cards through CUDA, so CUDA’s binary rules apply. Do the compatibility checks above again whenever a new architecture arrives, whatever language the kernels were written in.
