The NVIDIA CUDA Toolkit is the versioned development kit you install to build CUDA applications. It contains the nvcc compiler, the CUDA runtime library, GPU-accelerated libraries such as cuBLAS, headers, and the debugging and profiling tools. The NVIDIA driver is a separate component. It lets the operating system talk to the GPU, and it ships and updates on its own schedule. Every CUDA workload needs a compatible driver. Running compiled CUDA applications needs only the driver and runtime libraries, not nvcc or the rest of the toolkit.
What the CUDA Toolkit contains, and what stays with the driver
NVIDIA describes the toolkit as a development environment for building GPU-accelerated applications, made up of GPU-accelerated libraries, debugging and optimisation tools, a compiler and a runtime. On Linux, the recommended cuda-toolkit package installs everything needed for native development: the compiler, debugger, profiler and math libraries, plus the visual profilers on x86_64. The development tools it ships include CUDA-GDB, Compute Sanitizer, the NVIDIA Visual Profiler and Nsight Visual Studio Code Edition.
nvcc is the toolkit’s CUDA compiler driver. A CUDA source file mixes C++ host code with GPU device functions. nvcc splits them apart, compiles the device code with NVIDIA’s own compilers into cubin and/or PTX, compiles the host code with whatever C++ host compiler is available, and embeds the device code in the host object as a fatbinary. On Linux, the gcc compiler is required to develop with the toolkit but not to run CUDA applications.
The CUDA driver API lives in a dynamic library (cuda.so or cuda.dll) that the device driver installs, not the toolkit. A useful way to hold the split: the toolkit is a build-time dependency you pin per project, and the driver is a property of the host that sets the ceiling for what can run on it.
The driver is no longer in the installer
None of the toolkit meta packages install or depend on the driver, and upgrading or removing one leaves the other alone. Windows made the same change earlier, from CUDA 13.1. One consequence on Linux: if the cuda meta package pulled in your driver as a dependency (CUDA 13.3 and earlier), upgrading to 13.4 or later leaves that driver installed but no longer required by any package. Before you run a package clean-up on a machine upgraded that way, check what it would remove. Install a compatible driver yourself, separately. Our guide to NVIDIA Linux driver installation for AI workloads covers that step, and choosing between NVIDIA’s open GPU kernel modules and the proprietary CUDA driver is a separate decision.
CUDA download: where to get the toolkit, including older versions
Current releases come from NVIDIA’s CUDA downloads page, where you pick your platform and download the toolkit. Older releases are on the CUDA Toolkit Archive, which lists previous toolkits along with their versioned online documentation. When this was written, the latest release on the archive was CUDA Toolkit 13.4.2 (September 2026), and the list goes back to CUDA Toolkit 1.0 (June 2007). The archive also carries old developer drivers, and NVIDIA tells you to check its driver site for more recent production drivers for your hardware. Pull an archived toolkit when a project or framework pins a specific version, not just to avoid upgrading the driver. Under the backward-compatibility rule below, a newer driver also runs applications built with older toolkits.
Installing on Linux
NVIDIA’s Linux guide lists four methods: package manager (RPM and DEB, recommended for most users), runfile (a standalone installer that works on any distribution), Conda, and pip wheels. The distribution packages are recommended where possible, because they work through the native package manager and the runfile does not. The toolkit is only supported on distribution versions qualified for that release, since it depends closely on the host compiler and C runtime.
| Method | Command | What you get |
|---|---|---|
| Debian/Ubuntu packages | apt update then apt install cuda-toolkit |
Full toolkit through APT |
| RHEL/Fedora packages | dnf install cuda-toolkit |
Full toolkit through DNF |
| Runfile | sh cuda_<version>_linux.run |
Interactive ncurses installer |
| Conda | conda install cuda -c nvidia |
All toolkit components in a Conda environment |
| pip wheels | per-component wheels | Runtime only; no developer tools by default |
By default the toolkit installs to a versioned directory such as /usr/local/cuda-13.4, and the /usr/local/cuda symlink points to whichever version was installed, so projects can follow the latest toolkit without changing their configuration. If nvcc: command not found comes back after a reboot, check whether the PATH was set in ~/.bashrc, which only interactive shells read. IDEs, build systems and cron jobs need the variables set globally, for example in /etc/environment. Several toolkits can sit side by side in their own versioned paths.
Installing on Windows
On Windows you need a CUDA-capable GPU, a supported version of Windows and a supported version of Microsoft Visual Studio before the toolkit itself. Install the driver first from NVIDIA’s driver site, then the toolkit. Choose between the Network Installer, which downloads only the components you select, and the Local Installer, a single large download. Skip the driver and, according to one third-party install guide, the install still reports success, but nvidia-smi and every CUDA program then report that no device was found. The same guide dates the Windows unbundling to CUDA 13.0. NVIDIA’s own Windows guide and release notes say 13.1, and we go with the vendor documentation.
Installing in WSL2
Never install a Linux NVIDIA driver inside the WSL distribution, because the Windows host already provides the driver and a Linux driver package overwrites it and breaks CUDA. Inside WSL2, install only the versioned cuda-toolkit-* package, not cuda-drivers or the unversioned cuda meta package.
Installing in containers
With containers, the host needs only the NVIDIA driver and the NVIDIA Container Toolkit, not the full CUDA Toolkit. A peer-reviewed study of performance problems in deep learning systems (Cao et al., 2021) ran its experiments the same way: only the GPU driver on the physical machine, and a separate CUDA Toolkit version in each Docker container. NGC images and the official nvidia/cuda Docker images give you a working nvcc without touching the host. NVIDIA’s NGC framework containers ship a CUDA Toolkit, cuDNN and framework that have been version-matched and validated together. Their image tags do not tell you the PyTorch or CUDA versions inside. Read the release notes or the Frameworks Container Support matrix for those.
Checking which CUDA toolkit version is installed (and why nvidia-smi disagrees)
nvidia-smi and nvcc report different CUDA version numbers. Both are correct, because they report different things. The “CUDA Version” field in nvidia-smi is the highest toolkit version the installed driver can run. It is not the toolkit you have installed. nvcc --version reports the toolkit that is actually installed. HPC sites document the same point: on the Bristol Centre for Supercomputing’s Isambard-AI, the version nvidia-smi shows is the driver’s maximum and does not reflect the toolkit loaded through modules.
| Check | Command | What it tells you |
|---|---|---|
| Installed toolkit | nvcc --version (Linux) or nvcc -V (Windows) |
The toolkit compiler on your PATH; if a version prints, the toolkit is installed |
| Driver ceiling | nvidia-smi |
Driver version and the highest CUDA version it can run |
| Active toolkit among several | update-alternatives --display cuda |
Which installed version /usr/local/cuda points to, and the alternatives |
| End-to-end function | build and run deviceQuery from cuda-samples |
Confirms the driver and toolkit can enumerate and talk to the GPU |
So if nvidia-smi shows a lower CUDA version than the toolkit you just installed, update the driver, not the toolkit. The samples need care as well. Since CUDA 11.6 they ship on GitHub, not inside the toolkit, and an old 12.x samples tag fails against a 13.3 toolkit on cudaDeviceProp fields removed in CUDA 13.0. That failure looks like a driver problem but is not one, so use the sample tag that matches your toolkit.
Which CUDA Toolkit version is compatible with my driver and GPU?
Start from one rule. A newer driver runs applications built with an older toolkit, because the driver API is backward compatible. Each toolkit sets a minimum driver version. From CUDA 11 onwards, minor version compatibility lets an application built with any toolkit in a major family run on that family’s minimum driver, with a limited feature set.
| Toolkit family | Minimum driver under minor version compatibility |
|---|---|
| CUDA 13.x | 580 or later |
| CUDA 12.x | 525 or later (newer drivers still work through backward compatibility) |
| CUDA 11.x | 450 or later (newer drivers still work through backward compatibility) |
For the current release, existing CUDA 13.x applications run on drivers 580 or later, but CUDA 13.4’s new features and newly enabled platforms need an R615 or later driver. NVIDIA’s release notes also list the driver branch paired with each toolkit release, and update releases within a minor version share that branch. Before CUDA 11, each CUDA 10.x minor release required the driver packaged with it.
The “limited feature set” has three concrete consequences:
- A feature that spans both the toolkit and the driver can fail on an older driver with
cudaErrorCallRequiresNewerDriver. The fix is a driver upgrade, or code that checks whether the feature is available. - Applications that compile device code to PTX will not work on older drivers.
- Minor version compatibility requires passing a target architecture to nvcc, for example
nvcc -arch=sm_xx.
Running a newer toolkit on an older driver
Forward compatibility goes further. The cuda-compat-<major>-<minor> package lets applications built with a newer toolkit run on an older base driver, even across major families, subject to platform and GPU support. The package is named after the highest toolkit it supports. On the 535 driver with a 12.5 application, for example, you install the 12.5 compatibility package. It works by putting the compat libcuda ahead of the system libcuda.so in LD_LIBRARY_PATH. The package only supplies the libraries and does not configure the system to find them. Isambard-AI describes it as a feature for NVIDIA’s data centre GPUs. NVIDIA’s forward-compatibility guide adds that New Feature Branch drivers are not supported targets.
One caveat concerns the upgrade path. During a full system upgrade that installs both toolkit and driver, remove any forward-compatibility packages, because older compat packages are not supported on new drivers. Module scripts that load a compat package should check first that it still applies. NVIDIA says this flexibility matters where driver updates follow different qualification schedules from toolkit updates.
The GPU side of the match
The driver is half of the question. The other half is the GPU’s compute capability, which defines the hardware features and instructions of each NVIDIA architecture. NVIDIA publishes a table mapping each GPU to its compute capability. Toolkits and drivers can deprecate and drop support for GPU architectures over their life cycle, and TensorFlow 2.3, for example, does not support compute capability 6.1 GPUs. In the Cao et al. (2021) study, mismatches between hardware and library versions caused 1.8% of the performance problems analysed, and mismatched library versions caused 4.0%. cuDNN publishes its own support matrix across toolkit, driver and hardware versions, and see how cuDNN relates to CUDA. For what the version number actually constrains, see CUDA compute capability explained. For all the axes together, the four-axis CUDA compatibility matrix is the planning view.
Resolving a CUDA runtime and driver version mismatch
One mismatch cause is an application built with a CUDA runtime newer than the installed driver can serve. Check these in order:
- Compare the “CUDA Version” in nvidia-smi with the CUDA version of the application or framework build. If the application’s version is higher, upgrade the driver.
- If you cannot upgrade the driver on a data centre GPU, use the forward-compatibility package named for the toolkit you need.
- If the error is
cudaErrorCallRequiresNewerDriver, the code uses a feature that needs a newer driver, even inside a compatible family. - On Windows, if the toolkit installed but no device shows up, the driver is probably missing.
- If a sample or old project fails to build on 13.x, check for removed API fields before you suspect the driver.
- If driver and toolkit match but the GPU is rejected, check whether the toolkit or framework still supports its compute capability.
Do I need the CUDA Toolkit to run PyTorch?
Usually not. An answer on the PyTorch Forums explains that the pip and conda binaries ship with every CUDA dependency they need, so a locally installed toolkit goes unused and only a compatible driver is required. You need the local toolkit when you build PyTorch from source or compile a custom CUDA extension.
Runtime API or driver API: when you need the cu* calls
The runtime API (the cuda* functions) and the driver API (the cu* functions) are very similar, and for the most part either can be used, but they differ in a few important ways. The runtime is built on top of the driver API. It initialises the context and loads modules implicitly, which keeps code simpler but gives up control. The driver API has to be initialised with cuInit(), gives explicit control over contexts, and lets you keep only the modules you currently need loaded or reload them at run time, whereas the runtime loads every kernel at start-up and keeps them loaded. It also works only with cubin objects, which makes it language-independent.
So when do you need the driver API? Most applications reach full performance without ever calling it. The cases that justify it:
- Plug-ins in a shared process. Runtime users in one process share a primary context. If one plug-in calls
cudaDeviceReset(), it destroys the context the others depend on. Creating and setting contexts through the driver API, then working in them through the runtime, avoids this, though every extra context costs memory and switching overhead. - Dynamic module management. The driver API lets code keep only the modules it needs loaded, or reload them dynamically.
- Interfaces that land in the driver API first. New interfaces sometimes appear in the driver API before the runtime, and CUDA Green Contexts, which partition a GPU’s resources spatially, are a driver API feature.
Mixing the two APIs is supported. If a context is made current through the driver API, later runtime calls use it, and cuDevicePrimaryCtxRetain() gives driver code access to the runtime’s primary context. That matters when driver API code sits alongside runtime-based libraries such as cuBLAS or cuFFT.
CUDA Toolkit tools for profiling kernel performance
The toolkit’s profiler for kernels is Nsight Compute, an interactive kernel profiler with both a UI (ncu-ui) and a command-line tool. It collects detailed performance metrics and can compare results against a saved baseline. A runfile install puts it under /usr/local/cuda-<cuda-version>/nsight-compute-<version> by default. In the Nsight Systems timeline, find a kernel on the critical path, then profile that launch in Nsight Compute. Compute Sanitizer and CUDA-GDB also ship in the toolkit.
NVIDIA’s profiling guide has two points that are easy to miss. Nsight Compute serialises kernel launches in the profiled application. And judge a GPU performance optimisation by absolute workload duration, not by utilisation percentages, because those can move in either direction when the total work changes. The guide also says to re-evaluate the dominant limiter on each new GPU architecture, since cache sizes, SM counts and bandwidth change which part of a workload is limiting. TechnoLynx’s GPU performance engineering team turns profiles like these into faster kernels and pipelines.
Two versions to record
Version mismatches are easy to fix once you can see them, so record two versions for every environment: the driver, read from nvidia-smi as the ceiling, and the toolkit or bundled runtime the code was built with. They ship separately and change on different schedules, which is why treating them as one number is the most common reason a result cannot be reproduced on a stack that looks similar. With the driver no longer bundled in the installer, you can no longer get away with ignoring that split.
