To check whether TensorFlow can see your GPU, run tf.config.list_physical_devices('GPU'). If it returns a list of GPU devices, the runtime has found the hardware. That only proves the GPU was detected. It does not show which device your operations run on. To confirm your operations actually execute on the GPU, turn on device-placement logging as the first statement of your program. When the list comes back empty, check three things in order: whether the build was compiled with CUDA, whether the driver, CUDA and cuDNN versions line up, and whether your platform supports the standard CUDA build at all. Once detection is settled, NVIDIA Nsight Systems and Nsight Compute analyse overall application structure and individual CUDA kernels.
How do I check if TensorFlow is actually using my GPU?
The detection test is a one-liner from TensorFlow’s own installation guide:
python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"
list_physical_devices returns the physical devices visible to the host runtime, and it can be called before the runtime initialises, which gives you a chance to call other configuration APIs first. One subtlety: the number of GPUs available to the runtime can change during initialisation, because devices may be marked invisible or split into several logical devices.
Detection and use are different questions. To see where work runs, put this before anything else executes:
import tensorflow as tf
tf.debugging.set_log_device_placement(True)
a = tf.random.normal([1024, 1024])
b = tf.matmul(a, a)
With logging enabled, any tensor allocation or operation is printed with its device placement. Logging also covers ops inside a tf.function when the function is called. In the log, check that the operations that matter were placed on a GPU device.
Why “no error” does not mean “on the GPU”
TensorFlow and Keras models run on a single GPU transparently, with no code changes. The flip side is that, to find out which device ran an operation, TensorFlow points you to placement logging. When an operation has both CPU and GPU kernels, the GPU is prioritised; tf.matmul lands on GPU:0 unless you request otherwise. If soft device placement is enabled, TensorFlow will also pick an existing supported device when the one you specified does not exist.
Two other habits mislead people. First, nearly full GPU memory is TensorFlow’s default: it maps nearly all GPU memory of every visible GPU, to reduce fragmentation. Second, on multi-GPU machines the GPU with the lowest ID is selected by default, so a job can be “on the GPU” while the card you meant sits idle. tf.config.set_visible_devices restricts TensorFlow to the GPUs you choose.
Was my TensorFlow build compiled with CUDA support?
tf.test.is_built_with_cuda() returns whether TensorFlow was built with CUDA (GPU) support. If it returns False, no driver or toolkit fix will help: you have a CPU-only binary and need a different package.
tf.sysconfig.get_build_info() reports more build detail. It returns a dictionary with keys such as cuda_version, cudnn_version and is_cuda_build (plus DLL names on Windows), although TensorFlow notes that the keys and values may change across versions and platforms.
Which TensorFlow version do I have?
tf.version.VERSION returns the installed version string, for example '2.16.1' on that release. tf.__version__ gives the same answer and is the check TensorFlow’s build guide uses to verify an installed package.
What each check proves
| Check | Call | What it proves | What it does not prove |
|---|---|---|---|
| Version | tf.version.VERSION |
Which TensorFlow release is installed | That the release supports GPU on your platform |
| CUDA build | tf.test.is_built_with_cuda() |
The binary was built with CUDA support | That a GPU is present or usable |
| Build info | tf.sysconfig.get_build_info() |
Build details such as cuda_version and cudnn_version |
That your installed driver is compatible |
| Detection | tf.config.list_physical_devices('GPU') |
The runtime sees a physical GPU | That operations run on it |
| Use | tf.debugging.set_log_device_placement(True) |
Which device each op and tensor was placed on | That the GPU is busy or efficient |
Why does TensorFlow not detect my GPU even though the NVIDIA driver is installed?
TensorFlow’s pip installation guide lists NVIDIA drivers 525.60.13 or newer on Linux (528.33 or newer for WSL on Windows), CUDA Toolkit 12.3 and cuDNN SDK 8.9.7 as the GPU requirements for the release it documents. Those numbers belong to that release, so read the page for the version tf.version.VERSION reports rather than copying them from an old answer.
How do I confirm the driver, CUDA, cuDNN and TensorFlow versions are compatible?
Work from both ends. get_build_info() reports the build’s cuda_version and cudnn_version. Then check your installed driver against NVIDIA’s minimum for that CUDA major family:
| CUDA toolkit family | Minimum driver (minor version compatibility) |
|---|---|
| CUDA 13.x | 580 or newer |
| CUDA 12.x | 525 or newer |
| CUDA 11.x | 450 or newer |
NVIDIA’s compatibility rule since CUDA 11 is that an application built with any toolkit in a major family can run on a driver that meets that family’s minimum, but with a limited feature set. An application relying on features that span both toolkit and driver can fail on an older driver with cudaErrorCallRequiresNewerDriver, and applications that compile device code to PTX will not work on older drivers at all. If the application needs PTX, NVIDIA’s answer is to upgrade the installed driver.
The GPU itself also has to be supported: TensorFlow’s guide lists NVIDIA cards with CUDA architectures 3.5, 5.0, 6.0, 7.0, 7.5, 8.0 and higher.
When the versions match and detection still fails
TensorFlow’s own guide says that when the detection test fails, the most likely cause is that components are not being detected or conflict with an existing system CUDA installation, and its fix is to add symbolic links so the right libraries are found. Docker is the easiest way to build GPU support for TensorFlow, because the host then needs only the NVIDIA driver and not the CUDA Toolkit. If the driver itself is the suspect, our guide to installing the NVIDIA driver on Linux for AI workloads covers a clean installation.
Can TensorFlow use a GPU on Windows or macOS?
Yes on both: through WSL2 or DirectML on Windows, and tensorflow-metal on macOS.
Windows. Starting with TensorFlow 2.11, the CUDA build is not supported on native Windows; to use TensorFlow on an NVIDIA GPU you build or install TensorFlow inside WSL2. Inside WSL2, the check is the same list_physical_devices call, and the driver minimum TensorFlow lists for WSL is 528.33.
The other Windows route is Microsoft’s tensorflow-directml-plugin, which runs on native Windows or WSL with AMD, Intel or NVIDIA GPUs. Its limits are worth knowing before you choose it: Python 3.10 is the maximum supported version (3.11 and later are not supported), and Microsoft states support for TensorFlow 2.9 and later.
macOS. Apple’s route is the tensorflow-metal plugin. Apple’s troubleshooting notes name these failures:
- If no matching installation wheel is found, Apple says to check that your Python version is supported (3.9 or 3.10).
No registered OpKernelforDT_COMPLEX64means complex data types are not supported by tensorflow-metal.Cannot assign a device for operation ... colocated with a group of nodesmeans an operation has no GPU implementation available.
“The CPU is faster than my GPU”
This is not always a failure. On small networks with small batch sizes, the CPU can be faster overall because of the overhead of dispatching computations to the GPU, and that overhead is amortised as batch or model size grows. Apple states this for Metal, so benchmark with a realistic batch size before concluding detection is broken.
Diagnostic order when the GPU list is empty
- Print
tf.version.VERSIONand note the platform (Linux, WSL2, native Windows, macOS). - On native Windows with TensorFlow 2.11 or later, stop: move to WSL2 or the DirectML plugin.
- Run
tf.test.is_built_with_cuda(); ifFalse, install a GPU-enabled package. - Read
cuda_versionandcudnn_versionfromtf.sysconfig.get_build_info(). - Compare your installed driver version with the minimum for that CUDA family and with the release’s documented driver requirement.
- Fix conflicts with an existing system CUDA installation by adding symbolic links, or switch to the Docker route.
- Rerun
list_physical_devices('GPU'), then confirm placement with device-placement logging.
From detection to NVIDIA Nsight: seeing where GPU time goes
A GPU that is detected and has ops placed on it can still be underutilised. TensorFlow’s first suggestion is in the framework itself: the TensorFlow Profiler in TensorBoard, starting from a single-GPU profile and checking first whether the program is input-bound with the Input-pipeline analyzer. TensorFlow’s guide also says to check for unnecessary host-to-device and device-to-host copies. Utilisation percentages can shift when total work changes, a problem we cover in utilisation, bottlenecks and the illusion of idle GPUs.
Below the framework, NVIDIA splits the work between two tools: Nsight Systems for overall application structure and Nsight Compute for analysing individual CUDA kernels.
Nsight Systems: the timeline
Nsight Systems is a sampling profiler with tracing features. For TensorFlow it gives one timeline combining CPU core utilisation, thread states, CUDA kernels and NVTX annotations. With CUDA tracing on, it records CUDA Runtime and Driver API calls as well as GPU activity such as memory copies and kernel executions. A basic command-line run looks like this, adapted from the flags in NVIDIA’s and PyTorch’s published examples:
nsys profile -t cuda,nvtx,osrt,cudnn -o tf_session python train.py
The -t option selects which APIs are traced. To profile only the steps you care about rather than the whole run, --capture-range=cudaProfilerApi starts collection when cudaProfilerStart is called, or you can bound regions with NVTX ranges. NVIDIA’s account of profiling TensorFlow also has you run sudo sh -c 'echo -1 >/proc/sys/kernel/perf_event_paranoid' after each boot so that Nsight Systems can collect CPU utilisation and thread data, and nsys status -e checks that your environment meets the requirements for CPU sampling. NVIDIA’s nvtx-plugins-tf emits NVTX ranges from TensorFlow, which ties high-level Python code to the CUDA calls and GPU work beneath it on the timeline.
Nsight Compute: one kernel at a time
Once the timeline points at a suspect kernel, Nsight Systems can hand that kernel straight to Nsight Compute. Nsight Compute collects detailed performance metrics per kernel, and because the GPU can only collect a limited number of hardware counters at once, it may replay a kernel several times to gather them all. It also locks SM clocks and flushes caches by default, and measures durations differently from Nsight Systems, so expect the two tools to disagree on the same kernel’s time.
ERR_NVGPUCTRPERM means missing permission: by default NVIDIA drivers require elevated permissions to read GPU performance counters, so you either profile as root or enable non-admin profiling, and on WSL the counter access has to be enabled in the NVIDIA Control Panel of the Windows host. NVIDIA’s WSL guide lists profiler support on WSL 2 for Volta and later GPUs, with Windows 11 on driver r525+ or Windows 10 build 19044+ on driver r545+.
How to choose between these tools for a given bottleneck is the subject of our piece on profiling before provisioning. Beyond that, the work moves into GPU performance engineering: explaining why a GPU that is visible, correctly versioned and busy still delivers less than it should.
