“Does CUDA work with AMD GPUs?” gets a fast, wrong answer more often than almost any hardware question we hear. The short answer people repeat is “no, CUDA is NVIDIA-only” — and then they close the door on AMD entirely. That answer is technically correct about one narrow thing and misleading about the decision that actually matters. Here is the precise version. CUDA the proprietary runtime and driver stack does not execute on AMD hardware. But “does CUDA work with AMD” is almost never the real question. The real question is whether your workload — the thing built on top of CUDA — can run on AMD, and at what cost. For most inference stacks the answer is yes, with effort that ranges from a config change to a genuine engineering project depending on one variable: how much hand-written CUDA code you own versus how much you inherited from frameworks that already speak AMD. Does CUDA work with AMD GPUs? No, not the CUDA runtime itself. CUDA is NVIDIA’s proprietary programming model, compiler (nvcc), and driver interface. It targets NVIDIA GPUs and only NVIDIA GPUs. You cannot take a CUDA binary and run it on an AMD Instinct or Radeon card. That is where the binary “no” comes from, and where it stops being useful. Because in a production inference stack, very little of what you run is CUDA that you wrote. It is PyTorch calling into cuDNN, ONNX Runtime dispatching to a GPU execution provider, vLLM or SGLang orchestrating attention kernels. Those layers were written to be portable. When AMD support is added to them, your code — the Python, the model definitions, the serving logic — often does not change at all. What changes underneath is the kernel dispatch target. So the honest answer is a decomposition, not a yes or no: the runtime does not port, but the workload frequently does. Treating this as a hardware brand loyalty question hides the thing you should be measuring — a serving-path portability question with latency and cost-per-request consequences. What is ROCm and HIP, and how do they let CUDA workloads run on AMD? ROCm is AMD’s open compute platform — the driver, runtime, and libraries that fill the role CUDA plays on NVIDIA. HIP (Heterogeneous-compute Interface for Portability) is the piece that matters for anyone with CUDA code. HIP is a C++ runtime API deliberately shaped to mirror CUDA’s API surface. cudaMalloc becomes hipMalloc; cudaMemcpy becomes hipMemcpy. The mapping is close enough that AMD ships a translation tool, hipify, that mechanically converts a large fraction of straightforward CUDA source to HIP. The important nuance: HIP is source-compatible with CUDA, not binary-compatible. You are recompiling, not re-running a binary. Code that lives near the CUDA API and uses common patterns tends to hipify cleanly. Code that reaches for NVIDIA-specific hardware features — warp-level primitives tuned for a specific SM architecture, Tensor Core intrinsics through WMMA, inline PTX assembly — is exactly where mechanical translation stops and engineering starts. There is also a compatibility layer conversation worth naming honestly, because search results will surface it. Projects that attempt to run CUDA binaries on non-NVIDIA hardware exist, but for a production serving path we treat them as fragile: the durable path is recompilation through HIP/ROCm or, better, staying at the framework level so you never touch a kernel at all. If you are weighing runtime targets more broadly, our comparison of CUDA vs OpenCL as a GPU target runtime for a port walks through the same trade-off from the API-model angle. Which parts of my inference stack already run on AMD? This is where the decision usually resolves, and it resolves faster than most teams expect. The mainstream inference stack has had AMD backends for a while. What determines your porting effort is how much of your stack sits in the “already supported” column versus the “you wrote it” column. Portability scan: what runs, what ports, what stalls Layer Status on AMD What it costs you PyTorch (eager + torch.compile) ROCm build exists; same Python API Usually a container/image swap ONNX Runtime ROCm / MIGraphX execution provider Config + provider selection vLLM, SGLang ROCm support upstream, model-dependent Verify your model + features are covered cuDNN / cuBLAS calls via framework Mapped to MIOpen / rocBLAS underneath Transparent if you stay framework-level Custom CUDA kernels (yours) Must hipify + retune The real engineering line item TensorRT-optimized engines NVIDIA-only; no direct AMD equivalent path Re-optimize on the AMD toolchain Inline PTX / WMMA Tensor Core code No mechanical port Rewrite against AMD matrix intrinsics Evidence class: observed-pattern across the serving-path ports we have scoped; not a published benchmark. Verify each row against your exact model, framework version, and feature set before committing. The pattern we see repeatedly is that teams assume they are in the bottom rows when they are actually in the top rows. A serving path that is “just” PyTorch plus vLLM plus standard operators frequently moves to AMD with a container swap and a validation pass — not a rewrite. The teams that face real work are the ones who wrote custom fused kernels for a latency-critical path, adopted TensorRT engine files, or dropped into PTX for a hot loop. That work is genuine, but it is bounded and it is measurable, which matters more than whether it is zero. How much effort does porting hand-written CUDA kernels to AMD actually take? There is no honest single number, so we scope it by category rather than promise a figure. hipify handles the mechanical API rename for a large share of conventional CUDA source. What it does not do is recover performance. A kernel that was hand-tuned for a specific NVIDIA architecture — block sizes, shared-memory layout, occupancy targets chosen for that SM — will compile under HIP and then run slower than it should until someone retunes it for AMD’s wavefront width, LDS sizing, and memory hierarchy. So the effort splits into two very different tasks. The mechanical port (get it compiling and correct on AMD) is often days, not weeks, for a well-structured kernel. The performance port (get it back to latency parity) is the open-ended part, and it only matters for kernels that are actually on your critical path. This is why profiling comes before porting, not after. You do not want to spend engineering-days retuning a kernel that contributes two percent of your p95 latency. Our GPU profiling methodology for finding the real bottleneck kernels is the discipline that tells you which kernels a port must protect and which you can leave alone. Will AMD hit the same p95 latency and throughput as my NVIDIA path? Sometimes yes, sometimes it needs tuning, and the only way to know is to measure on the target hardware. Anyone who quotes you parity or a fixed slowdown without profiling your workload is guessing. Memory bandwidth, matrix-engine throughput, and how well your specific operators map to AMD’s libraries all move the result. What we can say from a decision standpoint: the metric that matters is not raw peak FLOPS on a spec sheet — it is sustained p95 latency and throughput at your batch shape on the actual card. A serving path bound by memory bandwidth behaves very differently from one bound by matrix throughput, and AMD Instinct parts often carry strong HBM bandwidth that helps bandwidth-bound decode phases. The reframe is the same one that runs through all of our cost work: measure the operationally relevant number under realistic load, not the transient peak. How do I decide whether switching to AMD is worth it? Frame it as cost-per-request, not hardware preference. AMD instances can change GPU cost by a meaningful margin at fixed throughput — that is the entire reason the question is worth answering. But the saving is only real if the ported path holds latency parity and the porting effort is smaller than the ongoing cost difference. That is an arithmetic problem, and the inputs are knowable before you commit. A decision rubric for scoping an AMD move Profile the serving path first. Which kernels dominate p95? Are they framework operators or code you wrote? If it is all framework-level, your port risk is low. Inventory the custom-code surface. Count hand-written CUDA kernels, TensorRT engines, and PTX/WMMA usage. That count is your effort estimate anchor. Price the fixed-throughput delta. Compare AMD and NVIDIA instance cost at the throughput you actually need — not peak, sustained. Estimate engineering-days to close the gap. Mechanical port days plus performance-tuning days for critical kernels only. Compare against the ongoing saving. If the cost delta pays back the porting effort inside a reasonable horizon, it is worth it. If not, it isn’t — regardless of vendor politics. The unit-economics framing here matters as much as the porting mechanics. A CUDA-to-AMD move is ultimately measured against a cost-per-request KPI, which is exactly what our AWS vs Azure vs Google Cloud cost comparison for AI inference develops for the cloud-provider dimension of the same decision. FAQ Does CUDA work with AMD GPUs? No — CUDA is NVIDIA’s proprietary runtime and driver stack, and its binaries do not execute on AMD hardware. But that is a narrow fact about the runtime, not about your workload. Most inference stacks run on top of frameworks that already have AMD backends, so the workload can move to AMD even though CUDA itself cannot. What is ROCm/HIP and how does it let CUDA workloads run on AMD hardware? ROCm is AMD’s open compute platform, the counterpart to CUDA. HIP is a C++ runtime API deliberately shaped to mirror CUDA’s API surface, with a hipify tool that mechanically converts much conventional CUDA source. HIP is source-compatible, not binary-compatible — you recompile CUDA code as HIP rather than running a CUDA binary on AMD. Which parts of my inference stack already have AMD backends versus which need porting? PyTorch, ONNX Runtime, vLLM, and SGLang have ROCm support, and framework-level calls into cuDNN/cuBLAS map transparently onto MIOpen/rocBLAS. The parts that need real porting are code you wrote yourself: custom CUDA kernels, TensorRT-optimized engines, and inline PTX or Tensor Core intrinsics. Most teams sit closer to the framework layer than they assume. How much engineering effort does porting hand-written CUDA kernels to AMD actually take? It splits into two tasks. The mechanical port — getting a well-structured kernel compiling and correct under HIP — is often days. The performance port, retuning for AMD’s wavefront width and memory hierarchy to recover latency parity, is open-ended and only worth doing for kernels on your critical path. Will running on AMD hit the same p95 latency and throughput as my current NVIDIA path? Sometimes yes, sometimes it needs tuning — the only reliable answer comes from measuring on the target hardware. Peak spec-sheet FLOPS do not predict it; sustained p95 latency and throughput at your batch shape do. Bandwidth-bound decode phases often benefit from AMD Instinct’s HBM bandwidth, but operator-mapping quality varies by workload. How do I decide whether switching to AMD is worth it for cost-per-request? Treat it as arithmetic: price the fixed-throughput cost delta between AMD and NVIDIA, then compare it against the engineering-days needed to close any latency gap. If the ongoing saving pays back the porting effort inside a reasonable horizon, it is worth it. If not, it isn’t — the decision is a cost-per-request question, not a vendor loyalty one. What should I profile before committing to an AMD port? Profile the serving path to find which kernels dominate p95 latency, then classify each as a framework operator or code you wrote. Framework-dominated paths carry low port risk; custom-kernel-dominated paths need effort estimates per critical kernel. That profile is what turns a vague “should we switch” into a scoped, measurable decision. How to tell if this is the right call The trap in “does CUDA work with AMD” is that the binary answer ends the conversation before the useful part begins. The runtime does not port. The workload usually does. And the distance between those two facts is a profiling exercise, not a leap of faith. If you are weighing an AMD move, the honest first step is to profile the serving path and count your custom-CUDA surface, because that count — not the vendor question — decides whether this is a config change or a project. Our [inference cost-cut audit](Inference Cost-Cut Pack) profiles exactly that serving path and flags where CUDA-specific code blocks a hardware move, and the broader R&D engagements scoped to your problem carry the port through to measured latency parity. The failure class to avoid is a hardware swap decided on a spec sheet instead of a p95 number.
