Read side by side, ROCm and CUDA look interchangeable. Both expose kernels, streams, and a math library set, and HIP advertises a near-mechanical source translation. That is the easy half of the problem. The question that decides an AMD-versus-NVIDIA move is not whether your CUDA code compiles under ROCm — it usually will — but which kernels survive translation with their performance intact.
That distinction is where migration budgets get destroyed. A HIP-translated kernel that compiles, runs, and returns correct numbers can still land well below its CUDA baseline. At that point the team is no longer porting code; it is debugging an AMD architecture. Nobody budgeted for that.
What actually breaks after a HIP translation?
API surface translates. Memory-access behaviour does not. The recurring offenders in our experience:
- Warp versus wavefront width. Kernels written against a 32-lane assumption behave differently on a 64-lane wavefront — shuffle patterns, ballot logic, and any hand-tuned reduction need re-derivation, not renaming.
- Occupancy tuning. Block sizes and register budgets chosen for a specific NVIDIA SM are arbitrary on a CDNA compute unit.
- Shared memory / LDS bank behaviour. Padding tricks that avoid bank conflicts on one architecture can reintroduce them on the other.
- Library kernel coverage. Where a CUDA path leaned on a tuned cuBLAS, cuDNN, or FlashAttention kernel, the ROCm equivalent may exist but not at the same tuned coverage for your shapes and precisions.
Framework-level work — PyTorch training and inference on ROCm builds — is generally the least painful part. Custom kernels are the expensive part, and they are exactly the kernels that were written because the generic path was too slow.
Quick answer: how to scope the decision
| Question | What to do | What it tells you |
|---|---|---|
| Which kernels matter? | Profile and pick the 2–3 that dominate wall-clock time | The port is only as risky as these |
| Do they translate? | Run HIP translation, confirm correctness first | Source compatibility is a gate, not a result |
| Do they perform? | Measure each against its CUDA baseline on comparable hardware | Share of baseline recovered per kernel |
| What is left? | Cost the rewrites against wavefront and LDS characteristics | A bounded engineering estimate |
| Is the trade worth it? | Compare that cost to the NVIDIA hardware premium | The lock-in premium, as a number |
Two or three representative kernels are enough. That test converts a multi-quarter migration gamble into an estimate with a range — and it belongs before hardware procurement, not after the cluster lands on the loading dock.
Which way to lean
Workloads that live inside framework operators, use standard precisions, and depend on library GEMM and convolution paths are reasonable ROCm candidates. Workloads carried by hand-written CUDA kernels, custom attention variants, or aggressive warp-level primitives should stay on CUDA unless the hardware saving is large enough to fund a rewrite you have actually scoped.
This is the same lesson that applies to OpenCL and SYCL: API translation does not port a memory-access pattern. We treat vendor-stack viability as a profiling question rather than a compatibility-matrix question, which is why it sits inside the API dimension of our GPU performance and optimisation work rather than in a procurement spreadsheet.
The open question for most teams is not whether ROCm works. It is whether they can name, today, what share of their CUDA baseline their three hottest kernels would recover — and if they cannot, the migration decision is still a guess.
