Triton and CUDA are not competing choices. Triton is a Python-embedded kernel DSL that compiles down through LLVM to GPU machine code; CUDA is the vendor API and toolchain underneath it. Once that relationship is clear, the decision stops being “which one wins” and becomes something narrower and much more answerable: who on your team owns the kernel, and how often do they need to change it?
The naive reading is that Triton replaces CUDA and therefore solves portability. It does not. Triton’s mainline maturity and performance tuning still track NVIDIA hardware most closely, and the memory-layout decisions that dominate GPU performance do not disappear because the syntax got shorter. Triton removes index arithmetic and shared-memory boilerplate. It does not remove the need to reason about tiling, occupancy, and data movement.
Is Triton a replacement for CUDA, or a layer on top of it?
A layer. A Triton kernel is still launched as a grid of programs over the same execution model, still bound by the same bandwidth and occupancy limits, and still ultimately compiled to machine code for a specific architecture. What changes is the authoring surface: you write block-level operations instead of per-thread index math, and the compiler handles vectorisation and much of the shared-memory staging.
That framing matters because it predicts where Triton stops being enough. When you need warp-level primitives, direct tensor-core intrinsics, or interop with vendor libraries such as cuBLAS and cuDNN, you are back in CUDA C++. This is not a Triton defect — it is the boundary of what a portable block-level abstraction can express.
Which kernels are worth writing in Triton?
| Situation | Where to author | Why |
|---|---|---|
| Fused attention or elementwise-reduction kernel your team edits weekly | Triton | Iteration loop shortens dramatically; the kernel is bandwidth-bound and Triton gets close |
| Custom activation, normalisation, or quantisation fusion | Triton | Little vendor-library equivalent exists; low expressiveness demand |
| Tensor-core-heavy GEMM competing with cuBLAS | CUDA C++ or the vendor library | Intrinsics and hand-tuned scheduling still matter here |
| Warp-level shuffles, cooperative groups, multi-stream orchestration | CUDA C++ | Not expressible at Triton’s block abstraction |
| Chosen because “it will run on other vendors later” | Reconsider | Portability is the wrong reason; expect a rewrite |
On bandwidth-bound kernels, a competent Triton implementation typically lands within single-digit to low-double-digit percent of a tuned CUDA baseline in achieved bandwidth utilisation — a gap most inference cost targets absorb without complaint (observed pattern across our GPU optimisation engagements; not a published benchmark). On tensor-core-heavy GEMMs the gap is wider, and there the honest answer is usually to call the vendor library rather than author anything at all.
The mistake that costs a rewrite
Choosing Triton for portability reasons rather than iteration-speed reasons is the failure mode we see most often. The portability argument sounds free, so nobody stress-tests it, and the decision quietly stops being traceable. Then a target platform changes, the mainline kernel does not perform there, and the team discovers that the memory-access patterns baked into their block sizes were tuned for one architecture all along.
Frame it as an engineering-time trade instead. Triton buys engineer-days back on kernels under active iteration. That is a real, defensible reason to adopt it, and it survives scrutiny in a way “we might switch vendors” does not.
For where kernel authoring sits inside the broader picture — profiling, memory movement, and the framework-level decisions that usually matter more than the kernel language — see our work on GPU performance engineering. Kernel-level findings from a performance audit are what should decide whether a bottleneck is worth re-authoring at all; the language question comes second.
So the open question is not Triton or CUDA. It is whether your bottleneck is actually a kernel you own — and in our experience, more often than teams expect, it is not.
