DirectML and CUDA are not two versions of the same thing. DirectML is an inference API bound to DirectX 12 and the Windows graphics stack. CUDA is a full compute programming model with its own kernels, memory model and mature profiling tooling. The choice between them is not a portability preference — it is decided by your workload class.
The common reading is that DirectML is “CUDA for everyone”: the same models running on AMD, Intel and NVIDIA hardware with nothing given up. That reading holds for exactly one shape of problem, and breaks badly outside it.
What is DirectML actually doing under the hood?
DirectML executes a graph of predefined operators through the DirectX 12 device that is already present on any modern Windows machine. Vendor drivers implement those operators. You hand it a graph; you do not hand it kernels. That is the whole surface.
CUDA sits a level lower. You write kernels, you decide memory placement, you reason about occupancy, and you can measure all of it with Nsight. The CUDA-specific memory patterns that dominate GPU optimisation work simply have no expression in an operator-graph API — there is no place to put them. This is the structural distinction, not a maturity gap that a future DirectML release closes.
When DirectML wins, and when it becomes a ceiling
The case where DirectML is clearly right: a fixed inference graph shipped inside a Windows desktop application, running on end-user GPUs you cannot enumerate in advance. DirectML removes a hardware qualification problem that CUDA cannot solve at all, because CUDA restricts your support matrix to NVIDIA parts.
The case where it becomes a ceiling: custom operators, training, or any bottleneck that lives in memory movement rather than in the operator list. Teams that pick DirectML for the portability headline and then find their hot path is a custom operator have made the same unexamined decision as teams that default to CUDA without asking who their users are.
| DirectML | CUDA | |
|---|---|---|
| Layer | DirectX 12 inference API | Compute programming model |
| Hardware reach | Any DX12 GPU — AMD, Intel, NVIDIA | NVIDIA only |
| Custom kernels | Not expressible | First-class |
| Memory placement control | None | Explicit |
| Profiling depth | Graphics-stack tooling | Nsight, mature |
| Best fit | Fixed inference graph, unknown end-user GPU | Custom kernels, training, tuned inference |
How to quantify the decision
Measure three things before committing, and measure them on target hardware rather than the development machine:
- Percentage of your installed base covered without falling back to a CPU path.
- Per-frame or per-inference latency on each vendor’s mid-range hardware.
- Engineering days spent on driver and runtime qualification.
If you retain a CUDA backend alongside DirectML, the number that justifies two backends is the measured latency delta — not the assumption that CUDA is faster. In our experience, the delta is worth maintaining a second path in tuned, latency-critical inference and rarely worth it for occasional background inference in a desktop app (observed across TechnoLynx engagements; not a published benchmark).
DirectML also does not stand alone in a Windows stack. ONNX Runtime can target it as an execution provider, which is usually how it reaches a production application, and that is the layer where a TensorRT or CUDA execution provider becomes the alternative to compare against.
We treat this as the Windows slice of a broader question we work through in GPU performance engineering: whether the current backend is structurally suited to the hardware your users actually own. If your support matrix and your bottleneck point at different backends, which one are you measuring against?
