MPS is not CUDA for Macs. It is a Metal Performance Shaders backend with a different memory model, a narrower operator surface, and a habit of falling back to the CPU without telling you loudly enough. Setting device="mps" instead of device="cuda" changes far more than one string.
What does CUDA vs MPS mean in practice?
CUDA is NVIDIA’s compute platform: a driver, a compiler, a kernel language, and an explicit host-to-device memory model. MPS — Metal Performance Shaders — is a library of GPU primitives on Apple’s Metal API, exposed in PyTorch as a device backend. So the honest framing is that you are comparing a full compute stack against a backend that maps a subset of framework operators onto Apple silicon.
Two differences do most of the damage:
- Memory. Apple silicon uses unified memory, so there is no discrete host-device copy to stage. Code written around explicit
.to(device)transfer patterns, pinned host buffers, and overlapped copy streams loses the thing it was optimised for; the CUDA-shaped tuning does not transfer, and neither do the assumptions behind it. - Operator coverage. MPS covers a narrower set of operators than the CUDA path. Unsupported ops either raise or fall back to CPU execution, and a fallback inside a hot loop produces a latency cliff that looks like a mystery rather than a missing kernel.
This is the same divergence point that separates CUDA from OpenCL and SYCL, which we develop in the comparison of CUDA, OpenCL and SYCL as portability strategies: portability of the API surface is not portability of the memory and kernel patterns underneath it.
Quick answer: which backend for which role
| Question | CUDA | MPS |
|---|---|---|
| Memory model | Explicit host↔device copies | Unified memory, no discrete copy |
| Operator coverage | Broadest in mainstream frameworks | Narrower; gaps fall back to CPU |
| Typical role | Datacentre training and serving | Local development, small-batch inference |
| Main risk | Cost of idle capacity | Silent CPU fallback under production shapes |
| What to test first | Sustained throughput at target batch size | Fallback list plus numerical tolerance vs CUDA |
Can you develop on MPS and deploy on CUDA?
Yes, if you test the boundary instead of assuming it away. In our experience the failures show up late — under production shapes and batch sizes, not on a laptop smoke test — which is exactly when they are most expensive. Three checks bound the risk before any code is shared across both backends:
- Enumerate the operators in your model that fall back to CPU under MPS, and record them as a list with an owner.
- Measure the per-batch latency delta between the MPS path and the CUDA path at your real batch size, not batch size one.
- Run a numerical-tolerance comparison between the two backends on the same inputs, and write down the tolerance you accept.
That turns an untracked “it works on my Mac” risk into a known set of gaps. Whether Apple silicon is a legitimate inference target rather than only a development machine depends on the same evidence: if coverage is complete for your model family and batch sizes stay small, it can be; if the fallback list is long, it is a dev box. Backend suitability is one of the dimensions we examine in our GPU engineering work, alongside memory behaviour and sustained throughput.
The open question for most teams is not which backend is faster. It is whether anyone has written down what diverges between them.
