A CUDA binary compiled for one compute capability will not run fast on hardware it was never targeted for — and in many cases it will not run at all. No translation layer recovers that at load time. This is the single fact that makes GPU compilation flags something other than boilerplate. Most build pipelines treat the compiler invocation as a fixed string that someone wrote once and nobody has looked at since. Copy the default nvcc line, ship the binary, assume the compiler did the right thing. That assumption is where silent lock-in and lost throughput accumulate. Flags are not a formatting detail bolted onto the end of a build command — they are part of the API decision, on the same footing as which kernel you wrote and which memory layout you chose. What does a compilation flag actually control on a GPU build? A compilation flag is an instruction to the compiler about what target to produce and what liberties it may take while producing it. On a CPU that mostly means “which instruction set may I emit” and “how aggressively may I reorder and vectorise.” On a GPU the stakes are higher, because a GPU compiler emits code for a specific device architecture family, and the mapping between source and hardware is far less forgiving than the CPU case. Three flag categories dominate GPU builds, and they answer three different questions: Architecture targeting — which GPUs will this binary run on? (-arch, -gencode in nvcc; --offload-arch in Clang.) Optimisation level and numerical behaviour — how fast, and at what cost to reproducibility? (-O, --use_fast_math, -ffast-math.) Portability model — is this one artifact for one vendor, or one artifact across vendors? (the choice between the CUDA toolchain, SYCL/DPC++, and OpenCL.) Get the first wrong and the binary fails to launch or falls back to a slow path. Get the second wrong and you ship silently different numbers. Get the third wrong and you have locked yourself to one vendor without ever deciding to. These are architectural commitments, and a build system that never audits them is inheriting decisions it never made. What do -arch and -gencode actually control, and how do you choose targets? NVIDIA GPUs expose a compute capability — a version number like sm_80 (A100), sm_89 (Ada / RTX 40-series), or sm_90 (Hopper H100). nvcc compiles CUDA C++ down through two intermediate forms: a virtual architecture called PTX, and a real architecture called SASS (the actual machine code, the sm_XX cubin). The -arch and -gencode flags decide which of each you embed. The distinction that trips teams up: PTX is forward-compatible via just-in-time compilation, SASS is not. If your binary contains sm_80 SASS and you run it on an sm_90 card, the driver will not run the sm_80 cubin on Hopper — it needs either a matching cubin or embedded PTX it can JIT-compile at load time. If neither is present, the launch fails outright. A -gencode clause lets you request both. This pattern embeds real SASS for two architectures plus forward-compatible PTX for the newest: nvcc -gencode arch=compute_80,code=sm_80 \ -gencode arch=compute_89,code=sm_89 \ -gencode arch=compute_90,code=sm_90 \ -gencode arch=compute_90,code=compute_90 \ mymodel.cu -o mymodel The last clause (code=compute_90) embeds PTX rather than SASS, which is the escape hatch for GPUs newer than anything you compiled native code for — the driver JIT-compiles that PTX on first launch. It works, but the JIT step costs a stall at first kernel launch, and the code it produces has not been tuned by ptxas for the specific target the way ahead-of-time SASS is. Relying on PTX JIT as your primary path is how a fleet-wide upgrade turns into a wave of first-launch latency spikes that nobody can explain. The practical rule for choosing targets: enumerate every compute capability in your actual GPU fleet, emit native SASS for each one, and add a single trailing PTX clause for the highest capability as a forward-compatibility safety net. Compiling for architectures you do not own inflates binary size and build time for nothing; omitting an architecture you do own is a runtime failure waiting for the wrong node to schedule the job. Quick reference: NVIDIA architecture targeting Flag pattern Emits Runs on Failure mode if wrong arch=compute_XX,code=sm_XX Native SASS for sm_XX Exactly sm_XX Launch failure on any other capability arch=compute_XX,code=compute_XX PTX only sm_XX and newer (via JIT) First-launch JIT stall; untuned code -arch=sm_XX (shorthand) SASS + PTX for one arch sm_XX and newer Bloats build if over-specified Default (no arch flag) Toolkit’s default sm_ Only that default capability Silent under-targeting on new fleets (Compute-capability-to-product mapping is per NVIDIA’s published CUDA documentation; verify against your installed toolkit version, since defaults change across CUDA releases.) Which optimisation and fast-math flags trade accuracy for throughput? The optimisation level (-O0 through -O3) governs how aggressively the host-side compiler reorders and inlines. On the device side, the bigger lever is the family of fast-math flags. nvcc’s --use_fast_math (and Clang’s -ffast-math) enables a bundle of relaxations: it maps sin, exp, sqrt, and division to faster hardware intrinsics with lower precision, flushes denormals to zero, and — critically — permits the compiler to reassociate floating-point operations as if they were mathematically associative, which IEEE-754 arithmetic is not. The throughput gain is real. In configurations we have profiled, moving transcendental-heavy kernels onto fast-math intrinsics can shave a meaningful fraction off kernel time (observed pattern across our porting engagements; the exact figure depends entirely on how transcendental-bound the kernel is — a purely memory-bound kernel sees close to nothing). The cost is that your results change. Sometimes imperceptibly; sometimes enough to break a numerical regression test or shift a model’s output past a decision boundary. When is the trade-off safe? A useful frame is to ask what depends on the last few bits: Safe by default: rendering, most inference on models that were already quantised, kernels whose output feeds a downstream argmax or thresholding step that swamps small numerical differences. Audit before enabling: iterative solvers where reassociation changes convergence, any path with a numerical reproducibility requirement, physics or financial simulation where accumulated error compounds. Do not enable blindly: anything where a regulator, an auditor, or a bit-exact reproducibility contract is downstream. The failure signature here is a build that “got faster” in a benchmark and then produced subtly different results in production that nobody attributed to a flag, because the flag lived in a Makefile nobody reads. If you enable fast-math, enable it deliberately, per kernel where the compiler allows it, and hold a numerical regression suite against the change. We treat this the same way we treat a precision downgrade — as a first-class engineering decision, not a free speedup. The interaction between optimisation flags and numerical behaviour on ported paths is the same one that shows up when -O3, -march, and fast-math change ported inference output, and it is worth understanding the mechanism before you flip any of them. How do flags differ across nvcc, Clang/SYCL, and OpenCL pipelines? The three toolchains answer the portability question differently, and their flags reflect that. nvcc is NVIDIA-only by design. Its architecture flags name NVIDIA compute capabilities and nothing else. This is the fastest path on NVIDIA hardware and the most locked-in — a nvcc-built binary has no concept of an AMD or Intel GPU. Clang with SYCL / Intel’s DPC++ takes single-source C++ and can target multiple backends. Instead of -gencode you write -fsycl -fsycl-targets=... and list backends: spir64 for a generic SPIR-V intermediate that Intel and other OpenCL/Level-Zero runtimes consume, nvptx64-nvidia-cuda to route through PTX onto NVIDIA hardware, amdgcn-amd-amdhsa for AMD via ROCm. One source tree, several targets — but each target still has its own architecture sub-flag (--offload-arch=gfx90a for an AMD MI210, for instance), so you have not escaped architecture targeting, you have just made it multi-vendor. OpenCL pushes the decision to runtime. You ship portable kernel source (or SPIR-V) and the OpenCL runtime on the target device compiles it for whatever hardware is present, with build options passed as a string to clBuildProgram — -cl-fast-relaxed-math being OpenCL’s analogue of --use_fast_math. The portability is genuine; the cost is a runtime compile step and less ahead-of-time tuning. If you are weighing these models against each other, our breakdown of what CUDA and OpenCL each mean in practice when porting AI workloads covers the trade space in more depth than a flag table can. Flag-model comparison across toolchains Concern nvcc (CUDA) Clang / SYCL (DPC++) OpenCL Architecture flag -arch / -gencode (NVIDIA only) -fsycl-targets + --offload-arch (multi-vendor) Chosen at runtime by the driver Fast-math flag --use_fast_math -ffast-math -cl-fast-relaxed-math (build option) Compile timing Ahead-of-time SASS + optional PTX JIT Ahead-of-time per target + optional JIT Primarily runtime compile Portability Single vendor AMD + Intel + NVIDIA from one source Broadest; any conformant device Tuning ceiling Highest on NVIDIA High per target Lower (less AOT tuning) There is no universally correct row. The right column depends on whether you are optimising for peak on one vendor or for reach across a mixed fleet — which is precisely the decision the flags encode. What happens at runtime when the compute capability is wrong? Three distinct failure modes, and they look different in the logs: The first is a hard launch failure. The binary contains no SASS for the present GPU and no PTX to JIT from, so the driver returns an error like no kernel image is available for execution on the device. This is the good failure — it is loud, it happens immediately, and it points straight at the architecture flags. The second is the JIT stall. The binary carries PTX, so the driver can run, but it JIT-compiles at first launch. On a fresh process that is a one-time cost per kernel; on short-lived processes or serverless workers that spin up per request, you pay it repeatedly, and it shows up as mysterious first-request latency that never appears in steady-state benchmarks. NVIDIA’s compiled-code cache mitigates this within a machine but does not survive a fresh container. The third is the quiet one: the binary runs, produces correct output, but on a compatibility path slower than the tuned code you could have shipped. A kernel that hits full occupancy with native SASS may fall back to a generic path that runs several times slower, and nothing in the output tells you. This is the failure that a throughput audit surfaces and that a functional test never will. Avoiding all three comes down to the same discipline: know your fleet’s compute capabilities, emit native code for each, carry PTX for forward compatibility, and pre-warm or persist the JIT cache where cold starts matter. Flag review is one of the first structural checks in a GPU performance audit precisely because mis-targeted architecture flags and default optimisation levels are cheap to find and cheap to fix — no kernel rewrite required. For teams whose workload spans laptops, edge boxes, and datacenter GPUs at once, the multi-platform case has its own sharp edges, covered in compilation flags for multi-platform edge inference. FAQ How does a compilation flag work, and what does it mean in practice for GPU builds? A compilation flag tells the compiler what target to produce and what optimisations it may apply. On a GPU build, flags decide which device architectures the binary can run on, how aggressively the compiler trades numerical precision for speed, and whether the artifact is tied to one vendor or portable across several. They are architectural commitments, not formatting details, and a build that never audits them inherits decisions nobody made deliberately. What do CUDA architecture flags like -arch and -gencode actually control, and how do I choose targets for my GPU fleet? They control which NVIDIA compute capabilities the binary contains code for, in the form of native SASS machine code and/or forward-compatible PTX. Choose targets by enumerating every compute capability in your actual fleet, emitting native SASS for each, and adding one trailing PTX clause for the highest capability as a forward-compatibility net. Over-specifying inflates build size for nothing; omitting a capability you own is a launch failure waiting to happen. Which optimisation and fast-math flags trade numerical accuracy for throughput, and when is that trade-off safe? --use_fast_math (nvcc), -ffast-math (Clang), and -cl-fast-relaxed-math (OpenCL) map transcendentals to faster lower-precision intrinsics, flush denormals, and permit non-associative floating-point reassociation. The trade-off is safe when downstream steps like thresholding or argmax swamp small numerical differences, or on already-quantised inference. It is unsafe for iterative solvers, bit-exact reproducibility contracts, or anything with a regulator or auditor downstream — enable it deliberately and hold a numerical regression suite against the change. How do compilation flags differ across nvcc, Clang/SYCL, and OpenCL build pipelines? nvcc targets NVIDIA compute capabilities only, ahead-of-time, and is the most locked-in. Clang/SYCL (DPC++) takes single-source C++ and targets NVIDIA, AMD, and Intel via -fsycl-targets, each with its own architecture sub-flag. OpenCL ships portable source or SPIR-V and defers compilation to the runtime on the target device, passing build options as a string. Each model trades peak tuning against reach. How do flag choices affect binary portability across AMD, Intel, and NVIDIA GPUs? A nvcc binary has no concept of non-NVIDIA hardware, so its flags bind you to one vendor. SYCL/DPC++ lets one source tree emit for all three vendors, but you still specify each target’s architecture. OpenCL is the most portable because the device runtime compiles for whatever hardware is present. The flag model you pick is the portability decision — there is no neutral default. What happens at runtime when a binary was compiled for the wrong compute capability, and how do I avoid JIT recompilation stalls? Three outcomes: a hard launch failure when no matching SASS or PTX exists (the loud, easy case); a JIT stall when PTX is present but must be compiled at first launch (costly on cold-start or serverless workers); or a silent slow fallback that produces correct results several times slower than tuned code. Avoid all three by emitting native SASS for every capability in your fleet, carrying PTX for forward compatibility, and pre-warming or persisting the JIT compile cache where cold starts matter. Flags are the cheapest performance lever on a GPU build — no kernel touched, no algorithm changed — which is exactly why they are the easiest to leave on autopilot. The next time a binary “just works” on a new GPU node, the question worth asking is not whether it ran, but whether it ran on the tuned path or quietly settled for the compatibility one.
