Flip on -O3 and -march=native, rebuild, and the inference path is supposed to run faster everywhere. That belief survives right up until the same code gets compiled to WebAssembly and the number refuses to move. A GCC compiler flag is not a throughput dial you bolt on at the end of a project. It is an instruction to the compiler about what kind of machine code to emit and what assumptions it may make while emitting it. Some of those assumptions recover real compute on the target you actually ship to. Others recover nothing — or quietly change your numerical results — and a handful are silently ignored the moment your build target changes from native x86/ARM to a sandboxed WASM runtime. The difference between a build-tuning pass and cargo-cult flag-stacking is knowing which is which before you attribute a speedup to a flag. How does a GCC compiler flag actually work? When you invoke gcc (or g++, or emcc wrapping it under Emscripten), the flags fall into a few distinct classes, and each class controls a different lever: Optimization level (-O0 through -O3, plus -Os and -Ofast) — how aggressively the compiler transforms your code. Target architecture (-march, -mtune, -mavx2, etc.) — which instructions it is allowed to emit and which microarchitecture it schedules for. Floating-point behaviour (-ffast-math, -ffp-contract, -fno-math-errno) — how strictly it honours IEEE 754 semantics. Code-size versus speed (-Os, -Oz under Clang, inlining and unrolling controls) — the trade between a small binary and a fast one. Naive tuning treats these as additive: more flags, more speed. In practice they interact, and the deployment target caps what any of them can buy. The correct mental model is that a flag authorises a transformation; whether that transformation helps depends on the workload, the target ISA, and the runtime boundary. This is the same reasoning we apply across nvcc, Clang, and SYCL builds when tuning GPU compilation flags — the compiler is a tool, not an oracle, and the target defines the ceiling. What do -O0, -O1, -O2, and -O3 change for an inference build? The optimization level is the flag people reach for first, and it is the one most misunderstood. The jump from -O0 to -O2 is almost always worth taking for a compiled inference kernel — it turns on inlining, loop transformations, dead-code elimination, and vectorisation that materially reduce instruction count on hot paths. The jump from -O2 to -O3 is where the reasoning has to get more careful. -O3 adds more aggressive loop unrolling, vectorisation of more loop shapes, and function-cloning heuristics. On a compute-bound tensor loop that vectorises cleanly, this can recover real throughput. On a branch-heavy or memory-bound path, -O3 sometimes regresses performance by inflating code size, blowing the instruction cache, and increasing register pressure — an observed pattern across build-tuning passes, not a benchmarked constant. That is why the honest recommendation is to measure -O2 versus -O3 on your actual kernel rather than assume the higher number wins. Level Turns on Right for an inference build when… -O0 No optimization; debug-friendly Never for shipping; only for stepping through in a debugger -O1 Basic transforms, low compile cost Compile-time is the constraint, not run-time -O2 Inlining, vectorisation, most safe transforms Default for a shipping inference bundle -O3 Aggressive unrolling, wider vectorisation Compute-bound loops that vectorise; verify against -O2 -Os Optimize for size Cold-start / footprint-constrained bundles (edge, WASM) -Ofast -O3 plus -ffast-math Only when relaxed FP semantics are acceptable — read the next section first The decision-grade point: -O2 is the defensible default, -O3 is a hypothesis you test, and -Ofast is a flag with a hidden floating-point clause bolted onto it. What does -march do, and why can -march=native break a portable build? -march tells GCC which instruction-set extensions it may emit. -march=x86-64-v3, for example, authorises AVX2 and FMA; -mtune separately tells the scheduler which microarchitecture to optimise instruction ordering for without restricting the instruction set. Targeting the right SIMD width is where a lot of real compute lives on a modern inference path — the same lesson that shows up when you compare what SIMD width buys a ported inference path. The trap is -march=native. It tells GCC to emit instructions for the machine doing the compiling. On a build server with AVX-512, that produces a binary that will fault with an illegal-instruction error on any deployment host that lacks AVX-512. For a build you ship to heterogeneous targets, -march=native is not an optimization — it is a portability bomb with a fast fuse. The correct move is to pick an explicit baseline ISA (-march=x86-64-v2 or v3) that every deployment host is guaranteed to support, and only go higher when you control the hardware. And here is where the WASM boundary bites hardest: -march has almost nothing to target under WebAssembly. WASM defines its own instruction set with an optional fixed-width SIMD extension. -march=native is meaningless — there is no “native” ISA inside the sandbox. An Emscripten build either enables WASM SIMD (-msimd128) or it does not; the elaborate -march/-mtune tuning that recovers compute on native x86 is silently discarded once the same code goes through the Emscripten toolchain. Which flags change inference numerical results? The floating-point flags are the ones that can quietly change your model’s output. -ffast-math is the headline offender because it is a bundle: it enables reassociation of floating-point operations, assumes no NaNs or infinities, drops errno handling on math functions, and lets the compiler treat FP arithmetic as associative when it mathematically is not. For an inference path, that matters. Reassociating a reduction — say, the accumulation inside a softmax or a layer-norm — can shift the last few bits of the result. Usually that is invisible. Occasionally it changes an argmax tie-break, flips a classification near a decision boundary, or diverges a long autoregressive decode. The flag that made your kernel 8% faster (an illustrative figure — measure your own) can also make your validation set produce a handful of different tokens. The disciplined approach is to treat FP-relaxing flags as a numerical decision, not a speed decision: Establish a numerical reference — the -O2 build without fast-math, run against a golden output set. Enable the relaxed flag and re-run the same set. Quantify the divergence (max abs error, changed-prediction count) and decide whether it clears your tolerance. If it does not, use -ffp-contract=fast alone (which allows FMA fusion but not full reassociation) as a middle ground. This is exactly the kind of numerical trade-off the porting and performance-assessment methodology treats as a first-class question rather than a rounding error — the profiling baseline tells you whether the flag moved the number and whether it moved the answer. How do speed flags trade against code size and cold-start? For a server that loads a model once and serves for hours, binary size barely matters — you optimise for steady-state throughput and reach for -O2/-O3. For a deployed bundle that has to cold-start — an edge device, a serverless function, or a WASM module downloaded and instantiated in a browser — code size is a latency term, not a footnote. -O3 inlines and unrolls aggressively, which inflates the binary. A larger WASM module takes longer to transfer, longer to parse, and longer to compile inside the browser’s baseline compiler before the first inference can run. In that regime, -Os (optimize for size) or Clang’s -Oz can produce a lower total time-to-first-result even though the steady-state kernel is marginally slower, because the cold-start dominates a short-lived session. This is why release-readiness checks in an AI-infrastructure release pipeline should flag code-size and cold-start regressions that an aggressive optimization pass can silently introduce — a build can get faster on the bench and slower for the user. Which GCC flags carry over to a WASM/Emscripten build? This is the question that separates a real port-decision pass from wishful flag-stacking. Emscripten wraps LLVM/Clang, not GCC directly, but it accepts the familiar flag vocabulary — and quietly reinterprets or ignores much of it. Flag class Native x86/ARM Under WASM/Emscripten -O0…-O3, -Os Full effect Honoured — level still governs LLVM optimization -march / -mtune Selects ISA + scheduling Largely ignored — no native ISA in the sandbox -march=native Targets build host Meaningless — no “native” target exists -msimd128 n/a WASM-specific — the only real SIMD lever here -ffast-math / -ffp-contract Relaxes FP Honoured — same numerical caveats apply -mavx2, -mfma, etc. Emits those instructions Ignored — those instructions don’t exist in WASM The takeaway is blunt: the optimization level and the floating-point flags carry over; the architecture-targeting flags mostly do not. Chasing AVX-512 or -march=native speedups and then porting to WASM means you were tuning for a machine you are not deploying to. The compute you thought you recovered evaporates at the sandbox boundary, capped by the interpreter/runtime overhead that no flag reaches. If your target is the browser, the relevant conversation is about what compiler flags do to WASM inference performance, not about squeezing the last AVX lane out of a native build. How do we confirm a flag set clears the target before committing to a port? Flag selection is only defensible if it feeds a decision. In a port-decision pass, the build-tuning step exists to answer one question: does the chosen flag set clear the latency and footprint target on the actual deployment target — not the build host — before a port is funded? That means: Baseline first. Measure the unoptimised (-O0) or current-production path on the real target so every subsequent speedup is attributed, not assumed. One flag change at a time. Stacking -O3 -march=x86-64-v3 -ffast-math in one commit and seeing a speedup tells you nothing about which flag earned it — or which one silently changed your outputs. Measure on the deployment target. A speedup on an AVX-512 build server is a fiction if you deploy to a WASM sandbox or an AVX2-only edge box. Guard the numerical result. Any FP-relaxing flag has to pass the golden-output check, not just the stopwatch. Do this and the build-tuning step produces a defensible read on how much compute the compiled path can actually recover — which is precisely the input the Inference Cost-Cut Pack needs before anyone funds a port. Skip it and you are stacking flags on faith. The broader framing of what these levers do and where they stop lives on our GPU engineering practice work. FAQ How does a gcc compiler flag actually work? A GCC compiler flag instructs the compiler on what machine code to emit and what assumptions it may make while emitting it. Flags fall into distinct classes — optimization level, target architecture, floating-point behaviour, and code-size versus speed — and each controls a different lever. A flag authorises a transformation; whether that transformation helps depends on the workload, the target ISA, and the runtime you are deploying to. What do the -O0/-O1/-O2/-O3 optimization levels actually change, and which is right for an inference build? Each level turns on progressively more aggressive transformations: -O0 disables optimization, -O2 enables inlining and vectorisation and is the defensible default for a shipping bundle, and -O3 adds aggressive unrolling and wider vectorisation. -O3 helps compute-bound loops that vectorise cleanly but can regress branch-heavy or memory-bound paths by inflating code size and register pressure. Treat -O3 as a hypothesis you measure against -O2, not a guaranteed win. What does -march / -mtune do, and why can -march=native break a portable build? -march sets which instruction-set extensions GCC may emit; -mtune sets which microarchitecture the scheduler optimises for without restricting the instruction set. -march=native targets the machine doing the compiling, so a binary built on an AVX-512 server will fault with an illegal-instruction error on any deployment host lacking those instructions. For heterogeneous targets, pick an explicit baseline ISA every host supports and only go higher when you control the hardware. Which flags affect floating-point behaviour, and how can they change inference numerical results? -ffast-math bundles several relaxations — reassociation of FP operations, no-NaN/infinity assumptions, and dropped errno handling — that let the compiler treat non-associative FP arithmetic as associative. Reassociating a reduction inside a softmax or layer-norm can shift the last few bits, occasionally flipping an argmax tie-break or diverging a long decode. Treat FP-relaxing flags as a numerical decision: check the relaxed build against a golden output set before shipping it. How do speed-oriented flags trade against code size and cold-start for a deployed bundle? -O3 inlines and unrolls aggressively, which inflates the binary; for a long-running server that barely matters, but for a cold-starting edge device or WASM module it adds transfer, parse, and compile latency before the first inference. In short-lived sessions where cold-start dominates, -Os (or Clang’s -Oz) can give a lower total time-to-first-result even if the steady-state kernel is marginally slower. Release-readiness checks should flag code-size and cold-start regressions an optimization pass can silently introduce. Which GCC flags carry over to a WASM/Emscripten build, and which are silently ignored under that runtime? The optimization level (-O0…-Os) and the floating-point flags (-ffast-math, -ffp-contract) are honoured under Emscripten. The architecture-targeting flags — -march, -mtune, -mavx2, and especially -march=native — are largely ignored or meaningless, because WASM defines its own instruction set with only an optional fixed-width SIMD extension enabled by -msimd128. Tuning for native ISA extensions and then porting to WASM means you optimised for a machine you never deploy to. How do we confirm a chosen flag set clears the latency/footprint target before committing to a port? Baseline the current path on the real deployment target so every speedup is attributed rather than assumed, then change one flag at a time to know which flag earned the gain. Measure on the deployment target — never the build host — and guard any FP-relaxing flag against a golden-output check. Done properly, the build-tuning step yields a defensible read on how much compute the compiled path can actually recover before a port is funded. Aggressive flags on faith are how a build gets faster on the bench and slower — or numerically different — for the user. The failure class here is attributed speedup that does not survive the target boundary: measure on what you ship to, one flag at a time, or the build-tuning step is theatre.
