You found -O3 -ffast-math -march=native in a forum thread, pasted it into your build, and the inference latency barely moved. That is the common outcome, and it is not because the flags are wrong. It is because none of them touched the part of your inference path that actually costs time — and a couple of them quietly changed things you did not intend to change. GCC compiler switches are not a performance dial. Each one is a specific trade: code size against speed, portability against tuning, numerical determinism against a few percent of throughput, build reproducibility against convenience. The flags earn their keep only when they target a bottleneck you have already measured. Reach for them blind and you ship a build that is slower to compile, harder to reproduce across machines, or numerically drifted — while the real hot loop sits untouched. This is the framing that separates a team that can attribute a latency win to a specific flag from one that guesses. If you are compiling an inference path — a native C++ extension, a Cython module, or a WebAssembly build via Emscripten — the switches below are the ones that matter, and the reasons they matter are more interesting than the flags themselves. How do GCC compiler switches actually work? A GCC switch is an instruction to the compiler about what kind of machine code to emit from your source. The source describes the computation; the switch shapes how that computation becomes instructions. Optimization switches let GCC reorder operations, inline functions, unroll loops, vectorize arithmetic into SIMD instructions, and eliminate work it can prove is redundant — all while preserving (mostly) the observable behavior of your program. The word “mostly” is where the trouble lives. Some switches preserve behavior exactly. Others relax the rules — floating-point associativity, for instance — in exchange for speed, and that relaxation can shift your inference outputs. So the first thing to internalize is that switches fall into two families: those that are behavior-preserving (safe to apply broadly) and those that trade correctness guarantees for performance (apply only when you understand the risk). The second thing: a switch that has no bearing on your bottleneck buys you nothing. If your inference path is dominated by memory-bandwidth-bound tensor loads, aggressive loop unrolling in a cold code path will not help. This is why we always start from a profile. The same principle applies whether you are tuning host code around a GPU kernel or a pure-CPU WASM module — you can read more about establishing that baseline in profiling the Python inference path before a C++ or WASM port. What is the difference between -O0, -O1, -O2, -O3, and -Os? The optimization levels are the switches most people reach for first, and the differences between them are frequently misunderstood. -O0 is no optimization: fast to compile, easy to debug, slow to run. It is the right choice while you are still finding bugs, and the wrong choice for anything you deploy. -O2 is the workhorse. It enables the large set of optimizations GCC considers safe and generally beneficial — inlining, common-subexpression elimination, strength reduction, basic vectorization — without the aggressive transforms that can bloat code or occasionally regress. For most compiled inference paths, -O2 is the sensible default, and the jump from -O0 to -O2 is where the bulk of the compute-time improvement typically appears. -O3 adds more aggressive loop transformations: fuller vectorization, loop unrolling, function cloning. It can help numerically heavy inner loops — the kind you find in a hand-written convolution or a tokenizer’s hot path — but it also grows the binary and, in our experience, sometimes produces no measurable gain over -O2 or even a small regression from instruction-cache pressure. Treat -O3 as a hypothesis to test against a profiled baseline, not a free upgrade. -Os optimizes for size. It applies most -O2 transforms but suppresses the ones that grow code. This matters more than it sounds for a WASM inference module, where binary size is download weight and instruction-cache behavior in a browser can dominate. Quick reference: what each optimization level buys and costs Level Compile time Binary size Runtime speed Determinism Best for -O0 Fastest Smallest-ish Slowest Exact Debugging only -O1 Fast Small Moderate Exact Quick sane builds -O2 Moderate Moderate Good Exact Default for inference builds -O3 Slower Larger Sometimes better, sometimes worse Exact Profiled numeric hot loops — test it -Os Moderate Smallest Good Exact WASM / size-constrained targets Note the determinism column: none of the optimization levels alone change your numerical results. -O2 and -O3 reassociate integer arithmetic freely because integer math is associative, but they respect IEEE floating-point semantics by default. The numerical drift comes from a separate switch, discussed below. This is an observed-pattern distinction drawn from build-tuning across porting engagements, not a benchmarked constant — your own profile is the authority. What does -march=native do, and why does it break portability? -march=native tells GCC to detect the CPU it is compiling on and emit instructions specific to that microarchitecture — AVX2, AVX-512, FMA, or whatever the local chip supports. When your inner loop vectorizes well, this can be a genuine win, because the compiler is free to use the widest SIMD registers available instead of a conservative baseline. The catch is in the word native. The resulting binary is tuned for the build machine’s exact instruction set. Run it on a CPU that lacks those instructions — an older server, a different cloud instance type, a colleague’s laptop — and it crashes with an illegal-instruction fault. For a native binary shipped to heterogeneous hardware, this is a portability trap: it works on the build box and fails in the field. For a WASM or cross-target build the problem is sharper still. WebAssembly is a portable bytecode compiled away from any specific host CPU; -march=native on the build machine is meaningless or actively harmful there, because the target is not the machine doing the compiling. When you are building for the browser through Emscripten, the SIMD story runs through WASM SIMD, not host -march. The general lesson — that instruction-set width is a portability decision, not just a speed knob — is the same one that shows up when you weigh what SIMD width buys a ported inference path. The portable alternative is -march=<baseline> with an explicit target (say x86-64-v2 or x86-64-v3), which pins a guaranteed instruction-set floor and lets you decide exactly which machines the binary must run on. You trade a little peak tuning for a build that runs everywhere you intend it to. How does -ffast-math change results, and when is the risk acceptable? -ffast-math is the switch that most deserves the caution it rarely gets. It is not one optimization; it is a bundle that relaxes IEEE 754 floating-point guarantees. It lets the compiler assume no NaNs or infinities appear, reassociate floating-point operations (which is not mathematically safe because floating-point addition is not associative), flush denormals to zero, and use faster, less precise reciprocal and square-root approximations. The consequence is that your inference outputs can change. Not catastrophically, usually — but a model that produced logit 2.4471 might now produce 2.4468, and if your pipeline has a downstream threshold or an argmax near a decision boundary, that drift can flip a prediction. Worse, the drift is not reproducible across compilers or -march settings, which quietly undermines any numerical comparison you run. So when is the risk acceptable? When you have quantified it. If you run your validation set through both the -ffast-math build and a strict build and confirm the output divergence stays inside a tolerance your application already accepts — a classification pipeline with wide margins, say — then the speedup can be worth taking. If you cannot bound the divergence, or your inference path is a stage in a larger deterministic system, leave it off. A safer middle ground is to enable only specific sub-flags (like -fno-math-errno) rather than the whole -ffast-math bundle, so you know exactly which guarantee you gave up. Which switches actually move a profiled bottleneck? Here is the diagnostic we apply before touching flags on a compiled inference build. It is deliberately ordered so that the cheap, high-certainty steps come first. Profile the unoptimized-but--O2 build first. Establish where compute time actually goes. If 80% of your latency is in one tokenizer loop or one matmul, that is your target. Flags that do not touch it are noise. Confirm the bottleneck is compute, not memory or I/O. Optimization flags reshape arithmetic and control flow. If you are bandwidth-bound loading weights, no -O3 will help — the fix is elsewhere (layout, batching, caching). Check whether the hot loop vectorizes. Use -fopt-info-vec to see what GCC actually did. If the compiler could not vectorize your inner loop, -march=native’s wider SIMD registers give you nothing until you restructure the loop so the compiler can use them. Test -O3 against -O2 on the profiled path only. Measure both. Keep -O3 only if the delta on your bottleneck is real and positive. Discard it if it grows the binary for no gain. Consider -march last, with an explicit portable baseline. Only after you know the loop vectorizes and compute is the constraint. Pin a target, do not use native for anything you ship broadly. Treat -ffast-math as opt-in, validated, and documented. Never enable it silently. If you take it, record the measured output divergence so a future engineer knows the guarantee you traded. The point of this order is that steps 1–3 tell you whether any flag will help, before you spend time A/B-testing flag combinations. Most of the wasted effort we see comes from teams that skip straight to step 4 and start permuting switches against a bottleneck they never located. Attributing a latency delta to a flag is only meaningful against a profiled baseline — which is exactly the discipline the inference cost-cut approach is built around, and the reason flag tuning belongs after the port-or-optimize decision, not before it. How do flag choices affect build reproducibility and deployment determinism? Two builds of the same source can differ, and compiler flags are a common cause. -march=native makes the output binary depend on the machine that built it — compile on an AVX-512 box and an AVX2 box and you get two different artifacts from identical source. That breaks the reproducible-build property that release engineering depends on: you can no longer prove that the binary in production corresponds to the source you audited. -ffast-math breaks a different guarantee — numerical determinism. The same input can produce subtly different outputs depending on how the compiler reassociated floating-point operations, which itself depends on optimization level and target. For an inference path that feeds a regulated or auditable decision, that non-determinism is a release-readiness problem, not a micro-optimization detail. Build-flag choices that affect reproducibility and numerical determinism are exactly the kind of thing that should surface in a deployment-readiness review, alongside the other portability and determinism concerns we treat as [release-readiness checks for a compiled inference path](GPU engineering). The practical discipline: pin your flags in the build system, not in a shell history; use explicit -march targets rather than native for anything that leaves the build machine; and if -ffast-math is on, that fact belongs in the release notes with its measured divergence, not buried in a Makefile. FAQ How does gcc compiler switches work in practice? A GCC switch instructs the compiler on what machine code to emit from your source — reordering, inlining, unrolling, and vectorizing operations. In practice, switches split into behavior-preserving ones (safe to apply broadly) and ones that trade correctness guarantees for speed. A switch only helps if it targets a bottleneck you have already profiled; applied blind, it adds build cost without moving latency. What is the difference between -O0, -O1, -O2, -O3, and -Os, and which matters for inference builds? -O0 is no optimization (debug only); -O2 enables the safe, generally beneficial transforms and is the sensible default for most inference builds; -O3 adds aggressive loop transforms that sometimes help numeric hot loops and sometimes regress from cache pressure; -Os optimizes for size, which matters for WASM modules. The largest compute-time gain typically comes from -O0 to -O2; treat -O3 as a hypothesis to test against your profiled baseline. What does -march=native do, and why does it break portability on a WASM or cross-target build? -march=native detects the build machine’s CPU and emits instructions specific to it, which can help a well-vectorized loop. But the binary then requires those exact instructions and crashes with an illegal-instruction fault on any CPU that lacks them. For WASM or cross-target builds it is meaningless or harmful, because the target is not the machine compiling — use an explicit portable baseline like x86-64-v3 instead. How does -ffast-math change numerical results, and when is that risk acceptable for inference? -ffast-math relaxes IEEE 754 guarantees — reassociating floating-point ops, flushing denormals, using approximate reciprocals — which can shift inference outputs and flip predictions near a decision boundary. The risk is acceptable only when you have measured the output divergence against a strict build and confirmed it stays inside a tolerance your application already accepts. If you cannot bound the divergence, leave it off or enable only specific sub-flags. Which GCC switches actually move a profiled compute bottleneck versus just adding build cost? Only switches that target the loop where your profile shows time actually going, and only when that bottleneck is compute-bound rather than memory- or I/O-bound. Confirm the hot loop vectorizes (via -fopt-info-vec) before expecting -march or -O3 to help. Flags that do not touch the profiled bottleneck add compile time, binary size, or non-determinism for no runtime benefit. How do compiler flag choices affect build reproducibility and deployment determinism? -march=native makes the output binary depend on the build machine, breaking reproducible builds and the ability to prove the deployed artifact matches audited source. -ffast-math breaks numerical determinism because reassociation depends on optimization level and target. Both are release-readiness concerns for an auditable inference path: pin flags in the build system, use explicit -march targets, and document any -ffast-math use with its measured divergence. When should I tune GCC flags versus accept the defaults for a compiled-WASM inference module? Accept -O2 (or -Os for size-sensitive WASM) as a default and only tune further once a profile shows a compute-bound hot loop that vectorizes. For WASM specifically, skip -march=native entirely and reason about SIMD through WASM SIMD, not host instruction sets. Custom flag tuning is worth it when the profiled bottleneck is real and the win is measurable against a baseline — otherwise the defaults are the lower-risk choice. If you take one thing from this: a compiler flag is a claim you should be able to defend against a measurement. Before your next round of -O3 versus -O2 experiments, ask whether you can name the loop the flag is supposed to speed up — and whether you have the profile to prove it moved. If you cannot, you are not tuning; you are guessing, and the guess usually costs more in reproducibility than it ever returns in latency.
