“Just build it with -O3” is the advice you’ll get in most threads about compiler flags gcc, and it is not wrong so much as incomplete. Turning on -O3 — or worse, piling on every optimization flag you can find — treats the compiler as a slot machine: pull the lever, hope faster and smaller falls out. For a compute-bound inference path, or one you’re squeezing into a WebAssembly bundle, that assumption breaks in ways that are quiet until they’re expensive. A flag can shave real latency, or it can bloat cold-start with no compute win, or it can silently change your numerical results. Which one you got is not something the flag list tells you. Only a profiled build does. The flags that matter for an inference build fall into a handful of classes, and each class trades against something you can measure. The job is not to memorize the flags — it is to read them against the runtime model of your workload, then confirm the change against a latency and footprint target you set before you touched the build. What does “compiler flags gcc” actually mean for an inference build? A GCC flag is an instruction to the compiler about how to translate your C or C++ source into machine code — or, when the toolchain is Emscripten, into WebAssembly. Some flags change what code gets generated (vectorization, inlining, loop unrolling); some change what instructions the target is allowed to use (-march, -mtune); some change what mathematical guarantees hold (the floating-point flags); and some change when optimization happens across translation-unit boundaries (link-time optimization). For an inference path the distinction that matters most is between flags that help a compute-bound hot loop and flags that only affect code you rarely execute. A model’s forward pass usually spends the overwhelming majority of its time in a small number of tight numerical loops — matrix multiplies, convolutions, activation functions. Optimizing those loops hard is where the win lives. Optimizing the model-loading path or the argument parser aggressively costs build time and binary size for no runtime benefit. This is the same reasoning that governs what “computationally expensive” means in an inference path and where the cost actually lives — you optimize where the time is, and you measure to find out where that is. Everything below assumes you already have a profiled baseline. If you don’t, the flag discussion is premature; start with profiling the Python inference path before a C++ or WASM port, because you cannot attribute a flag’s effect to anything without a before number. What do the GCC optimization levels actually change? The -O levels are bundles of individual passes, not a single dial. Here is what each level meaningfully does for an inference build, and where it fits. Level What it turns on Fit for an inference build -O0 No optimization; fastest compile, readable in a debugger Debugging only — never ship it; the hot loops run several times slower -O1 Basic optimizations, modest code-size impact Rarely the right choice for a compute path; leaves vectorization on the table -O2 Aggressive optimization without large code-size growth The safe default for most native inference builds; predictable, well-tested -O3 -O2 plus aggressive inlining, loop unrolling, vectorization Can win on tight numerical loops — but verify it, because it can also bloat the binary with no gain -Os Optimize for size (subset of -O2 that avoids size growth) The starting point for a WASM bundle where footprint dominates -Ofast -O3 plus -ffast-math and other standard-relaxing flags Fast but numerically unsafe — see the floating-point section before you touch it The naive move is -O3 everywhere. In practice, on many inference workloads -O2 and -O3 land within noise of each other, because the compiler already vectorizes the hot loop at -O2 and the extra -O3 unrolling just grows the instruction cache footprint. Whether -O3 helps your loop is a benchmark question, not a doctrine question (observed pattern across porting work; not a published rate). Build both, measure both, keep the one that clears your latency target at the smaller size. What do -march and -mtune do, and where do they bite? -march tells GCC which instruction-set extensions it may emit — AVX2, AVX-512, FMA, and so on. -mtune tells it which microarchitecture to schedule for without restricting the instruction set. The difference is portability: -march=native builds for exactly the CPU doing the compile. On a compute-bound loop with wide SIMD, this can be a real speedup because the compiler is allowed to use the widest vector units present. It is also a portability landmine: run that binary on an older CPU and it crashes with an illegal-instruction fault. The trade between SIMD width and portability is the same one that shows up when you read what SIMD width buys a ported inference path. -march=x86-64-v2 or -v3 targets a defined baseline of extensions, giving you portable binaries with a known SIMD floor. -mtune=generic (the default) schedules for no CPU in particular; pairing an explicit -march with a specific -mtune lets you set the instruction floor and the scheduling target independently. For a server you control, -march=native is often the right call — you know the target. For anything shipped to heterogeneous hardware, pick an explicit baseline. And for a WASM target this whole class mostly evaporates: WebAssembly has its own SIMD proposal (-msimd128 under Emscripten), and you are not selecting an x86 microarchitecture at all. Which flags affect floating-point behavior and reproducibility? This is the class that causes the most damage, because the failure is silent. -ffast-math (and its inclusion inside -Ofast) relaxes IEEE 754 guarantees: it lets the compiler reassociate floating-point operations, assume no NaNs or infinities, flush denormals to zero, and treat a + 0.0 as a. Each of those can change the bits your model produces. For some inference paths that is harmless — the model is robust to the last few ULPs. For others it is not: a classifier near a decision boundary, an accumulation over a long sequence, a quantized path where small errors compound. The dangerous part is that a -Ofast build often looks correct in a smoke test and diverges only on inputs you didn’t try. Numerical stability is a first-class property of the build, not an afterthought — reproducibility across optimization levels is something to confirm explicitly, the same way you would confirm it when choosing a lower-precision path like 4-bit floating point on GPUs. The disciplined approach: build with strict floating-point semantics first, establish your reference outputs, then selectively enable individual relaxations (-fno-math-errno, -ffinite-math-only) and re-verify against the reference. Reach for the whole -ffast-math bundle only when you’ve measured that your path tolerates it. A fast build that quietly changed your answers is worse than a slower correct one — you pay for it later, in a debugging session that starts from “why do prod and staging disagree.” When is LTO worth the build cost? Link-time optimization (-flto) defers part of the optimization work to link time, when the compiler can see across translation units. For an inference build this matters because it enables cross-module inlining and dead-code elimination — the layer boundaries in your code stop being optimization barriers. Two things usually improve: the hot path can inline small helpers that live in other files, and unused code gets stripped, which shrinks the binary. The cost is build time and memory. LTO link steps are slower and hungrier, sometimes dramatically so on a large codebase. For an inference deploy the trade is usually worth it: you build once and run the binary millions of times, so a slower link amortizes instantly, and the binary-size reduction feeds directly into a WASM footprint target. Thin LTO (-flto=thin on the Clang side, parallelized LTO on GCC) reduces the build-time penalty considerably and is often the pragmatic middle. If build-time cost is a concern, the gold linker is a related lever on the link side. Verify LTO the same way as everything else: it should reduce size and hold or improve latency. If it does neither, the cross-module wins weren’t there for your code, and you’re paying build cost for nothing. Which GCC-family flags matter for a WASM / Pyodide bundle? When the toolchain is Emscripten emitting WebAssembly — the situation for a Pyodide-style browser deployment — the flag priorities shift because the dominant cost changes. On the server, latency of the hot loop usually rules. In the browser, bundle size and cold-start often dominate, because the user waits for the .wasm to download and instantiate before any compute happens. The flags that matter for a WASM inference bundle: -Os or -Oz as the optimization baseline — size-first, because footprint gates cold-start. -O3 can grow the bundle with unrolling that never pays off in the browser. -flto — the dead-code elimination is worth more here than on a native build, because every kilobyte is download time. -msimd128 — enables WASM SIMD, which is the actual compute lever for a WASM hot loop; without it the numerical work runs scalar. Emscripten linker settings (-sINITIAL_MEMORY, -sALLOW_MEMORY_GROWTH, closure compilation on the JS glue) — these shape the runtime footprint as much as any GCC-level flag. This is where the flag list most obviously stops being self-explanatory. -O3 is the “obviously fastest” flag and it is frequently the wrong choice for a WASM bundle, because it optimizes for a cost — instruction throughput — that isn’t your binding constraint. The deeper treatment of this trade lives in what compiler flags do to Pyodide performance, and the broader native-vs-WASM flag comparison is covered in what -O3, -march, and fast-math actually change in a ported inference path. Reading either against the runtime model is the same discipline: the flag serves the constraint you actually have, not the one the flag name implies. How do you confirm a flag change cleared the target rather than just feeling faster? “It feels faster” is not evidence. Attributing a change to a flag requires holding everything else constant and measuring the two things the flags trade against: latency on the compute path, and binary or bundle size. A workable procedure: Fix a target first. Write down the latency ceiling and footprint budget the build must clear — for a WASM bundle, that includes a cold-start figure. Without a target, “faster” and “smaller” have no pass/fail. Establish a baseline. Build at -O2 (native) or -Os (WASM) with strict floating-point, and record reference outputs, latency distribution, and size. Change one flag class at a time. -O3, then -march, then -flto, then a floating-point relaxation — never all at once, or you can’t attribute the effect. Measure latency as a distribution, not a single run. Report a percentile under realistic input, not a best-case burst; warm and cold paths differ, especially in WASM. Diff numerical outputs against the reference on every optimization-level change. A latency win that moved your results is not a win. Record the size delta per flag change and check it against the footprint budget. The output of this is a small table of attributed changes: this flag bought this much latency at this much size cost, with numerical stability confirmed. That is defensible build-configuration evidence, and it is exactly what feeds a port-decision. The methodology behind the baseline itself — how you know the profiled numbers mean what you think — comes from the same porting and performance-assessment work that supplies the profiling discipline. When the question is whether a set of flags clears the latency and footprint bar before a port is funded, that evidence is the deliverable, and it maps directly onto the build-configuration slice of the Inference Cost-Cut Pack. For the wider engineering picture of tuning GPU and CPU builds, the GPU optimization landing page is the anchor. FAQ How should you think about compiler flags gcc in practice? A GCC flag instructs the compiler how to translate source into machine code (or, under Emscripten, into WebAssembly). Flags fall into classes — optimization level, architecture targeting, floating-point behavior, and link-time optimization — and each trades against something measurable like latency, binary size, or numerical reproducibility. In practice the flags matter most on the compute-bound hot loops of an inference path; the win comes from reading them against your workload’s runtime model, not from turning everything on. What do the GCC optimization levels actually change, and which fits an inference build? The -O levels are bundles of passes, not a single dial. -O2 is the safe default for native inference builds; -O3 adds aggressive inlining and unrolling that can win on tight loops but can also bloat the binary with no gain; -Os/-Oz optimize for size and are the starting point for WASM bundles; -Ofast adds numerically unsafe relaxations. On many inference workloads -O2 and -O3 land within noise of each other, so you build both and keep the one that clears your target at the smaller size. What do -march and -mtune do, and how do they trade portability against per-target speed? -march sets which instruction-set extensions the compiler may emit (AVX2, AVX-512, FMA); -mtune schedules for a microarchitecture without restricting the instruction set. -march=native can speed up a wide-SIMD loop but crashes on older CPUs, so it fits servers you control; an explicit baseline like -march=x86-64-v3 gives portable binaries with a known SIMD floor. For a WASM target this class mostly evaporates, replaced by -msimd128. Which GCC flags affect floating-point behavior and numerical reproducibility in an inference path? -ffast-math (and its inclusion in -Ofast) relaxes IEEE 754 guarantees — reassociation, no-NaN assumptions, denormal flushing — any of which can change your model’s output bits. The failure is silent: a -Ofast build can pass a smoke test and diverge only on untested inputs. Build with strict semantics first, capture reference outputs, then enable individual relaxations and re-verify against the reference before trusting the whole bundle. What role does LTO (-flto) play in binary size and inference latency, and when is it worth the build cost? LTO defers optimization to link time so the compiler can inline across translation units and strip dead code, which typically shrinks the binary and can improve the hot path. The cost is slower, more memory-hungry link steps. For an inference deploy — built once, run millions of times — the trade is usually worth it, and thin/parallel LTO reduces the build penalty; verify it reduces size and holds or improves latency, or you’re paying build cost for nothing. Which GCC-family flags matter when an Emscripten/WASM toolchain emits a Pyodide-style WebAssembly bundle? In the browser, bundle size and cold-start usually dominate over hot-loop latency because the user waits for the .wasm to download and instantiate. That flips the priorities: -Os/-Oz as the size-first baseline, -flto for dead-code elimination, -msimd128 to enable the actual WASM compute lever, and Emscripten linker settings that shape runtime footprint. -O3 is frequently the wrong choice here because it optimizes for a constraint you don’t have. How do we profile a build to confirm a flag change cleared the latency/footprint target rather than just feeling faster? Fix a latency ceiling and footprint budget first, then establish a strict-floating-point baseline with recorded reference outputs. Change one flag class at a time, measure latency as a distribution under realistic input, diff numerical outputs against the reference on every optimization-level change, and record the size delta per flag. The output is a table of attributed changes — this flag bought this latency at this size cost with stability confirmed — which is the defensible evidence a port decision needs. The flag list will always look like a menu of free wins. It isn’t. Every flag class trades against latency, footprint, or numerical stability, and the only way to know which trade you actually made is to hold everything else constant and measure against a target you set before you started. Build the attributed table, and the port decision writes itself; skip it, and you have a build that feels faster right up until the day prod and staging disagree.
