A tutorial hands you -O2 -march=native and you paste it into the build for your ONNX Runtime backend. The binary compiles, the numbers look right on your laptop, and you ship it. Weeks later a defect surfaces on one edge device and nowhere else — a detection score drifts a fraction, a threshold flips, and a downstream decision changes. The build machine never saw it. The flag line you copied was not neutral; it baked assumptions about your CPU into an artifact meant to run somewhere else. That is the core problem with treating GCC flags as a fixed incantation. Each flag is a tradeoff you are making on purpose or by accident — optimisation level, target architecture, floating-point behaviour, vectorisation. For a workload that runs identically everywhere this rarely bites. For an inference runtime shipped across a fleet of heterogeneous edge devices, the wrong flag introduces a second, hidden per-target divergence on top of the one quantisation already creates. Understanding what each flag changes is how you avoid a “works on my build machine, fails on the target” class of defect that otherwise only appears during per-platform validation. What does a flag in GCC actually do? A GCC flag is an argument passed to the compiler that switches on a behaviour, sets a value, or selects a policy. -O2 is a policy bundle — it turns on dozens of individual optimisation passes at once. -march=x86-64-v3 is a target selection — it tells the compiler which instruction set extensions it may emit. -ffast-math is a semantic relaxation — it permits the compiler to break strict IEEE-754 floating-point rules in exchange for speed. Grouping them mentally by what they touch matters more than memorising the list, because the ones that touch numerical semantics and target architecture are the ones that hurt portability. The trap is that most of these flags are silent. GCC does not warn you that -march=native just committed your binary to AV-512 instructions the target device lacks. It compiles cleanly and runs fine until it lands on hardware that raises an illegal-instruction fault, or — worse — until a floating-point reordering changes a result by an amount small enough to pass unit tests but large enough to shift a real decision. If you are already reasoning about how SIMD width buys a ported inference path, the compiler flag is the layer that decides whether that width is even available on the target. Which GCC flags change the numerical output of an inference build? This is the question that separates a portability nuisance from a correctness bug. Some flags only affect speed or binary size — get them wrong and you lose performance, not accuracy. Others change the bits that come out of a floating-point computation. Across the builds we have configured for native inference runtimes, the flags worth auditing before anything ships fall into a small set. Flag What it changes Affects numerical output? Portability risk -O2 Bundle of safe optimisations; preserves IEEE-754 semantics No (by design) Low -O3 Adds aggressive loop transforms, more auto-vectorisation Rarely, via reassociation edge cases Low–medium -Ofast -O3 plus -ffast-math Yes High -ffast-math Relaxes IEEE-754: reassociation, no NaN/Inf handling, flush-to-zero Yes High -funsafe-math-optimizations Subset of fast-math; reorders FP operations Yes Medium–high -march=native Emits every ISA extension the build host supports No (but changes which code path runs) Very high -mtune=<cpu> Schedules for a CPU without restricting the ISA No Low -ffp-contract=fast Allows fused multiply-add contraction Yes (rounding differs) Medium The claim that matters here: -O2 and -O3 are designed to preserve floating-point results, while any flag in the -ffast-math family explicitly does not — and -Ofast silently pulls -ffast-math in behind an innocent-looking name. This is an observed-pattern claim from build configuration work, not a benchmark, but it is a well-documented property of GCC’s optimisation model rather than folklore. The reason it matters across edge targets is that a numerical difference produced by -ffp-contract or reassociation is not the same on every microarchitecture — the fused multiply-add path exists on some targets and not others, so the same source and the same flags can produce different bits per device. Why does -march=native break portability? -march=native asks GCC to detect the CPU of the machine doing the compilation and emit code tuned to exactly that chip, including every instruction-set extension it supports. On a modern x86 build server that means AVX2, often AVX-512, FMA, and more. The artifact is now hard-committed to those instructions. Ship it to an edge device with an older or lower-power core — an Arm-based board, an Atom-class x86, a device without AVX-512 — and the binary either faults on an illegal instruction or, if the loader is lenient, behaves undefined. The fix is to name the target explicitly instead of the build host. For a known, fixed target CPU, use -march=<that-cpu>. For a fleet with a known baseline, pick the lowest common denominator — the x86-64 microarchitecture levels (x86-64-v2, x86-64-v3) exist precisely for this — and pair it with -mtune if you want scheduling hints for a more common member of the fleet without restricting the instruction set. The rule of thumb: -march is a portability contract; -mtune is a performance hint. Confusing the two is the most common way a build gets silently locked to the wrong hardware. This is the same discipline that applies when you reason about compilation flags for multi-platform edge inference: the moment you have more than one target, “optimise for this machine” stops being a safe default and becomes a bug generator. What is the difference between -O2, -O3, and -Ofast for inference? -O2 is the sane default for inference runtimes. It enables optimisations that are safe with respect to floating-point semantics and produces fast, portable code. -O3 layers on more aggressive loop unrolling and auto-vectorisation; for tight numerical kernels it can help, but the gain is workload-dependent and sometimes negative because larger code hurts instruction-cache behaviour. In configurations we have tested, -O3 over -O2 gives an inference hot loop anywhere from no measurable change to a modest speedup — you have to measure it on the target, not assume it (observed-pattern, not a published benchmark). -Ofast is the one to be careful with. It is -O3 plus -ffast-math, and that second half quietly changes your numerical results. If you have quantised a model and validated its accuracy band, -Ofast can move you outside that band without any warning, because the floating-point relaxations interact with the reference results your quantisation was measured against. For an inference workload where a specific numerical accuracy is a requirement, -Ofast is a decision to re-validate, not a free performance flag. The general lesson mirrors what happens with what -O3, -march, and fast-math actually change in ported inference: the optimisation level and the math semantics are separate axes that the flag names blur together. When is -ffast-math safe? -ffast-math is safe when you can prove three things about your workload: it never relies on NaN or infinity propagation, it does not depend on the exact rounding of associative floating-point operations, and its accuracy requirements have slack larger than the error the reordering introduces. Signal-processing preprocessing and some feature-extraction paths tolerate it well. Post-quantisation inference kernels that feed a hard threshold usually do not, because a sub-ULP change in an accumulator can cross the threshold and flip a class. The insidious part is where you catch the problem. A fast-math-induced difference is small. It passes a loose unit test, it looks fine in a demo, and it only shows up when a specific input on a specific device produces a result on the wrong side of a decision boundary. That is per-platform QA territory — expensive, late, and hard to attribute back to a compiler flag. Enabling -ffast-math (or -Ofast) is a commitment to re-run your accuracy validation on every target, not just the build machine. If you would not want to pay for that validation cycle, do not enable the flag. How do build-time flags interact with runtime quantisation? This is where edge inference gets subtle. Quantisation is already a runtime-specific decision — INT8 or FP16 behaviour depends on the target’s numeric support and the runtime’s kernels. When you then compile the surrounding native code with flags that also change floating-point behaviour, you have stacked two independent sources of per-target divergence. Distillation and quantisation-aware training were supposed to keep quality variation inside a defined band; a fast-math build flag adds a second hidden divergence that the model-level work never accounted for. The practical consequence is that build flags and quantisation decisions must be validated together, not separately. A model that passes accuracy checks with a reference build can drift when the production runtime is compiled with -Ofast and shipped to a device whose FMA contraction differs from the build host. Getting -march, the optimisation level, and the math flags fixed before per-target validation is what keeps quality variation inside the band and avoids a full re-validation cycle per device. Our work on GPU and inference optimisation reviews build and toolchain configuration as part of evaluating whether an inference pipeline behaves consistently across constrained targets — the flag line is not an implementation detail, it is part of the numerical contract. A build-flag checklist before a multi-target ship Before an inference artifact leaves the build machine for a fleet: Replace -march=native with an explicit target or a baseline microarchitecture level. Never let the build host decide the ISA for a fleet. Default to -O2; adopt -O3 only after measuring on the target. Bigger optimisation levels are not automatically faster for inference hot loops. Treat -Ofast and -ffast-math as re-validation triggers. If you enable them, budget a per-target accuracy pass — they change numerical output. Audit -ffp-contract. FMA contraction differs across microarchitectures; if your accuracy band is tight, pin it. Validate build flags and quantisation together. They are two sources of the same per-target divergence, and testing them in isolation hides the interaction. FAQ How does a flag in GCC work in practice? A GCC flag is an argument that switches on a behaviour, sets a value, or selects a policy for the compiler. Some, like -O2, bundle many optimisation passes; some, like -march, select which instructions may be emitted; some, like -ffast-math, relax numerical rules. In practice the flags that touch target architecture and floating-point semantics are the ones that change portability and correctness, while the rest mostly affect speed and binary size. Which GCC flags change the numerical output of an inference build, and why does that matter across edge targets? The -ffast-math family (-ffast-math, -Ofast, -funsafe-math-optimizations) and -ffp-contract=fast change floating-point results; -O2 and -O3 are designed to preserve them. It matters across edge targets because the resulting difference is not identical on every microarchitecture — FMA contraction and reassociation depend on hardware, so the same source and flags can produce different bits per device, potentially flipping a thresholded decision. Why does -march=native break portability, and what should I use instead when targeting multiple devices? -march=native emits code for the build host’s exact CPU, including instruction-set extensions the target device may lack, causing illegal-instruction faults or undefined behaviour on other hardware. For a fixed target, name it explicitly with -march=<cpu>; for a fleet, pick the lowest common baseline (for x86, the x86-64-v2/v3 levels) and use -mtune only as a non-restrictive scheduling hint. What is the difference between optimisation levels (-O2, -O3, -Ofast) for inference workloads? -O2 is a safe, portable default that preserves floating-point semantics. -O3 adds aggressive loop and vectorisation transforms whose benefit is workload-dependent and best measured on the target. -Ofast is -O3 plus -ffast-math, which changes numerical results — treat it as a decision to re-validate accuracy, not a free speedup. When is -ffast-math safe, and when does it introduce quality variation I would only catch in per-platform QA? It is safe when the workload never relies on NaN/infinity handling, does not depend on exact floating-point rounding, and has accuracy slack larger than the reordering error. It becomes dangerous for post-quantisation kernels feeding hard thresholds, where a sub-ULP change can flip a class — a small difference that passes unit tests and only surfaces in per-platform QA on a specific device and input. How do build-time flags interact with runtime-specific quantisation decisions? Quantisation is already a per-target numerical decision, and fast-math build flags add a second, independent per-target divergence on top of it. Together they can push a validated model outside its accuracy band when the production runtime is compiled aggressively and shipped to hardware whose floating-point behaviour differs from the build host. That is why build flags and quantisation should be validated together, before per-target QA. Compiler flags are the least glamorous part of an edge deployment and one of the easiest places to introduce a divergence no model-level work can explain. The question worth carrying forward is not “which flags are fastest” but “which of my flags changed a number, and is that change the same on every device I ship to” — because portability of an inference artifact is decided as much in the build line as in the model.
