A C++ workload feels slow, and the reflex is to blame the language or the algorithm. Someone drafts a plan to move the hot loop to CUDA, or to hand-write SIMD intrinsics, or to lift the whole thing onto a GPU. Before any of that, there is one question worth answering first: what is the compiler already doing? If the answer is “we built it at -O0 and measured that,” the migration case is being made against a number that does not exist in any sane production build.

That is the failure this article is about. A GCC optimization flag is not a tuning knob you reach for at the end — it is what defines the baseline everything else is measured against. Get the baseline wrong and every downstream estimate (speedup, payback window, engineering cost) inherits the error. Correct flags routinely move a CPU baseline by 1.5–4x with zero code change (observed range across porting engagements; not a benchmarked figure for any specific workload), which is often enough to reset — or cancel — a port entirely.

What does a GCC optimization flag actually control?

The GNU Compiler Collection does not translate C or C++ line by line. It builds an intermediate representation, runs a sequence of analysis and transformation passes over it, and emits machine code. Optimization flags decide which passes run and how aggressively. That is the whole story: a flag is a policy over the pass pipeline, not a magic accelerant bolted onto the output.

The passes that matter most for a compute-bound workload fall into a few groups. Inlining removes function-call overhead and — more importantly — exposes code across call boundaries so other passes can work on it. Loop transforms (unrolling, interchange, fusion, vectorization) reshape the hottest part of most numeric code. Instruction selection chooses which CPU instructions to emit, and this is where the target-architecture flags earn their keep. When people say “the compiler vectorized the loop,” they mean the auto-vectorizer recognized a loop it could rewrite using the target’s SIMD registers — AVX2, AVX-512, or NEON depending on the machine — and did so.

None of this happens by default. At -O0, GCC deliberately keeps the output close to the source so debugging is predictable. The auto-vectorizer is off. Inlining is minimal. A tight for loop over a float array runs one element at a time, in the order you wrote it, with the loop variable spilled to the stack. It is correct and it is slow, and it tells you almost nothing about how fast the same code goes when the compiler is allowed to do its job.

What is the practical difference between -O0, -O1, -O2, -O3, and -Os?

The optimization levels are presets. Each one enables a bundle of individual passes, and the higher levels are supersets of the lower ones with a few exceptions. Here is the practical read, oriented toward someone establishing a baseline rather than someone memorizing the manual.

Level	What it turns on	When you use it	Baseline caveat
-O0	Nothing (default). Fast compile, faithful debugging.	Active debugging, gdb step-through.	Never a performance baseline. Measuring here inflates any port.
-O1	Basic passes: dead-code elimination, simple inlining, register allocation.	Sanity check that optimization does not break correctness.	A floor, not a realistic build.
-O2	Most safe optimizations: full inlining heuristics, instruction scheduling, and (in modern GCC, since GCC 12) auto-vectorization.	The default for production.	This is your honest baseline.
-O3	Everything in -O2 plus aggressive loop unrolling, more inlining, extra vectorization.	Compute-bound numeric code, after measuring -O2.	Can help, can hurt (code bloat, i-cache pressure). Measure, do not assume.
-Os	-O2 minus size-increasing passes; optimizes for binary size.	Embedded, edge targets where code size matters.	Different objective; not a speed baseline.

The single most important row is -O2. It is what a reasonable release build uses, it is what your competitors and upstream libraries ship with, and it is the number a porting assessment should baseline against. The claim that -O3 is always faster than -O2 is one of the more durable myths in C++ performance work; in practice -O3 sometimes regresses because aggressive unrolling bloats the instruction cache or the extra vectorization does not pay for its setup cost. The only way to know is to measure both on the actual workload.

What do -march=native, -mtune, and -flto do, and when are they safe?

-O2 and -O3 tell GCC how hard to optimize. They do not tell it what machine it is optimizing for. By default GCC targets a conservative baseline instruction set so the binary runs on almost any CPU of that family — which means it will not emit AVX-512 instructions even on a chip that has them.

-march=native fixes that by telling GCC to optimize for the exact CPU doing the compilation, enabling every instruction-set extension that CPU supports. For a numeric loop that the auto-vectorizer can widen from 128-bit to 512-bit registers, this alone can be a step change. The catch is portability: a binary built with -march=native on an AVX-512 machine will crash with an illegal-instruction fault on a CPU that lacks AVX-512. That is fine when you build and run on the same hardware class, and a genuine hazard when you build in CI and deploy to something older. When the target set is heterogeneous, name a specific baseline (-march=x86-64-v3, say) rather than native.

-mtune is the softer sibling. It schedules instructions for a particular microarchitecture without restricting which instructions are used, so the binary stays portable while being tuned for one chip’s pipeline. -flto — link-time optimization — defers optimization to the link stage so the compiler can inline and specialize across translation-unit boundaries it could not see during separate compilation. On codebases split across many .cpp files, -flto is often where a surprising chunk of the remaining speedup lives, because it lets cross-file inlining happen. It costs link time and memory, which is a build-system concern, not a correctness one.

A realistic optimized native baseline, then, is something close to -O2 -march=native -flto, measured on representative input, compared against -O3 -march=native -flto. That pair is the honest starting line. This is also the point where the C++/WASM port decision framework becomes meaningful — you cannot compare a target runtime fairly until the native side is tuned.

Why does building at -O0 make a porting decision look favourable?

Here is the mechanism, stated plainly. A porting assessment reports a speedup: “moving this to the GPU gets us Nx.” That number is a ratio, and the denominator is the CPU baseline. If the baseline was built at -O0, the denominator is artificially small, so the ratio is artificially large. The port looks like a slam dunk because it is being compared against a strawman.

Consider a worked example with explicit assumptions. Suppose a preprocessing kernel measures 400 ms at -O0. A GPU prototype does the same work in 40 ms, so the assessment reports a 10x speedup and recommends the port. Now rebuild the CPU version at -O2 -march=native -flto and it drops to 120 ms — a 3.3x improvement from flags alone (illustrative figures; the exact multiplier is workload-dependent). The real GPU speedup against the honest baseline is 120/40 = 3x, not 10x. Against 3x, the engineering cost of a CUDA rewrite, the maintenance burden of a second code path, and the hardware dependency may no longer clear the bar. The port that looked mandatory is now a judgment call, and quite possibly a defer.

We see this pattern regularly. The compiler had already solved most of the problem for free, and nobody checked before scoping the rewrite. This is why a credible GPU performance audit starts by rebuilding the baseline at production flags — the integrity of every speedup and ROI figure downstream depends on that denominator being real. The same discipline applies when reading raw throughput numbers: our discussion of what GFLOPS on CPU actually measures covers the related trap of trusting a peak number the code never reaches.

How much speedup can flag tuning alone deliver before you consider a port?

There is no universal number, and anyone who gives you one is selling something. What is defensible is a range and the conditions that place a given workload within it.

• Memory-bound code (streaming over large arrays, low arithmetic per byte) sees modest gains from flags — often 1.2–1.5x — because the bottleneck is bandwidth, not instruction throughput. Vectorization cannot help what is waiting on RAM. See our note on memory-bandwidth-bound inference for why this ceiling is real.
• Compute-bound loops with regular access patterns and no data dependencies are the sweet spot for auto-vectorization; -O3 -march=native can deliver 2–4x here because the loop widens from scalar to SIMD.
• Branch-heavy or pointer-chasing code sees the least benefit — the compiler cannot vectorize what it cannot prove is safe to reorder.

The honest planning heuristic (observed across engagements, not a benchmarked rate): assume flag tuning can deliver somewhere in the 1.5–4x band for compute-bound C++ before you have written a single line of GPU code, and treat any assessment that skipped this as reporting an unknown, not a result. The SIMD headroom on the CPU preprocessing path is a concrete case of this same lever applied to media pipelines.

What are the correctness risks of aggressive flags like -ffast-math?

Not every fast flag is free. -ffast-math is the one to treat with real caution. It enables a cluster of assumptions — that floating-point operations are associative, that no NaNs or infinities appear, that signed zero does not matter — none of which are guaranteed by IEEE 754. Under these assumptions GCC can reorder and simplify floating-point arithmetic in ways that change results. For a rendering kernel where the last bit does not matter, that is a fine trade. For a numerical solver, a financial calculation, or anything where reproducibility across builds is a requirement, -ffast-math can silently change your answers.

The failure mode is nasty precisely because it is silent: the code runs, produces plausible output, and disagrees with the reference by an amount that only shows up in a downstream comparison or an edge case months later. If you use it, use it scoped to the specific translation units that can tolerate it, and validate against a non-fast-math reference. The related numerics question of how logit versus sigmoid choices behave on a serving-path port is a reminder that precision decisions travel with the code across a migration.

FAQ

How should you think about a gcc optimization flag in practice?

A GCC optimization flag selects which analysis and transformation passes run over the compiler’s intermediate representation and how aggressively. In practice it decides whether the auto-vectorizer runs, how much inlining happens, and which CPU instructions get emitted — so a flag is a policy over the pass pipeline, not an add-on that speeds up finished code.

What is the practical difference between -O0, -O1, -O2, -O3, and -Os?

-O0 disables optimization for faithful debugging and should never be a performance baseline. -O1 enables basic safe passes; -O2 enables most safe optimizations (including auto-vectorization in modern GCC) and is the realistic production baseline. -O3 adds aggressive unrolling and vectorization that can help or hurt depending on cache pressure, and -Os optimizes for binary size rather than speed.

What do -march=native, -mtune, and -flto do, and when are they safe to use?

-march=native targets the exact build CPU and enables all its instruction extensions — fast, but the binary crashes on older CPUs, so it is safe only when build and deploy hardware match. -mtune schedules for one microarchitecture while staying portable, and -flto optimizes across translation units at link time, often recovering cross-file inlining gains at the cost of link time.

Why can building at -O0 make a porting decision look artificially favourable?

A reported speedup is a ratio whose denominator is the CPU baseline. An -O0 baseline is artificially slow, so the ratio is artificially large and the port looks mandatory. Rebuild the baseline at -O2 -march=native -flto and the denominator shrinks, often cutting the apparent GPU speedup by several times and turning a slam-dunk port into a judgment call.

How much speedup can flag tuning alone deliver before you consider a port?

Compute-bound loops with regular access patterns can gain 2–4x from -O3 -march=native through vectorization; memory-bound code sees around 1.2–1.5x because bandwidth is the ceiling; branch-heavy code sees the least. As a planning heuristic, assume 1.5–4x is available from flags on compute-bound C++ before any GPU code is written — an observed range across engagements, not a benchmarked rate.

What are the correctness and portability risks of aggressive optimization flags like -ffast-math?

-ffast-math assumes floating-point associativity and the absence of NaNs, infinities, and signed zero — none guaranteed by IEEE 754 — so it can silently change numerical results. It is acceptable for kernels where the last bit does not matter but hazardous for solvers or anything requiring reproducibility. Scope it to specific translation units and validate against a non-fast-math reference.

Where do compiler flags fit in a performance and porting assessment baseline?

They define the denominator. A credible assessment rebuilds the CPU baseline at production flags (-O2 -march=native -flto) before measuring any target runtime, because every speedup, payback, and ROI figure inherits the baseline’s accuracy. Skipping compiler tuning ports away a problem the compiler could have solved for free.

The question worth asking before the rewrite

The interesting decision is rarely “GPU or not.” It is “against what baseline?” A port measured against -O0 is comparing the target runtime to a build nobody would ever ship, and the resulting number is a fiction that survives into the ROI slide. Rebuild at optimized native flags first, then ask whether the remaining gap justifies a second code path, a hardware dependency, and the maintenance cost that follows. Often the compiler has already closed the gap — and the honest baseline is the artifact that tells you so.