SSE vs AVX: What CPU SIMD Portability Teaches About Performance-Portable Code

Recompile with -mavx2 and you get free speedups. That is the sentence that sinks more vectorised CPU code than any other single assumption, and it is wrong in three different ways at once. Instruction-set availability is not the same thing as performance portability. A binary built for AVX2 faults outright on a CPU that only exposes SSE4. AVX-512 can clock the core down while it runs, so a wider vector unit finishes a mixed workload slower than the narrower one it replaced. And if your loads are unaligned, the extra register width you paid for evaporates into split-load penalties. The register got wider; the discipline that turns width into throughput did not come along for the ride.

This is the same failure that shows up when teams treat cross-vendor GPU portability as a translation problem. Moving CUDA to ROCm moves the API surface but not the performance characteristics — occupancy, memory coalescing, and launch overhead all shift underneath you. Moving SSE code to AVX moves the register width but not the memory-access discipline that actually determines throughput. Teams that treat SIMD width as a compiler flag accumulate the same vendor-specific debt on CPU that they accumulate on GPU. The lesson generalises: reaching a different instruction-set target is not the same as designing an algorithm that stays performant across targets.

What SSE, AVX, AVX2, and AVX-512 actually are

The x86 SIMD lineage is a sequence of register-width and instruction expansions, not a clean generational replacement. SSE introduced 128-bit XMM registers in 1999 and its later revisions (SSE2 through SSE4.2) added integer, double-precision, and string operations. AVX (2011) widened the vector registers to 256 bits (YMM) and introduced a three-operand VEX encoding, but the first AVX only vectorised floating-point. AVX2 (2013) extended the 256-bit path to integer operations and added gather loads and fused multiply-add. AVX-512 widened again to 512-bit ZMM registers with masked operations and per-lane predication — but it arrived fragmented into a dozen feature subsets (F, DQ, BW, VL, VNNI, and more) that ship inconsistently across Intel and AMD parts.

Per Intel’s and AMD’s published instruction-set references, these are the register widths and element counts that matter when you reason about a vectorised loop:

Extension	Register	Width	fp32 lanes	fp64 lanes	int8 lanes	First shipped
SSE / SSE2	XMM	128-bit	4	2	16	1999–2001
AVX	YMM	256-bit	8	4	(fp only)	2011
AVX2	YMM	256-bit	8	4	32	2013
AVX-512	ZMM	512-bit	16	8	64	2016 (Intel), 2022 (AMD Zen 4)

The naive throughput model reads straight off this table: AVX2 processes twice the lanes of SSE, AVX-512 twice again, so a well-vectorised kernel should run 2–4x faster as you climb. That model is a ceiling, not a forecast. The gap between the ceiling and the achieved number is where the interesting engineering lives, and it is the same gap that separates peak FLOPS from delivered throughput on a GPU. We cover the CPU side of that in what CPU GFLOPS means for peak versus achieved throughput; the reasoning transfers directly.

Why doesn’t recompiling for AVX deliver a proportional speedup?

Three mechanisms erode the register-width advantage, and they compound.

Memory bandwidth, not compute, is usually the wall. Doubling your vector width does nothing if the loop is already saturating a memory channel. A kernel bound by DRAM bandwidth reads and writes the same number of bytes whether it uses 128-bit or 512-bit registers; the wider registers just wait longer between useful cycles. This is the CPU expression of a claim we make constantly about accelerators: the reported peak metric predicts almost nothing about the sustained number under realistic load (observed pattern across the porting engagements we run; not a benchmarked rate). If your arithmetic intensity is low — a handful of FLOPs per byte loaded — SIMD width is not your bottleneck and widening it is wasted effort.

Unaligned loads split into two operations. SIMD load and store instructions want their data on natural boundaries — 16 bytes for XMM, 32 for YMM, 64 for ZMM. When the data isn’t aligned, the hardware issues a split access that touches two cache lines, and the penalty scales with register width because the wider the load, the more likely it straddles a boundary. A loop that was fine at 128-bit alignment can regress when recompiled to 256-bit if the buffers were never aligned to 32 bytes. The compiler flag changed; the allocator did not.

AVX-512 can downclock the core. This is the mechanism most teams miss. Running 512-bit instructions — particularly the heavy floating-point and multiply-heavy variants — draws enough power that many Intel server parts drop the core frequency to stay inside thermal and voltage limits. On workloads where AVX-512 instructions are sparse relative to scalar and lighter vector code, the frequency penalty can outweigh the width gain, so the AVX-512 binary finishes a mixed workload slower than the AVX2 one. We walk through a concrete instance of this trade-off in the context of media encoding in FFmpeg AVX-512 performance on AMD Ryzen, where the answer genuinely depends on the microarchitecture and the instruction mix.

When does AVX-512 hurt, and how do you know?

The downclocking behaviour is microarchitecture-specific, so the honest answer is “measure it,” but there are structural signals that tell you whether to bother measuring at all.

Diagnostic checklist — is AVX-512 worth it for this workload?

Is the hot loop compute-bound (high FLOPs-per-byte) rather than memory-bound? If memory-bound, AVX-512 will not help — stop here.
Do AVX-512 instructions make up a large, contiguous fraction of the hot path, or are they interleaved with scalar code? Sparse, interleaved use is where the frequency penalty bites hardest.
Is your target a specific known CPU (you control the deployment) or a fleet with mixed silicon? Downclocking severity differs sharply between Intel Skylake-SP, Ice Lake, and AMD Zen 4 — a fleet answer must hold across all of them.
Have you measured wall-clock time on the actual target, not just instruction counts? A profiler showing more retired vector instructions is not the same as a faster program.
Are you using the lighter AVX-512 subsets (masking, VL for 256-bit operations on 512-capable cores) rather than the heavy 512-bit FP path? The lighter subsets often carry the frequency penalty too, but far less of it.

If you cannot answer the first two questions with “yes,” the AVX-512 build is a coin-flip at best and a regression at worst. This is exactly the kind of reasoning the GPU performance engagements we run apply on the accelerator side: the extension is available, but availability is not a performance argument.

How do you ship one binary that runs well across CPUs?

The portable-code answer to a fragmented instruction set is runtime dispatch: compile the hot kernels multiple times — once per SIMD tier — detect the CPU’s capabilities at startup with CPUID, and select the widest supported path. GCC and Clang support this through function multiversioning (__attribute__((target_clones("sse4.2","avx2","avx512f")))) and through the resolver pattern; libraries like Intel’s oneDNN, NumPy’s SIMD layer, and FFmpeg do exactly this by hand for their inner loops.

Runtime dispatch is what lets a single distributed binary avoid the SIGILL fault on SSE-only hardware while still using AVX2 or AVX-512 where present. But dispatch alone is not sufficient — the dispatched kernels still have to be written for portable performance, which means the alignment and access-pattern discipline below applies to every version.

Comparison: three strategies for shipping vectorised CPU code

Strategy	Runs on old CPUs?	Uses new SIMD when present?	Binary count	Main risk
Compile for baseline (SSE4.2)	Yes	No — leaves 2–4x on the table	One	Under-uses modern silicon
Compile for newest (`-mavx512f`)	No — faults on older CPUs	Yes, on capable parts	One	SIGILL on deployment; downclock risk
Runtime dispatch (multiversioning)	Yes	Yes, per-tier selection	One (fat)	Build complexity; each path must be tuned

The third row is the only one that is genuinely portable in both senses — it runs and runs well — and it is the direct CPU analogue of writing a GPU kernel that adapts to occupancy and memory hierarchy rather than assuming a fixed device. The same instinct that makes teams reach for a single -march=native build is the one that makes them assume a CUDA kernel will port cleanly; we unpack the GPU version of that assumption in what software porting actually means when you move a workload to new hardware.

Which memory choices keep vectorised code fast across instruction sets?

Alignment and access pattern are where portable SIMD performance is won or lost, and they are decisions you make in the data layout, not the compiler flags.

Allocate hot buffers on the widest boundary you might dispatch to — 64 bytes covers everything through AVX-512 — using posix_memalign, aligned_alloc, or an aligned allocator in your container types. This costs a little memory and buys you aligned loads on every SIMD tier simultaneously. Prefer structure-of-arrays over array-of-structures so that a vector load pulls contiguous same-type elements rather than gathering strided fields; gather instructions exist in AVX2 and AVX-512, but they are far slower than a contiguous load and undo much of the width benefit. Keep the inner loop’s working set inside L1 or L2 where the vector units can actually be fed, because once you spill to DRAM the width of your registers stops mattering — you are back to the bandwidth wall.

These are not AVX-specific tricks. They are the CPU expression of the memory-access discipline that governs every accelerator, which is why we treat CPU SIMD characterisation as one input to a cross-target portability picture rather than a separate skill. How the same reasoning divides work across CPU, GPU, and WASM targets is the subject of heterogeneous architecture for inference across CPU, GPU, and WASM.

FAQ

How does sse vs avx work in practice?

SSE and AVX are successive x86 SIMD instruction-set extensions: SSE uses 128-bit registers, AVX/AVX2 use 256-bit, and AVX-512 uses 512-bit, each processing more data elements per instruction. In practice the wider registers only translate into speed when the workload is compute-bound, the data is aligned, and the target CPU actually exposes the instruction set — otherwise availability buys nothing.

What are the concrete differences between SSE, AVX, AVX2, and AVX-512 register widths and instructions?

SSE provides 128-bit XMM registers (4 fp32 lanes); AVX widens to 256-bit YMM (8 fp32) but initially only for floating-point; AVX2 extends 256-bit to integers and adds gather and fused multiply-add; AVX-512 widens to 512-bit ZMM (16 fp32) with masked, per-lane predicated operations. AVX-512 also ships as fragmented feature subsets that vary across Intel and AMD parts.

Why doesn’t recompiling for AVX automatically deliver a proportional speedup over SSE?

Three mechanisms erode the width advantage: memory-bandwidth-bound loops read the same bytes regardless of register width, unaligned loads split into two cache-line accesses whose penalty grows with width, and AVX-512 can downclock the core. A profiler showing more retired vector instructions is not the same as a faster program.

When does AVX-512 hurt performance through frequency downclocking, and how do I know if it applies to my workload?

AVX-512 hurts when its heavy 512-bit floating-point instructions are sparse and interleaved with scalar code, so the frequency drop outweighs the width gain. The behaviour is microarchitecture-specific, so you confirm it by measuring wall-clock time on the actual target CPU — not by counting instructions — and by checking whether the hot loop is compute-bound and contiguous in its AVX-512 use.

How do I ship one binary that runs performantly across CPUs with different SIMD support (runtime dispatch)?

Use runtime dispatch: compile the hot kernels once per SIMD tier, detect CPU capabilities at startup with CPUID, and select the widest supported path. GCC and Clang support this via function multiversioning (target_clones), and libraries like oneDNN and FFmpeg do it by hand — this avoids faulting on SSE-only hardware while still using AVX2 or AVX-512 where present.

What memory-alignment and access-pattern choices keep vectorised CPU code performant across instruction sets?

Allocate hot buffers on a 64-byte boundary to guarantee aligned loads on every SIMD tier, prefer structure-of-arrays layouts so vector loads pull contiguous same-type elements instead of using slow gathers, and keep the inner loop’s working set in L1/L2 cache. Once you spill to DRAM the register width stops mattering because you are bound by memory bandwidth.

How does the SSE-vs-AVX portability problem mirror the cross-vendor GPU portability problem?

Both problems confuse reaching a different target with staying performant across targets. Moving SSE to AVX moves the register width but not the memory-access discipline; moving CUDA to ROCm moves the API but not the occupancy and coalescing characteristics. In both cases treating the target as a compiler flag or a translation step accumulates vendor-specific debt instead of building code that adapts to the hardware it lands on.

The question worth carrying to the GPU side

The SSE-versus-AVX story is small enough to hold in your head, which is what makes it useful: one instruction set, one vendor lineage, and still three independent ways for a “free” recompile to cost you throughput. The accelerated-computing stack has the same structure at every layer. The right question is never “does this target support the wider path” but “have I designed the algorithm so that the wider path is actually faster on the silicon I will deploy to.” A CPU SIMD characterisation is one input to that answer; the cross-target portability analysis we run for GPU workloads is the same discipline applied where the stakes — and the vendor-lock debt — are larger.