CPU and GPU benchmarks measure different execution models CPU benchmark scores (Geekbench, Cinebench, Passmark CPU) and GPU benchmark scores (FP16 TFLOPS, 3DMark, render benchmarks) are not on the same scale and cannot be meaningfully compared against each other. They measure performance on different types of workloads under different execution models. For AI system benchmarking, what matters is how the CPU and GPU work together as a pipeline — not their individual scores. The question that drives procurement decisions is not “is this GPU faster than that CPU”, it is “which stage of my pipeline is starving the others”. Why the scores aren’t directly comparable CPU performance is measured primarily in single-threaded and multi-threaded throughput on operations that require sequential decision-making, branch prediction, and low-latency memory access. CPU architecture optimises for latency per instruction. GPU performance is measured in aggregate throughput across thousands of parallel simple operations. GPU architecture optimises for throughput per watt, not latency per instruction. Metric CPU benchmark focus GPU benchmark focus Core count 8–128 complex cores Thousands of simple cores Parallelism Task parallelism Data parallelism Memory model Cache-heavy, low latency High bandwidth, high latency Optimal operation Sequential with branching Uniform operations on arrays A Geekbench score and an FP16 TFLOPS number describe different physical capabilities of different silicon. Putting them on the same axis is a category error, and the procurement decisions that follow from that error are predictable: teams over-spec the component they can measure cleanly and under-spec the one they cannot. What a system-level AI benchmark actually measures A useful AI system benchmark measures the entire inference or training pipeline, including both CPU and GPU contributions: Data loading throughput (CPU + storage) Preprocessing throughput (CPU) Model forward pass (GPU) Postprocessing (CPU/GPU) End-to-end latency (all stages plus host-to-device transfers) In our experience profiling production pipelines, the rule of thumb is simple: if the end-to-end benchmark is slower than the GPU-only model benchmark by more than 20%, there is a CPU or I/O bottleneck worth investigating. This is an observed pattern across the engagements we have run, not a published threshold — but it has held up consistently enough that we use it as a triage filter before any deeper profiling. How do you design a system-level benchmark that captures both CPU and GPU? System-level benchmarking requires measuring the full pipeline rather than individual components. The methodology is to instrument the production code path with timing markers at each stage boundary (data loading, preprocessing, host-to-device transfer, inference, postprocessing), then collect per-stage latency distributions over a sustained run. The key measurement is the throughput ratio between adjacent stages. If the CPU preprocessing stage produces data at 500 samples per second but the GPU inference stage can consume 800 samples per second, the system throughput is 500 samples per second — bottlenecked by CPU preprocessing. Upgrading the GPU would waste budget; upgrading the CPU or parallelising preprocessing would improve system throughput. We implement this instrumentation using Python’s time.perf_counter_ns() at stage boundaries, collecting results into a ring buffer that reports P50, P95, and P99 latencies per stage every 60 seconds. The overhead is negligible — nanosecond-precision timing adds well under 0.001% to total execution time — and the visibility transforms capacity planning conversations. For multi-GPU systems, the benchmark must also measure inter-GPU communication overhead. NVLink-connected GPUs show observably lower all-reduce latency than PCIe-connected GPUs, which translates to meaningfully higher training throughput on communication-heavy models such as large-batch distributed training. We measure this using NCCL’s built-in nccl-tests suite, specifically the all_reduce_perf test at the message sizes the training workload actually uses, rather than at the default sizes the tool ships with. Measuring system-level bottlenecks in practice The most informative system-level measurement is the pipeline utilisation profile: what percentage of wall-clock time each stage of the pipeline is actively processing versus waiting for upstream or downstream stages. We instrument pipelines with per-stage timestamps and compute three metrics: stage throughput (how fast each stage could run in isolation), stage wait time (how long each stage waits for input), and stage queue depth (how much work is buffered between stages). A stage with high throughput but high wait time is over-provisioned. A stage with low throughput and zero queue depth is the bottleneck. For multi-GPU training, the pipeline profile includes communication stages — gradient synchronisation and parameter broadcasting. In the engagements where we have measured this on PCIe-connected multi-GPU systems training large transformer models, a non-trivial fraction of wall-clock time is spent in all-reduce operations rather than in compute. Moving to NVLink interconnect reduces that share substantially, recovering training throughput equivalent to adding compute capacity — without buying another GPU. The exact recovery depends on model size and batch configuration, which is precisely why a system-level benchmark is needed to size the investment. These measurements require no specialised tooling. Python’s time.perf_counter() at stage boundaries, logged to a CSV file, provides sufficient resolution for everything except the very tightest inference loops. The investment is roughly thirty minutes of instrumentation work, and the insight persists for the life of the deployment. Why isolated component scores mislead procurement Individual GPU or CPU scores give an incomplete picture of AI system performance. Memory bandwidth, PCIe bandwidth, CPU preprocessing throughput, and storage I/O all interact to determine actual system throughput, and isolated component benchmarks often mislead procurement decisions because they describe a component in isolation rather than the role that component plays in a specific pipeline. System-level benchmarking that combines CPU and GPU metrics — measuring end-to-end pipeline throughput rather than isolated component performance — produces the most actionable data for hardware selection. A system where the CPU preprocessing stage takes longer than the GPU inference stage has a CPU bottleneck regardless of how fast the GPU benchmark score is. Measuring both simultaneously, under realistic data pipeline conditions, reveals which component investment delivers the greatest throughput improvement. This is also why GPU utilisation reported by nvidia-smi is a deeply unreliable proxy for system efficiency. The GPU-Util field reports whether a kernel was active in the last sampling window, not whether the GPU was doing useful work at full memory bandwidth. A pipeline that keeps the GPU 95% “utilised” while the device is mostly waiting on host-to-device transfers looks healthy in nvidia-smi and is, in fact, paying for capacity it cannot use. The hidden cost of GPU underutilisation is rooted in this same gap between component-level metrics and system-level reality. LynxBench AI treats stage-level timing across the CPU and GPU halves of the AI Executor as required disclosure of a system benchmark, because a single end-to-end number cannot tell the operator whether the bottleneck is data loading, host preprocessing, device compute, or transfer overhead. The question to put to any system-level CPU-plus-GPU benchmark is whether the breakdown by stage is reported alongside the aggregate — or whether the result is a single throughput number that obscures which component the next investment should target. FAQ
