Geekbench’s ML benchmark is better than its CPU score — but not sufficient Geekbench 6 added an ML benchmark subtest that runs inference operations on CPU and GPU. This is more relevant to AI than the general compute subtests because it exercises the instruction types (matrix multiply-accumulate, activation functions) that AI frameworks use. However, it remains insufficient for production AI hardware decisions. What Geekbench ML benchmark tests In our testing, this pattern holds across a range of hardware configurations. The Geekbench ML benchmark runs standardized inference tasks: Task Operation type Hardware exercised Edge-size object detection INT8 inference NPU, GPU INT8 units Background removal Segmentation GPU Style transfer CNN inference GPU Portrait segmentation Small CNN CPU/GPU Platform targets: Tests are run on CPU, GPU (via Metal/OpenCL/DirectML), and dedicated neural processors (NPU/ANE on Apple Silicon, Neural Engine on Qualcomm). Why it’s insufficient for production AI Model size mismatch: The Geekbench ML models are small edge-inference networks. The models used in production AI (7B–70B LLMs, high-resolution diffusion models, large vision transformers) have fundamentally different compute profiles. Fixed-batch inference only: Geekbench runs single-inference mode. Production serving at scale cares about throughput at realistic concurrency levels (10–100 concurrent requests). No framework stack: Geekbench runs models directly; production workloads run through PyTorch or TensorFlow with their overhead, kernel selection, and graph optimization behaviors. Short duration: Thermal throttling effects that dominate sustained performance are not captured in short-duration benchmark runs. When Geekbench ML score is useful Comparing consumer devices (laptops, workstations) for basic AI inference capability A fast first filter to eliminate obviously underpowered hardware Comparing Apple Silicon generations (the Neural Engine efficiency difference is captured) Not useful for: Comparing discrete GPU performance for LLM serving Hardware procurement decisions for training infrastructure Predicting production throughput at scale What to run for production AI evaluation For LLM inference: benchmark the target model at the target context length using the actual serving framework (vLLM, TGI, Ollama). For training: run the actual training script on a sample batch for 10+ minutes to capture sustained performance. Why benchmarks fail to match real AI workloads provides the structural analysis of why the gap exists between standardized benchmarks and production AI performance. What would a useful AI benchmark look like? A useful AI benchmark for hardware evaluation would measure four things that Geekbench does not: sustained throughput (not burst), at production-representative batch sizes, using production-representative model architectures, with memory bandwidth characterisation under load. The closest existing tool is MLPerf, but its fixed model configurations and submission-optimised results make it better for comparing vendor claims than for predicting your specific workload performance. A practical AI benchmark for hardware evaluation should use your actual model, your actual batch size, and your actual data pipeline — run for at least 30 minutes to capture thermal steady-state behaviour. We have developed internal benchmarking scripts that run a sequence of three tests: a 2-minute burst test (matches what Geekbench measures), a 30-minute sustained test (reveals thermal throttling and power limit impacts), and a memory bandwidth saturation test (reveals the memory wall). The ratio between burst and sustained throughput — what we call the “sustain ratio” — typically ranges from 0.75 to 0.95 depending on cooling adequacy. Hardware with a sustain ratio below 0.8 has a thermal design problem that will reduce effective capacity in production. Geekbench’s ML benchmark score is useful as a first-pass filter: if a system scores anomalously low relative to its hardware specification, something is misconfigured. But hardware procurement decisions require workload-specific benchmarking, not generic scores. From Geekbench scores to hardware decisions Geekbench scores are most useful as anomaly detectors: if a system with known-good hardware scores 30% below the expected range for that hardware, something is misconfigured. Common causes we have diagnosed through Geekbench anomalies include: BIOS power management settings limiting CPU boost frequency, RAM running at lower-than-rated speed due to XMP profiles not being enabled, and thermal throttling from improperly mounted CPU coolers. For AI hardware decisions, we supplement Geekbench with three targeted measurements: bandwidthTest from the CUDA samples (measures GPU memory bandwidth), a 30-minute sustained inference run at production batch size (measures thermal steady state), and a data loading benchmark (measures the CPU-to-GPU pipeline throughput). These three tests, combined with a Geekbench score to confirm general system health, provide a complete hardware evaluation in about 2 hours. The Geekbench score alone would take 5 minutes but would leave the AI-critical performance characteristics unmeasured. For AI-specific hardware evaluation, we supplement Geekbench with targeted micro-benchmarks: bandwidthTest from the CUDA samples (measures GPU memory throughput), p2pBandwidthLatencyTest (measures GPU-to-GPU communication for multi-GPU systems), and a custom batch inference timing script that measures the exact metric that matters — inferences per second at the target batch size. Together, these four tests (including Geekbench as a general health check) require about 45 minutes and provide a complete picture of AI hardware capability that no single benchmark can deliver.
