Geekbench AI runs ten AI workloads, each at three data types, on whichever CPU, GPU or NPU you select, through a framework available on the device. It reports three overall scores: Single Precision, Half Precision and Quantized. Each score is the geometric mean of workload scores that have been normalised to a baseline machine (1,500 means parity with it) and adjusted for accuracy. That makes a Geekbench AI score a reasonable summary of short on-device inference tasks under the conditions the benchmark chose. It does not predict how fast a local open-weight model will generate tokens, which depends on memory bandwidth, memory capacity and runtime support.
The Personal Edition installs with pip install lynxbench-ai on Python 3.11 or later, on Linux or on Windows via WSL2, and runs a fixed model catalogue on your own hardware. A run takes 15 to 30 minutes and produces a release-named 26Q3 result. It gives a measured answer on stock PyTorch for the machine in front of you. The getting-started guide for the Personal Edition covers the install and the first run.
What Geekbench AI runs: workloads, backends and frameworks
Primate Labs describes Geekbench AI as a cross-platform benchmark that runs identical workloads on Android, iOS, Windows, macOS and Linux. You choose the GPU, the CPU or a dedicated NPU as the device under test, and you pick from the AI frameworks available on that device, such as Core ML or QNN. Linux is among the supported platforms.
By default each workload runs for at least five iterations and at least one second. Accuracy is checked against a fixed reference: the predictions of the Full Precision model running on an Intel i7 CPU serve as ground truth, and each task is scored with its standard evaluation metric. Workload scores are calibrated against a Lenovo ThinkStation P340 with a Core i7-10700, then adjusted by a factor based on accuracy.
Each of those design choices has a cost. A geometric mean across ten tasks smooths over the one task that matters to you, and a run of at least one second per workload describes a short burst, not an hour of generation.
Why one machine scores so differently on Single Precision, Half Precision and Quantized
The three scores run the workloads at different numeric formats, and the same hardware can behave very differently at each. Narrower formats fit more arithmetic into the same silicon and move less data through memory, so the same chip can have very different peak throughput depending on the format, and an FP32 figure cannot be compared with an INT8 figure as if both measured the same work. AMD’s HIP documentation makes a related point: precision changes both the FLOPs and the bytes of a kernel, and kernels above the GPU’s ridge point are compute-bound.
On an NPU such as the one in the Ryzen AI 7 350, the small-matrix multiply instructions are the only way to get anywhere near peak TOPS, and Daniel Estévez (2026) takes the marketed 50 TOPS to be an INT8 figure against a theoretical 58.98.
Quantisation does not help every workload equally. The roofline analysis in the “LLM Inference Unveiled” survey separates three cases: quantisation helps a lot when a layer stays memory-bound, helps when it moves the layer from memory-bound to compute-bound, and does nothing when the layer was already compute-bound. Geekbench AI’s accuracy adjustment adds one more term, since each workload score is adjusted by a factor based on accuracy against the full-precision ground truth. Expedera (2022) warns that engines which quietly lower precision in heavy layers can look faster than engines that keep it throughout, so precision has to be read alongside any speed figure.
Our published 26Q3 data shows how uneven precision support is on a single executor. In the 26Q3 results, the NVIDIA GeForce RTX 3080 scores 398.2 for DistilBERT inference at fp16, 278.1 at int8 and 0.0 at fp8, where 0.0 means that precision produced no result on that executor. Scores at different precisions answer different questions, and whatever a lower precision gains in speed still has to be checked against output quality.
Does the runtime Geekbench AI picks change the score?
It can. Primate Labs says developers use Geekbench AI to find the best combination of framework and model, so results vary with the framework chosen. The workloads are identical across platforms, but the framework that executes them is the user’s choice. A comparison between a phone using QNN and a laptop using another framework compares two hardware-plus-runtime pairs, not two chips.
Hivenet sums up the dependency this way: performance equals hardware plus model, precision, runtime, software configuration and workload, and dropping any one term makes the comparison less useful. When a vendor runtime is the best path on its own silicon, the higher score it gives that platform is legitimate. It also means the score belongs to that stack, which your own tools may not use.
How far to trust your score against the Geekbench Browser charts
Primate Labs presents the browser as a way to compare results instantly, but the Primate Labs material cited here does not describe how submissions are controlled for benchmark version, framework, power plan or cooling. Peak figures also assume conditions that real runs break: rising temperature, power limits, memory stalls, and a laptop that drops performance on battery. For both reasons, treat a comparison with Geekbench Browser results as a rough placement, not a measurement.
Before you read anything into a gap, check these on both results:
- the same Geekbench AI version
- the same device selection (CPU, GPU or NPU) and the same framework
- the same score (Single Precision, Half Precision or Quantized), never a mix
- the same platform and operating system
- for laptops, mains power rather than battery
If any of these differ, the gap describes the configuration as much as the hardware. The MLPerf Inference paper (Reddi et al., 2020) shows the alternative approach to ML benchmarks: a prescribed set of rules built specifically to make results comparable across very different architectures.
Why a good Geekbench AI result can still fail to predict your local model
When a local LLM generates text, the decode stage repeats for every token, and decode is typically limited by memory bandwidth, not arithmetic. RooflineBench (2026) describes prefill as typically compute-bound and autoregressive decoding as shifting the bottleneck to memory bandwidth, because model weights and the KV cache have to be moved to the compute cores for every token. The “LLM Inference Unveiled” survey finds every decode-stage computation memory-bound, running well below the GPU’s computational capacity. Even large-batch LLM inference stays memory-bound, with most GPU compute left underutilised.
Geekbench AI’s ten workloads each run for at least a second and are folded into a geometric mean. Hivenet makes the same point about TOPS: a TOPS figure tells you nothing direct about how fast an LLM generates text, because tokens per second depends on model, quantisation, prompt and output length, memory bandwidth, batch size, serving engine, kernels and software version. A processor with very high theoretical throughput can still be the wrong hardware, and a modest one can be exactly right.
Worked example: a memory ceiling that more TOPS cannot move
This example follows the arithmetic in LocalAIMaster’s NPU TOPS ranking:
- A 7B model at Q4_K_M takes about 0.6 GB per billion parameters, or roughly 4.2 GB of weights.
- Generating each token reads every weight once.
- On a system with about 135 GB/s of memory bandwidth, 135 / 4.2 gives a hard ceiling of about 32 tokens per second.
- Doubling the NPU from 45 to 85 TOPS does not move that ceiling, because the memory bus is the bottleneck.
Runtime support decides whether the NPU contributes at all. According to LocalAIMaster, Ollama’s hardware documentation lists NVIDIA, AMD via ROCm, Apple Metal and Vulkan backends, with no XDNA, Core Ultra NPU or Hexagon backend. A 50-TOPS NPU with no runtime targeting it produces zero tokens per second. The fit also depends on the device. RooflineBench reports that a 1.5B model can come close to saturating a Jetson Orin or a Raspberry Pi while leaving an RTX 3090 badly underused, because their ridge points differ.
Which number answers which question
| Question you are asking | What answers it |
|---|---|
| How does this device perform on Geekbench AI’s on-device workloads, under its chosen framework? | A Geekbench AI score, read within one score type and one framework |
| Is this hardware remotely large enough, and does it support the precision I need? | Vendor peak specifications: memory, supported precision, generation |
| How does this executor behave on common model architectures on the public PyTorch stack? | A LynxBenchAI 26Q3 result |
| How many tokens per second will my model give in my runtime? | Timing your own model, prompt pattern and runtime on the machine |
| Where is my workload actually bottlenecked? | Profiling the running workload |
Measuring what Geekbench AI leaves out on your own hardware
The final hardware decision needs a benchmark that uses your own model and traffic pattern. For LLM inference that means time to first token, time per output token, tokens per second, concurrency, memory use and cost. Our methodology treats empirical execution of real software workloads as the reference standard, with specification-based estimates as a fallback for cases where execution is not possible. The broader method, with its separate steps, is set out in how to benchmark your PC for AI.
LynxBenchAI benchmarks AI performance on your own hardware with stock PyTorch. The 26Q3 catalogue covers MobileNetV3, DistilBERT, DistilGPT2, Flan-T5 Small and GraphGPS on Peptides, plus dense and sparse matrix multiplication. Each test is one continuous timed window after a discarded warm-up. A run needs about 15 GB of free disk and downloads about 2 GB of model data once.
A LynxBenchAI result shows how that hardware performs with stock PyTorch. It is not a measurement of your workload. Every result carries the name of the release that produced it, currently 26Q3, and is read only against results from the same release name. The Personal Edition is released and free for non-commercial use. Press, Pro and Enterprise editions are not released and are contact-gated.
When a measured number falls well short of what the hardware should manage, the next question is where the bottleneck is. That is GPU performance engineering, and profiling GPU kernels to find the real bottleneck comes before optimising. The question that remains open for any personal machine is whether the model you care about fits in memory, and how fast that memory can feed it. A Geekbench AI score cannot settle that.
FAQ
Are TOPS and TFLOPS the same?
No. TOPS usually counts integer operations at a stated precision such as INT8, while TFLOPS counts floating-point operations, and figures at different precisions do not measure the same work. Vendors also use different counting conventions, so an unlabelled TOPS figure is only safe to compare within one vendor’s chips.
Is 80 TOPS good for AI?
There is no universal threshold. A TOPS figure is good only if the hardware meets the workload’s memory, latency, precision and software requirements. For local LLMs, the fastest-rated NPU is usually not the fastest machine.
Is a 4GB graphics card enough?
It depends on the model, because the weights have to fit in memory before speed matters at all. A 7B model at Q4_K_M needs roughly 4.2 GB for its weights alone, before the KV cache, so it does not fit in 4 GB.
