An AI GPU benchmark comparison is a function of the workload chosen, the precision, the software stack and the saturation condition. Change any of those and the comparison between two GPUs can reverse. A LynxBenchAI run reports four figures. Three are category scores (Training, Inference and Compute), and each one tells you on its own how that executor handles that class of work. The fourth, GT, is an overall figure built from the three. A device can be strong at one class of work and weak at another, and a single number hides that.
LynxBenchAI’s Personal Edition installs with pip install lynxbench-ai. The published 26Q3 results you would compare against are on the LynxBenchAI benchmark and leaderboard.
What does it mean to treat AI benchmarks as decision infrastructure rather than as scores?
It means running a benchmark to answer a question someone is about to act on, not to get a number to quote. For one person buying or diagnosing a card, the questions are “which card should I buy for the work I actually do” and “why is my card slow at this specific workload”. For an organisation, they are procurement and deployment questions about practical performance, software readiness and reproducibility. Both need the same thing: evidence tied to a workload rather than to a specification sheet.
It also sets the limit of what a benchmark is for. The run supplies the figures, and the judgement about what they mean for your build stays with an engineer who knows the workload. We make the wider procurement-side case in benchmarks as decision infrastructure, not scores.
What does a decision-grade benchmark need to encode beyond its headline number?
It needs the context that produced the number and the decision the number is meant to inform. The context has four parts: the workload chosen, the precision evaluated, the software stack on each candidate, and the saturation condition held constant. The intent is the work you are buying for. If a result does not say which class of work it covers, it cannot tell you whether it applies to yours.
LynxBenchAI fixes the workload and precision per release. A release pins the model catalogue, the precisions, the correctness thresholds and the scoring formula, and every result carries the name of the release that produced it. The intent part is yours. You decide whether the decision depends on Training, Inference or Compute, and then you read that category.
Reading AI GPU benchmarks as four scores, not one
The three category scores are meant to be read on their own, not treated only as inputs to GT. GT is an ordinal aggregate. It is not a percentage, not a physical quantity and not a 0 to 100 rating, and it is not normalised against a reference device. A GT gap tells you which device ranks higher overall. It does not tell you how much faster one device is.
Published 26Q3 figures show why the categories need to stay separate:
| Device | GT | Training | Inference | Compute |
|---|---|---|---|---|
| AMD Radeon RX 9060 XT | 76.3 | 37.2 | 122.7 | 112.7 |
| NVIDIA GeForce RTX 5060 Ti | 146.4 | 133.2 | 240.4 | 61.7 |
| NVIDIA H100 80GB HBM3 | 939.5 | 709.3 | 1678.1 | 487.1 |
| AMD Instinct MI300X | 986.7 | 727.2 | 1304.0 | 1023.3 |
In the 26Q3 results the RTX 5060 Ti scores higher than the RX 9060 XT on GT, Training and Inference, while the RX 9060 XT scores higher on Compute (112.7 against 61.7). The MI300X scores higher than the H100 80GB HBM3 on GT, Training and Compute, while the H100 80GB HBM3 scores higher on Inference (1678.1 against 1304.0).
Other benchmark designers also report separate scores. The MLPerf Inference paper (Reddi et al., 2019) defines four scenarios (single-stream, multistream, server and offline) and reports that an accelerator “may stand out in one category while underperforming in another”. The same paper found that adding the server scenario’s latency constraint costs some systems as little as 3% of their offline throughput and others as much as 50%. SPEC CPU 2026 is organised as four suites, with SPECspeed measuring the time to complete single tasks and SPECrate measuring throughput. So these benchmarks give one system several scores.
Novabench takes a different approach. It combines rendering, compute and VRAM results into one GPU score with a geometric mean, after a scaling step calibrated against reference hardware. Its own documentation says that “there is no such thing as a fully objective single-number GPU benchmark” and points users to the per-workload sub-scores when their work does not match its assumptions. That design suits general graphics use. LynxBenchAI differs in two ways: its categories are AI classes of work, and its aggregate has no reference-device normalisation. If your decision is about AI work, the AI category score is the figure the decision should rest on.
Compute is not a stand-in for inference
A higher Compute score does not mean a higher Inference score. In autoregressive decoding at small batch sizes, every generated token reads the model weights and the attention cache from memory and does little arithmetic per byte. That makes the step limited by memory bandwidth rather than by peak arithmetic throughput.
The 26Q3 results show the split. The NVIDIA H200 and the H100 80GB HBM3 score almost the same on Dense MatMul at fp16 (1257.0 and 1264.3). Their Inference scores differ: 1914.6 against 1678.1, and on the DistilGPT2 fp16 inference test 4070.9 against 3797.7. Whatever separates those two devices on inference, dense arithmetic throughput does not account for it.
When a decision-grade benchmark and its headline score disagree about which infrastructure choice to make, which should a technical leader trust, and why?
Trust the score for the class of work the decision is about, and below that, the per-test results for the architectures closest to yours. GT is still useful as a coarse summary when you are shortlisting across many devices. Once the shortlist is down to two, GT is the least informative of the four figures, because it averages away the very trade-off you are trying to decide.
The per-test level can reorder devices even within one category. In the 26Q3 results the H100 80GB HBM3 scores 564.1 on MobileNetV3 fp32 training against the MI300X’s 275.4, but the MI300X scores 1571.3 on DistilGPT2 fp32 training against the H100’s 1159.0. A team training decoder-only transformers and a team training convolutional vision models would read the same Training category differently.
The catalogue covers architecture types rather than applications. It includes a convolutional network (MobileNetV3), encoder-only, decoder-only and encoder-decoder transformers (DistilBERT, DistilGPT2, Flan-T5 Small) and a graph transformer (GraphGPS on Peptides), with dense and sparse matrix multiplication standing in for raw compute. A result is broadly indicative of the out-of-the-box PyTorch experience on that hardware. It does not run or measure your application. The result covers the fixed catalogue of one named release. The gap between a leaderboard figure and your own model is covered in inference benchmarks versus workload evaluation.
Instrument settings can pick the winner
How the benchmark itself is configured can move results. One 2026 arXiv study, “There Is No Neutral Harness”, ran 12 open-weight instruction-tuned language models against the same 3,679 items under 26 harness configurations that a practitioner would defend. Gemma4-31b, with fixed weights and fixed items, scored anywhere from 31 to 89 percent depending only on the harness. Four of the 12 models reached rank one under some configuration, which means “the choice of harness selects the winner”. The mean config-lucky fraction was 0.85, so most of the accuracy credited to a model was credited only under some configurations. Shuffling options preserved 0.60 of credited accuracy, changing prompt format preserved 0.40, switching between generation and likelihood scoring preserved 0.31, and varying all three together left 0.11.
The study has clear limits. It covers four-option multiple-choice benchmarks and instruction-tuned models between 3 and 70 billion parameters, and the authors say they do not claim the magnitudes transfer to other formats. They also describe the 26-configuration grid as a judgement that a reader can redraw. It is a study of LLM benchmark harnesses, not hardware. Hardware benchmarks have the same structure, though: a choice inside the instrument does work that the printed number does not reveal. The authors recommend reporting a score as a band with its robust accuracy, comparing models on the items both answer stably, and disclosing the harness in full, scoring method first.
For hardware, the equivalent is running the same test on every system. MLPerf does this in its closed division, where submitters use the same models, datasets and quality targets. LynxBenchAI does it per release by pinning the catalogue, precisions, thresholds and scoring formula. For a broader look at what makes a benchmark measure what it claims, see what makes an AI benchmark meaningful.
When comparing two systems for an AI workload, what makes the comparison valid?
A comparison is valid when both systems ran the same instrument, at the same release, with the same settings. UL Benchmarks puts the first condition plainly: “The aim of benchmarking is to run the same test on every system”. UL also warns against comparing custom-setting runs with default runs, “since the amount of work is no longer the same”. SPEC’s fair use rules allow comparisons only between SPEC metrics, require the basis of comparison to be stated, and forbid comparing different benchmarks or different major releases of the same benchmark.
Platform belongs in the comparison too. In the 26Q3 results the RTX 5090 on native Linux has a GT of 597.5 and the RTX 5090 under WSL has 545.7. For the RTX 3080 the direction changes by category: the native Linux entry scores higher on Training (163.7 against 157.4), while the WSL entry scores higher on Compute (72.5 against 64.8). The same card under two stacks gives two executors, so a valid comparison names both.
Why results compare only within one release name
LynxBenchAI scores are comparable within a release name and not across release names. The current release is 26Q3, the next will be 27Q1, and no fixed cadence is promised. LynxBenchAI ties each new release name to a major version of the benchmark, and a major version is free to change the scoring method itself, so numbers from two releases are not readings on the same axis. Put a 26Q3 figure next to a figure from a later release and the difference may come from the instrument rather than the hardware.
Established benchmark bodies draw the same line. UL guarantees comparability across application versions only while the major digit of the benchmark version is unchanged, and says that once it changes, “scores should not be compared across versions”. For SPEC’s rules, a major release is a change in the year component of the product name. SPEC states that there is no formula for converting between major versions and that normalised historical comparisons “are inherently approximate”. SPEC CPU 2017, for example, is being retired with the release of SPEC CPU 2026.
Does FP32 benchmark data carry over to an INT8 deployment?
No. Precision is a separate measurement, and the ratio between precisions differs from one executor to the next. In LynxBenchAI 26Q3, Training is measured at fp32 and bf16, and Inference at fp16, int8 and fp8. In the 26Q3 results the RTX 3080 scores 775.1 on MobileNetV3 inference at fp16 and 213.3 at int8. The AMD Ryzen 5 9600X runs the other way, at 2.6 for fp16 and 13.8 for int8.
Precision can also reverse the order of two devices. On Flan-T5 Small inference at fp16, the NVIDIA B300 SXM6 AC scores 6991.6 against the NVIDIA B200’s 6389.7. At int8 the B300 scores 826.5 against the B200’s 2364.5. A score of 0.0 means that precision produced no result on that executor, as for fp8 on the RTX 3080. If you plan to deploy at INT8, read the int8 results.
Accuracy is a separate question from speed. MLPerf Inference requires almost all implementations to stay within 1% of the FP32 reference model’s accuracy. Its authors report that for 8-bit integer arithmetic this target was “easily achievable without retraining” on their models. That finding concerns MLPerf’s reference models. Whether your own model survives quantisation needs its own check.
How long should a PC-for-AI benchmark run before the numbers are trustworthy?
The run needs to be long enough for clocks and power management to settle. MLPerf Inference requires every benchmark to run for at least 60 seconds, so that it measures the equilibrium behaviour of power-management systems and dynamic voltage and frequency scaling. It also sets a minimum number of queries per scenario, so that results are statistically robust and capture steady-state behaviour.
In LynxBenchAI 26Q3, each test is one timed window after a discarded warm-up, not a median of repeats. Because each test is a single window, small differences between two runs on one machine should be treated with caution.
What can a reader conclude when a device does not appear on the public leaderboard?
Every run submits automatically to a public leaderboard, so a claim about a device can be checked against runs other people produced with the same instrument under the same release. When a device is missing from the board, that absence is information too. It means that no run of that device under 26Q3 has been published.
Keep two cases apart. A missing device is a gap in coverage. A listed device with a 0.0 at one precision is different: that precision produced no result on that executor, as with int8 inference on Apple M1 for Flan-T5 Small. MLPerf shows how absences can carry meaning. Its submissions may cover a subset of tasks, and in version 0.5 GNMT received no multistream submissions, which the authors attribute possibly to the constant arrival interval being unrealistic for machine translation. The ordering on any leaderboard belongs to the measurement, and a position on it is not a recommendation. If the device you care about is missing, run the Personal Edition on it, and your result fills the gap. For more on how public leaderboards are read in general, see what public ML leaderboards do and don’t tell you.
What should I record so my benchmark result stays usable six months later?
Record enough that someone else could rerun the test or reject the comparison without asking you. SPEC sets a similar bar for public results: the source must be stated and publicly available, and the date the data was retrieved must be given.
| Record | Why it matters later |
|---|---|
| Release name (for example 26Q3) | Scores compare only within one release name. |
| Device and platform (native Linux, WSL, macOS) | The same card under two stacks is two executors with different results. |
| Software stack: PyTorch, driver and CUDA or ROCm versions | Changing the stack can change the comparison. |
| All three category scores, not only GT | GT is ordinal and hides category trade-offs. |
| Per-test results at the precision you deploy | Precision ratios differ between executors. |
| Default or custom settings | UL treats custom runs as not comparable with default runs. |
| Date and where the result is published | A dated, public source lets others check the claim. |
| The decision the run informed | Without it, nobody can tell whether the evidence still applies. |
Which tool should you use to benchmark a PC for AI?
LynxBenchAI’s Personal Edition measures Training, Inference and Compute on a catalogue of architecture types, on the public PyTorch stack your own models would run on, and it places your result beside the published 26Q3 results. Its limits are stated above: it covers the release’s fixed catalogue, not your application, and it is indicative of the out-of-the-box experience rather than a hand-tuned peak. To run it, pip install lynxbench-ai and follow the getting-started steps.
Other tools answer other questions. MLPerf Inference lets submitters optimise the reference models and use their own toolchain, then audits the results. That makes it the right source for a vendor-tuned peak, but not for the stack your model will run on. Novabench, a general-purpose GPU benchmark, weights rendering equally with compute and VRAM throughput in its headline score. What a popular mixed suite can and cannot show for AI is covered in Geekbench for AI workloads.
Once you have run a benchmark and recorded the results, one question is left: which category does your decision depend on, and is your production stack close enough to public PyTorch for that category to transfer? If both answers are clear, the four scores can decide the purchase. If the stack differs a lot, the result tells you where to start measuring, and the measurement of your own model is still to do.
