A citable FP16 benchmark GPU comparison from LynxBenchAI is a ratio between two devices’ FP16 inference or FP16 compute scores taken from the same named release. You quote it with both absolute scores, the release and the Index name, as in “on the LynxBenchAI 26Q3 Index”. It is not an FP16 training speedup, because LynxBenchAI trains in FP32 and BF16 only. Nor is it a spec-sheet ratio, and it comes with no confidence interval, because each published result is one timed window after a discarded warm-up.
LynxBenchAI gives you that comparison by measuring every covered device on one public PyTorch model catalogue at identical precisions and publishing the results per device in the AI hardware directory of all LynxBenchAI results. The first step is to open both device entries and confirm they belong to the same release. The results show how each device runs the public PyTorch stack your own models run on, so they are broadly indicative for custom PyTorch models, but LynxBenchAI does not run or measure your own workload.
How do I compare FP16 performance between two GPUs in LynxBenchAI?
Read the per-test FP16 scores, not the category score. In the 26Q3 results, FP16 appears in five inference tests (MobileNetV3, Flan-T5 Small, DistilGPT2, DistilBERT and GraphGPS (Peptides)) and in the compute category’s Dense MatMul test. The inference category score of 985.8 for the NVIDIA GeForce RTX 5090 on native Linux also covers INT8 and FP8 tests, so it is not an FP16 number and should not be quoted as one.
LynxBenchAI reports every precision separately and enforces a per-precision correctness threshold before a result counts at all. A 0.0 in an FP16 cell means that precision produced no result on that executor, which is a finding to report rather than a slow card.
Then match the platform. An executor is a device plus its software stack, and the 26Q3 results list the RTX 5090 twice: on native Linux with an inference category score of 985.8, and under WSL with 891.8. Put a native-Linux entry for one card next to a WSL entry for another and you have mixed two variables, silicon and software stack, into one ratio.
A worked FP16 benchmark GPU comparison from the 26Q3 results
The 26Q3 results include the RTX 5090 on two platforms, so the worked example compares those two entries. The arithmetic is the same for two different cards. Every ratio below is native Linux divided by WSL, so WSL is the base case.
| FP16 test (26Q3) | RTX 5090, native Linux | RTX 5090, WSL | Ratio (native / WSL) |
|---|---|---|---|
| Inference, MobileNetV3 | 2622.5 | 2037.5 | 1.29x |
| Inference, Flan-T5 Small | 2079.2 | 1813.1 | 1.15x |
| Inference, DistilGPT2 | 1176.8 | 1043.9 | 1.13x |
| Inference, DistilBERT | 1486.0 | 1455.3 | 1.02x |
| Inference, GraphGPS (Peptides) | 537.2 | 510.0 | 1.05x |
| Compute, Dense MatMul | 421.6 | 406.0 | 1.04x |
The scores are read from the 26Q3 data, and each ratio is the first score divided by the second, for example 2622.5 / 2037.5 = 1.29.
Two things stand out. The ratios run from 1.02x to 1.29x across tests on the same pair of executors, so an FP16 comparison is a statement about a model and a precision, not about the card in general. Averaging those six ratios into one headline would also go against the guidance in Scientific Benchmarking of Parallel Computing Systems: summarise the underlying rates rather than the ratios, and use the geometric mean only when the rates are unavailable.
Wording the citation
A sentence that holds up names the release, the Index, the test, both absolute scores and the base case: “On the LynxBenchAI 26Q3 Index, the RTX 5090 on native Linux scores 2622.5 in FP16 MobileNetV3 inference against 2037.5 under WSL, 1.29x the WSL result, from one timed window per executor.” The same paper’s general rule: never report ratios without absolute values, and always make clear what the base case is. A 2026 audit of benchmark papers asks for the same thing in another field: an explicit version tag or commit hash for the benchmark behind a number. For LynxBenchAI, the release name every result carries does that job.
Steps before you quote an FP16 ratio
- Confirm both entries come from the same release. Scores are comparable within a release name and not across release names, because a new release is free to change the scoring methodology.
- Confirm both entries run on the same platform (native Linux against native Linux, WSL against WSL).
- Pick the FP16 test that matches your question: a specific inference model, or Dense MatMul for raw FP16 compute.
- Write down both absolute scores, then the ratio, with the base case named.
- Attach the release and Index name (“LynxBenchAI 26Q3 Index”).
- State the limits: inference or compute, not training; one timed window, no published spread; a LynxBenchAI score, not tokens per second.
Why can’t I take the speedup from spec-sheet TFLOPS?
Spec-sheet TFLOPS describe a ceiling, not what a model achieves. NVIDIA’s A100 datasheet lists 312 TFLOPS for FP16 Tensor Core work, and 624 TFLOPS with sparsity. The same paper notes that vendor-specified numbers are only guarantees not to be exceeded.
Measured work shows how far real models fall below those ceilings. The team that ran the 2018 DAWNBench competition found that PyTorch’s ResNet-50 trained in standard FP32 on a V100 achieved 7.6 TFLOPS against a peak of 15.7, mostly because memory-bound kernels such as BatchNorm and ReLU take a significant share of runtime. In FP16, about 85% of the time went on kernels that never touched the Tensor Cores, and no kernel used them fully. The HPC AI500 authors measured a mixed-precision speedup of only 2.16x on image classification, and traced part of the gap to data-layout conversion between TensorFlow and CUDA kernels. They also warn that kernel micro-benchmarks can mislead: mixed precision raises a convolution’s FLOPS while affecting the time-to-quality of the real task.
A ratio of two spec sheets therefore compares two ceilings, not two models running. Extrapolating from a third card that was measured has the same flaw, because it assumes the unmeasured card falls short of its peak in the same way as the measured one, which is exactly what the evidence above says you cannot assume.
Why the ratio of two LynxBenchAI scores is not a throughput ratio
Aggregation weights throughput by the memory a run actually moves, so a lower-precision run has to earn its speed-up rather than be credited for being cheap. Batch size is not fixed either: LynxBenchAI raises it on each device until throughput plateaus inside a defined noise band. Describing 1.29x as “29% more images per second” claims something the score does not report.
The top-level GT figure is an ordinal aggregate, so a GT difference is never a percentage. The 26Q3 GT of 597.5 for the native-Linux RTX 5090 against 545.7 under WSL tells you the order of the two entries and nothing about how large the gap is. For a figure you can cite, use the per-test FP16 scores; the guide to interpreting LynxBenchAI benchmark results explains how to read and reproduce them.
Can a consumer card and a datacenter card share one comparison?
Yes, within one release. LynxBenchAI runs an identical model catalogue at identical precisions on every device in a release and adapts only the batch size, so a laptop GPU and a data-centre accelerator are each measured at their own saturation point rather than at one workload that suits one of them. What the ratio answers is how each device performs when saturated on that model at that precision. It does not tell you how they compare at a batch size you have picked for your own service. Putting the two on a per-watt or per-dollar axis is a separate normalisation question, and the published score does not do that for you.
What the figure leaves out: FP16 training and run-to-run spread
No FP16 training number exists to cite. In the 26Q3 results the training tests report FP32 and BF16 only; the RTX 5090 on native Linux, for example, scores 736.9 in BF16 DistilBERT training against 712.6 under WSL. If your article needs a training comparison, quote the BF16 figure as BF16. Do not relabel it, and do not infer an FP16 training ratio from the FP16 inference ratio.
The published result also has no spread. Each test is one timed window after a discarded warm-up, not a median of repeats. That matters, because performance varies between runs: in that paper’s HPL example the variation reached 20%. For nondeterministic data its rule is to report confidence intervals. MLPerf Training requires five runs for vision tasks and ten for others, drops the fastest and slowest, and reports the mean of the rest. NVIDIA’s AIPerf documentation treats a coefficient of variation below 10% as trustworthy. It judges a difference as real when the confidence intervals do not overlap, or when the gap is larger than the CV.
You cannot run either test on a single published window. So say it plainly in the citation (“one timed window per device; no run-to-run spread published”) and do not invent an interval. In our judgement, a large gap like the 1.29x MobileNetV3 ratio is reasonable to cite with that caveat. A 1.02x gap like DistilBERT’s is smaller than the run-to-run variation the sources above report for other benchmarks, so do not present it as a difference. If a reader or a vendor challenges the figure, your best defence is that both numbers come from one release under one methodology, and that the release name lets anyone check them.
Who can measure the GPU pair the leaderboard doesn’t cover?
LynxBenchAI measures FP16 inference and compute under one methodology for every device it covers, so the first place to look for your pair is the per-device entries of LynxBenchAI. If your exact pairing is missing, ask the LynxBenchAI team for coverage through the TechnoLynx contact instead of extrapolating from neighbouring cards. If you have both cards, the LynxBenchAI Personal Edition benchmarks them on your own hardware with stock PyTorch. Cite a Personal Edition run as your own measurement, with the version you ran, and not as a published release result.
Keep the limits in view. LynxBenchAI measures hardware on a public model catalogue and does not benchmark your production model, serving stack or latency targets. If your claim has to be about your own application, you need your own repeated runs with confidence reporting of the kind NVIDIA’s AIPerf documentation describes. If you need training throughput to a quality target, the right precedent is MLPerf’s time-to-train methodology.
FAQ
Why is FP16 faster?
On GPUs with Tensor Cores, FP16 arithmetic runs on units that perform more operations per second, usually with FP32 accumulation. Mixed precision needs close to half the memory of FP32, which leaves room for larger batches. How much of that potential a real model captures depends on its kernels, as the measured gaps above show.
What is FP16 and BF16?
Both are 16-bit floating-point formats. FP16 has more mantissa bits and a narrower exponent range, while BF16 keeps FP32’s exponent range at lower precision. In LynxBenchAI, FP16 is measured on inference and compute, and BF16 on training and compute.
