LynxBenchAICompute Leaderboard

Design Principles

Why the figures look the way they do, and why they sit below vendor slides.

• Stock PyTorch, Not a Tuned Peak

  A result is broadly indicative of the out-of-the-box PyTorch experience on that hardware, and so of custom PyTorch models too. MLPerf shows a vendor-tuned peak; this shows the stack your model will run on, which is why the figures sit below vendor slides.

• One Scale, Laptop to Data Centre

  Small models, chosen to cover architecture types (a CNN, three kinds of transformer, a graph network), run on every device from integrated graphics up. Only the batch size grows, until throughput plateaus, so a laptop GPU and an accelerator are each measured at their own plateau and share one ranking.

• No Vendor Path, No Free Omissions

  Models are prepared once, before any target device is known, and the same artefacts ship to every vendor. A workload that fails, or a precision a device cannot run, scores zero and stays in the total.

• What It Holds Under Load, Not Its Best Instant

  A warm-up phase is discarded, then completed iterations are counted inside one continuous timed window. Burst and turbo behaviour does not set the score.

• Hardware and Software, Measured Together

  Every result is bound to the device, the backend it ran through and the driver, framework and runtime on that machine. No score is attributed to silicon alone.

• Speed Only Counts When the Answer Is Right

  Each precision has to pass a correctness threshold before its result counts, and throughput is weighted by the memory a run actually moves, so a lower precision has to earn its speed-up.

The full methodology →