What does a LynxBenchAI result actually contain?
Five things, and each one narrows what the result may be used for.
| Part of the result | What it fixes | Why it matters when you read it |
|---|---|---|
| Release name | The model catalogue, the precisions, the correctness thresholds, and the scoring formula | Two results belong on the same axis only if they carry the same release name. The current release is 26Q3; the next is 27Q1. |
| AI Executor | The device and the backend, driver, framework and runtime it ran through | No score is attributed to silicon alone. The same chip under a different stack is a different measurement. |
| Four scores | Training, Inference, Compute, and GT | Three category scores that stand on their own, plus one ordering figure aggregated from them. |
| Per-precision detail | What each precision achieved, separately | A device that is fast at one precision and cannot execute another shows up as exactly that. |
| Completion record | Which catalogued tests completed and which did not | A test that fails or times out is recorded as a zero, not dropped. |
Everything below is an expansion of that table.
Why is a result only comparable within a release name?
Because a release is what fixes the thing being measured. A LynxBenchAI release names and freezes the model catalogue, the precisions each model runs at, the correctness thresholds a run must clear, and the formula that turns throughputs into scores. Every result carries the name of the release that produced it.
Scores are comparable within a release name. They are not comparable across release names, because when the catalogue or the formula changes, the measurement changes with it — the numbers on either side of that boundary are answers to different questions that happen to be printed in the same font.
Releases have names, not dates. There is no fixed cadence and none is promised. 26Q3 is the current release and 27Q1 is the next one by name; neither is a schedule.
The practical version: before you draw a conclusion from the gap between two numbers, check that both say Results: 26Q3. If one of them does not, the gap is not a finding.
Does the same methodology run on NVIDIA, AMD, Intel and CPU?
Yes — one instrument, one set of rules, four backends.
| Hardware | Backend the result records |
|---|---|
| NVIDIA GPUs | cuda |
| AMD GPUs | cuda via ROCm |
| Intel GPUs | xpu |
| CPUs | CPU |
The same catalogue runs on all of them, with the same bounded optimisation applied identically. The benchmarked models are prepared once, before any target device is known, and the identical prepared artefacts ship everywhere; where two vendors’ kernels disagree about what is permissible, the stricter constraint is applied to all of them rather than per vendor.
That is what makes a cross-vendor comparison mean something. A number produced by one vendor’s preferred test on its own preferred workload and a number produced by another’s are not comparable no matter how carefully each was measured. Here they are produced by the same instrument, which is a weaker-sounding claim and a much more useful one.
CPU-only is a first-class point on the same scale, not an afterthought — which is what lets you ask whether a GPU upgrade is worth it at all relative to the processor already installed.
Why four scores instead of one number?
Because devices are lopsided, and a single rating averages that away into something that matches nobody’s workload.
| Score | What it reports | How to read it |
|---|---|---|
| Training | How the Executor handles training workloads | Meaningful on its own. Read it if you fine-tune locally. |
| Inference | How the Executor handles inference workloads | Meaningful on its own. Read it if you serve models. |
| Compute | How the Executor handles raw compute workloads | Meaningful on its own. Read it if your load is closer to dense or sparse maths than to a model. |
| GT | An overall figure aggregated from the three | An ordering, not a magnitude. |
The three category scores are not merely ingredients of GT. Each is intended to be read by itself as how well that Executor handles that class of work. A card that trains badly but infers well appears as exactly that.
GT is ordinal. It orders Executors; it does not quantify how much better one is than another. It is an unbounded aggregate with no reference-device normalisation, so it implies no 0–100 scale and no percentage reading. If GT values happen to cluster in a familiar-looking range, that is a property of the current catalogue’s magnitudes, not a designed scale. Treat a GT gap as ahead of and not as N% faster than; for the second question, drop into the category and precision detail that produced it.
How are the four scores combined into GT?
By a shifted geometric mean, weighted so that Training and Inference count 0.4 each and Compute counts 0.2. Category scores are formed the same way from the tests beneath them.
The shift is what makes the combination behave. Each input is offset by a constant equal to a tenth of the mean of all the inputs, the offset values are combined geometrically, and the offset is then removed. Because that constant is derived from every value rather than from the surviving ones, nothing is dropped on the way in — a zero participates on the same footing as any other number.
The property that buys is monotonicity. Raising any single input can never lower the aggregate, and that holds across 200,000 randomised trials with no violations. So a workload that fails or times out scores zero and stays in the calculation: running a workload slowly always beats not completing it.
Compute carries the lower weight because it measures raw matrix operations rather than a model workload. Inside that category the two matrix tests are put on a comparable footing before they are combined. Dense and sparse matrix operations sit orders of magnitude apart in natural throughput because of how each one reaches memory, not because one is better work than the other, so the dense test is scaled by 0.1 and the sparse test by 10.0 as each is scored.
Every weight here is fixed by the release and applies identically to every device and vendor. None of this makes GT a magnitude — an ordering whose arithmetic is published is still an ordering.
What is a result bound to — and what falls outside that binding?
Every result is bound to the AI Executor that produced it: the device, the backend it ran through, and the driver, framework and runtime present on that machine, together with the release name and the per-test completion record. That binding is the result’s signature.
The scope of that signature is exactly those fields, and it is worth being precise about where it stops. The signature does not extend to anything the fields do not record — chassis and cooling, ambient temperature, power profile, or whatever else the machine was doing at the time. Two runs of the same card under the same release can differ for reasons the signature does not capture.
That is not a hole to be papered over; it is the reason a single result is read against a population rather than on its own. Your run sits alongside other submissions of the same device, made by other people on other machines, so an anomalous result looks anomalous instead of authoritative.
There is a second consequence, and it is about who gets to produce the number at all. Where no self-service benchmark exists, the performance figures available for a piece of hardware come from whoever sells it, or from tests chosen by whoever publishes them. Once anyone can run the same benchmark on their own machine, a supplied figure becomes one data point among several rather than the only one on offer. Disagreement with a published figure becomes testable instead of arguable. That is a claim about the shape of the available evidence, not an accusation about anyone’s honesty.
Who produces the number, and who checks it?
Three roles, deliberately separated.
- Whoever defines the method. TechnoLynx fixes the catalogue, the precisions, the correctness thresholds and the scoring formula, and freezes them under a release name. That is the part we own, and it is published rather than asserted.
- Whoever measures. The owner of the machine. The Personal Edition is free for non-commercial use, so producing a result needs no privileged hardware access, no lab, and no involvement from us. The party that defines the method is not the party that has to produce your number.
- Whatever holds the record. Runs submit automatically to a public leaderboard, so results accumulate in one inspectable place.
The value of that leaderboard is not the ordering. It is that a performance claim about a device can be checked against runs other people produced with the same instrument under the same release — and that a device’s absence from the board is itself visible information. If the hardware you are evaluating is not there, that is a fact about the available evidence, and it is one you can change yourself by running it.
What does the measurement window actually cover?
This is the part most benchmark descriptions leave implied, so here it is plainly.
For each test case, the workload is first scaled: batch size is raised until throughput stops improving inside a defined noise band. Every device is therefore measured at its own saturation point rather than at one fixed workload that happens to flatter a particular class of hardware. The model architecture and the numerical precision are held constant while that happens.
Then the measurement itself. A warm-up phase runs and is discarded. A single continuous timed window opens, and the benchmark counts how many iterations complete inside it. That count over that window is the throughput reported.
Three things follow, and we would rather state them than let a reader assume otherwise:
- It is not a thermal steady-state measurement. LynxBenchAI does not wait for a machine’s thermals to settle, and does not claim to report what happens after they do. The warm-up exists to get past cold-start effects, not to reach a thermal plateau.
- It is one window per test, not a median of repeated trials. There is no median-of-N scheme. A result is a single measurement window per test case, which is why the population on the leaderboard does the work that repetition would otherwise do.
- Failure is scored, not skipped. A workload that crashes or times out is recorded as a zero rather than dropped, and a precision a device cannot execute is scored as zero rather than quietly omitted. An incomplete run can never out-score a slow but complete one — that is a property of how the scores are combined, not a convention.
The integrity move here is not that the window is long. It is that the window is declared. A measurement whose boundaries are stated can be argued with; one whose boundaries are implied can only be believed or disbelieved.
What a LynxBenchAI score does not cover
Worth checking each of these against the conclusion you are about to draw.
- It is not your application. The score reflects a fixed model catalogue under one named release, not the model you actually run.
- It is not a cross-release number. A result from one release name says nothing about a result from another.
- It is not a 0–100 rating. GT is unbounded and unnormalised against any reference device.
- It is not a repeated-trial average. Each test is one timed measurement window.
- It is not a thermal-endurance test. See the section above — the window is declared, and it does not begin after thermal settling.
- It is not a purchase recommendation. The methodology tells you what to measure and what a measurement means. What to buy is a decision about your workload, your budget, and your deployment, and no benchmark makes it for you.
When a decision depends on something in that list, the honest answer is to measure your own case rather than extrapolate from someone else’s.
Where does this methodology come from?
LynxBenchAI is built by TechnoLynx, an AI engineering company founded in Budapest in 2019. Since then it has delivered AI and performance engineering for clients: GPU acceleration and cross-platform porting, production computer vision, generative models, edge and on-device inference, and codec and signal-processing work. That is the same problem class the benchmark measures — what a real stack does under real load, as against what its specification implies.
The evidence for that is public and checkable in two directions. The delivery record is on the TechnoLynx case studies page, which runs from 2020 to 2025 and includes engagements published under NDA; it is a record of engagements, with gaps, and not a claim of continuous or unbroken activity. The methodology’s own output is public too — the LynxBenchAI leaderboard and the A-Z device directory show every measured result, including the ones that are unflattering.
One anecdote, offered as an anecdote
The habit behind this methodology is older than the company, and this part is a recollection rather than a citable fact.
Long before TechnoLynx existed, our founder built CLBenchmark, a cross-vendor OpenCL benchmark. What stayed with him from that period was not any individual result but a pattern: once one instrument ran the same workload on everybody’s hardware, arguments about whose numbers were correct turned into questions about what the workload actually did — and the fastest way to get a stubborn performance problem taken seriously was to hand over something the other side could re-run for themselves. That experience is the reason this benchmark is self-service and the reason its boundaries are written down rather than assumed.
The checkable part is narrower than the story: he sat in the Khronos OpenCL and SYCL working groups, and is a named contributor to the SYCL specification. The rest is an anecdote, and we would rather label it as one than dress it up as evidence.
