Two engineers open the same tab. One reads “L40S vs A100” and wants a single number that says which card to buy. The other reads it and asks which of the three category scores under the release actually resembles the workload they are about to deploy. The second engineer will make a better decision, and not because they know more about either card.
Both devices carry a published LynxBenchAI result under the 26Q3 release — the L40S and the A100 80GB PCIe. Each result is broken into Training, Inference, and Compute category scores. That structure is not a presentation choice. It exists because a card built to maximise inference density and a general-purpose datacentre accelerator do not sit at the same point on the training/inference spectrum, and collapsing them into one comparative figure destroys exactly the information a reader needs.
Why a headline “L40S vs A100” number hides the decision
A single comparative figure has to average across categories. When the two devices being averaged were designed for different jobs, the average tells you about the averaging, not about the hardware.
Consider what the collapse does mechanically. If the Inference category favours one device and the Training category favours the other, a blended headline lands somewhere in the middle and reads as “roughly comparable.” A reader running a continuous serving workload gets no signal that the Inference score was the only one that mattered to them. A reader fine-tuning a mid-sized model gets the same non-signal in the opposite direction. Both walk away with a number that is arithmetically defensible and operationally useless.
This is the specific failure class: a comparison averaged across categories reports the shape of the average rather than the fit of the hardware to the workload. The fix is not a better weighting scheme. It is refusing to blend, and instead asking which category the workload resembles before looking at any score at all.
The category structure survives that question. Training, Inference, and Compute are separable because they exercise different parts of the executor — sustained gradient traffic and optimizer state pressure in one case, batch-shaped forward-pass throughput and memory-bandwidth-bound attention in another, and lower-level numerical throughput in the third. A device can be strong in one and unremarkable in another without any inconsistency.
What has to be true for the two results to be comparable at all
Two published results are directly comparable only when they were produced under the same release name — 26Q3 against 26Q3. That constraint is not bureaucratic caution. A release name fixes the catalogue of test cases, the artefact preparation, and the bounded optimization budget applied to each. Compare a 26Q3 result against a result carrying a different release name and you are comparing two different questions.
Within the release, the thing that makes the comparison checkable is that optimization effort was bounded and applied identically to the prepared artefacts that ran on both devices. Neither card got a hand-tuned kernel the other did not. This is the difference between a comparison a reader can audit and a comparison they have to trust — and it is why a vendor-published spec comparison is not the same claim as a same-release measured comparison, even when both mention the same two part numbers.
Each result is also tied to the specific backend, driver, framework, and runtime present on the machine that produced it. The unit being measured is not “an L40S” in the abstract; it is an L40S plus a software stack, at a moment in time. Change the CUDA version, swap TensorRT for a plain PyTorch eager path, or move from one cuDNN release to another, and you have changed the thing being measured. We treat that tuple as the unit of performance rather than the silicon alone, which is covered in more depth in our discussion of why GPU performance is a hardware-software tuple rather than a property of the chip.
Validity checklist for a two-device comparison
Run this before drawing any conclusion from two published results side by side:
| Check | Passes when | Fails when |
|---|---|---|
| Same release name | Both results carry the same release (26Q3 vs 26Q3) | One result is from a different release; the catalogue or budget may differ |
| Category-level read | You are comparing Training to Training, or Inference to Inference | You are comparing a blended headline figure |
| Executor recorded | Backend, driver, framework, and runtime are visible for both | Only the device name is given |
| Bounded optimization | The same effort budget was applied to both prepared artefacts | One device was hand-tuned, the other was not |
| Workload resemblance | Your workload plausibly resembles the category you are reading | Your workload is unlike anything in the catalogue |
Four passes and one fail on the last row is a common and honest outcome. It means the comparison is valid but your extrapolation from it is not — a different problem with a different remedy.
Which category should you read first?
Start from the workload, not the card. The question is not “which is faster” but “which category does my deployment live in, and what does the release say about each device in that category.”
- Continuous serving of a fixed model, latency-bounded, high request concurrency. Read the Inference category. Batch shape, precision, and memory bandwidth dominate here; the Training score is noise for this decision.
- Fine-tuning or full training runs, sustained for hours. Read the Training category. Sustained thermal and power behaviour under long load matters more than any burst figure, which is why sustained practical peak rather than transient peak is the operationally relevant framing.
- Mixed fleets that do both, on the same capacity. Read both categories separately and accept that the two devices may split the decision. That is a legitimate answer and often the correct one.
- Numerical or non-neural GPU compute alongside model work. The Compute category is the relevant surface; neither of the other two predicts it well.
Nothing in that list declares a winner, and the published results are not structured to produce one. What they do support is a narrower, more useful conclusion: for this category, under this release, on this executor, here is what each device produced.
The spec sheet answers a different question
Vendor specifications are accurate and useful for what they describe — memory capacity, HBM or GDDR bandwidth, form factor, power envelope, PCIe generation, tensor-core peak rates at a given precision. They describe the device’s ceiling under ideal conditions.
They do not describe what a framework, a driver, and a compiled graph actually extract from that ceiling. A peak FLOPS figure at a stated precision is a bound, not a prediction. The gap between the two is where most procurement surprises live, and we have written separately about what a GPU datasheet cannot tell you about measured AI performance.
Treating a vendor’s own comparative claim as equivalent evidence to a same-release measured comparison is the substitution to avoid. The two are not competing sources of the same fact; they are answers to different questions. A spec sheet tells you what the hardware can do. A same-release result tells you what one particular hardware-plus-software configuration did, on a named catalogue, under a declared optimization budget.
When neither published score covers your workload
This happens often, and pretending otherwise is the least useful thing a benchmark can do. If your deployment is a retrieval-augmented pipeline with an unusual sequence-length distribution, or a vision model with a custom pre-processing stage that dominates wall time, or a serving pattern with bursty concurrency the catalogue does not represent, then the published Inference score is context — not an answer.
The honest move is to run your own workload on the executor you are considering. The Personal Edition of LynxBenchAI is installable and free for non-commercial use, and gives you the same measurement discipline — bounded optimization, recorded executor, named release — applied to a machine you control. What it does not give you is permission to assume your workload scales like the catalogue. Nothing gives you that except running it.
For teams that need the hardware-fit question answered against a specific production workload rather than a published catalogue, that is applied engineering work rather than a measurement question — our GPU inference optimisation and hardware-selection practice is where that scoping happens.
FAQ
What does “L40S vs A100” actually compare once a published benchmark separates Training, Inference, and Compute scores?
It compares three things, not one. Under the 26Q3 release both the L40S and the A100 80GB PCIe carry a published result broken into Training, Inference, and Compute category scores, and each of those is a separate comparison. A single headline “L40S vs A100” figure averages across all three, which reports the shape of the average rather than the fit of either card to a specific workload.
Why does comparing a card built for inference density against a general-purpose accelerator require checking the category, not a headline number?
Because the two devices target different points on the training/inference spectrum by design. A blended figure will land somewhere in the middle and read as “roughly comparable” even when one device clearly suits the reader’s category and the other does not. Reading the category directly recovers the information the average destroyed.
What has to be true for an L40S result and an A100 result to be validly compared against each other?
They must carry the same release name — 26Q3 against 26Q3 — so that the test catalogue and the optimization budget are the same question. Optimization effort must have been bounded and applied identically to the prepared artefacts that ran on both devices. And each result must record the backend, driver, framework, and runtime that produced it, because those are part of what was measured.
Why can’t a vendor-published spec sheet for either device answer the same question a same-release LynxBenchAI comparison can?
A spec sheet describes the device’s ceiling — memory capacity, bandwidth, power envelope, peak rates at a stated precision — under ideal conditions. It does not describe what a framework, driver, and compiled graph actually extract from that ceiling on a named test catalogue. The two are answers to different questions, and a vendor’s comparative marketing claim is not equivalent evidence to a measured same-release comparison.
If a reader’s workload doesn’t resemble the benchmark catalogue’s Training, Inference, or Compute categories, what should they do instead of extrapolating from either device’s published score?
Run the workload. The published categories become context rather than an answer once the deployment’s sequence-length distribution, pre-processing cost, or concurrency pattern falls outside what the catalogue represents. The Personal Edition is installable and free for non-commercial use, and applies the same bounded-optimization and recorded-executor discipline to a machine you control.
When does an L40S deployment make more sense than A100 capacity for a given workload, based on the published categories rather than the spec sheet?
When the workload lives in the Inference category — continuous serving of a fixed model, latency-bounded, high concurrency — and the 26Q3 Inference score for the L40S supports that read on an executor resembling your own. For sustained training runs, the Training category is the surface to check instead, and it may point the other way. Mixed fleets legitimately split the decision between the two devices rather than resolving it.
Two cards, two questions, one workload
The useful output of an L40S-versus-A100 comparison is not a ranking. It is a sharper statement of your own problem: my workload is inference-shaped, latency-bounded, and lives in one category of the catalogue, so that is the only category whose numbers I should be reading. Once the question is that specific, the published results stop being a scoreboard and start being evidence.
What remains genuinely uncertain is how far your workload’s resemblance to a catalogue category actually extends. That is not something either device’s published score can tell you, and it is the one part of the decision that only your own measurement closes.
