An AMD GPU vs NVIDIA comparison for AI never measures silicon on its own. What gets timed is the GPU together with the CUDA or ROCm stack that drives it: the driver, the runtime, the math libraries and the framework build installed on that machine. This is why a spec sheet does not tell you the AI performance a card delivers. It is also why a vendor-against-vendor number tells you little until it names the stack on both sides. LynxBenchAI uses the AI Executor (hardware times software) as its unit of measurement. Every score is bound to the device, the backend it ran through and the driver, framework and runtime present on the machine, and no score is attributed to silicon alone.

FLOP/s, memory bandwidth and capacity do set physical limits. What the spec sheet leaves out is the software between your model and those limits, and that software is not neutral. On both vendors, software decides how much of the limit you reach, and the amount differs between vendors and between versions.

AMD GPU vs NVIDIA on paper and in delivered throughput: the MI300X case

One public example is SemiAnalysis’s training benchmark of the AMD MI300X against the NVIDIA H100 and H200. SemiAnalysis concluded that the MI300X’s on-paper advantage “was not realized due to a lack within AMD public release software stack and the lack of testing from AMD”. In its BF16 matrix-multiplication tests, the H100 and H200 reached roughly 720 TFLOP/s against a marketed 989.5. The MI300X reached about 620 TFLOP/s against a marketed 1,307, which left it 14% slower despite the much higher headline figure. That MI300X number came from a custom Docker image hand-built by an AMD principal engineer. The out-of-the-box MI300X result was slower still, and reaching even that level required environment flags that are not set by default.

SemiAnalysis tested several MI300X builds: a public stable release with PyTorch 2.5.1, a public nightly, a custom “VIP” image and an unmerged development build that AMD submitted later. The H100 and H200 were tested only on public stable releases. According to SemiAnalysis, getting the MI300X within 75% of H100/H200 performance took multiple AMD teams fixing bugs. It also took a Dockerfile of about 60 commands that builds hipBLASLt, Triton, PyTorch and TransformerEngine from source in around five hours, whereas the NVIDIA setup was a pre-built image.

That evidence makes the point better than a principle could. “MI300X versus H100” is not one comparison. It is a different comparison for each software build of the MI300X. Even on NVIDIA’s side, SemiAnalysis found delivered throughput below the marketed TFLOP/s, though by much less than on AMD’s. Horace He’s explainer on deep-learning performance shows a related gap: the headline figure assumes tensor-core matrix multiplication, and on an A100, work that is not matrix multiplication reaches 19.5 teraflops rather than the stated 312.

Which layers make up an AI software stack, from driver to framework, and which does a run record?

An AI software stack is a set of layers between the model code and the silicon. Each layer can raise or lower the ceiling the layer above it reaches. NVIDIA’s CUDA Best Practices Guide describes its software environment as three parts: the CUDA Toolkit (libraries, CUDA runtime and developer tools), the user-mode CUDA driver (libcuda.so on Linux) and the kernel-mode NVIDIA GPU device driver. AMD describes ROCm as its open software stack for GPU computing, with runtimes, compilers, system utilities and optimised math and compute libraries.

Layer	NVIDIA side	AMD side	Why it needs recording
Kernel-mode driver	NVIDIA GPU device driver	AMD GPU driver, with driver support influenced by the GFXIP version	One NVIDIA driver bug cut some workloads by about 10% or more
User-mode driver and runtime	libcuda and the CUDA runtime (cudart)	ROCm runtimes	The CUDA runtime guarantees neither forward nor backward binary compatibility across versions
Math and DNN libraries	cuBLAS, cuDNN (cuDNN can ship separately from the toolkit)	rocBLAS and hipBLASLt	Which library a call reaches can change GEMM throughput on the same card
Compiler	nvcc	ROCm compilers	Flags such as fast-math trade precision for speed
Framework build	PyTorch CUDA build, torch.cuda	PyTorch ROCm build, which reuses torch.cuda through HIP	The backend string alone does not identify the vendor

The last row is where the CUDA and ROCm difference goes missing from a result. The PyTorch documentation says PyTorch for HIP “intentionally reuses the existing torch.cuda interfaces”, so ported code needs few changes. As a result, a log line reading “device: cuda” could come from either vendor’s GPU. The differences are still there underneath. On the MI300X, TF32 goes through hipBLASLt with hardware-level numerical differences from the NVIDIA path, and one NVIDIA precision mode raises an error on ROCm. PyTorch also warns that CUDA and HIP version APIs do not map to the same values and should not be used interchangeably in version checks.

LynxBenchAI records the layers that decide the number. Each result names the device, the backend (cuda for NVIDIA, cuda via ROCm for AMD, xpu for Intel, or CPU) and the driver, framework and runtime versions on the machine. The Intel xpu backend follows the same pattern from the other direction: ported PyTorch code changes cuda to xpu. The Intel Arc driver stack and oneAPI/SYCL is a clear case of the driver layer deciding what the silicon is allowed to do.

How do GPU drivers and runtimes shape execution performance?

Drivers and runtimes manage how work reaches the GPU. A driver bug can move a result. The CUDA runtime handles driver version checking, context management, module management, kernel configuration and parameter passing. Each CUDA Toolkit version requires a minimum NVIDIA driver. Applications that compile device code to PTX will not run on older drivers.

Regressions are the clearest evidence that this layer matters. Phoronix reported NVIDIA’s confirmation of a memory-management bug introduced in its 378 Linux driver series and still present in the 387 releases, with a workaround planned for the 390 series. Some users saw a few frames of drop, while some GPUs and workloads saw drops of around 10% or more. A benchmark result that does not name its driver cannot show whether it was taken inside a window like that.

Why a version change can move results by a large margin on the same hardware

In the examples below, the larger shifts came from offloading more work to the GPU and from moving configurations between architectures. The GROMACS 2018 study on arXiv measured a modest effect from toolchain versions alone. Across three nodes, GCC 6.4 with CUDA 9.1 ran about 1.5% to 4% faster than GCC 5.4 with CUDA 8 (3.5% on the GTX 1080Ti node, averaged over 10 runs). Across application releases and node generations the gain was larger: GROMACS 2018 moved the PME mesh calculation onto CUDA GPUs, and consumer GPU nodes went from delivering two to three times more trajectory per price in 2014 to three to six times more. Configuration tuning is architecture-specific in the same way. The BAT 2.0 study found that moving an optimal kernel configuration between four NVIDIA GPU architectures kept between 58.5% and 99.9% of optimal performance.

How does framework behaviour change kernel scheduling and memory use?

Framework behaviour influences kernel scheduling and memory use. That makes it a performance layer, not a thin wrapper around the driver. The PyTorch team notes that frameworks pay a significant overhead to submit each operation to the GPU. When individual GPU operations shrink to a few microseconds, that submission latency leaves the GPU poorly utilised. At small batch sizes, CPU-side overhead can exceed GPU run time, and the GPU sits idle between kernel calls. Capturing work as a CUDA graph skips Python, C++ and driver dispatch on replay. In two MLPerf workloads, the PyTorch team reported speed-ups of up to about 1.7x from CUDA graphs.

Library routing inside the framework also changes throughput. SemiAnalysis found that on AMD, torch.matmul reached the faster hipBLASLt library while F.Linear still defaulted to the slower rocBLAS. The bug lasted for months until AMD fixed it. Framework memory behaviour can also erase a hardware advantage. When SemiAnalysis tried PYTORCH_TUNABLE_OPS on the MI300X, it leaked over 25 GB of HBM out of 192 GB, which “essentially” wiped out the capacity advantage over the H100. Precision controls change between releases as well. PyTorch 2.9 introduced finer-grained controls for TF32 behaviour, and the PyTorch documentation shows TF32 running about 7x faster on an A100, at roughly two orders of magnitude more relative error. Autotuning adds its own variation. With cudnn.benchmark enabled, cuDNN may choose different convolution algorithms on later runs, even on the same machine.

Finding the layer that sets the ceiling

Which layer binds depends on the regime the workload is in, so identify the regime before changing a version. Horace He divides time into compute, memory and overhead. If a workload is memory-bandwidth bound, more FLOPS will not help. If it is compute-bound, cutting framework overhead will not help. He also gives a quick test for overhead: increase the batch size, and if runtime does not grow proportionally, you are overhead-bound. In practice, an overhead-bound result points at the framework and launch path. A memory-bound result points at memory traffic, which operator fusion reduces. A compute-bound result that falls short of expectation can come from the library and precision path the framework chose, as the rocBLAS and TF32 cases show.

How can a benchmark compare an NVIDIA and an AMD GPU fairly?

A fair cross-vendor comparison holds three things fixed: the instrument, the optimisation effort and the rule about what software is allowed. Only then can the remaining difference be attributed to the executor. LynxBenchAI runs the same model catalogue on NVIDIA GPUs (cuda), AMD GPUs (cuda via ROCm), Intel GPUs (xpu) and CPUs under one set of rules. Cross-vendor numbers therefore come from one instrument, not from each vendor’s preferred test. Optimisation effort is bounded. The models are prepared once, before any target device is known, and the identical prepared artefacts ship to every vendor’s hardware. Where two vendors’ kernels disagree about what is permissible, the stricter constraint applies to all of them. A workload that fails or times out is recorded as a zero rather than dropped.

The software rule is that LynxBenchAI measures software anyone can obtain. A submitter may use a different driver or a newer runtime, provided every component is a publicly released version. A build produced for LynxBenchAI, or tuned against it and shipped to nobody else, falls outside the methodology. Under that rule, a score describes out-of-the-box behaviour, not what a configuration assembled to produce a number does. The rule is declared rather than technically enforced. The client does not verify that a submitted stack is publicly obtainable. The submission record captures driver, framework and runtime versions, which makes the rule auditable after the fact rather than at run time.

MLPerf takes a similar approach. Its inference rules define the system under test as hardware plus software resources, including “compilers, libraries, and drivers”. For “Available” submissions, they require the software versions of all components to be specified. A submission whose hardware and software are not reasonably available to the public becomes more likely to be audited.

How should I read a published NVIDIA-vs-AMD benchmark before trusting its number?

Check what the result says about the software before reading the number. If the result is silent on the stack, the comparison is underspecified, whichever vendor it favours. Each item below can be checked from the published material alone.

1. Device and backend named separately. “cuda” alone does not tell you the vendor, because PyTorch on ROCm reuses the torch.cuda interface.
2. Driver version on both sides. NVIDIA’s 378 to 387 driver series carried a confirmed memory-management bug that cut performance.
3. CUDA or ROCm version and the math libraries. On AMD, the choice between rocBLAS and hipBLASLt changed GEMM throughput.
4. Framework version and build source. Check whether it is a public stable wheel, a nightly or a custom image. SemiAnalysis’s MI300X results differed across all three.
5. Non-default flags and environment variables. SemiAnalysis’s MI300X result depended on flags that are not set by default.
6. Equal optimisation effort. SemiAnalysis’s H100 and H200 numbers were out of the box, while the MI300X numbers came after months of AMD tuning.
7. Precision and numerics path. TF32 on MI300 differs numerically from the NVIDIA path.
8. Repeat count and spread. A 2022 study of GPU variability found 8% average (and up to 22%) performance variation across GPUs of identical SKU, for both AMD and NVIDIA.
9. Release or version of the benchmark itself. LynxBenchAI 26Q3 results are compared only against other 26Q3 results.

For how GPU benchmarks turn workloads into scores in the first place, the explainer on why spec-sheet benchmarking fails for AI covers the mechanics this page assumes.

How LynxBenchAI binds the MI300X and H100 results to their stacks

In LynxBenchAI, every published NVIDIA and AMD score is an executor score. It carries the device, the backend (cuda for NVIDIA, cuda via ROCm for AMD) and the driver, framework and runtime versions it ran on. In the 26Q3 results, the AMD Instinct MI300X scores higher than the NVIDIA H100 80GB HBM3 on GT, at 986.7 against 939.5 on the LynxBenchAI 26Q3 Index. The MI300X also scores higher on training (727.2 against 709.3) and compute (1023.3 against 487.1), while the H100 scores higher on inference (1678.1 against 1304.0). The NVIDIA H200 scores above the MI300X on GT (1045.0), training (812.0) and inference (1914.6), and below it on compute (480.8).

The per-test figures show the ordering changing from model to model. On MobileNetV3 fp16 inference, the MI300X scores 733.8 against the H100’s 2833.0. On DistilGPT2 fp32 training, the MI300X scores 1571.3 against 1159.0. GT is an ordinal aggregate, so the gap between 986.7 and 939.5 is an ordering, not a percentage.

SemiAnalysis and the 26Q3 release measured different software builds on different workloads. Which of those differences accounts for most of the change, neither data set can separate. That is the thesis in practice. Each 26Q3 figure is bound to the stack it ran on, and says nothing about any other stack. The 26Q3 results are browsable from the AI hardware directory of LynxBenchAI results.

For a memory-bandwidth-bound inference workload, when does AMD make sense despite the software gap?

AMD makes sense for memory-bound inference when two conditions hold. The workload must actually sit in the memory regime, where extra bandwidth or capacity is the thing that helps. And the specific path your model takes through ROCm must deliver that bandwidth on a public stable release. The first condition comes from Horace He: in a memory-bandwidth-bound regime, more FLOPS will not help. The second comes from SemiAnalysis’s MI300X tests, where PYTORCH_TUNABLE_OPS leaked over 25 GB of the 192 GB of HBM and essentially erased the capacity advantage. The variability study adds a further reason. Memory-bound PageRank varied about 1% across GPUs, against 22% for compute-heavy ResNet-50, and the study suggests memory-bound workloads tolerate worse-performing nodes.

The 26Q3 results show category dependence even at consumer level. The NVIDIA GeForce RTX 5060 Ti scores higher than the AMD Radeon RX 9060 XT on GT (146.4 against 76.3), training (133.2 against 37.2) and inference (240.4 against 122.7). The RX 9060 XT scores higher on compute (112.7 against 61.7). On DistilBERT training, the RX 9060 XT scores 15.3 at fp32 and 95.5 at bf16, against 206.4 and 168.2 for the RTX 5060 Ti. A precision-dependent gap of that size is not a fixed hardware ratio, and the release does not isolate whether the stack causes it.

So, is AMD or NVIDIA better for AI? The question has no silicon-only answer. It has an answer for a named executor, a named workload category and a named release, and that answer can change when any of them changes. If your model is not in the published catalogue, the results do not answer it, and you would profile your own workload on the candidate stacks. Getting a model to run well on ROCm, or achieving cross-platform GPU performance portability off CUDA, is engineering rather than measurement, and TechnoLynx does that work under GPU Performance Engineering.

If a driver update changes a score on the same machine, what can and cannot be concluded from that single before/after pair?

A single before/after pair across a driver update shows that the executor’s score changed. It does not prove that the driver caused the change, or by how much. Each side is one timed measurement window on one machine after a discarded warm-up, and anything else on that machine that moved between the two runs moved with it. That window does not capture thermal state or clock behaviour. Run-to-run noise is not zero either. The 2022 variability study measured median per-GPU run variance of 0.44% and 0.12% on two NVIDIA clusters and 6.06% on an AMD MI60 cluster. A small difference from one pair can therefore sit inside the noise.

Mytkowicz and colleagues showed that changing a seemingly innocuous part of an experimental setup can lead a researcher to the wrong conclusion. They found such measurement bias in every architecture and compiler they tried. A large, repeatable change in the same direction across several runs justifies investigating that layer. A single pair justifies a second measurement, not a verdict. In LynxBenchAI terms, GT is ordinal, so a GT delta across a driver change is not a measure of “how much faster the driver made it”.

Upgrades can also make things slower. NVIDIA’s 378 driver series introduced a regression that later releases had to work around. In GROMACS, the PME offload added in the 2018 release was often slower on legacy GPU hardware. The F.Linear routing bug on AMD lasted across months of public PyTorch builds. Newer software is often faster, but not reliably so.

How should I weigh regression risk against potential gain before updating a GPU driver or runtime?

Weigh the update by what it unlocks and how easily you can undo it. Treat any speed claim as a hypothesis to measure on your own executor. NVIDIA’s compatibility rules shape the risk. A newer driver with an older CUDA Toolkit is the simplest upgrade path. A newer toolkit on an older driver can fail on features that span both, or on PTX code. NVIDIA also notes that applications can redistribute the exact CUDA library versions they were tested against.

A sound order of work is: record the full current stack, keep a known-good environment you can roll back to, change one layer at a time, and repeat each measurement several times so the spread is visible. An update earns its place when it fixes a bug you are hitting or enables a path your workload uses (a precision mode, a fused kernel, an attention backend), and the repeated measurements agree. Raising a software ceiling on a production inference stack, as opposed to measuring it, is the subject of performance tuning for AI inference in practice. One layer below the framework, which GCC compiler flags change AI workload performance is another variable that is easy to leave unrecorded.

Measuring your own stack effect with the LynxBenchAI Personal Edition

LynxBenchAI Personal Edition measures a stack change on your own NVIDIA or AMD GPU directly: a full pass takes 15 to 30 minutes on ordinary consumer hardware. The Personal Edition is released and free for non-commercial use. It measures the published catalogue on your executor, meaning your device and your stack. It runs the fixed, pre-prepared benchmark models, not your own model.

1. Write down the current driver, CUDA or ROCm version and PyTorch build. These are the versions a result is bound to.
2. Install with pip install lynxbench-ai on Python 3.11 or later, on Linux or on Windows via WSL2. Leave about 15 GB of free disk; the first run downloads about 2 GB of model data once.
3. Run the full pass, then run it again unchanged to see your own run-to-run spread.
4. Change one component, such as the driver or the runtime, to another publicly released version.
5. Run again, at least twice, under the same release.
6. Compare per-test scores within that release, and read GT only as an ordering.

The result tells you how your executor’s scores moved across a change you made, on one machine, with the noise you measured. It is evidence about your stack, not a controlled experiment about the driver.