A vendor hands you an HPL-MxP number and calls it the cluster’s LINPACK score. Treat that figure as a headline capability metric and you will size the wrong machine. HPL-MxP is not a faster HPL — it is a different solve, exercising a different precision path, and on the same nodes it can post a number several times larger than the classic High Performance LINPACK you may be mentally comparing it to. That gap is the whole point of the benchmark, and also the whole trap. It is not measurement noise or a better-tuned run. It comes from HPL-MxP deliberately doing most of its arithmetic in low precision and then repairing the result with iterative refinement. If your production workload leans on that same low-precision, tensor-core-heavy path, the number tells you something useful. If it does not — if you are FP64-bound or interconnect-bound — the headline figure describes a machine you will never actually run. What HPL-MxP is, and how it differs from plain HPL Both benchmarks solve a dense system of linear equations, Ax = b, by LU factorization. That shared shape is why people assume HPL-MxP is just HPL with a speed knob turned up. The difference is in the precision of the arithmetic and in what happens after the factorization. Classic HPL performs the factorization and solve in IEEE double precision (FP64) throughout. Its result is the machine’s sustained FP64 GEMM throughput, expressed in FLOP/s, and it is the number that has ranked the TOP500 list for decades. That path stresses the FP64 units on a GPU and the memory hierarchy that feeds them. HPL-MxP — the mixed-precision variant that anchors the companion HPL-MxP / Mixed-Precision benchmark list — factors the matrix in a lower precision (typically FP16 or a comparable reduced format, often through tensor cores) and then recovers a full FP64-accurate answer through an iterative refinement loop. The bulk of the arithmetic runs where the hardware is fastest; the correction runs where accuracy is decided. The reported score reflects the low-precision-dominated throughput, not the FP64 path a legacy dense solver actually exercises. That distinction is the reason two “LINPACK” numbers for the same cluster can disagree by a wide margin, and why quoting one where the other is meant is a genuine sizing error rather than a rounding quibble. Why does mixed-precision iterative refinement recover FP64-equivalent accuracy? The intuition that low precision means a low-precision answer is where most misreadings start. It is wrong, and understanding why is what makes the HPL-MxP number legible. Iterative refinement works on the residual, not the solution directly. You factor A cheaply in low precision to get an approximate solution x₀. That solution is inaccurate, but you can compute the residual r = b − Ax₀ in high precision, solve the much easier correction system A·d = r using the low-precision factorization you already have, and update x₁ = x₀ + d. Repeat, and the error shrinks each pass because the correction step only ever has to represent the gap between where you are and the true answer — a quantity that gets smaller and stays well-conditioned. The expensive part, the O(n³) factorization, happens once and in low precision. The refinement loop is O(n²) work per iteration and converges in a handful of passes for well-conditioned matrices. So you pay for the cheap arithmetic on the dominant term and buy back FP64-equivalent accuracy with a small high-precision correction. This is the same numerical trick that makes reduced-precision training and inference viable, and it is worth reading alongside how FP4 works on GPUs and when it pays off — the accuracy-recovery mechanism is different, but the reason low precision is safe to use is closely related. The catch is baked into the phrase “for well-conditioned matrices.” HPL-MxP generates matrices that refine cleanly. A real solver facing an ill-conditioned system may need far more refinement iterations, or may not converge at the same rate — which is one of several reasons the benchmark’s convergence behaviour is not a promise about your code. Why is the HPL-MxP score often several times higher than plain HPL? Reduced-precision units on modern accelerators run far faster than the FP64 units on the same die. On NVIDIA data-center parts, tensor-core FP16/BF16 throughput is a large multiple of native FP64 throughput per NVIDIA’s published specifications — the exact ratio depends on the part and generation, but the direction is consistent and it is the direct cause of the benchmark gap. HPL-MxP routes the dominant factorization work onto those fast units; HPL keeps it on the slow ones. The observable consequence is straightforward: on the same GPU nodes, an HPL-MxP score three times or more above the plain HPL score is normal, not anomalous — a ratio you can measure directly by running both benchmarks and dividing (benchmark; run both on your candidate nodes to get the number for your hardware, since it moves with the part). The gap is a property of the precision path, and it exists whether or not any of your workloads can use that path. That is exactly why the ratio is diagnostic. It tells you how much headroom the reduced-precision units carry over the FP64 units on a given cluster. What it does not tell you is whether your jobs live on the fast side of that divide. When is HPL-MxP a valid proxy for your workload? The rule underneath all of this: benchmark precision must match workload precision before the number means anything. HPL-MxP is a fair proxy for workloads whose dominant compute is dense, low-precision matrix multiply with tolerance for iterative accuracy recovery — which describes a large fraction of transformer training and inference, where reduced-precision GEMM through tensor cores is the hot path. It is a poor proxy for workloads dominated by FP64 arithmetic (many scientific solvers, certain financial and simulation codes) or bottlenecked on interconnect and memory movement rather than raw FLOP throughput. Use the table below to decide which number to trust for a given workload before you read any headline figure. Which LINPACK number should you trust? Your workload’s dominant cost Read HPL-MxP as Read plain HPL as Also check Reduced-precision dense GEMM (transformer train/inference) Relevant upper bound Understates achievable throughput Sustained vs peak; memory bandwidth FP64 scientific solve Misleading — ignore the headline Relevant capability figure FP64 GEMM rate specifically Sparse / irregular compute Misleading for both Loose upper bound at best Memory-access pattern, not FLOPs Communication-bound multi-node Misleading for both Misleading for both Interconnect fabric, collective bandwidth Mixed FP64 core + reduced-precision hot loop Overstates Understates Profile the split empirically Evidence class for this table: observed-pattern — a decision rubric drawn from workload profiling practice, not a single benchmarked result. Confirm the precision split for your own code by profiling before you commit. The honest position is that no single LINPACK number is your workload. Both HPL and HPL-MxP are dense-GEMM benchmarks; neither exercises the sparse access patterns, the collective-communication phases, or the data-feed bottlenecks that decide real throughput. This is the same reason AI FLOPS figures need care before they drive cluster sizing — a peak or benchmark FLOP number is an upper bound the workload rarely touches, not a prediction. How should HPL-MxP results factor into cluster sizing and interconnect decisions? The practical failure to avoid is sizing node count against an HPL-MxP figure when a meaningful share of your workload cannot use the reduced-precision path. You buy for the benchmark’s precision profile and receive a machine that is slower on your actual jobs — often the FP64-bound or communication-bound ones — with no benchmark record explaining the shortfall. A defensible reading follows a short sequence. First, estimate the fraction of your target workload that is genuinely tensor-core-bound — dense, reduced-precision GEMM — versus FP64-bound or interconnect-bound. Second, weight HPL-MxP and plain HPL accordingly rather than quoting either headline. Third, if the workload has a significant communication phase, treat both LINPACK numbers as compute upper bounds and size the fabric separately: dense benchmarks tell you almost nothing about collective bandwidth, which is where multi-node training frequently stalls. The broader HPC Challenge benchmark suite exists precisely because a single dense-GEMM number cannot cover memory and communication behaviour, and its component tests are a better starting point when the fabric is the question. We see this pattern regularly in procurement conversations: a headline TFLOP figure arrives already framed as capability, and the useful work is separating what it measures from what the buyer needs. Reading the benchmark correctly is one concrete decision point inside the larger question of what GPU infrastructure to commit to — the ground the GPU acceleration and infrastructure practice is built to reason through — and it feeds directly into the cost case, since the same cost-per-useful-FLOP framing that separates AWS from Azure for GPU workloads depends on knowing which FLOPs your jobs can actually reach. FAQ What should you know about HPL-MxP in practice? HPL-MxP solves a dense linear system by factoring the matrix in low precision (typically FP16, often on tensor cores) and then recovering a full FP64-accurate answer through iterative refinement on the residual. In practice it reports the throughput of that reduced-precision-dominated path, so its number reflects how fast the fast units are — not the FP64 path a legacy solver runs. What is the difference between HPL-MxP and the classic HPL LINPACK benchmark? Both solve Ax = b by LU factorization, but classic HPL does everything in FP64 and reports sustained double-precision throughput, while HPL-MxP does the bulk of the arithmetic in low precision and repairs accuracy with refinement. The result is that HPL-MxP measures a different precision path on the same hardware, which is why the two scores can diverge sharply. Why does mixed-precision iterative refinement let HPL-MxP recover FP64-equivalent accuracy from low-precision compute? Refinement works on the residual, not the solution: you compute the error in high precision, solve a small, well-conditioned correction system using the low-precision factorization you already have, and update the answer. The expensive O(n³) factorization happens once in cheap arithmetic, and a few O(n²) correction passes buy back FP64-equivalent accuracy — for well-conditioned matrices, which is what the benchmark generates. Why is an HPL-MxP score often several times higher than plain HPL on the same GPU cluster? Reduced-precision units on modern accelerators run at a large multiple of native FP64 throughput per NVIDIA’s published specifications, and HPL-MxP routes its dominant factorization work onto those fast units while HPL keeps it on the slow FP64 units. A ratio of three times or more on the same nodes is normal and reflects the precision path, not a tuning difference. When is HPL-MxP a valid proxy for my real workload, and when is it misleading? It is a fair proxy for workloads dominated by dense, reduced-precision matrix multiply that tolerate iterative accuracy recovery — a large share of transformer training and inference. It is misleading for FP64-bound scientific solvers, sparse or irregular compute, and communication-bound multi-node jobs, because those live off the fast precision path the benchmark measures. How should HPL-MxP results factor into GPU cluster sizing and interconnect decisions? Estimate the fraction of your workload that is genuinely tensor-core-bound versus FP64- or interconnect-bound, then weight HPL-MxP and plain HPL accordingly instead of quoting either headline. Treat both dense-GEMM numbers as compute upper bounds and size the interconnect fabric separately, since neither LINPACK variant measures the collective bandwidth where multi-node training often stalls. Before a single node is committed, the question worth answering is not “how big is the HPL-MxP number” but “what fraction of my workload can reach the precision path that number measures, and where does the rest of it bottleneck.” Get that split right and the benchmark becomes a sizing input; get it wrong and it becomes a headline capability figure describing a machine you will never run.
