A CTO forwards you a slide. The vendor’s top-of-rack machine posts a mixed-precision HPL-MXP figure several times larger than its double-precision HPL number, and the sales narrative writes itself: this is the machine for AI. Buy it, and training gets faster. The number is real, the benchmark is legitimate, and the conclusion is still wrong — not because HPL-MXP lies, but because it answers a question that is adjacent to yours, not identical to it. HPL-MXP measures how fast a machine solves a large dense system of linear equations using mixed-precision arithmetic with iterative refinement. That correlates with AI-training throughput far better than the classic double-precision HPL ever did. But it remains a proxy. The gap between “this machine is fast at HPL-MXP” and “this machine will train your model efficiently” is exactly the gap where over-provisioned procurement decisions go to die. What HPL-MXP actually measures The classic High Performance Linpack (HPL) benchmark — the one behind the TOP500 list for three decades — solves a dense linear system in IEEE double precision (FP64) and reports sustained FP64 FLOPS. It was designed for an era when scientific simulation demanded full double-precision accuracy end to end, and it became the de facto measure of “supercomputer speed.” HPL-MXP (also written HPL-MxP, and historically HPL-AI) keeps the same underlying problem — factor a large dense matrix, solve the system — but performs the heavy factorization in lower precision, typically FP16 or bfloat16, then recovers double-precision accuracy in the solution through an iterative-refinement loop. The refinement step repeatedly computes a residual and corrects the approximate solution until it converges to the same accuracy the FP64 solver would have produced. The reported figure is an effective FP64-equivalent FLOPS rate: how much full-precision work the machine effectively delivered by doing most of the arithmetic cheaply and cleaning up afterward. Two things follow directly, and both are citable when someone waves the number at you: HPL-MXP’s headline figure is typically several times larger than the same machine’s classic HPL FP64 figure, because the bulk of the arithmetic ran in FP16/BF16 where the hardware has far more throughput — this is a benchmark-class ratio, reported by the HPL-MXP reference runs, not an operational measurement of your workload. The benchmark only “counts” a run as valid if the iterative-refinement loop converges to double-precision accuracy. It is a mixed-precision path to a full-precision answer, not a low-precision answer. That second point is where naive readings start to fail. HPL-MXP is not “the machine’s FP16 speed.” It is the machine’s ability to use FP16 as a fast inner engine while a correction loop guarantees the final numerics. Your AI training run has no such guarantee and no such loop. Why mixed precision was introduced — and what it borrowed from AI The mixed-precision variant exists because the hardware changed. Tensor cores, matrix engines, and their equivalents across vendors deliver enormous FP16/BF16 throughput precisely because deep-learning training demanded it. HPL-MXP was, in a sense, reverse-engineered from AI: it exists to give the TOP500-adjacent world a benchmark that reflects the low-precision matrix hardware that AI workloads drove into existence. That shared lineage is why HPL-MXP tracks AI-training capability better than FP64 HPL does. A machine built to train transformers has tensor engines sized for FP16/BF16 matrix multiply, and HPL-MXP exercises exactly that path. If you are comparing two accelerators and one posts a dramatically higher HPL-MXP figure, it very probably has more or faster low-precision matrix throughput — which is genuinely relevant. The trap is treating a directional correlation as a workload prediction. Real training is a memory-bandwidth-bound, communication-bound, sequence-shaped problem for large stretches of its runtime. HPL-MXP is a compute-bound dense-GEMM problem with a highly regular access pattern. The two overlap in the compute engine and diverge almost everywhere else. If your model is bottlenecked on HBM bandwidth or on the interconnect during gradient all-reduce — as many large-model training runs are — a higher HPL-MXP score buys you throughput you cannot use. We see this pattern regularly when a team specs hardware by peak matrix FLOPS and then watches utilization sit far below the headline during real epochs. The same bandwidth-versus-compute confusion shows up in serving, which is why when memory bandwidth is the real bottleneck in AI inference is worth reading alongside this. How does HPL-MXP’s iterative refinement diverge from real AI training? This is the question the number does not answer on its own, so it is worth being explicit. Iterative refinement is a numerical-stability construct. The FP16 factorization introduces error; the refinement loop measures that error against the FP64 problem and iterates until the final answer is FP64-accurate. The benchmark therefore rewards hardware that can do fast FP16 GEMM and fast FP64-ish residual work, in a tight, deterministic loop with a known convergence target. Neural-network training has no equivalent convergence guarantee and no residual-correction loop that restores full precision. When you train in mixed precision — with automatic mixed precision in PyTorch, or bfloat16 throughout — you keep a master copy of weights in higher precision and apply loss scaling to keep gradients representable, but you are not iteratively refining toward an FP64 reference. You are optimizing a non-convex loss, and the low-precision arithmetic contributes noise that interacts with your optimizer, your learning-rate schedule, and your model’s numerical sensitivity in ways HPL-MXP never touches. Concretely, the divergence points that matter for procurement: HPL-MXP’s precision profile is fixed and validated. Your model’s tolerance for FP16 vs BF16 vs FP8 is empirical and model-specific. A machine that tops HPL-MXP with FP16 tells you nothing about whether your model trains stably in FP16. HPL-MXP’s problem is dense and regular. Attention, sparse layers, and variable sequence lengths produce access patterns HPL-MXP does not represent. HPL-MXP measures a single machine solving one problem. Distributed training spends significant time in collective communication (NCCL all-reduce, all-gather) across NVLink and the fabric — a dimension HPL-MXP exercises only weakly, if at all. HPL-MXP vs. classic HPL vs. your training workload The comparison below is the surface to hand a procurement team. It is deliberately structured so it can be lifted out of context and still make sense. Dimension Classic HPL (FP64) HPL-MXP Your AI training run Arithmetic precision FP64 throughout FP16/BF16 inner, FP64-accurate result via refinement Mixed (BF16/FP16/FP8), no refinement-to-FP64 What it stresses FP64 compute Low-precision matrix engines + refinement loop Compute, HBM bandwidth, interconnect, all in shifting proportion Access pattern Dense, regular Dense, regular Model-dependent, often irregular Interconnect load Moderate Moderate Heavy during collective ops (all-reduce) Convergence target Exact solve Guaranteed FP64-accurate solution Empirical, non-convex, no guarantee Correlates with AI training? Weakly Directionally, on the compute axis — Safe to procure on alone? No No This is what you must characterize Read left to right, the point lands on its own: HPL-MXP is a better proxy than FP64 HPL, and a proxy is still not a workload. The rightmost column is the one your feasibility assessment has to fill in with measured requirements, not benchmark inheritance. Why a high HPL-MXP score is not sufficient procurement justification Here is the failure mode stated plainly. A team quotes the HPL-MXP number, commits to a machine, and then discovers during the first real training runs that the bottleneck is HBM bandwidth or fabric contention — neither of which the benchmark represented. The FP16 matrix engines the number celebrated sit partly idle. Now you own double-precision-heavy or compute-heavy capacity you are not using, and you are short on the bandwidth or interconnect that actually governs your throughput. This is over-provisioning one axis while under-provisioning another, and it is expensive precisely because it looks defensible on the slide. The defensible alternative is to treat HPL-MXP as one input to a feasibility assessment, not as the assessment. That means characterizing your own workload first: what precision does your model actually train in, where does it spend wall-clock time, is it compute-bound or bandwidth-bound in the phases that dominate, and how much of the run is collective communication? Only then does the HPL-MXP number become useful — as a check on the compute axis, cross-referenced against the memory and interconnect specifications the benchmark under-represents. If you have never separated peak compute from sustained achievable throughput, the reasoning in what GFLOPS actually measures and when it predicts inference speed transfers directly to the training case. Grounding an infrastructure decision this way produces a concrete artifact: a hardware decision document that ties each capacity choice to a measured workload requirement, rather than to a headline FLOPS figure. That document is what survives the moment a project stalls and someone asks why this machine was bought. Turning that discipline into an engagement is what our AI consulting and R&D services exist to do — the sober counterweight when a vendor is selling peak numbers. A short rubric for reading any procurement benchmark Before you let a single benchmark figure drive a hardware decision, run it through five questions: What precision did the benchmark run at, and does it match your model’s training precision? If the benchmark is FP16 and your model needs BF16 for stability, the headline is inflated relative to your reality. Is the benchmark’s problem compute-bound, and is your workload? If your training is bandwidth- or communication-bound in its dominant phases, a compute benchmark over-states the machine’s usefulness to you. Does the benchmark exercise the interconnect the way distributed training does? Single-node dense solves do not. What hardware characteristic does the benchmark under-represent? For HPL-MXP, that is usually HBM bandwidth and fabric bandwidth under collective load. Can you trace the purchase to a measured requirement? If the only evidence is the benchmark number, you do not yet have a decision — you have a slide. FAQ How does HPL-MXP work in practice? HPL-MXP solves a large dense linear system by doing the heavy matrix factorization in low precision (typically FP16 or BF16) and then running an iterative-refinement loop that corrects the solution until it reaches full double-precision accuracy. The reported figure is an effective FP64-equivalent FLOPS rate — how much full-precision work the machine effectively delivered by computing cheaply and cleaning up afterward. In practice it signals how strong a machine’s low-precision matrix engines are, which is relevant to AI but not identical to it. How does HPL-MXP differ from the classic HPL benchmark, and why was mixed precision introduced? Classic HPL performs the entire solve in FP64 and reports sustained double-precision FLOPS; HPL-MXP performs the factorization in FP16/BF16 and recovers FP64 accuracy through iterative refinement, so its headline figure is typically several times larger on the same machine. Mixed precision was introduced because hardware changed: the tensor and matrix engines built for deep learning deliver far more low-precision throughput, and HPL-MXP exists to reflect that AI-driven hardware in a TOP500-adjacent benchmark. What does HPL-MXP’s iterative-refinement scheme actually measure, and where does it diverge from real AI-training workloads? Iterative refinement measures a machine’s ability to use fast low-precision GEMM as an inner engine while a correction loop guarantees an FP64-accurate final answer against a known convergence target. Real training has no such refinement loop or convergence guarantee — it optimizes a non-convex loss where low-precision noise interacts with the optimizer and model sensitivity — and it is often bandwidth- or communication-bound rather than compute-bound, so the two overlap in the compute engine and diverge almost everywhere else. Why is a high HPL-MXP score not sufficient justification for an AI infrastructure purchase? Because the score characterizes a compute-bound dense solve, not your workload. If your training is dominated by HBM bandwidth limits or collective-communication overhead, a higher HPL-MXP figure buys throughput you cannot use, leaving you over-provisioned on compute and short on the bandwidth or interconnect that actually governs your run. The number is one input, not the decision. How should a serious team use HPL-MXP as one input to a feasibility and infrastructure-risk assessment? Characterize your own workload first — its training precision, where it spends wall-clock time, whether it is compute- or bandwidth-bound in dominant phases, and how much time is collective communication. Then use HPL-MXP as a check on the compute axis only, cross-referenced against the memory and interconnect specs it under-represents. The output should be a hardware decision document that ties each capacity choice to a measured requirement. What hardware characteristics does HPL-MXP under-represent for real training? HPL-MXP under-represents HBM memory bandwidth, since its dense regular access pattern does not stress bandwidth the way irregular or memory-bound training phases do, and it under-represents interconnect bandwidth, because its single-machine dense solve exercises the fabric far more lightly than distributed training’s collective all-reduce and all-gather operations. It also says nothing about whether your specific model trains stably at a given precision. The honest closing question for any team holding an HPL-MXP slide is not “how big is the number?” but “which axis of my workload does this number describe, and which axes has it left silent?” That framing is the feasibility check the A5 AI Project Risk Assessment is built to document — infrastructure justified against measured requirements, not against a benchmark nobody interrogated.
