A winning MLPerf submission is not automatically the right hardware for your GenAI use case. The number most teams read first — throughput or latency — tells you how fast a system can go, not what it costs to keep it going. MLPerf Power adds a mandated energy measurement recorded alongside the performance run, which turns the honest question from “which is the fastest hardware?” into “what is the performance-per-watt under a defined load?” For a high-volume inference deployment, that distinction is the difference between an ROI estimate that survives contact with the production electricity bill and one that does not. We work with teams scoping GenAI feasibility, and the failure we see most often is not a modeling failure. It is an accounting one: a use case gets approved on a per-token cost that only holds at the throughput ceiling, then the energy and cooling bill at real production volume quietly drags the return negative. Reading MLPerf Power correctly is the guardrail against that outcome. What does MLPerf Power actually measure? MLPerf is a suite of benchmarks published by MLCommons, the industry consortium that also standardizes the datasets and rules submitters follow. A standard MLPerf Inference submission reports how a system-under-test performs on a fixed task — a model, a dataset, and a scenario — in throughput (queries or samples per second) or latency (time to first token, time per output token, end-to-end response time). MLPerf Power is not a separate benchmark. It is an optional measurement layer that runs during a performance submission. When a submitter opts in, wall-power draw is measured at the AC input to the system-under-test using a certified power analyzer, sampled continuously across the same run that produces the performance score. The result is reported as energy per unit of work — joules per query, or equivalently a performance-per-watt figure derived from throughput divided by average power. The key structural point is that the power number is bound to a specific, disclosed workload. It is not a nameplate TDP and it is not a peak burst. It is the measured energy the system consumed while sustaining the benchmarked scenario. That framing — empirical execution under a defined load as the reference, rather than a spec sheet — is the same discipline that separates a defensible feasibility call from an aspirational one. Three things to hold onto before reading any MLPerf Power result: The measurement is at the wall (AC input), so it already includes power-supply losses and, in the datacenter-class rules, a portion of node overhead. It does not include facility cooling, which you must add separately. The number is scenario-specific. An Offline result (batch, throughput-maximizing) and a Server result (latency-bounded, request-driven) on the same hardware and model can differ substantially in performance-per-watt. Performance-per-watt is a ratio, and both terms move when you change batch size, sequence length, or model. Comparing two submissions is only valid when those variables match. How is a power submission different from a standard performance run? A standard MLPerf performance submission answers “how fast?” The power submission answers “at what energy cost per unit of that speed?” — measured on the same run, so the two numbers are internally consistent rather than stitched together from different tests. This matters because the two rankings do not always agree. A system can top the throughput chart and sit mid-pack on performance-per-watt, because it reached that throughput by pushing clocks and drawing power at a rate that scales worse than linearly with the extra work. The reverse also happens: a more modest accelerator can win on energy efficiency while losing on raw speed. Which ranking is relevant depends entirely on what dominates your cost — and for high-volume inference, recurring energy usually does. Dimension Standard MLPerf performance run MLPerf Power (opt-in layer on the same run) Question answered How fast? (throughput / latency) At what energy cost per unit of work? Primary metric Queries/sec, samples/sec, TTFT, TPOT Joules per query; performance-per-watt Measurement point Task completion timing AC input via certified power analyzer What it feeds Capacity and latency sizing Recurring run-cost / ROI estimate What it excludes Energy accounting Facility cooling; your own workload’s tail behavior If you are also weighing where a benchmark like this fits against client-side and training measurements, our breakdown of what MLPerf Client measures and how to read it and the companion piece on MLPerf Training for sizing AI agent infrastructure cover adjacent parts of the same suite. How do I turn performance-per-watt into inference cost? This is where the benchmark becomes a feasibility input rather than a leaderboard entry. Start from the two numbers a power submission gives you: sustained throughput and average power under the benchmarked scenario. From there, a defensible run-cost estimate is arithmetic, provided you keep the assumptions explicit. Worked example (explicit assumptions) Assume a system-under-test with these disclosed figures — treat every number here as illustrative, not a benchmark of any specific device: Sustained throughput on the benchmarked scenario: for example, if a system measured 5,000 output tokens/sec in the Server scenario. Average measured power at the wall during that run: on the order of 1,000 W (illustrative). Electricity price: roughly $0.12/kWh (a mid-range commercial rate; substitute your own). A cooling/facility overhead multiplier: PUE of about 1.3 is a common datacenter planning figure — you must confirm your own. Then, per hour, the system delivers 5,000 × 3,600 = 18 million tokens while drawing 1.0 kWh at the wall, or ~1.3 kWh including cooling at the assumed PUE. Energy cost per hour ≈ 1.3 × $0.12 = $0.156. Cost per 1M tokens (energy only) ≈ $0.156 ÷ 18 ≈ $0.0087 per 1M tokens. This is an observed-pattern-style planning calculation built from illustrative inputs, not a measured price for any product; its value is the method, not the figure. Two lessons fall out of the arithmetic. First, energy is rarely the dominant line item on its own for low-volume use, but at production scale it compounds relentlessly — it is a recurring operating cost, not a one-off. Second, the multiplier that matters is the gap between hardware options: in configurations we have reasoned through with clients, a 2–3x performance-per-watt spread between accelerators is enough to move a high-volume use case from automatable-with-positive-ROI to speculative, without any change in model quality (observed-pattern; the exact threshold depends on your token volume and electricity price). Getting the throughput term right also depends on how many tokens your workload actually generates. Our token size calculator walkthrough and the field notes in DeepSeek inference and what it costs in production are useful for pinning that number down before you trust a cost-per-token figure. How do I match a benchmarked scenario to my real workload? A performance-per-watt number is only meaningful if the run that produced it resembles your workload. MLPerf scenarios encode very different operating regimes, and reading across them is the most common way teams mislead themselves. Use this checklist before treating any MLPerf Power figure as a feasibility signal: Scenario match. Is the result Offline (batched, throughput-maximizing) or Server (latency-bounded, request-driven)? A real-time chat assistant lives in the Server regime; a nightly document-processing job lives in Offline. Reading an Offline efficiency number for a Server workload flatters the hardware. Model match. Was the submission run on the model family and size you intend to deploy? Efficiency does not transfer cleanly from a 7B model to a 70B one, or from a dense model to a mixture-of-experts. Batch and concurrency match. The benchmarked batch size and concurrency set the operating point on the throughput-vs-latency curve. Your production concurrency profile may sit somewhere the submission never measured. Precision match. Was the run in FP8, INT8, BF16? Precision choices move both throughput and power. If your accuracy budget forbids the submission’s precision, its efficiency number does not apply to you. Overhead scope. Confirm what the wall measurement includes and add facility cooling separately if it is not in scope. When several of these fail to line up, the honest move is to treat the published figure as a directional bound and plan a small in-house measurement on a representative workload before committing to a deployment target. Benchmarks narrow the search space; they do not close it. FAQ What does working with mlperf power involve in practice? MLPerf Power is an opt-in measurement layer that records wall-power draw with a certified analyzer during a standard MLPerf Inference run, so the energy figure is bound to the same workload as the performance score. In practice it converts a throughput or latency result into a performance-per-watt figure, which is what you need to estimate recurring inference cost rather than peak speed. What exactly does MLPerf Power measure, and how does it differ from a standard MLPerf performance submission? It measures energy per unit of work — joules per query, or performance-per-watt — at the AC input of the system-under-test, sampled across the same run that produces the performance number. A standard submission answers “how fast?”; the power layer answers “at what energy cost per unit of that speed?”, and the two rankings do not always agree because a system can top throughput while sitting mid-pack on efficiency. How do I read a performance-per-watt figure and translate it into inference cost for my own GenAI workload? Take the sustained throughput and average power from the run, convert power to kWh, apply your electricity price and a facility PUE multiplier, then divide by tokens produced to get cost per 1M tokens. The figure is only as good as its assumptions, so keep electricity price, PUE, and the token count for your workload explicit — energy is a recurring cost that compounds at production volume. How do I match a benchmarked scenario to my actual use case so the number is meaningful? Check that the scenario (Offline vs Server), model family and size, batch/concurrency profile, and numerical precision of the submission all resemble your intended deployment, and confirm what the wall measurement includes. When several of these do not line up, treat the published figure as a directional bound rather than a feasibility verdict. How does inference energy cost factor into classifying a use case as automatable, speculative, or research? Energy-per-inference is what turns a throughput number into a defensible run-cost estimate, and run-cost is a primary input to whether the automatable portion of a use case clears a positive ROI bar. For high-volume workloads, a 2–3x performance-per-watt gap between hardware options can move a use case from automatable to speculative without any change in model quality. What are the limits of MLPerf Power results — when should I not trust them as a feasibility signal? Do not trust a result when the scenario, model, batch size, or precision differs from your workload, or when facility cooling is out of the measurement’s scope and you have not added it back. The figures narrow the hardware search space; they do not substitute for a small measurement on a representative workload before you commit a deployment target. Where this leaves the feasibility call The reason we push infrastructure buyers toward MLPerf Power rather than peak-throughput charts is that a GenAI feasibility decision is a run-cost decision in disguise. Whether the organisation can operate and pay for the inference infrastructure this benchmark measures is a prior question — an [organisational-readiness assessment](AI Readiness Scorecard) that sits ahead of any per-use-case call — and only once that is settled does performance-per-watt become the number that decides whether a use case is automatable or merely aspirational. That grounding runs through our approach to generative AI feasibility, where energy-per-inference feeds the ROI estimate for the automatable portion of a workload directly. The open question for any team reading a leaderboard is narrower than “which hardware wins?” It is this: does the benchmarked scenario match your workload closely enough that its performance-per-watt figure can carry the run-cost estimate your ROI depends on — and if it does not, what small measurement do you run before you trust it?
