In a hall with a fixed power budget, watts decide how many GPUs you can install. Divide the usable IT power by each unit’s sustained draw and you have the ceiling for that SKU. Liquid cooling for GPU data centers changes that ceiling in two specific ways. A team that expects liquid cooling to double the GPU count of a power-limited hall is mixing up rack density and hall capacity, and this article keeps the two apart.
How many of each GPU SKU fits a fixed hall power budget?
Start with per-GPU watts and work upwards. NVIDIA’s A100 drew 400 W per chip, the H100 700 W and the B200 1,000 W, while a GB200 NVL72 rack pulls 120 to 130 kW in total. Our sizing rule for a single SKU: usable IT power, less a reserve, divided by the sustained wall draw of one unit gives the maximum unit count. Our rule for mixed fleets: split that budget between SKUs and multiply each SKU’s throughput on your workload by the number of units that fit.
LynxBenchAI gives a device-level reference for the throughput half of that sum: its published results show how each device runs the public PyTorch stack your own models are built on, so they are broadly indicative for custom PyTorch models too. Before you divide the budget, check whether your candidate SKUs appear in the LynxBenchAI published results. They do not run or measure your own workload.
NVIDIA lists the H100 SXM at up to 700 W and the H100 NVL at 350 to 400 W, both configurable. Lenovo’s ThinkSystem guide gives 350 W for the H100 80GB PCIe adapter and 400 W for the H100 NVL, both passively air-cooled, against 700 W for the SXM boards, listed water-cooled or passively air-cooled. In an 8-GPU HGX H100 node the GPUs alone account for about 5.6 kW before host CPUs, memory, NICs and fans, and the full node draw depends on the integrator build. An 8-GPU B200 server uses 8 kW in GPU power alone, with CPUs, memory, network cards, fans and power-supply losses on top.
Worked example: GPU-only ceilings per megawatt
The table takes 1 MW of IT power as the unit (NVIDIA’s MaxLPS overview also uses a fixed 1 MW power basis) and divides it by GPU board power only. Host, network, storage and reserve all come out of the same megawatt, so real counts are lower.
| A100 | 400 W | 2,500 | 312 |
| H100 SXM | 700 W | 1,428 | 178 (5.6 kW each) |
| B200 | 1,000 W | 1,000 | 125 (8 kW each) |
| H100 PCIe | 350 W | 2,857 | n/a |
| RTX 6000 Ada | 300 W | 3,333 | n/a |
Cooling enters this arithmetic before the division, through overhead. AMCompute’s example notes that PUE multiplies every watt of IT load: at a PUE of 1.8, a 10 MW IT deployment draws 18 MW at the facility, while at 1.15 the overhead falls to 1.5 MW. Now invert it. If the site is capped at 1 MW at the meter, a PUE of 1.8 leaves about 556 kW for IT (1,000 ÷ 1.8), which is 793 H100 SXM GPUs on GPU power alone. A PUE of 1.15 leaves about 870 kW (1,000 ÷ 1.15), which is 1,242 GPUs. Both PUE values are AMCompute’s, and they are assumptions here. Your site’s real figure belongs in the formula instead. On a meter-capped site, that overhead gap is one way the cooling choice changes the unit count.
Inside the IT load, fan power matters as well. ASHRAE TC9.9 (2021) reports that fan power of 10% to 20% of server power is not uncommon in dense air-cooled servers, and because fans share the UPS with the server, moving from 2% to 10% fan power cuts usable UPS capacity by 8%. NVIDIA claims that running Vera Rubin NVL72 racks on 45°C warm-water cooling, which lets many climates use dry coolers instead of compressors, frees enough power for up to 10% more NVL72 racks in the same budget. That is NVIDIA’s claim about its own platform. It is not a measured result for your hall.
Two deductions come before the division. The first is a reserve for transients and redundancy. NVIDIA, for instance, says its DGX H100 needs power capping enabled to run safely in an N+N power-supply configuration. The second is the gap between rated and sustained draw on your own jobs. Vendor figures such as the H100’s are maximums or defaults, and the sustained draw on your workload sets the count.
How liquid cooling for GPU data centers changes rack density
Liquid cooling raises the power one rack can hold, and the published thresholds disagree on exactly where air stops working. RDP puts the practical ceiling for air at roughly 30 to 40 kW per rack, says direct-to-chip liquid becomes necessary above about 35 kW, and treats immersion as the practical option beyond about 100 kW. JLL research, as summarised by Network World, puts plain air at up to about 20 kW, rear-door heat exchangers at up to about 100 kW, direct-to-chip between about 100 and 175 kW, and immersion above about 175 kW. In a separate JLL article, Andrew Green says that, with rack densities growing beyond 70 kW, liquid cooling, mainly direct-to-chip or immersion, is currently the only viable solution.
| Rack density | Cooling the sources point to | GPU configurations in this range |
|---|---|---|
| Up to ~20 kW | Standard air with hot/cold aisle containment | A few PCIe GPU servers; Uptime’s 2024 survey average rack is 8 kW |
| ~20–40 kW | High-speed fans, rear-door heat exchangers or in-row cooling; air’s practical ceiling | Four 8-GPU H100 nodes already carry 22.4 kW of GPU power alone (4 × 5.6 kW) |
| ~40–100 kW | Direct-to-chip per RDP; rear-door exchangers per JLL; liquid as densities pass 70 kW, per JLL’s Green | Schneider’s modelled 73 kW liquid-cooled AI racks |
| ~100–175 kW | Direct-to-chip | NVL72 racks at 120–132 kW, which cannot be air-cooled |
| Above ~175 kW | Immersion; two-phase immersion supports 150 to 250-plus kW |
Plan on a liquid loop once racks pass about 35 to 40 kW, and treat NVL-class racks as direct-to-chip from day one. ASHRAE ties the need for liquid at a rack to the physics of heat removal. ASHRAE notes that some servers need 100 cfm or more per U, so a best-of-breed floor tile delivering 1,900 cfm can feed only 19U of such equipment. NVIDIA’s DGX SuperPOD guide asks for at least 157 ft³/min of airflow per kilowatt.
At the node level, the density gain is concrete. Supermicro says its 4U liquid-cooled HGX H100/H200 system doubles the density of its 8U air-cooled system. In Supermicro’s HGX B200 examples, a 42U rack holds 64 GPUs liquid-cooled against 32 air-cooled. The GPUs, memory and NVLink are identical between the air and liquid HGX H100 builds, and only density and facility requirements differ.
Doubling the GPUs per rack means fewer racks and less floor space. In a power-limited hall it does not double the GPUs you can power. NVIDIA’s own design guide spells out the coupling: in an air-cooled facility, a cooling shortfall caps rack density and pushes the same servers across more racks. Liquid cooling decides the GPU count in three cases: when the hall runs out of rack positions or floor area before it runs out of power, when the SKU you want (an NVL72, say) cannot run on air at all, or when the overhead saving above moves enough watts into IT. Outside those cases its value is floor space, shorter cable runs (which AMCompute ties to lower communication latency in training) and sustained clocks.
Retrofitting an air-cooled hall costs more than the cooling invoice
The most useful current public benchmark puts a liquid-cooling retrofit at about $2 million per MW, against more than $11 million per MW for new greenfield liquid-cooled capacity (STL Partners, as reported by DatacenterScope). DatacenterScope warns that the denominator is only the capacity actually converted, not every MW in the building, and that power upgrades, migration, stranded space, lost revenue, commissioning and live-site risk often sit outside that headline figure.
The other costs are physical and commercial. CBRE reports that fit-out periods for air-cooled facilities converting to liquid can run past six months, often around live workloads. Concentrating power into a few very dense racks can use up electrical capacity before the rack positions are full, which leaves stranded white space. JLL’s Green says floor loading rises from 12 to 15 kPa to at least 20 kPa when the same compute sits in a liquid-cooled footprint, and that an existing facility is unlikely to be upgradable to 100% AI. Uptime Institute’s 2024 global survey found that 29% of operators are upgrading existing data hall space to meet high-density demand.
The less invasive paths cap out lower. Introl lists rear-door heat exchangers at $8,000 to $15,000 per rack for 15 to 30 kW, in-row units at $20,000 to $35,000 for 40 to 100 kW, and direct-to-chip at $50,000 to $80,000 per rack for 60 kW and above. A liquid-to-air CDU lets direct-to-chip racks run without a facility-water connection, but the heat still goes into the room, so the existing air system remains part of the capacity limit.
Retrofit spend only pays where the throughput gain from the denser or more efficient SKU mix covers it. Two further questions, a like-for-like PUE and cost comparison of two pod designs and an independent check of an integrator’s claims, sit outside this page. Each is its own exercise, and neither can be settled from the per-rack arithmetic here. The same applies to whether the interconnect holds up at campus scale.
Do RTX 6000 Ada cards need different cooling or circuits than RTX 4090s?
The RTX 6000 Ada cools with its own active cooler and chassis airflow. The difference that matters is power draw. NVIDIA’s datasheet gives the RTX 6000 Ada a 300 W total board power, an active cooler and a dual-slot, full-height form factor. ServeTheHome measured about 305 W under load and found that the RTX 4090 takes 50 to 60% more power to reach similar performance. The RTX 6000 Ada uses a blower-type cooler and depends more on chassis airflow, which suits workstation and server chassis with strong airflow. Puget Systems notes that liquid-cooling a multi-GPU workstation can reduce noise but adds substantial cost and does not remove the need for a high-capacity circuit.
Worked example: two dual-GPU workstations on one 20A circuit
- The NEC (article 210.20(A)) requires the breaker to be rated at no less than 125% of a continuous load, so a continuous load may use 80% of the breaker. A 20A, 120V circuit is 2,400 W, so the continuous limit is 1,920 W.
- To convert to wall watts, add about 100 W for board, memory, NVMe and fans to GPU and CPU power, then divide by PSU efficiency, using 90% as a working figure.
- Two RTX 6000 Ada cards at 300 W plus 100 W of platform is 700 W DC per workstation before the CPU, or about 778 W at the wall (700 ÷ 0.9). Two workstations come to about 1,556 W, which leaves roughly 364 W at the wall (about 328 W DC) for both CPUs together.
- Two 4090-based workstations at 900 to 1,000 W of GPU power each need 1,800 to 2,000 W DC for GPUs alone, which is about 2,000 to 2,222 W at the wall and over the 1,920 W limit before any CPU is counted.
On these figures, the RTX 6000 Ada pair fits only with modest CPUs and nothing else on the breaker. Two 4090 workstations on one circuit do not fit at all. A 240V 20A circuit gives 3,840 W continuous under the same 80% rule.
Fewer H100 nodes at full power, or more nodes with a power cap?
Under fixed headroom, more power-capped H100 nodes beat fewer full-power nodes only when the throughput lost to the cap is smaller than the throughput gained from the extra nodes, and only measurement on your own jobs settles which side you are on. TechnoLynx runs that comparison for teams sizing against a headroom figure: it defines the workload mix and the power constraint, sweeps nvidia-smi power caps on the target H100 nodes to characterise throughput per watt, and validates throughput, latency and thermal behaviour against the uncapped baseline before a production policy is set. The first step in that work for AI infrastructure and production SaaS teams is framing the workload mix and the headroom number.
The mechanism is a software watt ceiling per GPU, set with nvidia-smi -pl, which the GPU enforces by lowering clock and voltage together through DVFS. On the H100 SXM5 the default cap is the rated 700 W and the configurable range is roughly 200 to 700 W. The H100 PCIe ranges from 200 W to 350 W, or to 310 W in its 300 W connector power mode. NVIDIA’s DGX H100 documentation lists three sources of power limit (the VBIOS maximum, nvidia-smi from the host and SMBPBI out of band), and the GPU’s PMU applies the most conservative one. At fleet level, NVIDIA’s Dynamic Power Software keeps total GPU power inside an approved envelope and moves unused headroom to the GPUs that need it every few seconds. NVIDIA claims its MaxLPS framework, which combines DPS with facility design and performance-per-watt techniques, fits up to 40% more GPUs in the same site-power envelope.
The measured cost varies with the workload. Factryze reports that capping an H100 at 600 W typically loses 5 to 15% of step throughput on transformer pretraining, with FP8 workloads losing somewhat more. An arXiv study sweeping H100 and H200 from 200 W to 700 W in 100 W steps found large gains from 200 W to 400 W and diminishing returns from 500 W to 700 W, about 10% per step. On memory-bound loops the H100 reached peak effective bandwidth at around 350 W and held it even at a 200 W cap. Efficiency varied at caps of 300 W and below.
Take the Factryze range and run the arithmetic on GPU power alone. One megawatt holds 1,428 GPUs at 700 W and 1,666 at 600 W, about 16.7% more, on GPU power alone. With a 5 to 15% loss per GPU, aggregate throughput lands between about 0.99× and 1.11× the uncapped fleet (1.167 × 0.85 and 1.167 × 0.95). Host CPUs, NICs and fans sit outside the per-GPU cap, so the real per-node gain is smaller than that. At the top of the loss range, capping for density roughly breaks even. Memory-bound inference, which gives up little to a cap, is where more capped nodes most clearly win.
Three caveats belong in the sign-off. Factryze advises capping every GPU in a synchronous job uniformly, validating the cost on your own workload, and watching P99 step time, because a growing tail usually points to a thermal asymmetry that should be fixed at the cooling layer. The same source treats capping as a budget tool for limits imposed from outside (a building, a contract), not as a cure for buying the wrong density. Capping also sits alongside thermal throttling: NVIDIA’s H100 PCIe brief specifies a 50% hardware clock slowdown at a set GPU temperature threshold. Edge modules such as the Jetson Orin family substitute for H100 nodes only when the workload itself runs at the edge.
Who can measure GPU power behaviour and help size the fleet against our budget?
TechnoLynx helps teams sizing a GPU fleet against a fixed power budget by measuring how candidate GPUs actually behave on their workloads, at full power and under caps, so the unit count and SKU mix rest on sustained numbers rather than datasheet maximums. When choosing a partner, ask whether they measure on your workload mix and report tail latency and thermals alongside mean throughput. For the device comparison itself, LynxBenchAI’s published results across hardware are the starting point. You can check your candidates against the LynxBenchAI results before any engagement.
This work covers the compute side. Facility cooling design, electrical upgrades and structural reinforcement belong to mechanical and electrical engineers, colocation operators and server integrators. Supermicro, Dell, Gigabyte and HPE each ship their own HGX H100 thermal designs, with Supermicro offering both air-cooled and liquid-cooled builds. If the hall cannot be retrofitted in time and renting is on the table, benchmarking reserved and spot cloud pricing shows whether the rates you are quoted hold up.
The throughput number the datasheets leave out
The per-megawatt arithmetic gives you unit counts. It does not tell you how much work those units will do. Rated TFLOPS, such as the 91.1 TFLOPS single-precision figure NVIDIA quotes for the RTX 6000 Ada, are peak rates based on boost clock. The arXiv sweep is a reminder that two SKUs can swap rank depending on whether the job is compute-bound or memory-bound. In that study the H100 led slightly on DGEMM at equal power caps, while the H200 at 400 W beat the H100 at full power on memory-bound efficiency. For any power-constrained hall, then, the cooling method fixes which SKUs are deployable and how many watts reach them, and only throughput measured on your own workload settles which mix wins.
