Key Takeaways
- Air cooling struggles once racks move past roughly 30-50 kW, yet many AI deployments already exceed 100 kW per rack
- A single NVIDIA H100 GPU’s SXM version draws 700 watts (the PCIe version draws 350 watts), while the B200 SXM6 reaches 1,000 watts and up to 1,200 watts at full spec; the GB200 Superchip’s combined CPU-GPU package draws 1,200 watts, with the GB200 NVL72 rack targeting roughly 120 kW across 72 GPUs
- Liquid moves heat far more effectively than air, which explains why the industry is moving toward it so fast
- Direct-to-chip, rear door heat exchangers, and immersion cooling each solve the density problem differently, and choosing the right fit depends on facility goals
- This shift is a technical readiness question facilities should assess before their next hardware refresh forces the issue
Data centers built even five years ago were never designed for what AI hardware demands today. Racks that once ran at 5 to 10 kilowatts now house GPU clusters pulling ten times that amount, and the gap between what air can cool and what these chips produce keeps widening. Understanding why that gap exists, and what to do about it, matters for any operator planning the next hardware refresh.
GPU Racks Now Outpace Air Cooling
Walk into a facility running a modern AI training cluster and the difference is physical, not theoretical. Fans spin louder, hot aisles run hotter, and cooling units that once had capacity to spare now operate at their limits around the clock. This shift happens within single hardware generations, not over years of gradual change.
The shift carries business consequences too. Experts like Triton Thermal say that facilities that can’t support dense GPU deployments limit what their organizations can do with AI, whether that means training larger models or running more inference workloads simultaneously. Triton Thermal works with operators moving through exactly this transition, matching cooling strategy to the specific chips, racks, and facility constraints each deployment presents rather than defaulting to a one-size-fits-all fix.
What follows breaks down why air cooling runs out of road, what liquid cooling offers instead, and how to figure out if a facility is ready to make the switch.
AI’s Extreme Power Demands
How Hot GPUs Have Gotten
GPU power draw has climbed sharply with each new generation of AI accelerators. The NVIDIA H100 SXM version draws 700 watts, while the H100 PCIe version draws 350 watts. The newer B200 SXM6 reaches 1,000 watts, with full-spec B200 configurations reaching up to 1,200 watts. The GB200 Superchip, which combines CPU and GPU in one package, draws 1,200 watts, and a GB200 NVL72 rack housing 72 GPUs is planned at roughly 120 kilowatts total. Multiply these figures across the eight or more GPUs packed into a single training rack, and the heat output concentrated in one small footprint becomes hard to overstate.
Almost all of that electrical energy converts directly into heat rather than useful computing output, a reality rooted in basic physics rather than any flaw in chip design. Sustained, near-maximum GPU utilization during training runs that last days or weeks only compounds the problem, since there’s little downtime for heat to dissipate on its own. This sustained-load pattern sets AI racks apart from the traditional, intermittently busy servers that data centers were originally built around.
From 10 kW Racks to 100+ kW Racks
Traditional enterprise racks were engineered for 5 to 10 kilowatts of draw, with occasional peaks reaching maybe 15. AI training racks housing eight GPUs plus supporting infrastructure easily reach 30 to 50 kilowatts, and the densest configurations now exceed 100 kilowatts per rack. Average rack power density, industry analysts project, is set to climb from roughly 15 kilowatts today toward a range of 60 to 120 kilowatts for AI-specific workloads in the near future.
That jump represents a tenfold to twentyfold increase in cooling requirements compared to legacy infrastructure, and the consequences don’t stop at the rack. More power draw means more heat to remove, which means more cooling capacity, which in turn demands more electrical infrastructure to support it. Facilities planning AI deployments are finding that the GPUs themselves are only the first domino.
Why Air Hits a Physical Wall
Air’s Poor Thermal Capacity
Air carries heat far less effectively than liquid. Removing 40 to 50 kilowatts of heat from a single rack using air requires airflow volumes far beyond what traditional raised-floor systems and CRAC units were ever built to deliver. Pushing more air through the same space means higher velocities, which brings its own set of problems before it even solves the original one.
Temperature differentials add another constraint. Effective air cooling depends on supply air that’s meaningfully colder than the components being cooled, but dropping that supply temperature too far wastes energy and risks condensation forming on sensitive equipment. The workable range is narrow, and high-density AI racks move past it quickly.
Wind Tunnels and Wasted Energy
Facilities trying to force air cooling to work at AI densities often end up creating what amounts to a wind tunnel around high-density racks, running fans at maximum output just to keep pace. That approach treats a physics problem as a brute-force one, and it shows up directly in the energy bill. Air-based cooling already accounts for a large share of a typical data center’s total electricity use, and pushing it harder to compensate for undersized infrastructure only drives that number higher.
There’s also a hard ceiling involved. Beyond a certain density, no amount of additional airflow meaningfully improves heat removal, since air cannot move fast enough through a rack to carry away heat at the rate modern GPUs generate it. Hot aisle containment and other airflow management techniques help at the margins, but they don’t change air’s fundamental thermal limitations once density crosses that threshold.
Liquid Cooling Steps In
Direct-to-Chip, Rear Door, and Immersion Approaches
Liquid cooling solves the capacity problem because liquid removes heat far more efficiently than air. Several approaches have moved from experimental to mainstream in response:
- Direct-to-chip (cold plate) cooling circulates liquid through plates mounted directly onto GPUs and other hot components, carrying heat away at the source before it ever spreads into the surrounding air.
- Rear door heat exchangers attach to the back of existing racks, cooling exhaust air with liquid before it re-enters the room, making this one of the easier approaches to retrofit into existing infrastructure.
- Immersion cooling submerges entire servers in a dielectric fluid that safely contacts electronics directly, delivering the highest cooling density of the three but requiring purpose-built server hardware and enclosures.
Each method fits different situations. Facilities with moderate density increases might find rear door heat exchangers sufficient, while the densest GPU deployments increasingly call for direct-to-chip or immersion solutions to keep pace with next-generation accelerators.
Adoption Is Accelerating Industry-Wide
The shift away from air-only cooling is now widespread across the industry, with a growing share of data center operators already running liquid cooling in some form and a sizable additional portion planning to adopt it within the next couple of years. That pace reflects how quickly GPU density has outstripped what air can handle.
Major cloud and hyperscale operators have been early movers, but the shift is reaching mainstream enterprise and colocation facilities as well. What counted as an ambitious specification just a couple of years ago is quickly becoming the baseline expectation for any facility hosting serious AI workloads.
What Shifting to Liquid Requires
New Infrastructure and Expertise
Switching to liquid cooling calls for more than swapping one box for another. It requires liquid distribution infrastructure such as piping, manifolds, and pumps, along with heat rejection systems like cooling towers or dry coolers to send that absorbed heat somewhere useful. Leak detection and containment systems become essential safeguards, since introducing liquid near sensitive electronics raises legitimate operational questions that need engineered answers.
Staff also need different skills. Maintenance procedures, monitoring systems, and daily operational routines all look different once liquid flows through the rack rather than just air, and teams accustomed to air-cooled environments often need training or outside expertise to operate these systems with confidence.
Assessing Your Facility’s Readiness
Before committing to a liquid cooling overhaul, it helps to answer a few grounding questions:
- What GPU capacity does the organization expect to need over the next three to five years, and what rack densities will that require?
- Can the existing electrical distribution system, including circuits and transformers, support concentrated high-density loads?
- Is there physical space for liquid distribution units, manifolds, and heat rejection equipment?
- Do current monitoring and control systems provide the granularity needed to manage a liquid-cooled environment safely?
Answering these honestly often reveals a gap between what a facility was built for and what AI workloads now demand. That assessment should happen before GPU racks arrive, not after they’re already overheating.
Air Cooling Can’t Scale With AI
Air cooling served data centers well for decades because workloads stayed within a predictable, modest power range. AI changed that equation permanently, pushing rack densities and chip heat output past what fans and ductwork can realistically manage. Facilities that recognize this shift early, and plan their cooling infrastructure accordingly, put themselves in a far stronger position to support whatever comes in the next generation of AI hardware.
Thermal strategy deserves the same forward planning as compute strategy, since the two are now inseparable at these densities. For operators mapping out their next GPU deployment, reviewing options like next-generation AI accelerator cooling strategies with developers like Triton Thermal early in the planning process can help avoid costly retrofits down the line.
Triton Thermal
3350 Yale St.
Houston
Texas
77018
United States
