Liquid Cooling
Liquid cooling has moved from exotic to mandatory because air simply cannot carry away the heat that modern AI silicon produces. A single high-end accelerator now dissipates on the order of a kilowatt, and a full AI rack can draw a hundred kilowatts or more, packed into a volume that a few years ago held a tenth of that. Water and engineered fluids carry roughly three orders of magnitude more heat per unit volume than air, so once power density crosses a certain line, moving the heat with liquid is no longer an optimization but the only physically viable option.
Air cooling fails not because fans are weak but because air is a poor heat carrier. The heat a coolant can remove scales with its density and specific heat, and air is thin. As chips pushed past a few hundred watts and racks past twenty or thirty kilowatts, the airflow and heat-sink size needed became impractical, and the fan power itself started to dominate the energy budget. Liquid breaks this wall because a small flow of water through a cold plate removes what a hurricane of air could not.
Direct-to-chip cooling puts a cold plate right on the hot die. A metal cold plate sits on the processor package, and coolant is pumped through microchannels inside it, absorbing heat directly at the source. In single-phase operation the coolant stays liquid and simply warms up; in two-phase operation a refrigerant boils inside the plate, using the latent heat of vaporization to absorb far more energy per unit flow. The warmed coolant runs to a coolant distribution unit, which exchanges the heat into a separate facility water loop that carries it outside to be rejected.
Immersion cooling takes the idea further and submerges the whole server. Instead of plumbing each chip, entire boards are dunked in a bath of dielectric fluid that does not conduct electricity. Single-phase immersion pumps the warm fluid to a heat exchanger; two-phase immersion lets the fluid boil directly on the hot components and condense on a coil above, a completely passive heat path with no cold plates or fans at all. Immersion reaches the highest densities and removes moving parts, at the cost of fluid expense and a very different serviceability model.
The payoff is not just density but datacenter efficiency, which is why hyperscalers are converting. Because liquid can be run warm and still cool the chips, facilities can often reject heat without energy-hungry chillers, using outdoor air year-round, which sharply lowers the power spent on cooling and improves PUE. The captured heat is warm enough to potentially reuse for district heating. The trade-offs are real, including plumbing complexity, leak risk near live electronics, and new maintenance procedures, but for dense AI clusters the density and efficiency gains have made liquid the default rather than the exception.
| Approach | How heat moves | Density it enables | Complexity / trade-off |
|---|---|---|---|
| Air | Fans over heat sinks | Low (declining fast) | Simple, but hits a hard wall |
| Direct-to-chip (single-phase) | Warm water through cold plate | High | Plumbing to every socket, CDU loop |
| Direct-to-chip (two-phase) | Refrigerant boils in plate | Higher | Best per-flow, refrigerant handling |
| Immersion (single-phase) | Board submerged, pumped fluid | Very high | Fluid cost, serviceability change |
| Immersion (two-phase) | Fluid boils and condenses | Highest | Passive, but fluid + containment cost |
Read liquid cooling through a heat-carrier-capacity lens rather than a fancier-fan lens. Once you accept that air physically cannot move a kilowatt off a die, the whole design space opens in one direction: bring a dense coolant to the heat, either through a plate bolted to the chip or by drowning the whole board, and the choice between them is just how much density, efficiency, and serviceability you are willing to trade against plumbing and fluid cost.