Liquid cooling has been bubbling under for years, but it hasn’t changed the world yet. An announcement at last week’s DCD Internet show in San Francisco suggests that the simple approach might be best.
A new supercomputer is being built in Albuquerque, in a data center owhed by the Navajo Nation, which uses a liquid cooling system provided by an outfit called Clustered Systems. Aquila, the group bulding the so-called Aquarius system, promises a PUE of 1.05 and a density of 216 CPUs per rack.
Flag of Navajo Nation
Liquid cooling is still a niche
Is this a breakthrough? After several years watching the field, I’d have to say it’s far more likely to be an incremental step, because that’s the way liquid cooling goes.
Liquid is vastly better at cooling than air, but is much harder to manage, because it has to be plumbed round the electronics. Every so often I hear that all servers will ultimately be cooled by liquid - in 2009, IBM told me it would happen by 2019. But despite this, it’s still rare to see it happening in the real world.
Liquid cooling allows higher density in racks, and it also allows CPUs to be overclocked without overheating. It has therefore appeared in areas where performance is important - high-performance computing (HPC or supercomputing) and in high-end gaming machines.
Those are niche markets compared with general servers in data centers. The main server rooms remain stubbornly air-cooled, because rack densities aren’t increasing fast enough to need liquid cooling, and low-power servers are off-setting possible increases in power and heat.
There aren’t that many supercomputers (or liquid cooled gaming systems) in the world, so liquid cooling firms are growing slowly. A 451 Research report from 2014 lists a handful of firms doing “immersion” cooling, and a dozen firms doing “direct liquid cooling” (DLC), a broader definition which includes the use of an intermediate such as a cold-plate, transferring heat from the electronics to the liquid.
This approach has some big-name support from HP, whose Apollo system uses “heat pipes” to conduct heat from servers to the circulating coolant.
Source: Clustered Systems
Using a cold plate
Among the more established specialists is Clustered Systems, whose Exablade product takes conventional computing blades, and fits each powered component with a thermal riser (like a heat sink) which makes contact with a cold plate that is cooled by liquid, either a two-phase refrigerant or water.
Clustered has been quietly in operation since 2008, and the system it now calls TouchIT made a splash in 2010-11 by winning a cooling competition and promising 100kW per rack.
Innava is a subsidiary of Nova Corporation, a high-tech firm owned by the Navajo Nation - but operated outside its borders so US firms need not fair any tax or litigation troubles.
It sounds a great initiative. Innava is a mixed use data center, and is intended to boost the prospects of the Navajo and build technology skills in the tribe. But Aquarius is still very much in the HPC niche, and has no definite delivery date.
However, it does suggest that simpler liquid cooling systems, which use an intermediate like Clustered’s cold-plates, have a lot of potential.
A version of this article appeared on Green Data Center News