Cooling takes up a significant amount of a data center's power, with around 40 percent of a facility’s electricity bill going towards beating the heat.
Beyond cost, cooling can also be bad for the environment - in a period where grids are struggling and humanity’s emissions need to fall drastically, every watt saved helps.
Free cooling has long been pitched as a way to take advantage of already available cold sources to chill a data center, rather than chillers. For low-density racks, and facilities in colder climates, outside temperatures can be enough - especially in winter months.
But as densities increase, and heatwaves get longer, researchers and operators have increasingly looked to natural bodies of water for cooling.
Data center giant Digital Realty, for example, uses river cooling at its Marseille data centers.
The cooling solution took two and a half years to deploy - after all, it uses more than three kilometers of buried pipes and 27 heat exchanges. The system diverts water from an underground channel, La Galerie de la Mer, built in the 19th century to collect and channel rainwater from old mines.
The water itself stays at a natural temperature of 15°C (59°F) all year round and is pumped to the data centers to cool them via thermal exchange.
This cooling solution is used by Digital’s MRS2 and MRS3 facilities, and will also be used, by MRS4 and MRS5, which are under construction. It cost Digital Realty a total of €15 million ($16m).
When asked about the value of the river cooling system, Digital Realty’s chief data center technology and engineering officer Lex Coors told DCD that the answer was two-fold: financial and sustainability.
“First of all, you look at the current energy pricing, and the trends and projects for what will happen with that. At the time, energy was actually a low-cost item, but we were seeing some increase though it was low, and some instability in the pricing as well,” Coors explained.
“We saw the availability of energy going up and down, particularly for sustainable energy from either solar or wind. We also have our corporate social responsibility program, but we do not invest in things that make no sense - it would be bad for us, and for the environment.”
In addition to the energy costs, Digital Realty saw trends across Europe to demand data centers achieve a PUE (power usage effectiveness) of 1.2 or better.
“In Amsterdam, we were forced many years ago to design all of our data centers against a PUE of 1.2. Now, we are seeing the German law also coming into effect, pushing for 1.2 or better, and this actually helps us,” said Coors.
“I won’t say it will be below 1.2 in Marseille, because it's pretty warm there, but we are definitely seeing more of a push down on PUE, and now there is also the Energy Efficiency Directive for the European Commission coming in place.”
The Energy Efficiency Directive was first adopted in 2012 but has since been updated both in 2018 and 2023. The latest update significantly raises the EU’s ambitions for energy efficiency.
The change established a new “energy efficiency first” principle, meaning that efficiency must be the priority in all policy and investment decisions. “The hardcore financial people would say, why would you invest €15m in a project like this? But at that time we had people in the company who said: ‘We also need to do the right thing. It will pay back.’ And yes, it's paying back on multiple angles now.”
By using what is already accessible - natural water supplies, and the unused infrastructure of history - Digital Realty reckons that its data centers in Marseille run more efficiently, and save around 18,400 MWh annually while reducing CO2 emissions by 795 tons - which Lex Coors confirmed the company monitors carefully every month.
According to Digital Realty, the solution will eventually be part of a loop. Once it has cooled Digital’s servers, the water will be hotter, and one day will be provided to the Euromediterranée urban heating network where it can help provide warmth for 500,000 sqm of offices and homes.
This is something that is still currently in “discussion,” but not yet a practical reality, says Coors.
“These programs take a long time. First, it starts with an intention, and there is an intention between both Digital Realty and the local authorities to see what can we do with this warm water. For us, it's important to get the warm water as low [in temperature] as possible, because the lower it is when the water goes back to the sea, the longer we can use the water from the river without raising alarm bells from the center,” explained Coors.
“It's important for us, and also for the city, to say that even though there's not much of a winter in Marseille, there is always a demand for heated-up water. Sometimes it's for tap water, sometimes it's for another resource, sometimes for the heating system. The intentions are there, the conversations are there, but you can imagine that connecting to such a massive system takes time.”
Digital Realty is not alone in its pursuit of using local water supplies while not impacting the local ecosystem.
Nautilus, a company perhaps best known for its “floating data center” in Stockton, California, designs water-cooled systems for data centers located up to 1km from a free-flowing water supply.
That water supply can be anything - from the ocean to a lake, a river, or even a wastewater treatment plant. But, of course, it has to be customized according to that water supply.
“We’re not just about energy efficiency,” Rob Pfleging, Nautilus’s CEO and president, told DCD. “We’re also about aquatic wildlife preservation.”
In the case of the Stockton data center, located in the San Joaquin River, Nautilus teamed up with Intake Screens Inc (ISI) for the intake drum. ISI found that there were two endangered species in the river, and so designed the intake drum with velocities low enough that a fish can swim alongside it without getting sucked into the system.
The drum itself is also designed with 100 percent inert materials. Materials like copper or alloys might leach into the water stream, so ISI chose stainless steel.
Similarly, the pipes are either stainless steel or high-density polyethylene with at least 3mm wall thickness. Another reason those materials are used, is that they are innately “slippery,” making it hard for things to get attached to them and clog the system.
Once the water comes in through the drum, it is ionized with a negative charge, meaning that the microbiota in the water can’t attract and stick to each other and the system. The system is also attached to ultrasonic transducers which slightly vibrate the pipes enough to discourage microbiota from attaching, all of which means no chemicals need to be used, and protects the aquatic environment surrounding the data center.
In total, the barge only holds onto the water for 60 seconds and it leaves the data center less than four degrees Fahrenheit warmer than it arrived. None of the water is evaporated or consumed.
Nautilus’s solution is both open and closed-loop. The loop circulating river water is open to the river, while a second closed loop takes heat from the servers and passes it to the loop of river water.
According to Pfleging, those loops will never touch: “Where we're unique is because we are doing both open loop cooling and closed loop. There’s a plate frame heat exchanger. The two liquid streams never touch each other - they're just transferring energy through some metal plates,” said Pfleging.
“On that closed loop side, we move the cold water around the data hall, not with pumps under positive pressure, like 50 or 60 per square inch positive pressure, we actually move it around the data hall with a light vacuum. We pull it around the data hall instead of pushing it.”
According to Pfleging, that lower pressure means that the system does not leak. Even if the pipes were breached, instead of leaking water, they would suck in air. It's less taxing on the system itself, as continual pumping can cause valve packing, and it also prevents the growth of microbes in the system - many of which need oxygen to develop.
According to Pfleging, Nautilus’s free cooling can be used with any other cooling method in the data center. “We don't care what your particular method is. We're just about being the lowest cost most energy-efficient final mile heat rejection. So we support all of those [in-row cooling, rear door heat exchange, immersion cooling] but more importantly, we make them all better,” explained Pfleging.
He also added that one of the benefits of their system is the lack of need for propylene glycol or anti-freeze in the system to prevent microbe growth.
“Because we don't need to put that kind of stuff in our water it adds to the efficiency. When you add, say, 20 percent propylene glycol to the water it knocks its cooling efficiency down by five or six percent. So we're even more efficient than the standard direct-to-chip.”
The free-water cooling solution would not work universally, however. The water itself needs to be moving, and it needs to be cool enough to actually, well, cool. “If you wanted to put a data center next to the Nile, we aren’t going there, because that water is almost a hot tub,” agreed Pfleging.
That being said, the heat in a given location is not always the end of the road for deploying these types of solutions. As Pfleging explained, during a heat wave, water temperatures rise far slower than air temperatures. During the California heatwave, the Nautilus data center was still able to run at a PUE of 1.15.
“We're logical. We're not the answer 100 percent of the time, everywhere on the planet - I'm not going to put this in the middle of the desert, I'm going to put this reasonably close to a large body of water that meets our temperature profile.”
Nautilus’s data centers can range in size, from their Stockton facility to another project they are working on in Maine, or the Start Campus in Portugal expected to have up to 120MW which signed up Nautilus for cooling in early September 2023. All of which begs the question: Is nature cool enough?
Data centers, in general, are going to be located near enough high-population areas to serve that city or region. Similarly, as settlements have developed over the Centuries, the one thing those settlements have in common is the need for access to water.
Pfleging argues that around 80 percent of the world’s population lives within a kilometer of water (the longest distance the company is prepared to transport water to a water-cooled data center).
“There are very few times when someone comes to us and says; ‘I’d like to be here’, and we say no, we just can’t do that.”
For this reason, it seems that free cooling may not be a total and complete solution to the cooling issue, but it certainly has a wildly relevant part to play, and one which is at our fingertips - just outdoors.
