Today, when ChatGPT writes a 100-word email, it swallows more than the equivalent of a 16-ounce bottle of water. Separately, Lawrence Berkeley National Laboratory published a study late last year warning that by 2028, data centers in the US could consume an estimated 74 billion gallons of water annually.

Given the explosive buildout of hyperscale data centers, AI's physical demands are anything but abstract. As AI accelerates, every prompt and inference creates heat that needs to be cooled. Where will all that cooling water come from?

The growing strain

High-performance GPUs have transformed data center design. Facilities are far denser, not just in computing but also in thermal output. Engineers routinely see three to five times more heat rejection per square foot compared to legacy systems. Traditionally, racks were in the 5kw to 10kw range, but with AI development, we are seeing rack densities increase to an 80-100kw range or higher.

The higher the chip performs, the faster you can train the algorithm, but you still must get that heat out. Many operators are returning to direct-to-chip liquid cooling at the server rack level. What was once a fringe solution is now standard. Simply put, you can't air-cool a 100-kilowatt rack. And these systems require water — and potentially lots of it. When designing these systems, a choice must be made between evaporative or air-cooled heat rejection to the atmosphere. That means a tradeoff: more water and less energy, or more energy and less water.

Meanwhile, the tech industry is moving in short, 18-month cycles, far outpacing the timelines of most public water utilities whose planning routines remain relatively longer and more rigid. That mismatch creates risk for the operators and the communities being asked to shoulder new loads without clear answers about resilience, affordability, or return.

A quiet mismatch

Black & Veatch's 2025 Water Report reveals the input from 680 US water industry stakeholders, and more than half of the respondents said data centers aren't part of their near-or long-term planning, while a third weren't sure if the issue was even relevant.

As more data centers move into secondary and rural markets — drawn by the promise of lower land costs and abundant power, water and wastewater utilities have an opportunity to get ahead of demand by planning infrastructure upgrades that make these regions more attractive to developers. For communities that choose to welcome data centers as long-term neighbors in hopes of expanding their tax base, those infrastructure decisions are critical to future growth.

But while the shift toward these less urban locations opens new economic possibilities, it also exposes gaps. Utilities in these regions may lack the capacity to meet the high water demands of modern data centers. That’s where water industry stakeholders can step in to collaborate on planning, investment, and incentives that enable sustainable development at scale.

The new cooling arms race

The industry is now in a full-on thermal redesign, with air cooling reaching its limits. Because more data computation now happens within a single chip or server, we have much more heat to reject per square foot.

The answer — for now — is water. Moving from air to water cooling typically means five times the heat rejection per square foot. But that raises infrastructure implications, especially in areas with water constraints.

Water challenges are not new. For decades, power plants have utilized closed-cycle cooling water loops with refrigerants or a glycol mixture to cool systems. These systems can be further utilized for heat recovery in cogeneration facilities.

Hyperscalers are also experimenting with technology solutions. One example: Microsoft recently came up with a zero-discharge, air-cooled, water, or heat rejection system. Its design leverages higher return fluid temperatures from chips, allowing more heat to be rejected via air. It’s important to note, however, that while the cooling of the servers themselves can use zero water, the data center structure itself will still need to be cooled, which could potentially require the use of water.

In hardware, the next wave of tech evolution could go even further, incorporating a refrigerant across the chips and then recondensing the vapor back into a fluid.

But all these approaches share a caveat: they'll only work if the site is planned accordingly. Retrofitting at scale is still hard, expensive, and inefficient… if a site isn't planned around water reuse or non-potable sources from the start, that already limits its options.

Better alignment, bigger opportunity

The report further notes that 40 percent of utilities are considering public-private partnerships to enhance wastewater systems. However, many operators remain unsure, especially those without experience supporting tech infrastructure.

Better coordination between the sides can help accelerate the process — especially if the tech client can help fund the upgrades to avoid impacting ratepayers. More partnerships like that would be beneficial since most utilities typically don't have huge unallocated budgets to invest in new technologies.

Beyond dollars and planning, it's also a question of public trust, where the benefits of transparency and working closely with the community can resolve any logjams and pave the way for upgraded infrastructure, better water quality, and long-term community investment.

We’re in an era where water strategy, community alignment, and resilience planning are fast converging.

For communities that want to attract AI infrastructure, the question isn’t only “Do we have enough water?” It’s: “Can we support the type of cooling required by this kind of workload?” The amount of heat rejection now is going up substantially per square foot of data center just because there's so much more data computation happening within a single chip.

For data centers, success won’t hinge on GPU counts or teraflops alone; if AI is going to be the growth engine it promises to be, holistic water planning must be part of their equation.

As data centers expand into new markets, more utilities may find themselves engaged in conversations that weren’t on their radar even a few years ago. While some may still be evaluating how this trend fits into their long-term planning, now is the moment to begin that dialogue. When utilities, municipalities, and data center developers work together early, they can co-create infrastructure solutions that serve both innovation and the long-term needs of their region.