Artificial Intelligence (AI) has the potential to increase efficiency and innovation across the spectrum, as well as bring economic growth. According to McKinsey & Company, generative AI alone could add the equivalent of up to $4.4 trillion annually to the global economy, including up to $100 billion in the telecommunications sector, $130bn in media, and $460bn in tech.

And it’s not just about money. AI is helping to achieve major breakthroughs, positively impacting areas like healthcare and climate change, and developing innovative solutions to combat global climate change.

However, unlocking the promise of AI will take an entire ecosystem to support the technology. The data center architecture and the critical digital infrastructure supporting it will have to undergo a major transformation to support the demands of AI workloads.

The critical digital infrastructure that enables AI

Current trends suggest that 19 percent of power usage in data centers will be associated with AI by 2028. So, what does this mean for data centers, especially when it comes to architecture, energy usage, and cooling?

The meteoric rise of AI is already disrupting IT architecture and data center critical digital infrastructure. But what is being experienced now is only the tip of the iceberg; rack weight, data center power, and cooling are all evolving.

AI can mine data and unleash its hidden value. But this also means businesses have to be prepared for the profound changes in infrastructure necessary to support AI. The revolution in AI will lead to a revolution in how data centers operate. So, while most people are focused on the economic benefits of AI, obtaining these benefits will require predicting, understanding, and solving the emerging infrastructure challenges.

Energy Efficiency is crucial. But it won’t be a silver bullet

One of the key elements of enabling AI is microprocessors.

The chips used to train AI models, i.e., those used to teach an AI model by exposing it to data so it can make accurate inferences, require a significant amount of power and generate a corresponding amount of heat.

Graphics Processing Units (GPUs) are the current chip of choice for running AI workloads, and using GPUs in place of Central Processing Units (CPUs) to run parallel compute workloads is 100x more efficient in terms of power consumption per unit of compute.

However, compute requirements will keep increasing exponentially, far outpacing any increase in chip efficiency. So, while GPUs will produce much more compute for the same amount of power used, the demands of AI mean that the amount of total power used will still increase.

Making sense of the future

For many applications, the promise of AI justifies the increasing power inputs. New functionality will likely demand even bigger AI models to accomplish tasks like performing multi-step work rather than just responding to prompts or determining which algorithmically available answer best meets the user’s needs.

Given the many factors that seem to be driving both demand for AI and the use of larger models, it is difficult to see how chip or model operational efficiency gains will make up for the many compute-intensive developments in AI. The future may be more efficient, but IT load capacity is expected to increase despite gains in efficiency.

Trends to be aware of

Innovative power and cooling technologies: Data center operators can extend the life of existing infrastructure by implementing high-efficiency power and cooling solutions while planning for future AI-driven expansions. This means investing in scalable, modular systems that can adapt as AI workloads evolve.

Changes in output capacity: prepare for a world where output capacity grows. Rack loads, power distribution units (PDUs), uninterruptible power supplies (UPSs), switchgear, and coolant distribution units (CDUs) will all increase in capacity and have a corresponding knock-on effect throughout the overall infrastructure.

Changes in overall scale: facilities will get bigger. Today’s 3MW blocks will be tomorrow’s 20MW blocks because traditional server racks operating at 10-15kW are being replaced with high-density racks commonly reaching over 50kW - and in some cases, even 300kW.

The quest for always-on power: bigger also means more power and heat. More heat means hybrid and liquid cooling are the future. Since the GPUs of the future must be continuously cooled, always-on power is an imperative to provide power backup for the critical cooling systems supporting the GPUs.

AI means unconventional load profiles: AI is associated with power loads that can pulse from a 10 percent idle to a 15 percent overload in a flash.

Infrastructure refresh and upgrades: If a new build is not an option, data center operators will need to determine the best way to retrofit their existing operations to quickly and economically meet the demands listed above.

Major changes are coming in architecture, power, and cooling

While no one can be truly certain about the exact future of IT architecture, looking at the basic drivers of change is the best way to determine what that future will look like.

The upward trajectory in rack density triggered by the current wave of AI will not be a historical blip. To meet the growing demand for AI, rack architecture is forecasted to increase from the upper 30kW to 300-600kW densities in the near-term and possibly 1MW and above by 2030.

Things are changing faster now than at any time in the last 30 years. It’s key to be prepared for the changes that are already happening and to work with industry leaders to develop a strategy for future deployments. While there may be uncertainty around how quickly rack architecture will evolve, especially in the longer term, there is little doubt that changes in architecture will require new approaches to cooling.

However, the tremendous growth in power usage will mean that various forms of hybrid and liquid cooling are the future. Liquid cooling is significantly more efficient than air cooling, and that efficiency is extremely important to developing the infrastructure necessary for AI. By the end of the decade, data centers will likely primarily rely on liquid-cooling to the chip, self-contained immersion, and air-cooling for residual heat loads.

Cooling is not the only concern. The data centers of the future will also have to be flexible enough to deal with “pulse loads,” or periodic loads with high power inrush in a very short time. AI training clusters are known to exhibit short-duration spikes in electrical current during certain compute cycles, so overload provisions will become de rigueur.

Although it is difficult to pinpoint all the exact technological milestones that will occur this decade and beyond, the broad brushstrokes of the future are becoming clearer.

Rack densities will increase, cooling methods and management need to evolve, and power and load management are quickly becoming major challenges.

Now we need to apply what we know to inform the development of products and solutions that will resolve the needs of data centers both today and tomorrow.

Investing in scalable, modular systems that can adapt as AI workloads evolve is the sensible option. Modularity allows for a more agile response to changing needs. Instead of planning for unforeseen growth, organizations can build capacity to match requirements.