The power of AI has skyrocketed since its first use several decades ago. For better or for worse, industries are finding ways to embrace AI but must combat its associated issues. “How we, as people, interact with data is a shift in humanity; whether we're consciously using generative AI or not, we've been conditioned to interact with data so well over the past 20 years,” says Bill Kleyman, CEO at Apolo.

A recent Power Hour talk with DCD’s Alex Dickins, featuring speakers from Apolo and Legrand, highlights the surge of power consumption by AI models, infrastructure challenges, increasing rack density, and the crucial role of partnerships in addressing industry-wide issues. Notably, how the regulatory impacts and predictions on AI's future power are driving the need for innovative approaches to power management and collaboration within the sector.

Generative AI’s role in data centers

For the data center industry, generative AI provides applications for increasing density and improving performance by identifying areas of inefficiency with solutions like strategic air containment.

The growing need for data centers has brought significant infrastructure challenges, especially as increasing rack density requires more power. John Consoli, VP of sales, cabinets and containment at Legrand, emphasizes: “We have seen industry averages skyrocket from six to eight kW per rack, to 60 to 80 kW, and even up to 100 kW per rack in some cases.”

This dramatic increase in rack power density reflects the enhanced way we interact with data and the higher intensity of resource utilization driven by generative AI applications. This shift could lead to data centers consuming a much larger share of available power, potentially making them responsible for 20 to 25 percent of global power usage.

This projection underscores the critical need for addressing these infrastructure challenges – particularly in terms of power supply capacity, efficient cooling solutions like rear door heat exchangers, and overall energy efficiency – to sustainably manage the growing energy demands of data centers supporting AI workloads.

Improving power efficiency with monitoring technologies

Besides needing more IT equipment, larger and heavier rack loads demand better structural integrity to handle more fibers and cables, facilitate in-row cooling, and ensure safe transportation. Consoli highlights:

“We’re at a point where the cabinet can no longer be considered an afterthought or a commodity. To support AI, the cabinet must be thoughtfully designed and engineered as a fit-for-purpose component of the entire system solution."

Kleyman adds: “Average power rack density of the industry has more than doubled from about 6 to 12 kW per rack in one year. According to AFCOM’s latest State of the Data Center report, 60 percent are actively looking to increase their rack density; 58 percent are doing it by improving airflow; 42 percent are doing so with containment; and another 40 percent are using liquid cooling like rear door heat exchangers.”

To accommodate this challenge, Kleyman strongly advocates for contemporary hybrid architecture in data centers, utilizing vertical space for higher power density in DGX units. The reasoning being that investment in taller units like rear door heat exchangers can replace traditional architecture, like raised floors, which could give way to leaning and hazards.

In the broader data center ecosystem, both the quality and supply of power, including efficiency and noise issues, are crucial factors. Rather than relying on manpower and a seemingly endless sequence of manual protocols to identify and reset the power issue, modern AI tools can automate this process and avoid critical outages. Such tools can also conduct and store waveform capture records to help pinpoint the source of deviation.

The importance and impact of monitoring technologies, especially waveform technologies, lies in managing and improving power usage and efficiency in data centers. This applies to various parameters, such as power usage, temperature, humidity, and power quality.

Calvin Nicholson, senior director of product management at Legrand, proclaims: “You can't improve what you're not monitoring. Whether it’s understanding power usage, managing stranded capacity, monitoring temperature and humidity, or diagnosing power quality issues and server failures, it all comes back to thorough monitoring.

Measuring advanced power quality metrics like crest factor, total harmonic distortion, and voltage dips and swells at the intelligent rack power distribution unit (PDU) level, along with features like circuit breaker trip forensics with waveform capture, can make operations more efficient and help identify and solve problems at the PDU outlet or device level.”

It seems effective use of monitoring tools will help manage higher densities and AI applications, making operations smoother and reducing the complexity of managing data centers.

Shifting trends in workloads and infrastructure

Comparing the current state of generative AI and high-performance computing (HPC) with their state a year ago, customers want control over AI workloads, away from big cloud providers. Kleyman states: “Apolo is driven by its customers who say that sometimes they don't want to be on one of the big cloud providers – they have all of their critical infrastructure with us already.”

“The use cases that are driving enterprises today are going to be the reason that we're going to see proliferation, for new power designs and infrastructure designs in our ecosystems,” Kleyman continues.

Consoli fondly remembers the cultural shift away from CD Walkmans towards Internet music downloads, applications which would drive cabinet densities of four to six kW per rack, which was unprecedented at the time.

“We're going to see gigawatt data centers sparking the whole paradigm to change. We'll move on from what we're doing now to something different. And I believe it's going to happen very, very quickly,” Consoli exclaims.

Regulatory trends and the role of partnerships

Other aspects of the story of AI and power include developing regulations in Europe and the US. With Europe passing the world's first act to regulate AI, the importance of regulating and managing how we work with AI systems is becoming increasingly apparent. Experts predict that in the US and globally, we will see secondary and tertiary markets emerging to support AI systems more than ever before.

“We’ll see greater impact from local communities, and we have to make sure we nurture good relationships and conversations to ensure our developments serve them well,” Kleyman adds. In the US, laws and regulations at state and federal levels are evolving to support this market.

We will eventually see the ripple effects of increased data center power usage across different industries and regions. The increasing power demands are causing fundamental shifts in various industries, particularly those not traditionally associated with heavy power usage, such as tech companies that now also see themselves as AI or data companies.

In terms of partnerships within the data center industry, corporations are moving away from working in isolation and are actively seeking partnerships with vendors to address the evolving landscape of technology and regulation. This collaborative approach is driven by the evolving needs of the customers and the competitive marketplace. Consoli summarizes:

“Where we spend the lion's share of our time is working with a customer to engineer rack solutions that will accommodate these challenges. It's no longer us as the vendor across the table, negotiating with the customer; we've come around to the same side of the table, to collaborate with the customer, and focus on the problem.”

The power of AI continues to transform industries, posing both opportunities and challenges, particularly in power consumption and infrastructure. As our interaction with data evolves, collaborative efforts in the data center industry are crucial.

Tune into the full Power Hour here for a deeper exploration of:

• Common power and cooling challenges in the AI era and strategies to effectively address them
• The role of real-time insights into power quality and usage at the device and cabinet levels in boosting efficiency and uptime
• Cooling and cabinet solutions specifically designed to manage increased power densities