In this DCD>Talk, Nathan Mallamace, strategic business development manager in the rack department at Supermicro, highlights the company’s transformative role in the AI space, emphasizing the critical importance of partnerships across the entire ecosystem – from the chip level to data centers. "We wouldn’t be where we are without our partners," he notes, reflecting Supermicro’s collaborative approach to addressing emerging industry challenges.
The biggest challenges in AI-driven data centers
Mallamace identifies two pressing issues in AI-driven data centers: power management and liquid cooling. The increasing thermal design power (TDP) of chips leads to much higher power requirements for racks, from 10-15 kilowatts to as high as 120 kilowatts per rack. This surge drives the need for more efficient liquid cooling solutions, especially given the increasing regulatory requirements and power limitations that data centers must navigate.
Liquid cooling, particularly systems like coolant distribution units (CDUs) and coolant distribution manifolds (CDMs), has become essential in managing these demands. These components function as the “arteries” of a rack, circulating coolant in a closed loop to maintain optimal temperatures.
According to Mallamace, the need for liquid cooling has intensified as chip TDPs reach unprecedented levels, necessitating sophisticated cooling mechanisms:
“Supermicro has excelled in fine-tuning its understanding of this ecosystem, making systems like direct-to-chip liquid cooling more efficient than ever.”
Direct-to-chip cooling stands out for its cost-effectiveness, delivering substantial savings in capital and operational expenses: “The cost savings can be in the multi-millions of dollars over the first five years of a data center’s operation. The efficiency of water towers and evaporative cooling, key components of this system, provide significant cost benefits over alternatives like liquid-to-air or chiller-based cooling,” Mallamace adds.
How the environment impacts cooling technology
During the Talk, Mallamace highlights that the choice of cooling technology is heavily influenced by the local climate and environment: “Evaporative cooling, for example, thrives in dry, arid climates, but its efficiency declines in hot and humid regions.”
“In contrast, water-based cooling systems may face challenges in drier areas, but the technology still offers advantages in water and power management. Therefore, the selection of cooling technology should be tailored to the local environment to optimize efficiency.”
How the evolution of AI will shape data centers
Looking to the future, Mallamace predicts that AI’s rapid evolution will drive significant changes in data center infrastructure over the next two to five years. Current facilities, which support around 10-15 kilowatts per rack, will need to accommodate much higher power densities and implement advanced liquid cooling technologies to meet these demands. He foresees large-scale investments in data center infrastructure, with new facilities being designed around power and liquid cooling as integrated solutions.
In contrast, smaller data centers may face challenges in scaling up, but innovations in efficiency could bridge that gap.
In the long term, Mallamace believes the focus will be on improving CDU and pumping technologies, which are crucial for efficiently cooling high-density racks. He envisions a paradigm shift in data center design: “The rack is becoming the new ‘server,’ while the data center itself is increasingly viewed as the new ‘rack,’ scaling up exponentially to meet the computational demands of AI.”
In closing, Mallamace underscores the need for continuous innovation and collaboration with partners to develop the next generation of liquid cooling solutions, which will be vital as AI continues to reshape the future of data centers.
If you’d like to watch the full DCD>Talk with Nathan Mallamace of Supermicro, click here.