As research institutions worldwide race to accelerate discovery in AI, climate science, health, and other fields, the infrastructure supporting these breakthroughs is under greater strain than ever before. Performance demands are rising sharply, while sustainability, energy efficiency, and facility constraints have become critical design considerations.

In this context, the University of Waterloo’s Nibi supercomputer stands out as a defining milestone for the future of HPC in Canada, and globally.

A national research platform built through collaboration

Nibi is operated by the University of Waterloo in partnership with SHARCNET, a research computing consortium that provides high-performance computing services and expertise to academic researchers across Canada.

Hypertec worked closely with both organizations to design and deploy a next-generation HPC system that delivers high-performance computing capabilities while meeting ambitious sustainability goals. Today, the Nibi system supports a research community of more than 4,000 users nationwide.

This collaborative effort brought together academic leadership, research computing specialists, and infrastructure engineers to rethink how large-scale HPC systems can be deployed responsibly at production scale.

The first major immersion-cooled Intel Xeon 6 HPC & AI cluster

Nibi is Canada’s first large-scale immersion-cooled supercomputer and the first major HPC cluster based on Intel Xeon 6 (Granite Rapids AP) processors to be deployed using immersion cooling. The system comprises:

• More than 700 compute nodes
• 134,400 CPU cores powered by Intel Xeon 6 processors
• 288 Nvidia H100 GPUs (HGX SXM5 with NVLink and NVSwitch)
• Over 25 petabytes of all-flash storage
• Nine single-phase immersion cooling tanks, complemented by direct-to-chip liquid cooling for GPU subsystems

The architecture enables mixed workloads, including simulation, data analytics, and large scale AI training, while maintaining stable operating temperatures under continuous load.

From legacy constraints to production at scale

The project emerged as the University of Waterloo’s previous flagship system, Graham, approached end of life. The demand for advanced research computing continued to grow, but existing facilities faced limitations around power density, cooling capacity, and carbon impact.

Rather than expanding physical footprint or relying on increasingly constrained air-cooling methods, the Nibi system was deployed within the University’s mathematics and computer building, demonstrating how modern HPC can scale efficiently within existing infrastructure.

Today, Nibi is fully operational and ranks among the TOP500 research computing systems worldwide, making it one of the most sophisticated digital research platforms in Canada.

Sustainability as a design requirement

Traditional air-cooled data centers struggle to keep pace with the heat generated by modern HPC and AI systems. Nibi was designed differently from the outset.

Using immersion-born server technology developed by Hypertec, all major system components are submerged in a dielectric fluid that enables uniform heat removal, eliminates hot spots, and reduces reliance on energy-intensive cooling infrastructure. The result is:

• Up to 50 percent reduction in cooling energy compared to air cooling
• Approximately 40 percent smaller physical footprint for equivalent compute capacity
• 100 percent heat reuse, with waste heat captured and repurposed to warm the Mike and Ophelia Lazaridis Quantum-Nano Centre
• More than $150,000 in annual energy savings

To reflect this water-centric and sustainable approach, the system was named Nibi, meaning water in Anishinaabemowin (Ojibwe).

A model for next-generation research infrastructure

Nibi demonstrates that immersion cooling is a production-ready, scalable solution for modern HPC environments. By maintaining stable operating temperatures, reducing thermal throttling, and enabling higher compute density, immersion cooling helps institutions meet performance goals without compromising sustainability.

As Dr. Charmaine Dean, vice president, research and international at the University of Waterloo, noted: “We have the opportunity in Canada and globally to champion secure and effective computing environments, such as those produced in Canada by Hypertec.”

John Morton, director of technology at SHARCNET, added: “Sites doing large deployments should seriously consider immersion cooling. It provides very consistent cooling and a very sustainable solution, and Hypertec has been a great partner in this project.”

Nibi now serves as a national platform for advanced research, enabling breakthroughs across disciplines while providing a practical reference for sustainable HPC design at scale.

Learn more about Hypertec’s immersion cooling solutions and watch Nibi in production