The University of Durham will use AMD processors in its COSMA 8 supercomputer, part of DiRAC.
Distributed Research utilizing Advanced Computing (DiRAC) provides distributed high-performance computing services to those studying in fields such as astrophysics, cosmology, solar system science and nuclear physics.
DiRAC's computing resources are distributed over four University sites (Cambridge, Leicester, Durham & Edinburgh), each tailored to specific use cases. The DiRAC Memory Intensive (DiRAC MI) system at Durham University focuses on cosmological simulations, and so is known as COSMA, short for Cosmology Machine.
The first generation of COSMA came online in 2001, prior to the establishment of DiRAC, with the latest iteration is COSMA 8. Successive upgrades tend to overlap, so the current DiRAC service is actually three clusters (COSMA 6, 7, and 8), while local Durham users continue to use COSMA 5.
OPINION: How to provide a cloud-native experience for HPC applications
“If you're doing large-scale cosmological simulations of the universe, you need a lot of RAM,” said Alastair Basden, Technical Manager for the DiRAC MI service at Durham University. “They can have run times of months, and then, after they've produced their data, which will be snapshots of the universe at lots of different time steps and different red shifts, then years are spent in processing and analysis.”
COSMA simulates space with AMD
For the 8th generation of COSMA, Durham Uni opted to use AMD’s Epyc processors. The system utilizes dual 280-watt AMD Epyc 7H12 processors per node with a 2.6GHz base clock frequency and 64 cores, installed in Dell Cloud Service C-series chassis with a 2U form factor and custom CoolIT water cooling. COSMA 8’s initial installation comprises 32 nodes, each with 1TB of RAM, but will eventually have 360 nodes. The previous system used 5120 Intel Xeon Skylake processors.
“For our current system (COSMA 7), we have about 18GB of RAM per core,” said Alastair Basden, Technical Manager for the DiRAC MI service at Durham University. “When you compare that with typical systems of two to four gigabytes, that's a significant uplift. We do large-scale cosmological simulations of the universe, right from the start of the Big Bang up until the present day.”
Basden said Durham opted for AMD to enable much larger simulation data sets with faster execution to speed up the discovery process during cosmological investigations. DiRAC@Durham is also exploring GPU acceleration with AMD Radeon Instinct graphics cards and plans to use for them for running AI workloads like TensorFlow.
For more breaking data center news, features, and opinions, be sure to subscribe to DCD's newsletter
COSMA 8 entered service in a prototype stage last October. The rest of the system is undergoing installation and is expected to be ready by October 2021.
“This will help us to understand the nature of the universe, dark matter, dark energy and how the universe was formed,” said Basden. “It's really going to help us drill down to a fundamental understanding of the world that we live in.”
