It might be cold and inhospitable, but Antarctica and the South Pole still house a surprising amount of data centers and IT infrastructure.
Despite generating terabytes of data a day, limited satellite coverage means huge amount of information is sent north via hard drives flown out on planes. And though a proposed subsea cable could be transformational for researchers and base staff on the fringes of the Antarctic, will it be able to help those working at the South Pole?
Data centers in the Antarctic Circle
Located on Ross Island, the McMurdo Station research and logistics base is around 3,860 km (2,415 miles) south of Christchurch, New Zealand, and 1,360 km (850 miles) north of the South Pole. The base’s existing data center is undergoing a refresh that has been delayed by Covid-19.
First announced in 2018, the new Information Technology and Communications (IT&C) facility is a two-story, 11,000 square foot building set to house data center, communications, and ground station infrastructure.
Work on the building, referred to as ITC-Prime on the NSF site, began around February of 2019 and was due to span two Southern Hemisphere or “austral” summers and was originally anticipated to be completed in late 2020. However, due to Covid-19, the completion date has been pushed back to 2023.
“The completion of the new McMurdo Data Center facility was impacted by the Covid-19 worldwide pandemic,” Mike England, National Science Foundation media officer, told DCD last year. “Assuming continued Covid decline, NSF anticipates completing the facility by end of 2023 and data center functions would be moved soon after.”
The new facility is being developed as an addition to the existing Science Support Center facility. The new addition consists of two floors, with the lower floor housing electrical power equipment, backup UPS systems, and the telecommunications entrance and termination to inside cable distribution.
The second floor will be the data center floor, consisting of a hot aisle-cold aisle configuration for three different caged spaces with open airflow between cages: one area for the NSF McMurdo Station operations data center, one area for the combined NASA Near Earth Networks and NOAA Joint Polar Satellite System ground station equipment, and a more general-purpose area to house equipment supporting science computing equipment and the McMurdo local broadcast media systems which uses MPEG transport over IP networks.
Leidos is the prime contractor for the National Science Foundation’s United States Antarctic Program, with Datapath the subcontractor for support communication, and Parsons is the general contractor for Antarctic infrastructure improvement efforts.
Patrick Smith, manager of technology development and polar research support at the National Science Foundation, said the current data center was established in the early 2000s.
“We will use this as an opportunity to do a tech refresh and modernize our infrastructure for local compute,” he said.
The data center project is part of the Antarctic Infrastructure Modernization for Science (AIMS) project to replace obsolescent buildings – some of which date from the late 1950s – and consolidate most station operations into a more energy-efficient, centralized facility with a much smaller physical footprint.
Data infrastructure at the South Pole
However, some of Antarctica’s most interesting data infrastructure lies much further south. The Amundsen–Scott South Pole Station lies some 850 miles south of McMurdo Station.
Despite being the largest data generator on the continent, it suffers from extremely limited connectivity and has little prospect of being directly connected to a fiber cable to McMurdo or the rest of the world any time soon.
South Pole Station’s biggest data generators are the IceCube Neutrino Observatory and the South Pole Telescope, which between them generate terabytes of data a day. The telescope looks at the early universe and the millimeter band; it was one of the facilities that helped capture the first image of a black hole in 2019. IceCube is dedicated to studying the origin of the highest energy cosmic rays through neutrino reactions in the ice; its thousands of sensors are located more than 1km under the Antarctic ice and distributed over a cubic kilometer.
“If you're trying to understand the early universe you are looking in the millimeter bands, microwave frequencies, a few 100 gigahertz, they don't travel very far right in the atmosphere because they're absorbed by oxygen and by water,” says Professor Nathan Whitehorn of Michigan State University. “If you're trying to see what's coming from space, you need to be someplace that has very little oxygen and very little water and one of the best sites is at the South Pole.”
While McMurdo station has continuous but limited connectivity, South Pole Station is even more restricted both in terms of coverage and capacity.
Today the South Pole currently relies on communications from three satellites; NASA’s TDRSS F6, Airbus’ Skynet 4C, and the US DoD’s DSCS III B7 providing 4 hours, 6 hours, and 3.5 hours of coverage respectively (though these windows overlap and shift constantly throughout the year). Iridium provides more continuous coverage but can’t be used for anything beyond voice calls or small (<100kb) emails that cost dollars per message.
For more than 20 years until 2016, one of the main satellites the South Pole relied on was the GOES-3 weather satellite. Launched in 1978 and originally built for the National Oceanic and Atmospheric Administration as part of the Geostationary Operational Environmental Satellite system, GOES-3 was used as a communications satellite from 1989 after its imaging components failed, and was used by the NSF for links to the South Pole from 1995 when its drifting orbit brought it in range of the station there. Providing around 1.5Mbps up and down, it was decommissioned in 2016 after 38 years in operation.
The remaining TDRS satellite supports a 5Mbps S-band link in both directions for telephone calls, web browsing, large emails, file transfers, video teleconferencing, etc. and a 300Mbps Ku-Band link reserved exclusively for the transmission of science research data and a few other large station operation files. It is not available for the general station population. Skynet and DSCS satellites both operate on the X-band, the former offering 1.544Mbps and the latter a 30Mbps southbound connection and 10Mbps northbound connection.
“For half the year, the only time that you have access to the rest of the world is in the middle of the night, which is challenging as a working environment,” says Whitehorn.
“If you're trying to connect from McMurdo to the South Pole, you go up to geosynchronous orbit, down from geosynchronous orbit, through some networks in the United States, back up and then back down and then the round trip does that several more hops. So you can have ping times between McMurdo and the South Pole in excess of 10 seconds.
“If you're trying to just get the manual for a piece of equipment that you're trying to use, often that involves waiting for 12 hours, unless you thought of it ahead of time and have it printed out. I've ended up using handheld satellite phones over Iridium to call people and ask them to Google things on a fairly regular basis.”
As well as limited bandwidth, all three satellites are long in the tooth; DSCS-3 B7 is the youngest of the three machines but was launched in 1995; TDRSS F6 was launched in 1993, and Skynet 4C 1990. Most satellites have an expected lifespan of up to 18 years, and many of the satellites’ sister machines have already been retired. DSCS-3 B7 and TDRSS F6 were launched with a 10-year design life, and Skynet-4C just seven years.
The South Pole Telescope generates around 1.5TB of data a day and the IceCube telescope around 1TB. The South Pole Telescope maintains a 100 core scale computing cluster, while the IceCube project has something closer to a 200 core cluster to do some limited pre-processing and systems management, but the team is able to transmit around 100GB a day off-site via satellite. The rest is flown out once per year as raw data on hard drives.
“We have developed custom software to manage these various data transfer streams as well as to check that all data are received in the northern hemisphere, resending files if necessary,” says Dr. John Kelley, IceCube Manager of Detector Operations, Wisconsin IceCube Particle Astrophysics Center (WIPAC) at the University of Wisconsin–Madison. “Overall this process works rather well but there are many links in the chain where transfer hiccups are possible, and latencies of a day or two are typical.”
“You really have to just wait a full year before you have access to the data,” notes Whitehorn. “And you have to hope that it's good and there aren't problems that nobody knew about because they couldn't see it.”
Connectivity to Antarctica still limited
Subsea cables can now reach more 300 terabits per second (Tbps) but the McMurdo station has a satellite connection providing something closer to 25Mbps to share between up to 1,000 people in the Austral summer. Even the International Space Station, in Earth orbit, does better than our most southern continent; at 600Mbps - more than 20 times the bandwidth of McMurdo.
The National Science Foundation is strongly considering laying a subsea cable from Ross Island to New Zealand, something that provide ‘essentially infinite capacity’ to the base compared its current bandwidth. However, while the cable could connect the base, and even act as a ‘terabit hotspot' for last-mile connections out into the near field, the South Pole will remain a connectivity challenge.
“The South Pole Station has our biggest demand for data with the burgeoning astronomy and astrophysics programs there,” says NSF’s Smith. “Right now, we're supporting them by satellite, and there are discussions about various satellite solutions to meet future needs.”
Smith says when the NSF explored a potential cable to McMurdo a decade ago it also looked at the potential to lay a cable across the Antarctic continent between South Pole and McMurdo - a swathe of land covered in deep ice.
“That was a pretty tough challenge; the whole ice sheet moves, and at different rates; there's a mountain range you have to get over; there's glacial ice draining through that pass you would go through; and then even at sea level on the Ross Ice Shelf, there's a lot of motion and twisting action.”
He notes that satellite is likely to remain key for the South Pole for the foreseeable future.