On September 2, 2022, an electrical substation in Flevoland, the Netherlands, caught fire. A second substation swiftly ignited, and the local grid shut down.

Local data centers switched to their backup generators, as one would expect. But there were problems with the generators at A2B Internet. A simultaneous outage in the mobile network impacted their controls.

The issue brought the reliability of backup power to the fore. And this was not by any means an isolated incident.

There are multiple factors that mean data centers round the world could be switching on their diesel generators more often in the near future. And that might be a problem in itself.

Power grids under stress

In January, a power line failed in Kazakhstan disconnecting the whole country, as well as much of neighboring former Soviet states Uzbekistan and Kyrgyzstan.

In March, a malfunction at the Hsinta Power Plant in Taiwan caused blackouts in the south of the country, hitting Internet connectivity to the whole nation.

These outages may be coming our way more frequently, for a variety of reasons. European nations will very likely be experiencing a shortage of natural gas this winter due to the war in Ukraine, which could lead to electricity blackouts

And even before the war provoked an energy crisis, multiple nations and metro areas have been predicting a difficulty in serving demand for electricity. Ireland’s EirGrid, for instance, warned of “rolling blackouts” in 2021, due to increasing demand for electricity.

In London and Northern Virginia, this is because data centers have been approved faster than new electricity capacity can be commissioned, along with the infrastructure to distribute that energy.

In other cases, planners are warning that attempts to reach net-zero targets will create extra strain on the grid, because of a double effect: moving to net zero will increase demands on the grid, while destabilizing supplies.

New demands will be placed on the network as countries attempt to shift to electric cars and electric heating in homes and offices.

And at the same time, nations are trying to shift the power supply to their grid, over to renewable sources such as wind and solar, which are variable and can’t be managed as easily as a baseload source such as coal or gas.

As the effects of global warming begin to be felt, it’s likely that natural disasters such as floods and wildfires will be more frequent. In August 2022, huge floods in Pakistan took down the Internet there for an extended period, and plenty of earlier disasters meant data centers were taken off the grid.

In March this year, Australia’s New South Wales experienced outages due to flooding, while at the same time, wildfires in Tennessee cut off many residents and businesses.

In February, a giant storm in Texas created outages that were exacerbated by that State’s poorly-maintained electrical grid, leaving a lot of data centers struggling.

To add one more factor to the cocktail of risks facing the grid, it has long been expected that malicious attacks could bring down sections of the electricity network. Critical equipment is often provided with interfaces for remote management across the Internet - and these interfaces can be vulnerable.

In 2016, years before the current war, it’s believed that hackers brought down a section of Ukraine’s power grid. On 23 December, half the residents of Ivano-Frankivsk, were left without electricity for several hours.

Since the current hostilities began, Russian forces have hit power generators, and effectively treated the giant Zaporizhzhia nuclear plant as a hostage.

New role for backup

Against this background, there are signs that data center operators are treating their backup systems differently - at least in the developed world, where grid reliability has been a given till now.

In the US and Europe, diesel generators and battery rooms have always been there to cater for very rare emergencies when the electrical power fails or falters. They’re mostly seen as an insurance policy.

Now, data center operators are realizing that those backup systems may be called upon more often, perhaps approaching the experience of pioneering facilities in developing nations.

In Africa, it’s long been a fact of life that the grid is less reliable, and diesel generators are frequently fired up for backup. But that is changing, according to Funke Opeke, CEO of MainOne, in Nigeria.

“We build dedicated lines to take power from a substation, and that results in 96 percent or higher power availability from the grid,” Opeke told DCD this year, adding that the reliability of grids in other African countries may be less. “We get 95 percent in Ghana, and in Cote d’Ivoire we do better than that.”

While Africa may be emerging from an era of unreliable grids, Europe may be entering one. As winter blackouts approach, data center operators are buying up extra supplies of diesel fuel, in case they have to shift to their own generators for extended periods.

“We’ve been doing contingency planning since the war in Ukraine broke out,” Gary Aitkenhead, Equinix’s SVP of Europe, Middle East, and Africa operation told the Financial Times earlier this year.

“We don’t ever expect to have to run for more than a few hours, or at worst case a day, on diesel but we’re prepared to run for up to a week.”

Digital Realty is also storing more diesel at its sites across Europe.

Just as nations are filling their gas reserves, data centers are pumping more diesel into tanks that have previously been kept at 60 percent capacity, filling them up to more than 90 percent.

Is it a good approach?

There are problems with this. More diesel burnt at data centers will mean more emissions, putting the operators’ own net-zero ambitions out of reach.

It’s also inherently more unreliable. Once the diesel is running, and the power grid is down, then the data center has one less source of power available, and the diesel may be a single point of failure.

“Burning that much diesel isn't good for the environment or the stability of the data center,” says Opeke.

Other African providers are less optimistic about the grid’s stability, and have ambitions to bypass it.

“We don't believe that the grid can support the data centers of today, so we have to do something about it,” says Wale Ajisebutu of another Nigerian provider, 21st Century. “So our philosophy is microgrids, and captive power.”

In the long term, operators are aiming to shift to alternative sources which can keep the data center operating without creating emissions (see p4). If the backup generators are being used more regularly, this will increase the need to make that shift.

And in the short term, they will be doing all they can to secure their diesel supplies. If only because they remember how things went in New York in 2012.

When Hurricane Sandy hit in October of that year, Manhattan’s data centers discovered how crucial, and physically demanding, that supply effort could be.

When the storm hit, Internet provider Peer 1 had a problem. City regulations meant its generators were on the roof, with fuel pumped up 18 stories from main tanks in the basement.

The storm flooded Peer 1’s basement, and the pumps shorted. Data center manager Michael Mazzei had six hours’ fuel in a tank on the roof - and decided the only way to get more fuel reserves up there was by hand, in buckets.

Mazzei, his staff, and colleagues from other operators, all pitched in and Peer 1 remained online with a bucket chain providing its power for two-and-a-half days, till the pumps could be reinstated.

Ten years later, DCD caught up with Mazzei, and he was proud of the legendary achievement: given a similar situation, "I would have done it again," he told us.

But today’s Internet is more complicated than that, with multiple services connected across diverse facilities. Mazzei wouldn’t do it again.

At a very basic level, Equinix and Digital Realty may be glad they secured their diesel supplies.