Load shedding by the grid operator in South Africa has caused a cooling system failure at an MTN data center in Johannesburg.
African hosting firm Afrihost is suffering an outage after a cooling failure in an MTN facility it occupies in the Gallo Manor area of the city. Email services and websites hosted through Afrihost have been offline for upwards of thirty hours.
Starting at 09:07 local time yesterday, Afrihost reported disruption in the data center affecting its hosting environment, noting that affected servers will experience limited to no access to the outside world.
“Due to extended load shedding there was a cooling system failure at MTN’s Gallo Manor data center where we host some of our web servers,” the company said on its status page yesterday. “This unfortunately damaged certain Afrihost hosting servers.”
At the time of writing, 12.6 percent of the affected servers recovered.
Jacqui O’Sullivan, Executive for Corporate Affairs at MTN South Africa told DCD: “MTN can confirm that one of its coolers at the Gallo Manor data center was impacted following a power outage on Wednesday afternoon. However, the cooling system is redundant and therefore did not fully shut down and was in fact fully restored within 45 minutes. MTN is liaising with customers to investigate any impact on their operations.”
Eskom this week has been load shedding – disconnecting sites from the power grid ahead in a scheduled manner to control the load and prevent total grid collapse – at Stage 4, where more than 4,000MW of capacity is removed from the grid.
Power cuts have become increasingly regular as the grid struggles with capacity needs. A stage 6 load shed – where 6,000MW is cut – has been scheduled this week. Experts are warning a Stage 8 shed (i.e. 8,000MW) may be needed in the near future.
Eskom said it is struggling with a ‘high level of absenteeism’ amid an ‘unlawful strike’ that is slowing repairs. Staff are on strike amid wage discussions.
“We currently have 3,161MW on planned maintenance, while another 17,395MW of capacity is unavailable due to breakdowns,” Eskom said.
MTN tweeted: “We are working tirelessly to limit the impact of load shedding on our customers. The majority of MTN’s sites have back-up power but the frequency & severity of outages means those batteries are unable to sufficiently recharge.”
The telco said it is undergoing an “aggressive rollout” of batteries, generators, and alternate power supplies, and is reaching out to small businesses to supply generators for its network operations.
“There is no doubt the country is facing a power crisis but at MTN, we want to turn this crisis into an opportunity for small businesses by ‘crowd sourcing’ generators to further support our network,” said Charles Molapisi, MTN SA CEO.
Michele Gamberini, the Chief Technology and Information Officer at MTN SA, added: “Despite us having placed thousands of batteries at our sites across the country, the efficacy of those batteries greatly reduces once we pass stage 4 load shedding.”
The company said it has upgraded its battery back-up solutions on over 80 percent of its sites, but warned the current outage schedule does not allow enough time for batteries to charge. Battery back-up systems generally take 12-18 hours to recharge, while batteries have a capacity of about 6-12 hours, depending on the site category. At Stage 4, load shedding outages are to be implemented 12 times over a four-day period for two hours at a time, or 12 times over an eight-day period for four hours at a time.
MTN has also deployed more than 2,000 generators to counter the impact of Stage 4 (and higher) load shedding and currently using more than 400,000 liters of fuel per month to keep these generators operational.
MTN said it is inviting all businesses that are in possession of generators, to become potential suppliers to MTN. “Whether the business has two or 20 generators, MTN is looking to partner,” the company said.