American network service provider AT&T’s broadband and TV customers were unable to access the company’s U-verse platform yesterday, following a fire at one of the company's data centers in Texas.
A spokesperson for the City of Richardson, representing the principal city to the northwest of Dallas, issued a statement which said two fire brigades arrived at the 1666 Firman Drive data center at 9:48am EDT to put out an electrical fire “which started at a power switch and burned primary and backup electrical systems.” CBS local reporter Mary Ann Martinez tweeted an extract of the release.
Fire in the switch
An AT&T spokesperson is said to have tweeted that the blaze was the result of a lightning strike hitting the data center, but another company representative later declined to confirm what caused the fire.
Though CBS local reported that services had been restored by 8pm, outage reports on downdetector.com peaked to 1377 at 9:48pm EDT.
A large proportion of customers said their Internet service was down and comments suggested that the majority were located in the Dallas area. Many also reported that their TV services were down.
Electrical fires are a common cause of outages in data centers: in the past eighteen months alone, Georgetown University in Maryland, a healthcare center in Cairns, Australia, and an unnamed provider in Melbourne suffered downtime as a result of such incidents.
Perhaps more common still, are outages caused by the very systems designed to prevent the fires. When triggered, fire suppression systems can damage data center hard drives by emitting powerful shockwaves, as was the case at the ING Bank data center in Romania two years ago, as well as at an unnamed London university last year and at the Glasgow City Council’s data center in 2015.
Sometimes, fire suppression systems simply set off safeguarding systems that switch off the facility’s IT equipment to avoid any damage. This happened in one of Microsoft’s Azure data centers in October 2017, affecting customers in the company’s North Europe region; a DigiPlex facility in Sweden containing Nordic Stock Exchange servers suffered a similar fate earlier this year.