A fire test went awry in Romania, with ING Bank’s regional data center going offline due to a loud noise.

When the Inergen fire suppression system was turned on, the resultant sound caused a rare, but not unique, failure - the noise of escaping gas damaged dozens of hard drives.

Bang, bang my baby shut me down

The Romanian Insider reports that more than one million customers were affected by the downtime, which lasted 10 hours, before services were shifted to a backup data center.

Daniel Llano, Head of ING Retail Banking, said in a statement (translated): “We inform you that today, September 10, at 13:00, the data storage system started generating operating errors. Due to this unforeseen technical situation, the ATM network, the ING card acceptance system and the Home’Bank system have become unavailable.

“We apologize for the inconvenience and we assure you that, together with the technical platform supplier, we have made it our highest priority to solve this problem.”

Llano explained the cause of the failure: “Yesterday, during a planned test of fire extinguishing system in our data center, Inergen [inert gas] flooding affected more seriously and unexpectedly our servers and data storage system.”

“It was as high as their equipment could monitor, over 130dB,” a source told Motherboard, with another adding that it was “like putting a storage system next to a [running] jet engine.”

The vibration caused by the noise is able to damage nearby hard drives.

“It can now be established with a high degree of certainty that the faults in storage systems as a result of an inert gas extinguishing systems discharge were caused by the impact of high noise levels on the hard disk drives,” Siemens said in a white paper.

A similar shockwave caused by a fire suppression system is believed to have taken out a Glasgow City Council data center last year, while a previous DCD editorial pointed out the need for data center operators to speak openly about causes of failures that could be avoided if more were aware of them.

Ed Ansett of i3 Solutions Group noted that with fire suppression systems “nozzles should be baffled or placed away from the storage racks.”