OVHcloud is cleaning up and restarting servers in its SBG3 data center after the disastrous fire in Strasbourg two weeks ago, but still cannot give definite answers to all its customers' questions, according to the latest update from company founder Octave Klaba.
The undamaged SBG4 data center is now online, and servers from SBG1 are being cleaned and moved there, while SBG3 is coming back bit by bit, according to a set of tweets from Klaba this morning (26 March). SBG2 was destroyed in the 10 March fire, and OVHcloud eventually took the decision not to restart the fire-damaged SBG1 after a second fire in one of its battery rooms. The cause of the fire will be eventually reported after an inquiry led by police and insurers.
Respecting the cleaning process
"All the servers in SBG4 are UP again," said Klaba, also pointing out that many of the racks now show green on the status page for the largest data center on the Rhineside site, SBG3.
"Now, we started to move the servers from SBG1 to SBG4," went on the OVH founder, explaining that as servers are installed racks may be switched on and off, as seen in a timelapse video on YouTube showing two days' work on SBG3.
"As soon as we have a rack UP, the team start working on the servers then on the services," he said, adding that the process is slow and unpredictable: "We are working on the ETA per rack but honestly it’s complex to get one that we are sure to respect because of the cleaning process."
Racks from rooms 61E and 62E in the abandoned SBG1 data center are being shipped to other OVHcloud data centers in France, and are being cleaned up at a rate of three racks per day, he said: "We are looking how to speed it up."
The cleaning process involves chlorides, but is a proprietary one, he said, in response to a tweeted question.
At the software level, OVHcloud is restarting OpenVMS private cloud services, and OpenCloud public cloud, along with bare metal instances. It's getting about 20 customers' servers restarted every three hours, said Klaba (up from 20 per day at the start of the process), but that is still unpredictable because of different customer requirements: "Some [customers] just want data since they moved to [a different OVH data center] RBX. Lots of different cases."
Klaba posted a number of images of servers and hard drives from the damaged data centers which are in the process of cleanup.