OVHcloud has abandoned efforts to restart the SBG1 data center in Strasbourg, which was damaged when the SBG2 building burnt down on March 10.
After a second smaller fire caused smoke in a battery room on Friday night, the French cloud operator announced that it will not restart SBG1. Instead, it will move the servers in SBG1 to other OVH data centers on the Strasbourg site and elsewhere, and concentrate its efforts on restarting the other two data centers there, SBG4 and SBG3. It is now ten days since a devastating fire, suspected to have started in a UPS, ripped through SBG2 and resulted in the whole site being taken offline. Customers will have lost data, but OVHcloud hopes much can eventually be reconstructed.
OVHcloud pulls plug on SBG1
"We don’t plan to restart SBG1. Ever," tweeted OVHcloud founder Octave Klaba, before detailing where the servers in SBG1 will be taken. SBG3 and SBG4 are being restarted.
The majority of the SBG1 servers - those in about 160 racks, labeled 61A/D and 62A/D - will be moved to SBG4, the smaller of the two remaining data centers on the Strasbourg site, which is on an industrial park by the Rhine. Servers from the other 50 racks, labeled 61E/62E, will be cleaned up at OVH's Croix facility in Roubaix, and then restarted at its data centers in Roubaix and Gravelines.
OVHcloud is running a new power cable from SBG3 to SBG4: "We need 3 days to dig & connect power from SBG4 to SBG3," tweeted Klaba early this morning (Sunday 21 March).
In SBG3, the largest OVHcloud facility on the site, 60 technicians are cleaning and restarting the servers. As of 21 March, Klaba has reported that 40 percent of the virtual private servers (VPSs) in SBG3 are up and OVH is now working to restart the remaining VPSs. Two outside expert cleaning firms are helping.
Klaba cautioned that the process will be slow: "We clean up the network first. Then rack per rack. then the servers. the clean up of the server is a painful process once we have the PCI & PCC servers for the control plane, the teams can work on software, which will take time to restore. At the same time, the teams are working on BM."
Backups will be free
The day after the fire, Klaba revealed that UPS7, which served SBG2, was given maintenance by the supplier the previous day, and both UPS7 and UPS8 were ablaze a few hours later when the firefighters arrived. While this strongly suggests the UPS systems were involved in the fire, an investigation involving the police and OVH's insurers must be completed before there is any definitive information about the cause. The UPS systems, batteries, and fuses have been handed over to investigators.
On Friday night, smoke was seen in a room at SBG1, which contained batteries that were not connected. Firefighters attended and quickly controlled the situation. There were no injuries on Friday or in the original fire on March 10.
While users wait to find out whether their data can be restored, Klaba has promised that free backups will be included in all future services, because users are relying on cloud services without understanding the level of reliability. "Some customers don’t understand exactly what they bought," said Klaba last week.