French operator OVHcloud has restarted the servers in its SBG4 data center in Strasbourg, after the disastrous fire last week which took the whole location offline.
OVHcloud's four data centers on a Rhine-side industrial estate have been closed since the fire destroyed SBG2, but power and network have been restored to the site, and servers are being restarted. According to OVHcloud's Strasbourg status page, the smallest facility SBG4 is up again (green), while several servers are operational in SBG1. However all the racks on the status page for SBG3 are down (gray), and the page detailing the status of services in SBG reveals that some public cloud services, based on OpenStack, that were hosted in the SBG complex may be unrecoverable or will not be back till April 8
"OVhcloud cares more for your data than you do"
Storage in servers at the site will contain a lot of user data which should be recoverable, but to bring back the personal cloud storage (pCS) cluster at SBG will take a long while, OVHcloud founder Octave Klaba said on Twitter: "pCS is a big object storage cluster based on > 10000 disks. we need 80% of data to rebuild 100% of cust’s data. we have > 99%. we just need to restart openstack, keystone, etc then we will restart pCS."
The process has been made more complicated because, as Klaba explained in SBG1 some damaged servers have needed repairs from smoke damage: "Disk, CPU, RAM works well, but we need to replace the motherboards of all servers. To do that quickly, we will change all of them to just 1 standard motherboard with Atom and allow the custom to get & to migrate the data to another server."
Some customers were impatient that this process was taking so long, noting that as their sites disappeared from Google listings, their livelihoods were threatened. One user tweeted: "I don't know how I'm going to feed my family after this and more about how work is. At least take a test in SBG3, I beg you."
Other users explained backup procedures, repeating that backups in the same site were never proof against a whole data center going down. "If you lost everything by your provider losing one DC you didn't prepare and backup well enough," said one.
Another pointed out: "You don't seem to grasp the absolute gigantic task it is to restart a full DC either. You should just wait patiently and thank OVH for caring more about your data that you do."
Klaba has promised a video update later today, which will explain more about the root cause and the procedure to restore services.
