I'm just back from a visit to OVHcloud, the French provider which has still not fully explained the fire which burnt down its SBG2 data center in Strasbourg, more than two years ago. Legal cases brought by customers have begun to be settled in the Lille courts.
Among the allegations against the company, it has been claimed that OVHcloud did not have an adequate fire extinguishing system, although a report from the BEA-RI accident investigator said that fire detectors went off promptly.
During my visit, I was told that, after a year's evaluation, the company has been installing VESDA fire detection systems, and inert gas fire suppression systems. We also saw interesting work on liquid cooling and the circular economy, which DCD will be covering in more depth in future.
In the Roubaix 8 facility, we saw the new VESDA equipment, and sensors for temperature and humidity - and our visit was interrupted when work on a generator triggered a smoke detector.
Did OVHcloud have working fire prevention? In the absence of a full statement from OVHcloud, all the industry has to go on is observations like this, along with statements OVHcloud has been making in court, to argue against damage awards.
Sadly these don't always help the provider's standing.
Apologies to non-UK residents who are bored with British politics (and to UK residents who are heartily sick of it), but in some ways, OVHcloud's court defenses remind me of the UK's ex-Prime Minister Boris Johnson, who has recently been grilled over parties he attended during the pandemic. Johnson previously told Parliament that No 10 Downing Street obeyed the Covid rules at all times - but this was not true. In March, appeared before a Parliamentary committee that is deciding whether he lied deliberately.
Johonson's defense is that he knew about the parties, but his advisors told him they were perfectly legal. If I've understood that right, he is saying he asked advisors, because he didn't understand the simple lockdown rules (the rules he spelled out to the nation in public broadcasts). He is saying he went to parties and failed to notice there were too many people, standing too close together. In other words, he was both too stupid and too incompetent to understand the rules and spot their breach. And this is a man who hopes to continue a career in politics.
Some legal arguments really aren't worth using, because they just make you look even worse.
Disaster recovery that isn't
In the French courts, OVHcloud was ordered to pay damages to two of the many companies which lost data in the SBG2 fire
Reports of the trials include some very dubious arguments by OVHcloud which can only damage its reputation as a cloud services provider. It seems to have argued that it doesn't really understand the nature of backups.
Bati Courtage, a French property firm, lost servers in the disaster, but it had subscribed to an automated backup service from OVHcloud. So, after the fire, it asked for its backed up data.
After a month of repeated requests, OVHcloud finally admitted that the backups were destroyed along with the primary data, because they were "stored in the same building as the one where the main server was completely destroyed by the fire."
OVHcloud argued that the terms of its backup service allowed it to keep "local backups" in the same room.
The judge demolished this argument, pointing out that the OVH contract for automatic backup specifically said that "the back-up option is physically isolated from the infrastructure in which the VPS server is set up.".
OVHcloud has plenty of other data centers to send backups to, and locating them in the same data center makes a mockery of the service it is claiming to provide: "Keeping all the backup copies in the same place does not make it possible to protect the data, does not respect the state of the art of backup and does not allow the objective set by the contract to be achieved.," said the judge.
Like Bati Clourtage, SaaS company Bluepad also thought its data would be safe, because it was running its production server in the SBG1 data center, with a backup in the SBG2 facility which burnt down.
To its surprise, Bluepad heard that the production and backup servers were both in the destroyed SBG2 data center.
OVHcloud had signed an agreement to run the servers in the designated buildings, and then given Bluepad access to a management console that showed the servers were where they were supposed to be. After the fire, it admitted that the management console was wrong, and that it had not kept the servers where it had said it would.
OVH attempted to claim that the agreement and the console data weren't meant to be reliable, they were just a "simple internal reference".
The judge gave that short shrift. The location of the servers was contractual, and the agreement barred OVHcloud's from "modifying, without the customer's agreement, the location or geographical area provided for in the order.”
The ruling says Bluepad "never imagined that the company OVH, which presents itself as a European cloud leader, hosting specialist and which boasts ISO 27001 certification, may have misplaced a client's server or made a mistake in its location"
OVHcloud then added further to Bluepad's woes by retrieving the backup server, with the data intact - and then accidentally wiping the disk before returning it to the customer.
A defense of incompetence?
OVHcloud, a European cloud provider with ISO 27001 certification, is not merely admitting that it cannot reliably locate customer servers, or meet promises it made to customers. It is telling Bluepad that this is a reason to let it off the hook for damages caused when it misplaced those servers.
And with Bati Courtage, the cloud company had apparently sold a backup service, believing that a get-out a clause allowed it to deliver a backup service that did not work, because it failed to meet sensible definitions of a backup service.
Lawyer Etienne Wery says: "With such a clause, in the event of a claim, OVH is never required to carry out its mission." As he explains, you don't need a backup copy until there is a disaster. If your backup service ensures that both copies are destroyed in a disaster, it is not a usable backup service.
Johnson's defense over the so-called PartyGate scandal is "I couldn't understand the lockdown rules or follow them reliably, so I can't be blamed".
OVHcloud's arguments in court have a lot of similarities. Its console can't be relied on, its backups aren't where they should be. These are statements that essentially admit the company has trouble managing its technology and business agreements.
Johnson expects to be indulged for his failings. The case of OVHcloud is less clear, because we still don't have full statements.
Let's see what emerges.