Montreal-based service provider Web Hosting Canada suffered a lengthy outage on Saturday 28 August.

The company blamed the downtime on unauthorized activity by an undisclosed third-party service provider. Production servers and backup servers were affected, which may have caused permanent data loss for some. Services are still recovering.

“Based on our investigation to date, the morning of August 28 at approximately 6 a.m., an individual with a third-party service provider used their privileged account access to connect to one of our data center’s management portals and without authorization, initiated server reimaging on some of our backup servers, then on some of our production servers,” CEO Emil Falcon said.

The reason the individual brought down the data center was not disclosed.

Falcon continued: “Within only hours our incident response team had identified the issue and disabled access to the source account, preventing any further damage. The environment was secured, the individual fully locked out, and our disaster recovery plan immediately kicked into action, but [the] damage was already done.”

For the servers that are still unrecoverable, the company has brought in data recovery specialists - but warned that it seemed unlikely that everything would be restored.

On Reddit, customers shared stories of data loss or of sudden downtime. They also complained that they were not informed of the problem by WHC.

"This is a massive fail," one commenter said. "I'll be jumping ship. All professional sites I've done have been hosted there. Not any more. Not good disaster recovery when your disaster recovery fails."

Another added: "Yeah, my website and emails have been down for over 24 hours at this point and it looks like I'm part of the "non-recoverable" server. Awesome.

"I was never informed there were any issues until I noticed emails stopped coming in. This is despite having a different email address as my main contact that remains unaffected. I'm not too impressed by that. I wish they had taken a minute to send a "head's up" at least."

Others pointed out that it is best practice to have a local backup, or external disaster recovery site.