The Internet’s latest antihero, Marco Marsala, the web hoster who deleted all his customers’ data, appears to have been saved by a data recovery company, according to a story unfolding on a sysadmins’ support board. 

Marco Marsala, a web hoster with some 1500 customers, deleted all their data by running a single command rm -rf, which deleted all data on his hard drives. It also deleted his backups as these were mounted to the local system. Asking for help on the sysadmin support board Server Fault, he was told that he had probably destroyed his whole company, but after some days, it seems that a data recovery company has said it can retrieve the lost files. 

Deleting your company 

“I run a small hosting provider with more or less 1535 customers and I use Ansible to automate some operations to be run on all servers. Last night I accidentally ran, on all servers, a Bash script with a rm -rf {foo}/{bar} with those variables undefined due to a bug in the code above this line,” Marsala said on the board on April 10. 

The rm -rf command removes everything in a given directory; the addition of the f “forces” the command and suppresses warnings. All the drives that were local had all their files deleted. His backup was remote, but mounted locally so that was deleted as well. 

Board user Sven quickly advised him: ”You might have an extremely slim chance to recover from this if you turn off everything right now and hand your disks over to a reputable data recovery company. This will be extremely expensive and still extremely unlikely to really rescue you, and it will take a lot of time.” 

Most commenters agreed that he had probably destroyed his whole company, with one user telling him: ”You don’t need technical advice, you need to call your lawyer.”

Inevitably, the story got wider coverage, starting with the Independent, and including Slashdot, elevating Marsala to a status as a cautionary tale warning people to keep remote backups remote, and to test every command on a single system before unleashing it. 

However, yesterday, the story seems to have improved for Marsala. “We consulted a data recovery company who analyzed one of our 1500+ server disks for a reasonable fee, and after diagnoses, sent … a list of recoverable files,” he said on the board. “All files are here. Now we’re finding the money to pay the recovery service for all our servers.” 

Questions still remain, however. Server Fault users have asked why the command succeeded when it seems Marsala didn’t use the “–no-preserve-root” option, and the Slashdot post says “Something about the story feels wierd.