When Google had some trouble with lost data, we very quickly heard from people with ideas for how the firm could have made its data more reliable. They also promised green benefits.
It started when a Google data center suffered four lightning strikes to its power systems. This might seem unlikely, but “lightning never strikes twice” is just a myth. Repeated strikes during one storm aren’t that unusual - but it is probably fair to say this incident went beyond the normal design parameters of the data center - which appears to be Google’s St Ghislain facility in Belgium.
Susceptible storage
The storage systems at the data center carried on working, but they had to switch to battery backup power while the site’s power systems were restarted.
What seems to have happened is that some of these systems were “susceptible” to power failure if it involved repeated incidents or battery drain. With the disks in trouble, the data got written to permanent storage as quickly as possible, and Google engineers restored it over the next couple of days, but some people suffered some permanent data loss.
Google puts the loss at one-millionth of a percent of the Persistent Disk data at the facility, but for IT people, any data loss is serious, and it wasn’t long before DatacenterDynamics had a contribution from a vendor who knew what Google should have done differently.
Nik Stanbridge, VP of marketing at archive storage firm Arkivum took Google to task. “The statement that only 0.000001 per cent of disk space was permanently affected,” said Stanbridge, “will not be of any interest to the individual customers that have had their data wiped.”
If you need to have 100 percent reliability, you need a storage system based on tape, he said. You won’t be at all surprised to learn that is what his company uses.
As well as being more reliable, tape is not so susceptible to power problems, because most of the time, when the data is not being written or read, the system will be powered down. This also means that tape-based systems use less energy and are therefore greener.
Tape can’t do everything
But tape isn’t appropriate to every need, and I’m sure it couldn’t replace the hardware that failed here. You can’t drive Google Compute Engine virtual machines from a tape. Live working data hasn’t been kept on tape since the days of the Sinclair Spectrum.
The Google facility did indeed have reliable long term storage and I bet some of it is on tape. I also am pretty sure Google will be energy conscious enough to be aware that it can save energy by using tape. Tape is still widely used and is not going away any time soon.
Storage, after all, is the third biggest energy consumer in the data center, after cooling and servers, so cutting its energy use is well worth doing.
It’s good to have a reminder of all of this. If by any chance Google - or you, dear readers - aren’t making good use of tape where possible, then I’m sure we are all grateful for the reminder.