The data center industry is changing at a breakneck pace but one thing remains as constant as the northern star - the humble 3.5-inch hard drive. These devices have come a long way in terms of capacity and storage density, with the largest 3.5-inch HDD on the market today measuring a hefty 10TB, but they retain the shape and size inspired by the 3.5-inch floppy disk drives of the eighties, complete with all the attendant drawbacks and limitations.

In order to satisfy the requirements of future data centers, hard drives have to become larger, and we don’t just mean capacity, but their physical size too.

New look drives

In February Eric Brewer, VP of Infrastructure at Google, issued a strange call to arms: he promised the company would work with industry and academia to develop new types of disks that are a better fit for the needs of hyperscale data centers.

Among the most interesting suggestions was a taller form-factor that would enable manufacturers to fit more disk platters per casing to increase capacity, while simultaneously lowering the costs of packaging and components.

Brewer also suggested batches of multiple drives packaged as a single unit, with a very large cache and improved power distribution.

Yet another scenario called for hard drives to be made less reliable, as long as it resulted in lower prices. It turns out that in hyperscale environments, a higher target error rate would have little effect on overall durability, since all data is distributed and backed up across multiple physical locations.

Google is a huge consumer of hardware and has recently joined the Open Compute Project, which means it has gotten used to simply asking its partners to create the equipment it needs. But while anyone can build a server, hard drive manufacturing is an extremely complex, technical and capital-intensive task and there are just three vendors out there that can meet Google’s demands: Seagate, Western Digital and Toshiba.

Between them, these three companies make nearly all of the hard drives in the world. Western Digital is currently the largest, with about 43 percent market share, followed by Seagate with 39 percent, and Toshiba with a 17 percent share.

According to Uli Plechschmidt, managing director for cloud systems and electronic solutions at Seagate EMEA, in the past the storage industry was held back because large drive manufacturers couldn’t experiment with radical ideas – everybody wanted to buy a drive that fits comfortably in an existing category, nobody wanted the new (and potentially risky) device, no matter how mind-blowing it was.

“Now the handcuffs are off,” Plehschmidt told DatacenterDynamics. “We can really innovate, because there are now market segments which allow and are willing to pay, and sometimes even co-fund fundamental innovation on the drive side, whether it’s HDDs or SSDs. It’s a really exciting time.”

Google takes control

Eight or ten years ago, Google’s suggestions would have been dismissed out of hand, but now the cloud giant buys more drives than traditional storage vendor EMC, and paying close attention to its needs makes sense.

“There are two segments where we as a company are engaged directly – that does not necessarily mean selling directly – but being engaged directly, in order to really capture the bleeding edge of requirements. One is the big cloud service providers, the hyperscalers – the Facebooks, Googles, Microsofts, Emersons, Alibabas, Tencents and so on. With all of them, we have really big teams which really understand their requirements, because what they need today, probably in 3 to 5 years will become mainstream.

If Google says they need it – let’s really understand what do they need it for - and if that is a use case that many other cloud service providers might have, then it might be a good idea to develop something 

Uli Plechschmidt, Seagate

“And the same is true for the largest supercomputers on the planet, and the largest users of object storage. It’s the people who already have triple-digit petabytes, and plan be in the exabyte range by 2020-2022.”

These two groups are moving the hard drive industry forward and will continue to do so, even as consumers switch to flash. But back to the question of the tall drive: how feasible is it, really?

“If Google says they need it – let’s really understand what do they need it for - and if that is a use case that many other cloud service providers might have, then it might be a good idea to develop something, initially for Google but then to productize it and monetize it around the world.

“There must be a business case – that has not changed. But I think most IT companies nowadays are much more open if a large customer has some, at first sight, unusual ideas. We are doing a similar thing on the SSD side with Facebook. Basically Facebook tells us: ‘this is what we need, and we can’t get it from anyone else.’”

Drives for the cloud

And indeed, at the Open Compute Project Summit in March, Seagate showcased a prototype of the fastest solid state drive ever built, achieving throughput performance of 10GB per second. Plechschmidt added that, even though the device was built with a particular customer in mind, it will be available to other cloud service providers – some of which haven’t reached Facebook’s level of maturity yet.

Does that mean that, very soon, hard drives will finally change their familiar shape? It certainly does. Disk-based storage is the last remaining bastion of exclusive, proprietary tech. Hyperscale data centers deserve credit for kickstarting initiatives like the Open Compute Project and teaching the rest of the industry to trust in open source cloud platforms, white box servers and no-name networking equipment. This community is also working hard to encourage the creation of new hardware standards, like the upcoming 21-inch Open Rack.

Unfortunately we can’t order our HDDs from outsourcing specialists like Quanta or Penguin Computing, but it looks like the traditional hard drive vendors themselves are finally starting to listen to their customers, and realizing that there are important considerations in large data centers that go beyond speed and capacity.

This article appeared in the September issue of DatacenterDynamics Magazine.