Facebook has become a huge repository of photographs, and it is heavily used, as images are downloaded, shared and re-shared. With two billion photos shared each day on Facebook services, the company has had to find a way to handle them efficiently. Today the social giant has shared some of details of how it saved space and power, but still kept the library reliable and fast to access.

“Finding a place for these images to live so they can be instantly available is a recurring scale challenge for our infrastructure team,” say Krish Bandaru and Kestutis Patiejunas in an Engineering Blog post. ”But as the number of photos continued to grow each month, we saw an opportunity to achieve significant efficiencies in how we store and serve this content and decided to run with it.”

Older photos are less used

To make sure the photos are not lost, Facebook keeps a highly durable copy of older copies in specially designed “cold storage” facilities.

These facilities have been much talked about, and the company has previously discussed a scheme using robot arms to sift through Blu-Ray disks holding old images.   

This time, the company is using hard drives, to ensure faster access, but has rebuilt the infrastructure to limit the power used by the storage, by ensuring that servers and drives only power up when required.

In the past year, Facebook has opened two of these facilities in Prineville, Oregon and Forest City, North Carolina - both designed from the ground up for cold storage. 

Because the servers are often shut down, the data center needs only one sixth the power per square foot of a normal data center. Each data hall can hold up to 1 Exabyte (1000 Pbyte). They also don’t have fully redundant power, since they aren’t serving live data. 

Heavy duty racks

The halls use the Open Compute Project Open Vault specification, but only allows power to one drive per tray at a time. More than that might overload the slimmed-down power, and cutting it down this way means fewer fans and power supplies are needed in the racks. 

This allows racks with 2 PB storage, using 4 TB drives, running at a quarter the power usage of conventional storage servers. The biggest problem Facebook found, according to the blog, is that these racks were so heavy they crushed the normal rubber castors which the company uses on the racks. 

Surprisingly, it’s not hugely compressed, and uses error correction codes so 1GB takes up 1.4GB of space, in order to reduce the chances of “bit rot” and keep the image storage reliable. Facebook is also looking at the possibility of using other storage mechanisms, such as Blu-Ray disks or novel persistent Flash storage.