Netflix has completed its move to the public cloud, having shut down its last in-house data center and moved all its back-office services to the Amazon Web Services (AWS) public cloud.
The move, announced in the summer of 2015, happened because Netflix believes that the public cloud is more reliable and cost-effective than its own in-house data centers were, and running those data centers was never a core part of its business. The streaming media company does, however, continue to run its own content delivery network (CDN), consisting of low-latency equipment installed in edge locations close to its customers.
Database corruption
“We are happy to report that in early January, 2016, after seven years of diligent effort, we have finally completed our cloud migration and shut down the last remaining data center bits used by our streaming service!” according to a blog post by Yuri Izrailevsky, vice president of cloud and platform engineering at Netflix.
The move to the cloud actually dates back to the days when the company still majored on DVD rental instead of streaming media, says Yuri Izrailevsky: ”Our journey to the cloud at Netflix began in August of 2008, when we experienced a major database corruption and for three days could not ship DVDs to our members. That is when we realized that we had to move away from vertically scaled single points of failure, like relational databases in our datacenter, towards highly reliable, horizontally scalable, distributed systems in the cloud.”
Netflix chose to use AWS, and had migrated the majority of its systems including customer-facing services before 2015, says Izrailevsky. In the last year, it has moved its billing infrastructure, and databases for customer and employee information.
Thanks to the cloud, Izrailevsky says Netflix has eight times as many streaming members as it did in 2008, with viewing up by three orders of magnitude. ”Supporting such rapid growth would have been extremely difficult out of our own data centers,” says Izrailevsky; ”we simply could not have racked the servers fast enough. Elasticity of the cloud allows us to add thousands of virtual servers and petabytes of storage within minutes, making such an expansion possible.”
Netflix has expanded to 130 new countries this year, and become a global Internet TV producer and distributor, using Amazon’s AWS regions.
Overall, the main reason for moving to the cloud was not cost reduction, the blog post says, but the move has provided a great improvement in reliability. And it has incidentally given a massive reduction in the cost per stream of video.
Don’t forget the CDN
Netflix still has its own hardware of course. Video is delivered through Netflix Open Connect, a content delivery network (CDN), which places the company’s own design of appliance in ISPs round the world close to customers.
The hardware consists of commodity components assembled in custom cases. Partners include HGST, LSI and Micron for storage, and Chelsio for network assistance.
The system uses open source software, including the FreeBSD version 10 operating system, the Nginx web server, and the Bird Internet routing daemon. Audio and video components for each streaming title are served directly to the customer client software via HTTP.