Once again, a content delivery network (CDN) failure has demonstrated the importance of parts of the Internet that are unknown to a lot of people.
CDNs speed the Internet by serving it from local proxies. They are an idea that's 20 years old, continually updated to meet current needs - and they are fragile.
As I remarked last year, while the Edge marketing community hype exciting new products that may deliver a gleaming new infrastructure, the reality on the ground is that Edge services are already being delivered - by the installed base of services originally built to support the old, static Internet. In the real world, CDNs are the Edge.
These outages are unpredictable, but inevitable. Akamai failed before in June, and before that, we had a Fastly failure that took down large parts of the Internet. One year ago Cloudflare failed, having previously gone down almost exactly a year before.
All too often, they come about because of bad configurations in services like DNS, update errors, and unknown bugs which are triggered by updates. DNS errors were at the root of more than one of the incidents.
Diversify to stay online
Your services may be relying on a single service from a company you are barely aware of - and therefore on that company's ability to maintain disciplined codes and accurate updates.
One way around this is to diversify. As ThousandEyes pointed out after the Fastly outage: "Customers leveraging multiple CDNs were only partially impacted and eventually were able to fall back on alternative providers. Businesses using Fastly as their sole CDN, were taken offline completely."
That's good, but maybe not enough to save services because there's more to it. It turns out that there's an interrelationship between CDNs.
After the Fastly outage, ThousandEyes' Internet Report analyzed IP addresses, and found that other CDNs were using Fastly. So some services which used Amazon's Cloudfront CDN were hit, when Cloudfront offloaded some work to Fastly. Other CDNs may get tangled up as well, given that providers such as Microsoft and Alibaba also have their own such services.
"The Internet landscape has become so concentrated that even for redundancy amongst the CDNs and cloud providers themselves, only a few players are being used," said Alban Kwan of CSC Digital Brand Services, after the Fastly outage. "The result is that someone who is using CDN A (as an example) may actually be resolving some of the traffic via Fastly, and when Fastly went down, it also impacted clients who are using CDN A. Such deep-dependencies seem to go deeper than anyone may think."
A domain name service (DNS) problem is involved in the current Akamai issue, according to Akamai's own status page - and that's no surprise. The DNS service over the years has proven fragile, vulnerable to disastrous configuration errors and highly damaging DDoS attacks - such as Dyn in 2016.
The DNS is dull, and keeping your connection to it is a chore, but as Kwan points out, it's vital. There's a secure DNS option, and if possible, it's good to think where your DNS service comes from and not blindly rely on a single provider, or fail to update SSL certificates it relies on.
Narrow ownership, wider problems
This is not, as they say, rocket science. The Internet Society (Isoc) warned of the dangers of Internet consolidation producing "deep dependencies" and "flattening the Intenet" in its 2019 annual report Consolidation in the Internet Economy.
"Default one-stop shops, interoperability, and standards development and implementation at scale, as well as the flattening of the Internet’s infrastructure, are all the result of the concentration and consolidation in key areas," said Isoc. "This creates dependencies both within layers and cross layers of the Internet. The development of new applications, services, and businesses across the global economy is increasingly dependent on a small number of private platforms owned by the largest Internet companies."
Isoc warned that consolidation threatens more than our infrastructure. Excessive power owned by single platforms at all levels can create dangers of fake news on social media, and malign control of physical infrastructure when over-powerful web giants can effectively own vital services in small countries because they provided the investment to build them.
Isoc warns of a "domino effect" that could be caused by failures, particularly in sectors where "there is a risk of growing societal dependencies on a handful of powerful economic actors".
Consolidation is a normal process as any market develops. The danger with the Internet is that it is simultaneously consolidating multiple real-world markets into its virtual space. And that creates the danger of real-world consequences.
So Amazon has more than half of online retail spending in the US - and the majority of public cloud.
Google has the majority in online search, mobile operating systems web browsers, and user-posted video channels.
Facebook dominates social media, not just through its own platform but through the acquisition of WhatsApp and Instagram.
In China, Alibaba has more than 60 percent of China's e-commerce, and Tencent's WeChat is ubiquitous.
We all expect new services such as smart homes and the Internet of things to be dominated by some of these players - or maybe one or two new ones.
The evidence seems to be that this consolidation can produce ideal conditions not just for outages in physical services, but for systems that nurture online hate and foster political system crashes like the election of President Trump and the UK's Brexit vote.
Internet consolidation will cause a never-ending series of outages like yesterday's Akamai incident. It is also bringing us more grief in the world around us.