Akamai has apologized for yesterday's Edge service outage, that hit many high profile sites across the Internet, and said it happened due to a DNS bug.
The outage lasted just over an hour, from 15:46 to 17:09 UTC, and was caused when a configuration update set off a DNS bug, which disrupted customer websites including Steam, American Airlines, Fox News, HSBC, and many more. The problem was fixed by rolling back the software update.
An error, not an attack
"At 15:46 UTC today, a software configuration update triggered a bug in the DNS system, the system that directs browsers to websites. This caused a disruption impacting availability of some customer websites," according to the brief Akamai statement on Twitter. "The disruption lasted up to an hour. Upon rolling back the software configuration update, the services resumed normal operations."
The service added: "We apologize for the inconvenience that resulted. We are reviewing our software update process to prevent future disruptions."
Akamai also reiterated that the problem was not caused by any attack on its systems.
Content delivery networks (CDNs) are vital to delivery of Internet services, as they front-end almost all web sites, delivering their content to users quickly through local proxies. They rely on fast resolution of domain name service (DNS) requests, and are vulnerable to misconfiguration and errors in their DNS systems.
Because CDNs are invisible to end users, they can be overlooked in corporate resilience planning, The event underlines the need to avoid having a single point of failure.
Akamai has not yet specified the nature of the "bug" in the DNS system which was triggered by its configuration update. The statement implies that the problem was in the DNS system rather than in the configuration that was pushed out across Akamai's Edge services - that is, the configuration update was not dangerous in itself, but exposed a problem in the DNS.
If this is true, then Akamai will be expected to provide more details of the DNS problem and whether it could occur in other CDNs or Internet services