Archived Content

The following content is from an older version of this website, and may not display correctly.

Some customers of data center, hosting and managed hosting service provider iWeb saw their servers stay down for as long as 13 hours, following an incident that started Wednesday evening and whose consequences continued into Thursday afternoon.

Overall, about 3,000 servers were affected by the outage that started around 11 pm on Wednesday, but most of them (about 2,600) were brought back to life about 2.5 hours later, according to a post on the company's blog. Yet, few machines were still down as of about 2 pm on Thursday.

The outage was caused by a failed transfer to generator power at the data center. A fire nearby caused disruptions in utility power and the facility was automatically switched over to generator power. The transfer went normally for two of the three generators. The third generator started but the transfer switch failed to transfer power to it as expected.

Servers affected by the failed transfer constituted about one-third of the data center, or close to 1.5MW of load.

The entire data center was powered shortly after midnight. Around 12:40 am, sections of the data center that were not affected by the transfer-switch failure were back on utility power, while the third section was running on generators.

As of about 12:20 am, the iWeb team did not know exactly what caused the failure to transfer to the third generator. "We suspect the transfer switch control logic has been disrupted by the kind of power outage (the fact that it was related to a fire nearby)," the company's representative wrote in a blog post. "We are still investigating to find out why the power has not been transferred to the third generator."

In the same post, the provider said generator tests are conducted every week and a successful test had been completed one day before the incident without any signs of problems.