The US National Park Service data center, located in Denver, Colorado, went completely dark on 1 January as a result of a utility power outage, causing damage to hardware systems and data stored in the facility.
The NPS building housing the data center went dark around 3pm, and power was restored about two hours later, according to a note on the agency's website. “The power outage was due to an underground cable that either had water seepage or rodent invasion,” the note read.
An NPS spokesman said the agency was still in the process of gathering details about the incident and would have more information next week.
The agency is blaming Xcel Energy, the power provider in the region for the outage. The report on its website says the building has uninterruptible power supplies (UPS) – they kept the power going for 30 minutes when utility power was lost – but makes no mention of backup generators.
In a typical mission-critical data center, UPS supplies stored energy to give diesel generators time to switch on. Once they are on, they can keep the data center running as long as they are supplied with fuel.
The January outage seems to have caused a lot of damage, according to the NPS report. “The hardware in the data center has been damaged, and about a third of the data in the data center has been corrupted,” it read.
One of the reasons for the damage was the way the two utility power feeds were turned on, according to the report. NPS attributes file corruption to an “automatic shutdown process.”
Another reason cited is high temperature that occurred in the data center before it was shut down.
The agency's Office of the CIO has been working with Microsoft to restore the data. “Their (Microsoft's) response was positive, but they advised that just restoring the data from backup tapes will not resolve the issue and that in most cases the environment needs to be rebuilt due to the corruption in the system,” the note on the website read.
An outage is a costly affair for any data center. According to a recent study conducted by the Ponemon Institute and sponsored by the data center equipment vendor Emerson Network Power, one minute of data center downtime cost an average of about US$8,000 in 2013.
NPS, a nearly 100-year-old agency, manages all national parks and national monuments in the US.