Cookie policy: This site uses cookies (small files stored on your computer) to simplify and improve your experience of this website. Cookies are small text files stored on the device you are using to access this website. For more information on how we use and manage cookies please take a look at our privacy and cookie policies. Some parts of the site may not work properly if you choose not to accept cookies.

sections

Hurricane Sandy takes out Manhattan data centers

  • Print
  • Share
  • Comment
  • Save

At least eight New York data centers are struggling with connectivity and service issues as Hurricane Sandy moves inland.

InterNAP's data center at 75 Broad Street (known as LGA11) has been evacuated, according to a customer note sent by Matt Price, the company's director of hosting operations and support.

“The flooding has submerged and destroyed the site's diesel pumps and is preventing fuel from being pumped to the generators on the mezzanine level,” he told customers via a posting online. With the available fuel reserves good enough for seven hours of power at the most, he urged customers to “take necessary remote action”.

“Life safety is our number one priority,” he said. “We are making plans to completely exit the facility.” Customers were warned to shut down servers immediately.

Meanwhile, Internet service provider Datagram's Manhattan building suffered a flooding in the basement, taking out servers that host BuzzFeed, Huffington Post, Gawker and other sites. Though Buzzfeed was able to keep its service online to some extent, as a consequence of using Akamai's distributed content-delivery network, Huffington Post and Gawker suffered outages.

Not all customers seemed to be able to forgive the company for not insuring itself against this extreme weather. “Gawker's Datacenter is still flooded to hell. Go figure,” tweeted @damagedgears.

In response, Datagram put out a statement on social media: “Basement flooded, fuel pump off line - we got people working on it now. Five feet of water now.”

It is possible that many New York data centers had to run on backup power as the local electricity company, Consolidated Edison, reported a wide-scale blackout across the city.

Over 650,000 customers in NYC and Westchester County were without power, said John Miksad SVP for electric operations at Consolidated Edison. “This is the largest storm-related outage in our history,” Miksad said. Some unconfirmed reports and tweets spoke of a substation explosion in Manhattan.

Many businesses in the region were already shut down for a number of days, notably the New York Stock Exchange.

Meanwhile, on the wider network infrastructure, IP backbone operator Init7 reported a power outage in a data center facility at 111 8th Avenue. This, it said, would cause possible routing issues to and from the US. The facility, a key internet hub, houses many large data center players.

The problem was caused not by a lack of backup power supplies but by failure to implement it properly. Reports on Twitter suggested that a failure happened during the backup generator switchover.

However, not all its infrastructure is down, it posted on its web site. “While connectivity to NYIIX and EQUINIX-NYC exchanges are still up, connectivity to Miami and Los Angeles is lost,” said an announcement.

Another Manhattan hosting site at 121 Varick Street saw Steadfast Networks report that its facility would be “going down” while ETA was not available for its restoration.

Connectivity issues were also being reported for Savvis data centers in Boston and for web host InterServer's fiber-optic links in the New York area. StrongVPN’s New York Financial District data center also experienced major flooding.

Equinix came out better. Though there were claims that its NY9 internet business exchange (IBX) had a service interruption, three other Equinix facilities - NY1, NY2 and NY4 - switched to generator power with no problems.

There were some signs of a backlash from users in response to these announcements.

Some clients of Internap, such as gamers NuclearFallout Enterprises and chat service Zopim posted news of their service outages on Facebook and Twitter.

“It's rather sad that data centers are promising failsafes that don't actually work,” complained one angry user on Webhostingtalk.com.

“What I find frustrating is all the people who want to defend shoddy planning, and have some sort of kumbaya moment. It's not like they didn't have a week to prepare,” ran another comment.

“Don't bother mentioning SLAs. Those are the hosting world equivalent to used toilet paper. Totally useless,” the poster concluded.

 

Related images

  • Large chunks of Manhattan were in darkness thanks to Hurricane Sandy

Have your say

Please view our terms and conditions before submitting your comment.

required
required
required
required
required
  • Print
  • Share
  • Comment
  • Save

Webinars

  • Live Customer Roundtable: Optimizing Capacity (12:00 EST)

    Tue, 8 Sep 2015 16:00:00

    The biggest challenge facing many data centers today? Capacity. How to optimize what you have today. And when you need to expand, how to expand your capacity smarter. Learn from the experts about how Data Center Infrastructure Management (DCIM) and Prefabricated Modular Data Centers are driving best practices in how capacity is managed and optimized: - lower costs - improved efficiencies and performance - better IT services delivered to the business - accurate long-range planning Don;t miss out on our LIVE customer roundtable and your chance to pose questions to expert speakers from Commscope, VIRTUS and University of Montana. These enterprises are putting best practices to work today in the only place that counts – the real world.

  • Power Optimization – Can Your Business Survive an Unplanned Outage? (APAC)

    Wed, 26 Aug 2015 05:00:00

    Most outages are accidental; by adopting an intelligent power chain, you can help mitigate them and reduce your mean-time to repair. Join Anixter and DatacenterDynamics for a webinar on the five best practices and measurement techniques to help you obtain the performance data you need to optimize your power chain. Register today!

  • Power Optimization – Can Your Business Survive an Unplanned Outage? (Americas)

    Tue, 25 Aug 2015 18:00:00

    Most outages are accidental; by adopting an intelligent power chain, you can help mitigate them and reduce your mean-time to repair. Join Anixter and DatacenterDynamics for a webinar on the five best practices and measurement techniques to help you obtain the performance data you need to optimize your power chain. Register today!

  • Power Optimization – Can Your Business Survive an Unplanned Outage? (EMEA)

    Tue, 25 Aug 2015 14:00:00

    Most outages are accidental; by adopting an intelligent power chain, you can help mitigate them and reduce your mean-time to repair. Join Anixter and DatacenterDynamics for a webinar on the five best practices and measurement techniques to help you obtain the performance data you need to optimize your power chain. Register today!

  • 5 Reasons Why DCIM Has Failed

    Wed, 15 Jul 2015 10:00:00

    Historically, DCIM systems have over-promised and under-delivered. Vendors have supplied complex and costly solutions which fail to address real business drivers and goals. Yet the rewards can be vast and go well beyond better-informed decision-making, to facilitate continuous improvement and cost savings across the infrastructure. How can vendors, customers and the industry as a whole take a better approach? Find out on our webinar on Wednesday 15 July.

More link