Regardless of the theoretical appeal of having data centers participate in utility demand-response programs – perhaps more acute today than ever, considering the attention to data center power consumption and cost – the area remains an esoteric one as far as the broader industry is concerned.

Today, most data center operators that operate in service areas of utilities that have peak-demand pricing (far from all of them do) simply prefer to pay the higher peak-demand power rates than risk not meeting their service-level agreements, regardless of whether that risk is real or only perceived.

If data center demand response is to become a reality, operators have to be convinced that it is not only safe but that it also has financial benefits. This is what Girish Ghatikar and a group of researchers at the US Department of Energy’s Lawrence Berkeley National Laboratory (LBNL) in California are out to do. Ghatikar oversees demand-response research at the national lab.

First real-life tests

Between October 2011 and July this year, his team ran a series of tests at four California data centers to get some real data on whether demand-response would work in the data center and to identify the best ways to shed a data center’s overall power load. In August 2012, LBNL published a paper describing the study and the results, which were quite positive. In some instances the team was able to temporarily reduce a data center’s load by as much as 25%.

Success in four relatively small data centers is not enough to convince the industry, however. Ghatikar says one of the largest barriers to further adoption of demand response in data centers is still articulating “What is the value for a data center to participate?”

Another major barrier is the lack of sophisticated tools that can automate the process of shedding load and ramping it back up after the demand-response period is over. LBNL’s tests were all done manually, each involving way more people than any typical data center operator would be willing to pay to do it.

Not convincing enough

LBNL’s results are encouraging, but they represent too small of a dataset, as the researchers themselves admit in the paper. The industry would need to see more test results from a wider variety of data centers to consider demand response more seriously. The study’s goal was to “improve the understanding of the demand-response opportunities in data centers,” as well as to test a variety of strategies to realize these opportunities in field tests.

There are many ways to shed load in a data center, and the team at LBNL tested seven of them. The strategies included shutting down IT gear and computer room air conditioners (CRACs), idling servers by postponing processing, lowering the temperature set point, reducing cooling load along with reduction in IT load, and moving compute load from one set of servers to another. The data centers they used included LBNL’s own 350kW facility and a University of California – Berkeley’s (UCB) 550kW facility.

The other two were a 145kW NetApp data center and a 1.6MW supercomputer facility at University of California – San Diego’s San Diego Supercomputer Center (SDSC). All power figures above represent average power consumed by the IT equipment.

These were not the researchers’ ideal testing grounds, but they had to work with what they could get their hands on. Their choice was limited by a number of factors, including the study’s funding sources (California State government and two of the state’s largest utilities, among others), and the reluctance of major data center end users to run experiments in their mission-critical facilities. The operators would have to “take a little bit of risk, because it hasn’t been done before,” Ghatikar says.

Not for mission-critical

The limited choice of the type of data centers used in the study is one of the things that limits application of the results across the industry. Vali Sorell, VP and national critical facilities chief engineer at Syska Hennessy Group, says some of the strategies LBNL used to shed load would not be possible in most mission-critical facilities. Syska Hennessy is a major US-based engineering firm with a massive data center practice.

“Those are entities that can afford to just drop load,” Sorell says about the study’s four data centers. “Most of my clients... are not able to plan ahead [for a drop in load] or delay IT operations.” A typical IT group watching over certain applications will not even want to hear about it. “They will not tolerate being shut down for a few hours a day,” he says. “Even if it’s a graceful shut down, it will not be tolerated.”

At the time of writing, Syska Hennessy was working on a data center with a US company that would support a high-performance computing (HPC) infrastructure. The supercomputer will run for few weeks or months at a time, analyzing oil-exploration data. “They will not tolerate putting everything on hold for a six-hour window – whether the utility tells them to or not,” Sorell says. “This may not be a realistic strategy for them to do IT load reduction.” The same is true for companies that do business online. Postponing processing is not something they can do at will.

LBNL researchers do not deny this. Which demand-response strategy will work for a particular data center will depend on that data center’s function, as well as the types of IT equipment inside its cooling system, its management, the operator’s comfort with the strategy and the strategy’s value to the data center’s customer(s), they write in the paper.

IT and cooling in tandem

The largest demand-response opportunities, Ghatikar’s team concluded, are in reducing temporarily IT load and the cooling load. The cooling load can be reduced either automatically (in response to a temperature drop on the floor that results from the IT load reduction), or manually, in which case the facility’s overall electrical load drops faster, Ghatikar says.

When the cooling system’s response was automatic, the team simply turned off a portion of servers in a data center and monitored IT and cooling load status until the end of the “disaster-recovery event.” Once all servers were back on, the cooling system responded accordingly.

In the manual scenario, after they turned off the servers, they increased temperature set points in the data center based on temperature readings incrementally and held the set points at the higher level until the end of the demand-response window. At this point, they rolled the temperature set point back to the original value first and turned the IT equipment on second.

Since shutting down even a portion of the servers is not an option at most mission-critical data centers, Ghatikar says, putting servers into idle mode would be a suitable alternative for these end users. This would mean consolidating the workload onto fewer servers for the demand-response event’s duration and idling the rest. The way LBNL went about reducing compute load on some servers, however, was by scheduling some processing jobs to run after the demand-response window on those servers.

Opportunity in load migration

Keeping mission-critical loads in mind, another promising strategy is load migration, Ghatikar says. This means moving a processing load to a data center in a different geographic area, while a demand-response event is in progress in the primary facility. “Basically, you are not compromising a single ounce of your operations,” he says. And, many data center operators have the infrastructure in place to do it: disaster recovery sites. “Use DR for DR,” Ghatikar says, the first acronym meaning “disaster recovery” and the second one meaning “demand response”.

Latency, however, prevents this strategy from being implemented by most end users. LBNL researchers moved loads between the Bay Area and San Diego, California (about 500 miles apart). “Mission-critical facilities that have a real-time need for their data… will not build parallel or synchronous facilities that are so far apart,” Sorell says. While the distance is appropriate for disaster recovery, along with it comes latency, which makes seamless compute-job migration difficult.

Another issue with this and other strategies the researchers tested are staff costs, since all the tests were done manually. They took the manual route to understand the sequences required for each strategy and to make it easier to identify potential issues, Ghatikar explains. No private company is going to pay for the amount of staff necessary to implement these strategies manually, he says. “That’s not going to happen.” Another key to adoption of demand response by data centers is technology that automates the process and does it reliably.

 Learning to migrate seamlessly

One of the companies working on such automation technology is Power Assure. The data center infrastructure management vendor contributed technology and time to LBNL’s research effort, but the researchers only used its monitoring (of both IT and cooling loads) and analytics capabilities. Jim McCray, Power Assure’s VP of marketing, says the company is actively working on solutions for data center demand response.

While it has some capabilities in this area, it has not developed a demand-response-specific product just yet. Had a customer come to them and asked for one, “we would do a customized solution at this point in time,” McCray says. The client’s ability to participate in demand-response events would still be dictated by whatever the data center is doing during those peak-demand timeframes. “There are times when there’s no way you’re going to do demand response,” he says.

 This was LBNL’s second major study of demand response in data centers, building on the first one Ghatikar and his team conducted in 2010. The team fully acknowledges that a more diverse group of data centers would have to test the results to validate them. More research into integrating data centers with electrical grids through automation needs to be done, they write, as well as other areas.

 If data centers learn to migrate processing jobs between remote sites seamlessly, not only can they save on peak-demand pricing, more opportunities to use renewable energy will open as well. Wind and solar are intermittent power generators, but an adequate load-migration technology may potentially move the load to where most renewable power is being generated at any given moment. Power Assure sees load migration as having the biggest future in data center demand response and works on developing the appropriate technology. “That’s where we see this thing going,” McCray says.

 A version of this article first appeared in DatacenterDynamics FOCUS magazine. Visit the FOCUS registration page for a free subscription.