There is a strange-looking structure on the roof of the computer-science building at Rutgers University in Piscataway, New Jersey. It is a beige sheet-metal container sitting on a raised metal platform and covered by a slightly tilted solar array. It’s called Parasol, a small data center designed as a test bed for technologies that may – in about a decade’s time – be used to manage when and how data centers consume energy.

Parasol is a brain child of Rutgers professor Ricardo Bianchini, a 45-year-old Brazilian native who has been preoccupied with data center energy use since around 2000. Before Bianchini came to Rutgers that year, there was no such thing as research of data center energy use at the university. At the time, people were worried more about energy consumption of consumer devices – battery life of laptops and cellphones – he remembers.

Bianchini convinced the research community that data center energy use was going to become a problem. Today, Rutgers’ computer-science department has its own data center testing platform and Bianchini’s small crew of professors and graduate students are thinking big: looking for and finding ways to make data processing a green endeavor. In addition to testing technologies for green computing it has already developed, the team plans to use Parasol to look deeper into free cooling, different server technologies, solid-state drives and more.

Parasol: what’s inside

Parasol currently houses two racks of servers and networking gear. It uses a combination of free cooling and direct-exchange air conditioning and has the ability to adjust dynamically how much of each is used at any time. It also has extensive power-monitoring infrastructure to measure how much energy it draws from each of its three energy sources: the solar panels, batteries and the utility grid. The scientists have full control of how much power Parasol draws from each of the sources at any given moment using three manual switches.

Each of the 16 polycrystalline solar panels can produce up to 235W. Two inverters inside the container convert DC power the panels generate into AC power. The team expects the panels to produce up to 3kW of AC power after derating. Capacity of Parasol’s batteries is 32kWh. Power is distributed through three 208V Raritan Dominion PX power distribution units, which monitor each outlet’s energy draw and can turn each of them on or off.

The two standard 42U racks contain 64 Atom-based servers, consuming an average of 20W each, and a quad-core Intel Xeon machine to store all the monitoring data Parasol collects. The team plans to add more servers and PDUs in the future. The machines are linked by two 1Gpbs Ethernet switches by HP, 48 ports each, connected to each other and to the university network.

Parasol’s cooling setup consists of a Dantherm Flexibox free-cooling unit, which consumes 110W at half speed and 450W maximum, and 19,000BTU air-conditioning unit by the same company. The latter uses about 2kW of power unless it’s too cold outside, in which case it activates its 3kW heater.

Lately their efforts have been dedicated to maximizing use of solar energy to process massive compute workloads.

Good timing

It is well known that the main technological problem with renewable energy sources, such as solar and wind, is their intermittency. A solar array generates power only when there is enough sunlight, and wind turbines only when there is enough wind. Rather than seek ways to store clean energy and use it when a data center needs it, the team has been trying to answer a simple question: What if you do the bulk of your processing when the sun is shining? Thu Nguen, associate professor at Rutgers and a key member of Bianchini’s research team, says they have done a lot of work to figure out how best to “match the workload to an energy source that is variable over time”.

Parasol was conceived as a platform to test their ideas and to generate new ones. Over the past three years, the team has published a series of papers detailing the technologies it has developed, showing how they work.

They chose to focus on solar over other forms of renewable energy because they had access to plenty of data collected from the massive solar farm on the university’s campus. The system’s current capacity is 1.4MW, and it is being extended. Many of the campus’s parking lots will get roofs in the form of solar arrays. Bianchini says the project, once complete, will create a solar-array system that produces the equivalent of 60% of all power consumed by the campus.

So far, the scientists have created two energy-management systems that automatically match workload processing times to availability of green energy: GreenSlot and GreenHadoop. The former does this in the context of high-performance computing, the second automates scheduling of workloads for the popular cluster-computing framework Hadoop. Both systems predict the amount of solar energy that will be available in the near future and schedule processing jobs accordingly. The biggest difference between the two is that GreenSlot knows how many servers and how much energy it is going to need to process a given scientific workload beforehand. GreenHadoop does not have this information upfront and does its job scheduling based on a prediction it makes. “It sort of presents a few interesting challenges that weren’t there in the context of GreenSlot,” Bianchini says. They chose to work with Hadoop because of the open-source framework’s popularity.

“It is one of the data processing frameworks that has really become very commonplace in a lot of different areas,” Nguen says. He spent a lot of time working with Hadoop five years ago as director of web crawling at Ask.com – one of the earliest popular search engines. Before Ask.com, he consulted on crawling for Teoma, a company started by another Rutgers faculty member that was eventually bought by Ask.com. “They were building a fancy new search engine to compete with Google on some ideas that came out of Rutgers,” Nguen says about Teoma.

Nguen worked as associate professor at Rutgers for seven years and then, in 2006, became associate professor. About two years into this job he joined forces with Bianchini to focus on data center energy use. Because Hadoop is becoming more and more widespread, managing the way it uses energy is a good target for them.

How GreenHadoop works

The problem with Hadoop (energy wise) is that its goal is to complete a processing workload as fast as possible, without regard for energy use, Bianchini and Nguen’s research team writes in its paper on GreenHadoop. Hadoop uses all servers in a cluster and keeps each server active even if it is not processing anything.

GreenHadoop is smarter. It takes into account availability of green energy and the cost of brown (utility grid) energy when scheduling Hadoop jobs. Instead of processing a job as fast as possible, the goal is to process a job using as much green energy as possible or, when green energy is unavailable, use brown energy when it is cheaper.

GreenHadoop offers various priority levels for job processing. Highest-priority jobs get processed right away, while lower priority jobs are delayed until green energy is available or brown energy is cheaper. Unlike Hadoop, it uses an optimum amount of servers, powering the rest down.

One of the biggest problems in building GreenHadoop was predicting the amount of time and energy it takes to process a job. Traditional batch-job schedulers know how many servers a job will need and how much time it will take before processing starts. This data is not available to GreenHadoop. It calculates average running times and energy consumption based on historical data, using aggregate workload statistics instead of information about specific jobs or applications. The system then uses this data to estimate how many servers and how much energy it will need to process a group of jobs over a set period of time.

GreenHadoop is a “wrapper” around Hadoop, which means the original Hadoop code is not altered. When evaluating a workload, the system splits jobs into two queues: Run and Wait. Hadoop executes jobs in the Run queue, and the magic really happens in the wrapper, where all Wait jobs are processed. There, the system assigns each job a start time, figures out how many servers and how much energy processing a group of jobs will require and assigns green, cheap brown and expensive brown energy to them. It uses data collected during previous processing periods to predict how much energy a group of jobs will need.

Ahead of its time?

The computer scientists at Rutgers have written research code for GreenHadoop, which means they have made it only robust enough for research. “In production, we’d have to improve it quite a bit more,” Bianchini says. But that would be up to whoever decides to productize it. “We build just enough so we can do research, and if companies want to develop these ideas into products, then they do it themselves.”

In his opinion, renewable energy production has a way to go before something like GreenHadoop becomes commercially viable. “The two main issues with solar and wind is the space that they consume and the cost,” he says. At present, these are still impediments to such systems, “but because we’re looking five-10 years ahead, we’re projecting that these [impediments] are going to be less severe.”

This article first appeared in FOCUS magazine. To register for FOCUS digital editions, click here.