Archived Content

The following content is from an older version of this website, and may not display correctly.

In our previous article on power monitoring and control we described the P states in a processor as a mechanism to regulate server power consumption. We also discovered that these mechanisms are not very useful if a reference measurement in watts is not available to measure the stat. This is done through the use of an instrumented power supply.  Servers are modular in construction, and hence these power supplies conform to the PMBus standard, making any PMBus compliant power supply interchangeable with baseboards supporting Intel┬« Intelligent Node Manager technology to implement power capping in a server.  
intel graph

Likewise, non operating sleep states may be used to extend the power proportional computing capabilities of a pool of servers beyond the maximum 100 watts possible under the current implementation of Node Manager technology, or about 30 percent of full power, down 90 percent or more using the ACPI S5 sleep state.  A server in S5 will consume less than 25 watts instead of the 250 to 300 watts consumed at full throttle.
intel graph ii

Unfortunately, being able to monitor and control power consumption in one server at a time is not immediately useful in data centers with thousands of servers deployed.  What is needed is an aggregation mechanism that replicates these basic features for a group of servers. Such a capability is available with the Intel┬« Data Center Manager ("DCM") Software Development Kit (SDK).  DCM implements a control loop analogous to the one used by Node Manager, this time for a node pool.  The DCM "sensors"are implemented by a software module that polls the group of nodes, allowing higher level software using the SDK to set a target power consumption for the whole group.  The DCM policy engine regulates the power in the individual nodes in such a way that the collective power draw of the nodes in the group matches the target power for the group.

Actually, DCM can implement much more sophisticated policies, for instance by allowing users to define sub-groups, each with individual power consumption targets, as shown below.

Unfortunately, we are still short of the initial goal set in the first article of broadening the power efficient sweet spot beyond operation at maximum demand.  The missing link is an ability to link up application performance to the regulation of power consumption.  This can be done by higher level application management software overseeing the main application's key performance indicators or KPIs.  There is nothing esoteric in this principle: a server pool may have a collective capacity to deliver a certain number of millions of transactions per second.  If application demand goes up and down through the day, the application manager can enforce specific capping or temporary shutdown policies.  The easiest way of implementing these capabilities is by delegating them to DCM.  Control may not be as precise as applying it to individual nodes, but using DCM and not having to code policies for individual nodes can save implementation engineers considerable time.  The figure below illustrates this scheme.

The goal for the scheme above is to regulate the system to a target SLA as set forth in our first article and address the "Goldilocks"challenge: not setting the delivered transactions beyond the contracted SLA because there is no payoff to the service provider for going over, yet not going under to ward off possible customer complaints.

From a software engineering perspective, the big picture is illustrated in the figure below.  At the bottom-most level, Node Manager communicates with the CPU using hardware protocols.  The goal is to map chip-to-chip constructs to a standards based interface, IPMI (Intelligent Platform Management Interface) or DCMI (Data Center Management Interface.)  These protocols consist of especially encoded IP packets.  These packets get routed to the Management Engine through a server's assigned management port.  The IPMI and DCMI standards define the bit fields for these packets, so coding in IPMI or DCMI is not too much different than coding in machine language. As with machine language, IPMI and DCMI provide very fine grained control to access server platform power management features, but can be tedious to program and debug.

Available tools such as ipmitool or dcmitool raise the abstraction level one notch, but still not above that of assembly language.  DCM provides a Web Services interface to access its features.  

As Node Manager deployments increase, library bindings for multiple programming languages will become available as well, filling the niche for OS and application-level APIs.

Part 1 of this series: The power management landscape 
Part 2 of this series: Server power management