Data centers can consume up to 100 times more energy per unit area than a standard office building and often only around 15 percent of the original source energy (at the power plant) is used by the IT equipment. The Power Usage Effectiveness (PUE) metric, first defined by the collaborative organization The Green Grid, is now widely accepted as the industry standard for understanding and improving the energy efficiency of data center infrastructure systems.
The use of PUE is the norm for large-scale data centers such as those operated by Facebook and Google, and is requested from all ICT sector participants of the Carbon Disclosure Project (CDP) and the Greenhouse Gas Protocol (GHG Protocol).
The Federal government also uses the metric as a performance indicator for its data centers. However, many data centers have complex electrical and mechanical distribution systems making it difficult and expensive for the extensive metering generally required to establish PUE. The result is that that many small-and medium-sized data centers do not calculate PUE.
There exists, however, a solution for small-and medium-sized data centers to calculate PUE and to do so without metering beyond the UPS output. This article provides guidance on how to minimize costs by relying on existing meters, manually collected data, and estimates to satisfy The Green Grid’s PUE Level 1 (PUE1): Basic Metering Requirements.
PUE1 is also the minimum requirement for data center participants of the Department of Energy’s Better Buildings Challenge (BBC), an initiative to reduce the energy intensity of partner portfolios (including data centers) by 20% by 2020. Although the more extensive metering associated with PUE Levels 2 and 3 provides more accurate measurements, PUE1 provides valuable information to reduce energy use for data centers of all sizes and levels of complexity.
An overview of PUE
PUE is a measure of how efficiently computer data center infrastructure systems use energy. Specifically, it is the ratio of total energy used by the data center to energy used by IT equipment. PUE values can range from 1, which indicates that 100% of the total data center energy is being used to power the IT equipment, to infinity. However, research shows that PUE values are usually between 1.3 and 3. According to The Green Grid, “When viewed in the proper context, PUE provides strong guidance for and useful insight into the design of efficient power and cooling architectures, the deployment of equipment within those architectures, and the day-to-day operation of that equipment.” It is best leveraged to understand an individual data center over time and as a baseline to see the effect of design and operational changes.
PUE: Total Facility Annual Energy Use / IT Equipment Annual Energy Use
Total Facility Energy Use
- Energy associated with all of the IT equipment (as described below)
- Power delivery components: uninterruptible power supply (UPS) systems, switchgear, generators, power distribution units (PDUs), batteries, and distribution losses external to the IT equipment
- Cooling system components: chillers, cooling towers, pumps, computer room air handlers (CRAHs), computer room air conditioners (CRACs), and other air handling units (AHUs) serving the data center
IT Equipment Energy Use
- Energy associated with all of the IT equipment: Compute, storage, and network equipment (measured at the output of UPS equipment for PUE1)
Although the power, cooling, and other data inputs required for PUE may seem daunting to gather, remember that calculating PUE1 relies on meters that already exist in most data centers (e.g., UPS output) plus manually acquired data and estimates.
PUE1: basic metering
The Green Grid 2019, for PUE1, “the IT load is measured at the output of the UPS equipment and can be read from the UPS front panel, through a meter on the UPS output, or, in cases of multiple UPS modules, through a single meter on the common UPS output bus. Note that there may be transformers and other electrical distribution losses between the UPS and the IT equipment that will be ignored when using PUE1.”
Typically, a facility’s existing meters consist of built-in power meter(s) at the UPS output(s) to capture IT equipment energy use. Energy measurements are usually expressed in kilowatt hours (kWh), and data should be collected at least once a month. If there are no meters at the UPS output (or equivalent in data centers without full UPS coverage) they will need to be added; please see the Data Center Metering and Resource Guide for assistance in setting up and implementing a metering plan.
The Guide also offers solutions for potential obstacles that may arise. For instance, energy numbers may not be available at the UPS display, but if there are internal UPS meters then networking cards can be installed to read the UPS outputs and report to a server. Or, the UPS output may read in power (kilowatts) and not energy (kWh), in which case an option is often available to convert the meters to display kWh. Thus, with UPS output (or equivalent) measured in kWh, IT equipment energy use is captured for both “stand-alone” and “embedded” data centers.
The process for capturing the total (data center) facility energy use and calculating the PUE number varies depending on whether a data center is stand-alone or embedded, and how much metering beyond the UPS output is available. The data center personnel can generally help with this determination. A stand-alone data center has no, or negligible, energy use for anything in the building but the data center itself. An embedded data center, on the other hand, is located in a building with significant energy use for other purposes than the data center.
Stand-alone data centers
Total facility energy use is measured at the main electrical feeder (utility electrical service entrance) of the electrical and mechanical equipment used to power, cool, and condition the data center (M1 in Figure 1).
If the stand-alone data center does not have its own meter, metering needs to be added at this location. As mentioned previously, IT equipment energy use is measured at the UPS output(s) (M2), thus for stand-alone data centers
PUE1 = M1/M2.
Embedded data centers
PUE is more complex to determine when a data center is embedded in a larger building with shared electrical and cooling systems. Some additional metering is usually recommended to improve the accuracy of the PUE estimate. Please see the Data Center Metering and Resource Guide for PUE calculations that rely on metering beyond UPS output.
Even without additional meters, PUE1 can be calculated by using metered data from the UPS output, manual spot measurements, and estimates. The PUE calculations are slightly different depending on the characteristics of the data center; for example, whether the cooling plant is water-cooled or air-cooled and whether an economizer is in place (see Table 1).
The PUE calculations in Table 1 are often performed using (homegrown) spreadsheets to manipulate the data. The quality and extent of the calculations and reports depend on the operator’s knowledge and effort.
Table 1 footnotes:
1- M2 is UPS output (kWh) from meter readings,
2- M2/.9 is based on assumption of 10% loss in electrical distribution upstream of the M2 measurement.
3- Efan is the energy used by the CRAH or CRAC fans (kWh). (It can be estimated using motor nameplate. Efan = (motor horsepower x 0.746 kW/hp x 0.75 assumed load factor x operating hours)/motor efficiency. Actual energy measurement is preferred, especially for fans with variable-speed drives; many VSDs have built-in energy and power meters, which can be read at the same intervals as the UPS meters). Depending on power distribution to the CRAC or CRAH fans, a few central points can often be used to spot-check all of the units; one such power measurement is then assumed to be typical for the year.
Edited from Table 6.8.1C - ASHRAE 90.1 – 2010
Understanding data center type
Data center drawings and interviews with site infrastructure personnel can provide both general information (whether stand-alone or embedded data center) and specifics (e.g., what meters are already in place).
Electrical single-line drawings show the electrical distribution system for the building and existing metering. Meters to look for include: Main electrical feeder(s) to the data center and for electrical input to the IT equipment (meters often exist on the UPS output or PDU output). In stand-alone data centers, these two numbers are generally all that is needed to determine the PUE.
While somewhat useful, drawings (including “As-Built” or “Record”) are notoriously inaccurate especially for older data centers. A site visit is often needed to verify actual conditions, including inspections of electrical and mechanical systems, as well as operating conditions.
Although determining PUE1 as described above has limited accuracy due to reliance on simplified and manually acquired data, it serves well as a starting point to understand and improve the energy performance of the data center infrastructure systems. Calculating PUE1 can also act as a first step towards additional metering needed for a more accurate PUE1, and the more advanced metering and reporting required by PUE2 and PUE3.