Almost all electrical power delivered to the IT load in a data center ends up as waste heat that must be removed to prevent overheating. Virtually all IT equipment is air-cooled, meaning each piece of IT equipment takes in ambient or cooled air and ejects it as waste heat. A data center may contain thousands of IT devices, resulting in as many hot airflow paths representing the total waste heat output. The air conditioning system has to capture this flow efficiently and eject it from the room.

Traditional approach
The historical method for data center cooling is to use perimeter units that distribute cold air under a raised floor with no form of containment. This is known as targeted supply and flooded return air distribution. Air conditioners provide raw cooling capacity and serve as a large mixer, constantly stirring and mixing air in the room to bring it to a homogeneous average temperature to prevent hot spots from occurring.

This process is effective as long as the power needed to mix the air is a small fraction of the total data center power consumption – generally when the average power density in data is about 1 to 2kW per rack. With power densities of modern IT equipment pushing peak power density to 20kW per rack or more, simulation data and experience show traditional cooling with no containment no longer functions effectively.

There are design approaches that focus on room-, row- and rack-based cooling, integrated to minimize air mixing. This provides better predictability, higher density and higher efficiency as well as other benefits.

Every data center air conditioning system has two key functions: to provide the bulk cooling capacity and distribute the air to the IT loads. The first function is the same for room-, row-, and rack-based cooling, namely that the bulk cooling capacity of the air conditioning system in kW must exhaust the total power load (kW) of the IT equipment. The various technologies to provide this function are the same whether the cooling system is designed at room, row, or rack level. The major difference lies in how each performs distribution of air to the loads. Airflow is only crudely constrained by room design. The actual air flow is not visible in implementation and can vary considerably between different installations. Controlling the air flow is the main objective of the different cooling system designs.

Three basic architectures are shown in the generic floor plans depicted in Figure 1 below. The black square boxes represent racks arranged in rows and blue arrows represent the logical association of the computer room air handler (CRAH) units to the loads in the IT racks. The physical layout of the CRAH units may vary. With room-based cooling, the CRAH units are associated with the room; with the row-based cooling the CRAH units are associated  with rows or groups, and with rack-based cooling CRAH units are associated with the individual racks.


Figure 1: Floor plans show the basic concept of room- row- and rack-based cooling architecture.

The blue arrows indicate the relation of the primary cooling supply paths to the room

Room-based cooling
The CRAH units are associated with the room and operate concurrently to address its total heat load. Room-based cooling may consist of one or more air conditioners supplying cool air completely unrestricted by, for example, ducts, dampers and vents. The supply and/or return may be partially constrained by a raised floor system or overhead plenum.

This design is affected by the unique constraints of the room, including shape, ceiling height, obstructions above and under the floor, rack layout, CRAH location and distribution of power among the IT loads. Uncontained supply and return paths will result in poor performance prediction and performance uniformity, particularly as density is increased. And in many cases the full rated capacity of the CRAH cannot be used.

Row-based cooling
The CRAH units are associated with a row and are assumed to be dedicated to a row for design purposes. The CRAH units may be located in between the IT racks or mounted overhead. Compared with the traditional uncontained room-based cooling the airflow paths are shorter and more clearly defined. In addition, airflows are much more predictable, all rated capacity of the CRAH can be used and higher power density achieved. The reduction in the airflow path length reduces CRAH fan power, increasing efficiency. This is beneficial because in many lightly loaded data centers the CRAH fan power losses alone exceed total IT load power consumption.

A row-based design allows cooling capacity and redundancy to be targeted to the needs of specific rows. For example, one row of racks can run high-density applications such as blade servers, while another row satisfies lower power density applications such as communication enclosures. Furthermore, N+1 or 2N redundancy can be row specific.

For new data centers less than 200kW, row-based cooling should be specified and can be implemented without a raised floor. For existing data centers row-based cooling should be considered when deploying higher density loads of 5kW and above per rack. Systems can be configured as hot-aisle containment to extend power density capability. This further increases performance predictability by eliminating the chance of air mixing, as well as being immune to the effects of room geometry or constraints. It also simplifies the specification and implementation of designs, particularly at densities of more than 5kW per rack.

Rack-based cooling
CRAH units are associated with a rack and are assumed to be dedicated to a rack for design purposes. They are directly mounted to or within the IT racks. Compared with room- or row-based cooling, the rack-based airflow paths are even shorter and exactly defined, so air flows are immune to any installation variation or room constraints. All the rated capacity of the CRAH can be used and the highest power density of up to 50kW per rack can be achieved.

In addition to extreme density capability, the reduction in the airflow path length reduces the CRAH fan power required, increasing efficiency. A rack-based design allows cooling capacity and redundancy to be targeted to the actual needs of a rack. Additionally, N+1 or 2N redundancy can be rack specific.

The deterministic geometry of rack-based cooling gives rise to predictable performance that can be characterized by the manufacturer. This allows simple specification of power density and design to implement the specified density. Rack-based cooling should be used in all data center sizes where cooling is required for standalone, high-density racks. This approach requires a large number of air conditioning devices and associated piping, particularly at lower power density, when compared with the other approaches.

Hybrid cooling
There is nothing preventing room-, row- and rack-based cooling from being used together, as shown in Figure 2, below. This is considered a hybrid approach and benefits facilities operating with a broad spectrum of rack power densities.


Figure 2: Floor layout of a system using room- row-and rack-based architectures simultaneously

An effective use of row- and rack-based cooling is for density upgrades within an existing low density room-based design Groups of racks are retro-fitted with row- or rack-based cooling systems, rendering the new high density racks thermally neutral to the existing room-based cooling system. Another example is the use of a chimney rack cooling system to capture exhaust air at the rack level and duct it directly to a room-based cooling system. This has some of the benefits of a rack-based cooling system but can integrate into an existing or planned room-based cooling system.

Conclusion
Uncontained room-based cooling has practical and technical limitations for next-gen data centers. The need to adapt to changing requirements, reliably support high- and variable-power density and reduce electrical power consumption and other operating costs has directly led to containment strategies.

Contained room-, row- and rack-based cooling provide the flexibility, predictability, scalability, reduced electrical power consumption, reduced TCO and optimum power availability for next-gen data centers. It is anticipated data centers will use a mix of these methods. Rack-based cooling will be used in situations where extreme densities, high granularity or deployment or unstructured layout are key drivers. Uncontained room-based cooling will remain an effective approach for low-density applications where change is infrequent. For most users with newer, high-density server technologies, contained room- and row-based cooling will offer the best balance of predictability, high-power density, adaptability and the best TCO.

This article first appeared in Focus 35. To download a digital edition click here.