Choices for data center cooling architectures

Choices for data center cooling architectures

Higher power densities and the need to lower the data center’s electric bill demand more flexible, predictable cooling schemes

BY WENDY TORELL and VICTOR AVELAR
APC, West Kingston, RI
http://www.apc.com

Those who are planning to roll out new IT equipment—or an entire data center—should be mindful of what type of cooling will be required now and in the future. That’s because the latest high-density IT equipment creates conditions that room cooling was never intended to address, resulting in cooling systems that are inefficient, unpredictable, and in some cases ineffective. To address these problems, row- and rack-oriented cooling architectures have been developed.

Air-conditioner functions

In a data center, all air conditioners serve two functions: to create cool air (that is, provide the kilowatt cooling capacity required) and to distribute it to critical loads. There are several ways to create the bulk cool air capacity—chilled water, air-cooled, water-cooled, or glycol cooled—and any can all be used regardless of the architecture chosen.*

It is the second function, distributing cool air, that differentiates the architectures into three types. In the room-oriented architecture (Fig. 1, left), the computer room air conditioner (CRAC) units are associated with the room; in the row-level architecture (Fig. 1, center), the CRAC units are associated with rows; and with the rack-level architecture (Fig. 1, right), CRAC units are assigned to the individual racks.

Fig. 1. Floor plans illustrating room- (left), row- (middle), and rack-based (right) architectures.

The room-oriented cooling architecture consists of one or more air conditioners that either supply cool air to the equipment in a completely unrestricted fashion—without ducts, dampers, vents, or other means of flow control—or they can supply and/or remove air in a way that is partially constrained, say by a raised floor system or an overhead return plenum.

For many small computer rooms that use a room-oriented architecture, minimal attention is paid to airflow, and racks are sometimes laid out in an unplanned arrangement. For larger or more sophisticated data centers, raised floors may be used to direct and align the air into well-planned hot-aisles and cold-aisles

But even with a well thought-out layout, the effectiveness of this cooling architecture is heavily dependent on unique constraints of the room, such as the shape of the room, obstructions under the floor, location of air conditioners, and how the IT loads are arranged in the room. With such constraints, and with the constant additions and rearrangements that most data centers experience, the cooling predictability of this approach comes into question, especially as power densities increase.

Rack and row

With the row-oriented cooling architecture, the cooling systems are now brought closer to the row of equipment racks, with air conditioners dedicated to each row. By placing the air conditioners among the IT racks, or over the cold aisles (Fig. 2), the airflow paths are much shorter and more clearly defined, as compared to the room-oriented approach. Airflows become much more predictable, and higher power densities can be achieved. In addition, this architecture can be implemented without the expense of a raised floor.

Fig. 2. Row-oriented cooling can employ in-row (left) and/or overhead (right ) cooling.

In the rack-oriented cooling architecture, the CRAC units are dedicated to an IT rack. Airflow paths are even shorter than row-oriented cooling, and the airflows are completely immune to any room constraints. This architecture allows for rack power densities up to 50 kW, and has the flexibility to specify capacity and redundancy cooling requirements to the actual needs of specific racks. The main drawback to this approach is the large number of air conditioners needed, compared to the other approaches.

It isn’t necessary to adopt one of these architectures exclusivelyusing a combination proves beneficial in many instances. This is especially true when a data center operates with a broad range of power densities and system-availability requirements.

A typical scenario in which multiple architectures make sense is upgrading a low-density data center. Adding row- or rack-oriented architectures to supplement the existing room-oriented design lets the datacenter accommodate high-density loads without the need to modify the existing cooling system.

Architectures compared

In evaluating architectures, there are five core system characteristics that must be considered: agility, availability, life-cycle cost (TCO), serviceability, and manageability.

With ever-changing requirements such as increasing and unpredictable power densities, the system’s agility becomes increasingly important. Room-oriented cooling can be complex to upgrade or adapt since it is typically built out in advance and is unpredictable without complex CFD analysis. Further, any changes in the room can lead to insufficient cooling of critical loads, requiring extensive re-engineering to solve problems.

On the other hand, row- and rack-oriented cooling, being modular in nature, offer predictable deployment in row- or rack-level increments to target specific densities of unique loads within the data center. When data center changes/additions occur, new loads can be completely isolated from the existing cooling system.

Room-based cooling also presents a set of availability challenges. The more closely a system can “couple” heat removal to heat generation (the loads), the more the system can eliminate air mixing, to avoid hot spots in the data center. Row- and rack-oriented systems can do this more effectively than room-oriented systems can.

In addition, N + 1 redundancy is common in data centers to remove single points of failure, but the way this redundancy is provided is different for the three different architectures. With rack-oriented cooling, which is basically dedicated cooling for each rack, the only way to achieve N + 1 redundancy is to add another CRAC—or dual-path CRAC systems for each rack—a severe price penalty to pay for N + 1 redundancy across the data center, but effective for isolated ultra-high-density racks.

For room-oriented cooling, the typical thinking is that one additional CRAC around the perimeter of the room provides the required level of redundancy. However, at high densities, the ability of a particular CRAC to make up for the loss of another CRAC is strongly affected by room geometry—that is the farther the failed unit is from the spare unit, the more unlikely it is that the spare unit will provide the same amount of air flow to the affected racks.

Row-oriented systems allow for redundancy at the row level. This requires one additional CRAC for each row of racks. This would not be a cost effective choice at low power densities, say 1 to 2 kW/rack, but at higher densities up to 25 kW/rack, this becomes advantageous.

Reducing life-cycle costs, or total cost of ownership (TCO) is everyone’s goal today. Electrical costs are becoming a larger fraction of the TCO, as power densities and electric rates continue to rise.

Density drives up electrical costs because it drives down the efficiency of traditional room-oriented cooling systems. Electrical costs are consistently lower for the rack-oriented architecture, because CRACs are sized to and closely coupled with the load so unnecessary airflow is avoided (see Fig. 3).

Compared to those for room-oriented cooling, electrical costs for row-oriented systems are poor at very low densities. That is true because, at light loads, row-oriented systems require more CRACs, since one must be assigned to every row.

However, as density increases, the electrical costs with the row-oriented approach are lowest. Air paths are shorter, which means that high air-volume requirements are easily met by smaller more efficient motors. To deliver higher air volumes, room-oriented schemes would require larger motors to send air further and overcome higher frictional losses.

Fig. 3. Depending on the type of cooling architecture, electrical costs vary differently with rack power density.

With regard to serviceability and the manageability, room-oriented architectures are generally uniquely engineered solutions involving multi-vendor subsystems, unique floor layouts, and complex maintenance. Row- and rack-oriented cooling systems are designed using standardized modular components, which reduces downtime and simplifies service procedures. Row- and rack-oriented systems also provide near real-time models of current or future performance, which eliminates guesswork for capacity and change management. ■

*See “Choosing a datacenter cooling method,” electronicproducts.com/ShowPage.asp?FileName=farcapc2.sep2007.html