2. Improving Cooling efficiency in tomorrow's data centre
Kevin Hughes, APC Senior Systems Engineer

3. Agenda The Problem: High Density IT Equipment
Legacy Cooling System Limitations
The Solution: APC InfraStruXure In-Row Cooling Innovation
The Benefits of IR Cooling Methodology
Summary

4. The Problem Average rack in a typical data center is under 2 kW
Full deployment of high density servers (10-20 kW per rack) would greatly exceed the power and cooling ability of the typical data center
Providing 10-20kW of cooling per rack is technically impractical to infeasible using conventional methods
Survey by University of California at Berkeley
In 2002, Average power densities found in data centers in the US were;
Average facility had 1.1 kW / rack avg (35W/sqft) and 3kW/rack max
90% of facilities were < 1.8 kW (60w/sqft) with 6kw/rack max
The highest density facility in the survey was found to be 6kw/rack average (200w/ft2) with 12kW/rack max

5. What is the industry saying ?
“You cannot keep it a secret, performance-per-watt will become critical over next decade. […] for every watt that goes into a processor you will use another watt to cool it“,
Steve Prentice, research VP at Gartner June 2006
"There's no question that the issue of power and cooling is a growing concern. The assumptions used for building data centres have been blown away" ,
John Humphreys, IDC Analyst

6. Industry statistics Power & cooling #1 issue for CIOs
70% of respondents recently said power and/or cooling issues are now their single largest problem facing their datacenters
Projections are 48% of IT budget spent on energy, up from 8%
Source: Gartner CIO Survey

7. Legacy Limitation: Unpredictable Performance
Legacy Architecture
Redundancy:
Any unit failure results in loss of cooling to an area
N+1 at room level does not provide adequate cooling in failure modes
This image shows samples of CFD analysis output. We are looking at a sectional plane slicing through the room at a height of 5’6”. The color scale indicates dry bulb temperature. (point out hot aisles / cold aisles, referencing previous slide plan view as necessary). In the bottom right image, all 5 cooling units are running, and the room works. “The room works” is determined by judging against IT equipment inlet temperature criteria set forth by ASHRAE TC9.9 document, or, 68-77F. There are 5 failure modes to consider before making statement that the room is N+1 redundant. We’ve only shown 3 here, but the point is that a unique and UNPREDICTABLE re-configuration of airflow occurs when we shut off one cooling unit. From a room perspective, a heat balance still exists, but from a rack’s perspective, localized effects may result in excessive temps and hence, premature IT equipment failure. The point to make here is that;
1.) we didn’t get the redundancy that is often assumed to exist
2.) complex and expensive analysis is required to determine the performance of the room, and even worse…
3.)the analysis is invalidated by any change to the ‘as builts’. Roll in one new rack of servers, and the analysis must be redone.
Shouldn’t it be possible to develop a cooling architecture which yields predictable performance without expensive one-time engineering?
Sectional 5’-6” from Raised Floor

8. Cooling capability limitations of Legacy Architecture - Floor Tiles
Floor Tile Cooling Ability
Requires careful raised floor design, careful CRAC placement, and control of under-floor obstacles
cfm
Perf tile 12
Typical Capability
With Effort
Grate tile
Extreme
Impractical 10 8
Rack Power (kW) 6
Blade Servers
Standard IT Equipment
that can be cooled by one tile with this airflow 4
There are 2 important concepts shown by this slide
1.) Achieving per-tile airflow above 300 cfm requires special effort, including careful raised floor design, CRAC placement, and the control of under-floor obstacles such as pipes and wiring
2.) The slope of the lines for blade servers and standard IT gear has slope in unit’s of kW / cfm. This is an indicator of how much heat the designers of the IT gear intended for the equipment to dissipate for each cfm of airflow ingested. Blade servers have a higher slope, indicating they can throw off more heat per unit of volumetric airflow. This is generally a benefit and explained in more detail in notes on next slide. However, it should be emphasized that since blade servers are so much smaller, we can fit more of them in a single IT rack, and therefore the total cfm can still be very high. 2
[47.2] [94.4] [141.6] [188.8] [236.0] [283.2] [330.4] [377.6] [424.8] [471.9]
Tile Airflow
(cfm) [L/s]

9. Supplemental Solutions
Effective only in the short term
A new & different system to purchase, install, learn about, operate & maintain
More failure points
Increased Operator Error Risk
Relatively Inflexible
Redundancy is Questionable
Pretty self explanatory – there is only room for one XDO, or one ARU on a rack. What happens when it’s capacity gets exceeded, if it fails. You’ve put 2 systems (the XD_ and the perimeter units for example) in logical series – failure of either one now takes down the whole. Operator error is already #1 cause of downtime in DC, about 60%+ according to the Uptime Institute. What do you think will happen to operator error when we add complexity to the cooling system?? Sure doesn’t go down!!

10. The Solution: In-row rack-coupled CRAC

11. CFD model of in-row system:
Modeling failure of one CRAC

12. Basic building block: rack form factor CRAC
N+1 hot-swap variable speed fans
Captures hot air exhaust from nearby IT racks
Overhead or under-floor piping
Works with any brand of IT cabinet
Add units for redundancy or density
Opening Comments:
The InfraStruXure CDU has a couple of related products that make up the system in many cases.
Air Cooled Modular Chiller:
An APC Modular Air Cooled Chiller can be selected for use with the CDU as the method of heat rejection for the cooling fluid.
InfraStruXure InRow RC & SC:
The CDU is the fluid distribution method that APC recommends for use with the InfraStruXure InRow RC or SC.
The InfraStruXure InRow RC or SC is used to cool high density heat loads in medium to large datacenters, by positioning the CRAC closer to the Heat Source.
InfraStruXure Cooling Distribution Unit

13. InRow RC Product Design Benefits
Dual A-B Power Inputs
Power redundancy and protection
Variable Speed Hot Swappable Fans
Rightsizes cooling capacity, energy savings
Top or Bottom Piping Connections
Cooling Coil
Removes heat using chilled water
Air Filter
Removes airborne particles, protects coil
Opening Comments:
The following illustration displays the many benefits of the InRow RC. ~
~Variable speed hot swappable fans reduce energy consumption during off-peak cooling periods. These fans are easily replaced while the unit is in operation, and do not warrant a service call.
~A dual A-B power input, when plugged into redundant power sources, maximizes uptime.
~The cooling coil has been designed to maximize heat transfer utilizing a copper tube, aluminum fin construction, and chilled water as the source for heat rejection,
~Chilled water can be piped through the top or bottom of the unit, thus adapting to any IT environment.
~As APC enhances our data center management capability we are striving to provide more visibility to the IT manager. As a result a flow meter has been incorporated into the InRow RC to provide the IT manager with the output cooling capacity in kW.
~A fully modulating valve is microprocessor controlled to automatically direct the proper amount of chilled water into the cooling coil based on the cooling demand. The piping within the unit is set up to allow for the valve to be field configuration for 2-way or 3-way operation.
~The air filter removes airborne particles in order to protect the cooling coil. The InRow RC comes standard with a cleanable mesh filter. An optional 2 inch 30% efficient filter is available.
~Casters allow for mobility and easy installation.
~The unit is equipped with a primary and secondary drain pan to collect any condensation that may occur from the cooling coil. Condensation is detected in the drain pan using redundant float switches. Once condensation is detected, the condensate pump will remove water from the drain pan.
Flow Meter
Measures water flow to facilitate determining unit kW output
Casters
Allows unit to move easily
Field Configurable 2-Way or 3-Way Valve Operation
Condensate Management
Detects and removes condensation
InfraStruXure® InRow RC

14. In-row rack-coupled CRAC
Bayed to adjacent IT racks
Up to 60kW rating
N+1 hot-swap fans
Directly ducts hot air exhaust from connected IT rack
Front may be open or ducted
Mix into existing legacy data center
Add second unit for redundancy or capacity
Opening Comments:
The InfraStruXure CDU has a couple of related products that make up the system in many cases.
Air Cooled Modular Chiller:
An APC Modular Air Cooled Chiller can be selected for use with the CDU as the method of heat rejection for the cooling fluid.
InfraStruXure InRow RC:
The CDU is the fluid distribution method that APC recommends for use with the InfraStruXure InRow RC.
The InfraStruXure InRow RC is used to cool high density heat loads in medium to large datacenters, by positioning the CRAC closer to the Heat Source.
InfraStruXure Cooling Distribution Unit

15. In-row with hot aisle containment

16. CDU serving 12 x RC’s

17. InfraStruXure® Coolant Distribution Unit (CDU)
Main supply and return headers
Supplies and returns coolant to and from the Chiller
Isolation and balancing valve
Allows isolation for servicing and provides coolant flow adjustment
Top or bottom piping connections
Allows the CDU to be top or bottom piped
Individual supply and return lines
Supplies and returns coolant to and from CRAC units
Slide intent: Provide overview of the Product Family design and main feature/benefits
Notes for the presenter:
Direction: Clockwise beginning with the Isolation and Balancing Valve
Opening Comments
Easy to install (or move if required) since it ships pre-configured with circuit isolation valves
Water or Glycol, 1 to 12 InRow RC support, Top or Bottom Piping Configuration.
Isolation and Balancing Valve:
These valves allow individual circuit isolation and provides a means to balance the flow of fluid to multiple CRAC units.
Main Supply and Return Headers:
Provides a point of connection for the main supply and return lines to and from the chiller.
Pre-configured with 12 distribution lines, which allows scalability from 1 to 12 In-Row RC’s
Top or Bottom Piping Connections
The main supply and return connections provide the customer with the flexibility to install the piping from the chiller without the concern of the CDU connections
The individual supply and return connections can be configured for top or bottom piping,
Individual Supply and Return Lines:
Individual lines that allow for independent fluid circuiting for each CRAC.
These lines provide the last point of connection between the CDU and the CRAC to reduce the risk of leaks within the datacenter.
Note: The individual supply and return connection location must be defined when the order is placed.
Casters
Allows the CDU to be installed easily
Isolation valve
Allows isolation for servicing
Condensate pan with drain connection
InfraStruXure® Cooling Distribution Unit

19. What about installation and piping?
Water in the data centre?
What about leaks and condensation?
Complicated installation?
Retrofittable to existing data centre?
InfraStruXure® InRow RC

20. What is PEX? “Cross-Linked Polyethylene” Semi-flexible piping solution
Corrosion free, low head loss, highly reliable
Developed in 1960’s EU, 1980’s  NAM
Applications
Plumbing, district heating / cooling, hydronic radiant heating, snow melt, ice rinks, refrigeration warehouses
Standardized product
ASTM F877 (tubing), ASTM F1807 (fittings)
Approved material in all national plumbing codes
UL Flame Ratings
Why use in APC InfrastruXure™ cooling?
‘Pay as you grow’ compatibility
Increased Reliability
Position this list as factors mitigating the risk of overhead piping. Emphasize standardization messages blankets even this piping system “ as I’ll show on next slide”

21. Connections & Supports for PEX-AL-PEX
Piping bb
Standardized Fitting  Decreased Human Error

22. Alternative In-row rack-coupled CRAC Solutions

23. InfraStruXure® InRow RP
In-row precision air conditioning for medium to large data centers including high density applications.
Opening Comments:
~An unpredictable data center environment is common among IT managers. In today’s data centers, traditional cooling approaches involve complex air distribution systems that tend to be unpredictable and leave many customers guessing where the cold air goes. With the InfraStruXure InRow RP, APC has taken the guess work out of data center cooling. Placing the unit in the row of racks moves the source of cooling closer to the heat load. This minimizes air mixing and provides a predictable cooling architecture.
~The in-row architecture provides “predictable cooling solutions for an unpredictable environment” improving the availability data center.
The predictable solution for an unpredictable environment
InfraStruXure® InRow RP

24. InRow RP Product Design Benefits
Air Filter
Removes airborne particles, protects coil
Electric Reheat
Controls temperature during dehumidification
Top or Bottom Piping Connections
Cooling Coil
Removes heat using chilled water or refrigerant
Opening Comments:
The following illustration shows the many benefits of the InRow RP. ~
~Electric Reheat prevents overcooling during dehumidification cycles.
~The cooling coil has been designed to maximize heat transfer utilizing a copper tube, aluminum fin construction. There is both a chilled water and R-407C refrigerant based unit available.
~The air filter removes airborne particles in order to protect the cooling coil. The In Row RP is designed to accommodate up to a 4” 85% efficient filter for maximum filtration.
~The unit can be piped through the top or bottom of the unit, thus adapting to any IT environment.
~The Steam humidifier delivers exact amounts of humidification to maintain optimal environmental conditions.
~Variable Speed Compressor
~Casters allow for mobility and easy installation.
~Moisture removed from the air during cooling and dehumidification is pumped away to ensure continuous operation.
Steam Humidifier
Maintains moisture level in room
Variable Speed Compressor
Only on DX Version
Condensate Management
Detects and removes condensation
Casters
Allows unit to moved easily
InfraStruXure® InRow RP

25. InfraStruXure InRow SC
Air from plenum space pulled into unit via condenser fans
Heat rejected from condenser coil to plenum space
Condenser air absorbs heat from coil
Hot exhaust air from IT equipment pulled into unit via evaporator fans
Heat transferred to condenser coil
Evaporator coil absorbs heat from air
Opening Comments:
As you can see, this image shows the airflow pattern of the InRow SC air cooled self-contained air conditioner. As an InRow unit, the SC pulls in the hot exhaust air from the back for the rack, where it is pulled across the evaporator coil and cooled. The conditioned air then flows back into the room through the front of the unit. The heat removed is ejected into the ceiling plenum via the SC's condenser section. The rear duct brings in air from the plenum which then passes over the condenser coil, then that air is sent back into the plenum by the two condenser discharge ducts. The InRow SC is suitable for use on a hard floors or raised floors.
Cool air is supplied to row or room

26. High Density Case Study

27. InfraStruXure® Designer: Intelligent data centre design tool
Capacity and redundancy assessment is now possible during design
Testing rack-by-rack airflow analysis for various failure conditions

28. Summary
Your data centre’s Network Critical Physical Infrastructure (NCPI) must support dynamic business needs
High-density server deployment requires unique cooling and power solutions
In-row, close-coupled cooling maximises efficiency
NCPI Health Check increases availability and reduces TCO of existing systems
APC’s original In-Row Cooling is the most cost effective, agile solution to today’s cooling challenges
Complexity is not a necessity.
Do not tolerate unpredictable performance & inflexibility
Doing high density the right way costs less
         

29. Thank you white papers www.datacenteruniversity.com
We breezed through some of these subjects; but we have several white papers which go in detail on these.