Energy Efficient Data Centre Design Presenter: Tony de Francesco B.Eng Mech (Hons. Class 1) Systems Engineer ENERGY EFFICIENT DATA CENTRE DESIGN

Meet STULZ Worldwide Since its founding in 1947, STULZ has specialised in areas requiring technological expertise and entrepreneurial flexibility. ENERGY EFFICIENT DATA CENTRE DESIGN

STULZ Australia - Business Areas As an air conditioning specialist STULZ offers a wide range of customer solutions for the high-tech areas of IT, telecommunications and automation. In conjunction with an extensive range of precision air conditioning units, STULZ also manufacture a range of internal and external chillers for precision and comfort applications. Precision Air Condition Units Humidification Systems Standalone Steam Humidification Ultrasonic Humidification For Close Control Areas, such as Museums, Archives, Art Galleries, Storage Rooms, Testing Laboratories, and Process Areas. STULZ provides 24/7/365 days a Year service and support through our factory trained service technicians, local spare parts and a range of Comprehensive Service Agreements. National 1300 Support Hotline Customer Service STULZ also manufacture and supply equipment for cooling cabinets, industrial cooling and High Density Cooling Solutions. High Density Cooling Solutions ENERGY EFFICIENT DATA CENTRE DESIGN

Topics Room Design CRAC Unit Design 3. CHW System Design ENERGY EFFICIENT DATA CENTRE DESIGN

1. Room Design ENERGY EFFICIENT DATA CENTRE DESIGN

Source: ASHRAE - Thermal Guidelines for data Processing Environments, pg 10 ENERGY EFFICIENT DATA CENTRE DESIGN

Rack Cooling Basics Source: HP ENERGY EFFICIENT DATA CENTRE DESIGN

Typical Computer Room Air Distribution 22°C @ 50%RH Average, say 20°C @ 57%RH 35°C @ 23%RH 13°C @ 88%RH ENERGY EFFICIENT DATA CENTRE DESIGN

Required Air Flow Rate: V = 100,000 / (1.21 x 15) Rack Cooling Example ΔT 20 racks @ 5 kW/rack = 100 kW Therefore if, Q = 100,000 W DT = (35 – 20) = 15K Let ρA x cPA = 1.21 kJ/m³K Q V Required Air Flow Rate: V = 100,000 / (1.21 x 15) => 5,500 l/s total or 275 l/s per rack ENERGY EFFICIENT DATA CENTRE DESIGN

Typical Raised Floor Air Distribution Design 9200 l/s @ 22°C 9200 l/s @ 13°C 1750 l/s @ 35°C 3750 l/s @ 13°C 5450 l/s @ 13°C @ 35°C 5500 l/s @ 35°C 5500 l/s @ 20°C 4.67 l/s @ 7°C 12.5°C 20 racks x 5kW ea Q = 100kW Power In: 7.2 kW @ 50Pa ESP Power In: 34 kW @ COP = 3.16 Total Power In = Fan Power + Chiller Power = 7.2 + 34 = 41.2 kW for 100 kW of rack load => η = 2.43 ENERGY EFFICIENT DATA CENTRE DESIGN

Direct Rack Air Cooling Total Power In = Fan Power + Chiller Power 5500 l/s @ 35°C 5500 l/s @ 20°C Smaller Chiller! 5500 l/s @ 35°C 5500 l/s @ 20°C Smaller CRAC! 4.42 l/s @ 12°C 17.5°C 20 racks x 5kW ea Q = 100kW Power In: 1.8 kW @ 50Pa ESP Power In: 27 kW @ COP = 3.77 30% Power Reduction! Total Power In = Fan Power + Chiller Power = 1.8 + 27 = 28.8 kW for 100 kW of rack load => η = 3.47 ENERGY EFFICIENT DATA CENTRE DESIGN

Underfloor Supply Air Distribution Source: ASHRAE - Datacom Equipment Power Trends and Cooling Applications, Page 34 ENERGY EFFICIENT DATA CENTRE DESIGN 1

Ducted Return Air Distribution Source: ASHRAE - Datacom Equipment Power Trends and Cooling Applications, Page 35 ENERGY EFFICIENT DATA CENTRE DESIGN 1

Ducted Supply & Return Air Distribution Source: ASHRAE - Datacom Equipment Power Trends and Cooling Applications, Page 35 ENERGY EFFICIENT DATA CENTRE DESIGN 1

Example: Stulz Rack Air Removal Unit How Does It Work? Fan assisted system that draws air through rack from either the raised floor or from the cold isle and discharges the air out of the top of the door Features Airflow of 830l/s delivers ~15kW of cooling Returns air back to room at high temp, say 35°C ENERGY EFFICIENT DATA CENTRE DESIGN

Liquid Cooled Racks Source: ASHRAE - Datacom Equipment Power Trends and Cooling Applications, Page 38 ENERGY EFFICIENT DATA CENTRE DESIGN 1

Stulz Liquid Cooled Rack Solutions CyberChill Integrated Rack Cooling Solution RAW Rear Door Air/Water Heat Exchanger Racks designed for 10-22kW of heat load ENERGY EFFICIENT DATA CENTRE DESIGN

CyberChill: Basic Function cooling water connection ENERGY EFFICIENT DATA CENTRE DESIGN

RAW Module: Basic Function – Closed Loop RAW Module mounts to the rear door of the rack of your choice cooling water connection ENERGY EFFICIENT DATA CENTRE DESIGN

RAW Module: Basic Function – Open Loop RAW Module mounts to the rear door of the rack of your choice cooling water connection ENERGY EFFICIENT DATA CENTRE DESIGN

Stulz Liquid Cooling Intrastructure ENERGY EFFICIENT DATA CENTRE DESIGN

2. CRAC Unit Design ENERGY EFFICIENT DATA CENTRE DESIGN

Cooling Capacity: What does total & net sensible mean? Input Conditions Function Return Air 24° C; 45 % r. H. 8.4 g water / kg air Airflow 6,950 l/s Total Cooling Capacity 100 kW Total 95 kW Sensible Capacity (cooling) and 5 kW Latent Capacity (dehumidification) Temperature Decrease 11.4° C Humidity Decrease 0.3 g / kg Cooling Air 12.6° C; 89 % r. H. 8.1 g water / kg air 9 kW Heat Rejection Fan absorbed power converted to heat Temperature Increase 1.1° C Cooling Air 13.7° C; 83 % r. H. 8.1 g water / kg air 86 kW Net Sensible Cooling Capacity ENERGY EFFICIENT DATA CENTRE DESIGN

Effect of Supply Fan Efficiency Traditional belt driven fan technologies can reduce the available cooling capacity by as much as 10%! Up to 30% reduction in fan power can be achieved by using highest efficiency direct driven fan technologies in lieu of typical belt driven fans. A reduction in fan power of only 1kW would result in an energy cost saving of over $1000 p.a. and reduction of 11.8 tonnes CO2/yr (@ $0.10//kWhr and COP = 4) ENERGY EFFICIENT DATA CENTRE DESIGN

High Efficiency Direct Driven EC Fans Electronically commutated (EC) permanent magnet DC motor Variable Speed Driven Rotary motion of the motor is achieved via electronically switching device (electronic commutator) and not via mechanical carbon brushes and therefore wear-free requiring no maintenance Efficiencies of up to 92% An EC motor is simply a DC motor without mechanical brushes and commutator rings. EC Technology began almost 30 years ago in computer and telecom applications. ENERGY EFFICIENT DATA CENTRE DESIGN

Benefits of EC Fan Technology Reduced fan power provides increased real (net) cooling capacity and reduces load on refrigeration plant Variable airflow rate capability allows rates to be set at commissioning for optimum levels (rather than nominal levels) Non-Maintenable item: no v-belts or pulleys – Reduction of life cycle costs Inbuilt Softstart operation eliminating high inrush current. Particularly helpful in diesel generator mode. ENERGY EFFICIENT DATA CENTRE DESIGN

Benefits if EC Fan Technology (DX Operation) Data   A.C Belt Driven Fan EC Direct Driven Fan Airflow per Unit l/s 6,665 @ 20Pa Cooling Capacity (Total) per Unit kW 101.5 Cooling Capacity (Sensible) per Unit 94.6 Compressor Power Consumption per Unit 22.0 Evaporator Fan Power Consumption per Unit 9.2 5.9 Condenser Fan Power Consumption per Unit 3.1 COP 4.6 EER 2.7 Operating Hours per Year per unit hrs 8,760 Energy Pricing $/kWhr $0.10 Average Annual Compressor Operation 100% 96% Total Energy Consumption kWhrs/yr 300,293 263,256 Comparative Energy Costs $/yr $30,029 $26,326 Comparative Savings in Energy Costs $3,704 Comparative Savings in CO2 tonnes/yr 40.0 Return Air Setpoint: 24°C @ 45%RH ENERGY EFFICIENT DATA CENTRE DESIGN

Dual Fluid System Each individual CRAC unit has two independent cooling systems Possible cooling combinations include: CHW / CHW Water Cooled DX / CHW Air Cooled DX / CHW Cooling mode can be optimised to provide most efficient method of cooling Dual fluid system also provide increased levels of redundancy ENERGY EFFICIENT DATA CENTRE DESIGN

Benefits of Dual Fluid System Reduced energy consumption by sourcing chilled water from more efficient central plant whilst providing high operational safety/system availability Example Operating Logic: B/H Mode A/H Mode Emer. Operation Peak Load Operation Office A/C On Off Chiller CHW Circuit DX Circuit Compressor Condenser Office A/C Precision A/C Office A/C Office A/C ENERGY EFFICIENT DATA CENTRE DESIGN 1

3. CHW System Design ENERGY EFFICIENT DATA CENTRE DESIGN

Typical Data Centre Configuration CRAC 1 CRAC 2 CRAC 3 CRAC 4 5,000 l/s 5,000 l/s 5,000 l/s 0 l/s TOTAL: 35,000 l/s 5,000 l/s 5,000 l/s 5,000 l/s 5,000 l/s CRAC 5 CRAC 6 CRAC 7 CRAC 8 Duty / Standby Operation (N+1) ENERGY EFFICIENT DATA CENTRE DESIGN

Optimal Data Centre Configuration CRAC 1 CRAC 2 CRAC 3 CRAC 4 4,375 l/s 4,375 l/s 4,375 l/s 4,375 l/s TOTAL: 35,000 l/s 4,375 l/s 4,375 l/s 4,375 l/s 4,375 l/s CRAC 5 CRAC 6 CRAC 7 CRAC 8 CHW Fan Management Mode (N+1) ENERGY EFFICIENT DATA CENTRE DESIGN

Chilled Water Fan Management Airflow: 2 x 8,000 l/s Fan absorbed power: 2 x 8.0 = 16.0 kW Combined noise (@2m): 2 x 65.1dBA = 68.1dBA Example: 3 x ASD1500CW R/A = 24°C/45% CHW = 7/12°C water Standby 2 x 106.0 = 212.0 kW net sensible 3 x 70.7 = 212.1 kW net sensible Airflow: 3 x 4,850 l/s Fan absorbed power: 3 x 1.8 = 5.4 kW Combined noise (@2m): 3 x 54.5dBA = 59.3dBA Benefits: Energy savings Reduced noise levels Improved air distribution Dampers not required Annual Energy Savings: Fan Savings: 10.6 kW = $9,286/yr Est. Chiller Savings: 2.7 kW = $2,321/yr * Energy cost rate: 0,1 $/kWh & Chiller COP = 4 $11,607 / yr 125 tonnes CO2 / yr ENERGY EFFICIENT DATA CENTRE DESIGN

CASE STUDY: Westpac Data Centre Existing Units with Belt Drive Centrifugal Fans EC Fan Technology (N+2 Operation) (Energy Saving Mode) Total No. of Units Installed 9 No of Duty Units 7 Airflow per Unit (l/s) 4481 @ 100 Pa 5,055 @ 100 Pa 3,650 @ 100 Pa Total Cooling Capacity (kW) per Unit 65.4 60.7 46 Sensible Cooling Capacity (kW) per Unit 62.8 Fan Power Per Unit (kW) 5.7 3.5 1.6 Net Total Cooling Capacity (kW) 418 400 Net Sensible Cooling Capacity (kW) Total Airflow (l/s) 31,367 35,385 33,147 Total Fan Power (kW) 39.9 24.5 14.4 Refrigeration Power at system EER=4 (kW) 114.5 106.2 103.5 Total Power (kW) 154.4 130.7 117.9 Hours of Operation (h) 8,760 Total Energy Consumption (kWh) 1,352,106 1,145,151 1,032,804 Unit Energy Cost ($/kWh) 0.10 Annual Energy Cost ($) $135,211 $114,515 $103,280 Annual Energy Cost Difference ($) - -$20,696 -$31,930 Annual Reduction in CO2 (tonnes per year)  - 223.5 344.8 ENERGY EFFICIENT DATA CENTRE DESIGN

Summary: Identify user’s needs in order to best select equipment capacities/capabilities Consider energy impact of design/procurement decisions Select all system components (including IT Racks) to optimise system efficiency The design process must include input from, and coordination of, all facets of the design, i.e. mechanical, electrical, comms, IT suppliers, etc. Ensure design operation is documented and all staff (facilities management & IT) are trained in correct operation of system ENERGY EFFICIENT DATA CENTRE DESIGN

...thank you ENERGY EFFICIENT DATA CENTRE DESIGN