1. VIACOM Data Center Optimization Project
Executive Briefing
May 7, 2010
John Wallerich,

2. Agenda Review Project Requirements and Objectives
Dell’s Gated Project Methodology
ASIS State Assessment
TOBE State Analyses
APC InRow Cooling Analysis
Implementation Cost Estimates

3. Project Objectives IMPROVE Cooling Infrastructure Effectiveness
Data Center Capacity
IT Equipment Temperatures
Cost of Operations
TOOLS
Computation Fluid Dynamics (CFD)
3-D Graphical Software to model airflow and thermal dynamics

4. Cost Improvement Categories
Improvements to room density and OPEX
Cool load using less cold air volume
Reduce Bypass air
Reduce equipment Inlet air Delta-T
Raise Supply Air Temperatures
Reduce Fan Energy
Reduce DX Compressor loads
Extend useful life of this data center and avoid new build

5. Services Engagement Methodology An Interface to Investment Decision-Making
An efficient “gated” engagement approach designed to arrive at solutions with reduced cost and improved efficiency

6. Glossary of Terms BLANKING PANEL
A plastic or metal plate fitted to the front of a rack where no IT equipment is present.
BYPASS
This refers to the percentage of cold air that is generated by the cooling system, yet for one reason or another never passes through IT equipment.
CAC
Computer Room Air-Conditioner. This is a generic term that includes all of the cooling devices in the Viacom data center.
CFD
Computational Fluid Dynamics. This is the technical term for the software tool that we use to model both airflow movement and thermal dynamics for both the under-floor cold air supply plenum, the computer room itself, and if used, the hot air return plenum (above the false ceiling).
CFM
Cubic Feet per Minute. This is a measurement of air volume. For this study, this term is used to describe the amount of air processed by CACs as well as the volume of air flowing into the computer room through a perforated tile.
RETURN AIR
This is a term that refers to the air that enters to CAC cooling units to be cooled.
HOT AIR RETURN PLENUM
This represents an area above the false ceiling when it is used as part of an airflow management system.
COLD AIR SUPPLY PLENUM
This refers to the area under the raised floor where cold air is injected by the CAC units. The cold air is distributed into the room via perforated tiles.

7. Solution Options Summary
Solution Option #1: Deploy Hot Air Return Plenum
Solution Option #2: Install Airflow Management
Solution Option #3: Deploy Containment Strategy
Special Study: APC In-Row Containment Design

8. CFD Modeling
Computational Fluid Dynamics allows for the analysis of airflow and thermal dynamics with the detail and accuracy required for fine tuning of the data center cooling infrastructure

9. Under-Floor PRESSURE Dynamics
High Pressure
Low Pressure resulting from high velocity
High Pressure
Low Pressure resulting from high tile count
Low Pressure resulting from high velocity

10. ASIS: Thermal Slice at 6ft Elevation
Exhaust air contamination
Exhaust air contamination
Exhaust air contamination

11. ASIS: Max Inlet Temps and CAC % USED ASHRAE Temperature Guidelines

12. ASIS: Cooling Challenges
SAN Room
Exhaust air is flowing over tops of racks contaminating surrounding racks.
Hot air contamination
AS400’s
Racks are facing same direction so exhaust air contaminates rows behind it with increasingly warmer air.

13. ASIS: RETURN AIR STREAMS
CACs
3 CACs in center of room are drawing server exhaust from servers that are closer to other CACs

14. ASIS: AS400 AIRFLOW CONTAMINATION
Exhaust air from the front rows is directed at air inlets of rows behind them. This increases the inlet air temp and puts the back rows at risk of over heating.

15. ASIS: BEST PRACTICES OBSERVED
Use of Blanking Panels good Best Practice
Use of hot-aisle/cold-aisle allows for enhancements
Plumbing and wire trays are well implemented
Lack of combustible materials
Lead-Acid batteries not on raised floor
Site is clean and well maintained
Cable cut-outs and floor leakage is minimal

16. ASIS: Design Weaknesses
AS400 racks are not optimized (facing same direction)
SAN room suffers from very low UF pressure
Room cooling is greater than needed to compensate for weak airflow management design
CACs are of 2 different designs for high availability, yet redundancy is no longer an option
CACs are aging and lack VFDs and multi-stage compressors
Hot server exhaust contaminates server inlets as it flows towards CACs – no airflow controls
Room capacity is limited by cooling effectiveness

17. Solution Option #1: Design Basic Airflow Control: Return Plenum
By connecting the return ducts of CAC units to the area above the false ceiling, and installing return air grills above hot aisles and other heat sources, the area above the false ceiling becomes a hot air return plenum. Hot air is “pulled” from the room and returned to the CAC unit to be re-cooled. This is a foundational requirement to installing more advanced airflow control measures and yields a number of benefits.

18. Solution Option #1: Design Hot Air Return Plenum
CAC ducts connected to false ceiling turns ceiling void into a hot air return plenum
CAC Duct Extensions
Hot server exhaust air is removed from the room through ceiling grills and directed back to CACs at higher temperatures

19. Solution Option #1 Operational Statistics
Description ASIS SO#1
Number of Cabinets
365
379
Total Cooling Airflow
199,538 CFM
187,138 CFM
Total Equipment Demand
142,069 CFM
Cooling Capacity
1213kW
Cooling Capacity in Use
947kW
Maximum Inlet Temp
93F
87F
Number of Active ACUs 18
ACU Power Consumption
107.6kW
102kW
Note: Number of cabinets increased as some new racks were
added to establish row symmetry (racks in purple)

20. Solution Option #1: Benefits Impact of Enhanced Heat Removal
Hot exhaust air mixing with cool room air is reduced
Return air is warmer: CAC can reject more heat
Reduced contamination of nearby server inlets
Supply air temperature increases from 58F to 64F
Room can sustain increased IT power loads
Can cool using reduced supply air

21. Solution Option #2: Design ESD Airflow Control Curtains
Solution Option #2 introduces Electro-Static Discharge curtains to provide enhanced isolation of hot and cold air, and to direct hot exhaust air into the hot air return plenum.
These curtains are located between the tops of racks and the false ceiling and help to isolate hot and cold air masses
ESD curtains have been adopted as a low-cost yet highly effective mechanism for simple airflow management.

22. Solution Option #2: Design Install ESD Airflow Control Curtains
ESD Curtains
Note: false ceiling not shown

23. Solution Option #2: Design Install ESD Airflow Control Curtains
ESD curtains prevent hot exhaust air from flowing over the tops of racks into cold aisles where it would contaminate server inlets
Ceiling Grill
ESD Curtains
Hot Aisle

24. Solution Option #2 Design ESD Curtains and Airflow Management
False Ceiling is not shown, but hot air removal and effect on cold aisles is apparent as a result of deployment of ESD curtains

25. Solution Option #2 Benefits Ambient temperatures more stable
A thermal slice at a 6ft elevation now shows dramatic improvements as a result of ESD airflow control curtains. The result is further improvements in effectiveness of cold supply air and reduction of hot air contamination

26. Solution Option #2 Benefits Cooling Effectiveness Improved
Improved isolation of hot and cold air
CAC return air is warmer due to reduced mixing
Supply Temperature is increased from 60F to 68F
Rack Delta-T is reduced
Volume of supply air is reduced due to higher effectiveness

27. Solution Option #2 Operational Statistics
Description ASIS SO#2
Number of Cabinets
365
379
Total Cooling Airflow
199,538 CFM
187,138 CFM
Total Equipment Demand
142,069 CFM
140,237 CFM
Cooling Capacity
1213kW
Cooling Capacity in Use
947kW
943kW
Maximum Inlet Temp
93F
84F
Number of Active ACUs 18
ACU Power Consumption
107.6kW
97.5kW

28. Solution Option #3: Design Containment Strategy
Containment represents the most advanced airflow management system due to the creation of a closed airflow loop. This solution builds on Solution Options 1 and 2, and provides the ultimate solution in terms of achieving the stated project objectives. The concept behind containment is to completely enclose either a hot or cold aisle. Cold aisle containment prevents cold air exiting perforated tiles from going anywhere but through equipment. But the problem then becomes the effect of hot air entering the room. Hot aisle containment encloses the hot aisle and provides a number of major benefits.

29. Solution Option #3: Benefits Hot Aisle Containment Benefits
Creates slight negative pressure in hot aisle which “pulls” air through the air inlets
Pressure difference eliminates seeping of hot air through the rack into the air inlets
Entire room becomes cold air plenum: the volume of air supplied at a given perforated tile is no longer critical
Allows for racks in areas where there is low UF pressure to still get enough air as long as total supply is > total demand.
Rack Delta-T is very low allowing for increase in supply temps
Consumes 30% less air than Solution Option #1
Supply Temperature raised to 74F

30. Solution Option #3: Design Containment Strategies: CAC
PROS:
Reduces require air volume by 20%
Isolation of hot exhaust air
Does not require ceiling plenum
CONS:
Hot air enters room
Very little excess cold air capacity
Example of a Cold Aisle Containment Strategy. Notice the return air ceiling grills above the hot aisles

31. Solution Option #3: Design Containment Strategies: HAC
PROS:
Reduces require air volume by 30%
Isolation of hot exhaust air
Pulls air through servers
CONS:
Requires hot air plenum
Creates warm work area
Example of a Hot Aisle Containment Strategy.

32. Solution Option #3: Design Combined Containment Model
Example of a Contained and non-contained areas

33. Solution Option #3: Benefit Containment Benefits: CAC vs. HAC
Cold aisle Containment:
Reduces bypass air by enclosing cold air supply
Eliminates hot air exhaust contamination
Does not require hot air plenum
Reduces total supply air by 20% over SO#1
Hot Aisle Containment:
Maximizes air bypass reduction
Higher degree of airflow control (pull vs. push)
Removes all hot air from room
Reduces total supply air by 30% over SO#1
Increases rack density
Extremely even Delta-T across rack face

34. Solution Option #3: Benefit Increased Supply Air Temperature
Raising supply temps reduces costs of cooling, and requires less cold air volume.
Ambient room temp varies only slightly and is a comfortable 75F.
Heat rejection is 96% done by the CAC

35. Solution Option #3: Benefit Stable Equipment Supply Temperatures
A combined containment strategy has resulted in a significant increase in supply temperature and reduction of total cold air supply needed. This translates into reduced OPEX while providing a more stable operating environment as well as the ability to increase equipment power loading.

36. CAC Cluster Creates Low Pressure Zone
One factor that is limiting CAC effectiveness is caused by 3 CACs near the middle of the room. These units and their close proximity to each other creates a low pressure zone that pulls exhaust air from
Low Pressure Zone
CACs

37. Solution Option #3 Operational Statistics
Description ASIS SO#3
Number of Cabinets
365
379
Total Cooling Airflow
199,538 CFM
189,419 CFM
Total Equipment Demand
142,069 CFM
Cooling Capacity
1213kW
Cooling Capacity in Use
947kW
Maximum Inlet Temp
93F
81F
Number of Active ACUs 18 17
ACU Power Consumption
107.6kW
97kW

38. Special Design Review APC High Density Cluster Hot Aisle Containment: sub-optimum
APC’s InRow coolers are an efficient solution, but their effectiveness is lost on hot aisle containment
APCs leaking warm air
8 APC InRow Coolers with 12-12kW racks
Contained area is so hot, the coolers are sending 85F air into the room

39. Solution Design Review APC High Density Cluster Cold Aisle Containment
APC units are optimized when in a cold aisle containment design. Total load is 216kW over the hot aisle maximum load of 144kW and all equipment is experiencing inlet temps in the mid-70F range.
6 APC InRow Coolers with 12-16kW and 2-12kW racks. All rack
inlets are below 78F and hot air is not affecting local racks.

40. Solution Option Comparative Analysis
Description ASIS SO# SO# SO#3
Number of Cabinets
365
379
Total Cooling Airflow
199,538 CFM
187,138 CFM
178,138 CFM
165,419 CFM
Total Equip. Demand
142,069 CFM
Cooling Capacity
1213kW
Cooling Capacity in Use
947kW
Maximum Inlet Temp
93F
87F
84F
81F
Number of Active ACUs 18 17
ACU Power Consump.
107.6kW
102kW
97.5kW
89kW
AVG. Supply Temp. 60 62 63 68

41. Cost Basis Metrics and Calculations
1kW of IT Electrical Load - Annual cost: $6,000
(space, power, cooling)
Annual OPEX expense per 3kW rack $18,000
1F of Set Point increase = 1% energy reduction
(CAC compressor efficiency*)
If VFD’s were deployed, an additional 20% of total CAC energy
reduction would be realized as a result of reductions in fan energy.
Dependency is that Solution Option #3 would be deployed.

42. Data Center Design Comparative Models
CURRENT STATE METRICS
Number of Racks 365
Total Cooling Capacity kW
Total Cooling Load in Use 947kW
Total Connected Power 905kW
Server Temperatures F
CONTAINMENT DESIGN UNDER MAX LOAD
Number of Racks 397
Total Cooling Load in Use 1156kW
Total Connected Power 1114kW
Room Temperatures F
DELTA
+32
+209kW
-9F (max)

43. FINANCIAL ANALYSIS – Containment Costs
Ceiling Grill Construction
Grills (1540 $1/sqft) = $ 1,540
Containment Curtains and Doors* = $ 78,000
Additional Racks (12) =$ 18,000
TOTAL $ 97,540
*includes CAC ducts to ceiling plenum
Recommended:
Replace CAC’s with more efficient units $1,800,000

44. Containment Cost Reduction Estimates
OPEX Reduction
Reduce cooling costs by 6% $360/rack
365 racks X $360 = $131k/yr
Equipment replacement (annual) $20k/yr
Upgraded CAC cooling efficiency gains $48k/yr
TOTAL $199k/yr
CAPEX Reduction
Increase rack power loads by 30% (avoid new racks) $200k
Avoid adding new CACs $60k) $180k
Increase cooling system effectiveness (no cooler exp.) $500k
TOTAL $880k OTC
COST AVOIDANCE
New Data Center (20 sqft) Estimated at ~$30m

45. Recommendations
1. Deploy Solution Option #3: combined containment strategy.
This will reduce OPEX while extending the capacity of the existing
Cooling infrastructure and data center.
2. Virtualize as much of the environment as possible. This will have a
multiplier effect in terms of power reduction (each W of IT power
removed equates to 2-3 Watts of electrical and cooling overhead
reduction
3. Shorten the server refresh cycle. Consider a 3-year plan. This will
provide benefits both in terms of service levels from faster servers,
and will also have a multiplier effect in terms of power savings
as a result of more power efficient processors.

46. Additional Recommendations (1)
1. Replace all CACs with more efficient models:
VFD fans (10% reduction in speed = 50% reduction in power)
Multi-phase compressors (2 or 3 stage compressors)
2. Relocate new CACs for better distribution and coverage
VFD’s and Multi-phased compressors will help reduce power
Consumption for the cooling infrastructure particularly for data
Centers with containment strategies.
Containment allows for the same degree of cooling using less volume
of air. So, VFD’s, totaling over 80kW today could be cut in half.
Multi-phase compressors allow CACs to provide required cooling
running in a more power efficient mode (stage 1 vs. stage 2)

47. CAC SET POINTS and MIN SUPPLY TEMPS
ASIS
SO#1
SO#2
SO#3
MST SP
CAC-1 60 68 64 88 66 62
CAC-2 86 90
CAC-3
CAC-4
CAC-5
CAC-6 84
CAC-7
CAC-8 70
CAC-9 67 78 82
CAC-10
CAC-11 72 76
CAC-12
CAC-13
CAC-17 80
CAC-18
OFF
CAC-19
CAC-21 65
CAC-22
AVG
60.0
68.1
64.0
84.9
64.3
85.9
66.25
85.3

48. Thank You