1. #watitis2015 watitis.uwaterloo.ca @watitisconf TOWARD A GREENER HORIZON: PROPOSED ENERGY SAVING CHANGES TO MFCF DATA CENTERS Naji Alamrony nalamron@uwaterloo.ca

2. #watitis2015 AGENDA Objectives and Motivation Energy and Peak power reduction Reducing IT load Reducing Data Center energy waste Current State of MFCF Data Centers Proposed actions Conclusion

3. #watitis2015 OBJECTIVES Propose changes to current data centers Reduce energy consumption Minimal implementation cost Significant rewards Getting UW support/involvement

4. #watitis2015 PEAK POWER VS ENERGY Energy determines Size of energy bill Battery life Peak power determines Capacity Electrical Cooling

5. #watitis2015 MOTIVATION Monetary Energy bills Environmental greenhouse gases emission Capacity Efficient use of the datacenter building Cost of building new data center

6. #watitis2015 ENERGY CONSUMPTION COMPARISON Ref: Data Center Efficiency Assessment, NRDC, August 2014

7. #watitis2015 MFCF ENERGY CONSUMPTION Estimated Energy cost from $10,000 to $15,000/Month

8. #watitis2015 RESTRICTIONS Cost  start cheap but effective High availability Performance Virtualizability Capacity (Data center) Cooling Electricity Spatial  using existing data center

9. #watitis2015 REDUCING PEAK POWER/ENERGY

10. #watitis2015 BACKGROUND – IT LOAD POWER, EFFICIENCY VS. UTILIZATION Reference: The case for energy-proportional computing, Barroso, 2007

11. #watitis2015 BACKGROUND – IT LOAD

12. #watitis2015 BACKGROUND - IT MACHINES’ UTILIZATION – 1 DAY

13. #watitis2015 BACKGROUND - IT MACHINES’ UTILIZATION – 2 WEEKS

14. #watitis2015 REDUCING IT LOAD New machines: Use properly sized machines RAM CPU Storage (SSD vs. HDD) Power supply Use efficient class machines Blade server Micro server

15. #watitis2015 REDUCING IT LOAD (CONT.) When using blades consider cooling capacities (up to 30kW/rack) SSD drives is recommended More blades on a chassis more efficient

16. #watitis2015 REDUCING IT LOAD (CONT.) Improve utilization External power management policies ON/OFF policy Suspend polices Internal power policies Enable c-states (sleep modes) Enable p-states (DFVS) Virtualization Provides elasticity for cluster Consolidate light weight services

17. #watitis2015 INCREASE DATA CENTER EFFICIENCY REDUCING COOLING LOAD Measured air flow Input (vertical) = output (horizontal) Tight hot/cold aisle containment Open aisle (currently used) Cold aisle containment (better) Hot aisle containment (recommended) Higher cold aisle temperature ASHRAE recommends up to 27 ℃

18. #watitis2015 HOT AISLE CONTAINMENT

19. #watitis2015

20. ASHRAE PSYCHROMETRIC CHART

21. #watitis2015 SAFE TEMPERATURE LIMITS Device Temperature Specs ℃ CPU60 - 70 Memory85 - 105 GPU60 - 80 Hard drive50 - 60 I/O devices50 - 60

22. #watitis2015 FREE COOLING Use of free cooling Air-side economizer Water-side economizer

23. #watitis2015 COOLING DEGREE-DAY Cooling degree-days (CDD) A measure of how much (in degrees), and for how long (in days), the outside air temperature was above a certain level (base temperature) CDD is proportional to cooling energy Waterloo CDDs From http://www.degreedays.net/http://www.degreedays.net/ Base temp CDD 27 ℃ 13 23 ℃ 94 18 ℃ 352

24. #watitis2015 POWER DISTRIBUTION WASTE Require infrastructure change: Higher voltage distribution (480, 220v vs. 120) DC distribution Reduce electricity conversion stages

25. #watitis2015 MFCF SERVER ROOMS M3-3101 (~ 50kW)MC-3015 (~ 100 kW)

26. #watitis2015 CURRENT STATE OF MFCF DATA CENTER Lack of power saving measures No air flow Management/Analysis No efficiency measure (PUE/DCiE) No power-usage monitoring No use of machines utilization logs

27. #watitis2015 PROPOSED ACTION "To measure is to know - if you cannot measure it, you cannot improve it” - Kelvin

28. #watitis2015 PROPOSED ACTIONS (CONT.) Determine data center capacity Start with CFD study Monitor energy utilization to Get baseline efficiency PUE, DCiE, or DCP Determine highest Energy consumers Plan load distribution across datacenter Implement Hot aisle containment Increase cold aisle temperature (23 ℃ )

29. #watitis2015 PROPOSED ACTIONS (CONT.) Monitor machine utilization pattern Determine: Comatose servers ON/OFF and suspend policies Virtualizable machines High availability clusters (elasticity) Consolidating services

30. #watitis2015 PROPOSED ACTION (CONT.) Utilize machine’s remote management To apply ON/OFF and suspend policy To provide machines’ owners remote control Investigate free cooling options

31. #watitis2015 SUGGESTION FOR THE UNIVERSITY OF WATERLOO Move electricity bill payment down to faculty level Incentive Free cooling worth investigation

32. #watitis2015 CONCLUSION Many ways to reduce energy Start with cheaper, easier, most rewarding Weed out comatose server Define and implement ON/OFF policies Use Virtualization Tighten hot aisle containment Increase cold aisle temperature (23 to 27 ℃ ) Utilize economizers (air-side, water-side) Utilize measures for comparison: PUE, CDD …

33. #watitis2015 TO BE CONTINUED …. I am hoping that by next year, I will be able to present a different kind of presentation. One that includes data, results and actual changes to improve energy efficiency Ultimately leading to a greener future

34. #watitis2015 Questions? Email: nalamron@uwaterloo.canalamron@uwaterloo.ca Ext. 36847

35. #watitis2015 This is an optional area for you to put a message or caption for your photo. Delete it if you don’t need to use it.