1. Data Center Energy Efficiency SC07 Birds of a Feather November, 2007 William Tschudi wftschudi@lbl.gov

2. Benchmark results helped to find best practices The ratio of IT equipment power to the total (or its inverse) is an indicator of relative overall efficiency. Examination of individual systems and components in the centers that performed well helped to identify best practices.

3. Percentage of electricity delivered to IT equipment Average.57 Higher is better Source: LBNL Benchmarking

4. Performance varies The relative percentages of the energy actually doing computing varies considerably.

5. Typical power use Courtesy of Michael Patterson, Intel Corporation Computer/System Architecture System Architecture / Power Engineering Facility Design / Mechanical Design

6. Server Load/ Computing Operations Cooling Equipment Power Conversion & Distribution Alternative Power Generation High voltage distribution Use of DC power Highly efficient UPS systems Efficient redundancy strategies Load management Server innovation Energy efficiency opportunities are everywhere Better air management Move to liquid cooling Optimized chilled-water plants Use of free cooling On-site generation Waste heat for cooling Use of renewable energy/fuel cells

7. Best practices topics identified through benchmarking

8. A word about appropriate environmental conditions… ASHRAE published thermal guidelines ASHRAE published thermal guidelines –All IT suppliers participated and agreed –Guidelines allow most centers to relax setpoints –HPC community agrees Recommended and allowable ranges of temperature and humidity are provided – at the inlet to the IT equipment Recommended and allowable ranges of temperature and humidity are provided – at the inlet to the IT equipment High temperatures in the “hot aisles” and return to air conditioners are desirable. High temperatures in the “hot aisles” and return to air conditioners are desirable.

9. Humidity guidelines – at the inlet to IT equipment ASHRAE Allowable Maximum ASHRAE Allowable Minimum ASHRAE Recommended Maximum ASHRAE Recommended Minimum

10. Temperature guidelines – at the inlet to IT equipment ASHRAE Allowable Maximum ASHRAE Allowable Minimum ASHRAE Recommended Maximum ASHRAE Recommended Minimum

11. Air management best scenario – isolate cold and hot 70-75º 95-100º

12. Another isolation scheme

13. Measured fan energy savings – 75% If mixing of cold supply air with hot return air can be eliminated- fan speed can be reduced

14. Better temperature control can allow raising the temperature in the entire data center! ASHRAE Recommended Range Ranges during demonstration

15. See the problem areas Infrared thermography and CFD modeling can be used as visualization tools

16. Best practices – Free cooling with air economizers

17. Encouraging outside air economizers Issue: Issue: –Many are reluctant to use air economizers –Outdoor pollutants and humidity control considered equipment risk Goal: Goal: –Encourage use of outside air economizers where climate is appropriate Strategy: Strategy: –Address concerns: contamination/humidity control –Quantify energy savings benefits

18. Outdoor measurements IBM Standard EPA Annual Health Standard EPA 24-Hour Health Standard and ASHRAE Standard

19. Measurements inside the centers IBM Standard EPA Annual Health Standard EPA 24-Hour Health Standard and ASHRAE Standard

20. Data center w/economizer

21. Findings Water soluble salts in combination with high humidity can cause failures Water soluble salts in combination with high humidity can cause failures New ASHRAE particle limits drastically lower than one manufacturer’s recommendation New ASHRAE particle limits drastically lower than one manufacturer’s recommendation Particle concentration typically an order of magnitude lower than new ASHRAE limits (no economizer) Particle concentration typically an order of magnitude lower than new ASHRAE limits (no economizer) Economizers, without other mitigation, can cause particle concentration to approach new ASHRAE limits but filtration can mitigate this Economizers, without other mitigation, can cause particle concentration to approach new ASHRAE limits but filtration can mitigate this Large energy savings Large energy savings

22. Set Inlet Temperature To upper end of ASHRAE

23. NERSC/LBNL CRT Building Conceptual Design DRAFT: Subject to change Howard Walter

24. Schematic Design Concepts DRAFT: Subject to change; Do not re-distribute Howard Walter

25. Best practices – power conversion

26. Inverter InOut Bypass Battery/Charger Rectifier Internal Drive External Drive I/O Memory Controller  Processor SDRAM Graphics Controller DC/DC AC/DC DC/DC AC/DC Multi output PS Voltage Regulator Modules 5V 12V 3.3V 12V 1.5/2. 5V 1.1V- 1.85V 3.3V 12V PWM/PFC Switcher Unregulated DC To Multi Output Regulated DC Voltages Data center power conversions AC voltage conversions

27. Prior research illustrated large losses in power conversion Uninterruptible Power Supplies (UPS) Power Supplies in IT equipment

28. With over 25 industry partners direct DC powering of servers was demonstrated

29. Typical AC distribution today 480 Volt AC

30. Facility-level DC distribution 380V.DC 480 Volt AC

31. AC system loss compared to DC AC system loss compared to DC 7-7.3% measured improvement 2-5% measured improvement Rotary UPS

32. websites:http://hightech.lbl.gov/datacenters/