1. Data Centers - They’re Back… E SOURCE Forum September, 2007 William Tschudi wftschudi@lbl.gov

2. Acknowledgement to LBNL sponsors California Energy Commission – PIER program California Energy Commission – PIER program Pacific Gas and Electric Company Pacific Gas and Electric Company New York State Energy and Development Agency (NYSERDA) New York State Energy and Development Agency (NYSERDA) US - Environmental Protection Agency US - Environmental Protection Agency US – Department of Energy US – Department of Energy

3. Data Center research activities Research Roadmap Research Roadmap Benchmarking and 23 data center case studies Benchmarking and 23 data center case studies Self-benchmarking protocol Self-benchmarking protocol Power supply efficiency study Power supply efficiency study UPS systems efficiency study UPS systems efficiency study Standby generation losses Standby generation losses Performance metrics – Computation/watt Performance metrics – Computation/watt EPA report to Congress EPA report to Congress DOE Save Energy Now program DOE Save Energy Now program

4. LBNL data center demonstrations “Air management” demonstration Outside air economizer demonstration DC powering demonstrations

5. Data center energy roadmap July, 2003 – Jay Stein a major contributor July, 2003 – Jay Stein a major contributor The problem receded for several years as excess data center space was absorbed The problem receded for several years as excess data center space was absorbed Now the concerns are back – and they are badder than ever! Now the concerns are back – and they are badder than ever!

6. Performance varies The relative percentages of the energy actually doing computing varies considerably.

7. Percentage of electricity delivered to IT equipment Average.57 Higher is better Source: LBNL Benchmarking

8. Benchmark results helped to find best practices The ratio of IT equipment power to the total is an indicator of relative overall efficiency. Examination of individual systems and components in the centers that performed well helped to identify best practices. Lets talk about a few….

9. Server Load/ Computing Operations Cooling Equipment Power Conversion & Distribution Alternative Power Generation High voltage distribution Use of DC power Highly efficient UPS systems Efficient redundancy strategies Load management Server innovation Energy efficiency opportunities are everywhere Better air management Move to liquid cooling Optimized chilled-water plants Use of free cooling On-site generation Waste heat for cooling Use of renewable energy/fuel cells

10. Best practices topics identified through benchmarking

11. A word about appropriate environmental conditions… ASHRAE published thermal guidelines ASHRAE published thermal guidelines –All IT suppliers participated –Guidelines allow most centers to relax setpoints Recommended and allowable ranges of temperature and humidity are provided – at the inlet to the IT equipment Recommended and allowable ranges of temperature and humidity are provided – at the inlet to the IT equipment High temperatures in the “hot aisles” and return to air conditioners is desirable. High temperatures in the “hot aisles” and return to air conditioners is desirable.

12. Humidity guidelines – at the inlet to IT equipment ASHRAE Allowable Maximum ASHRAE Allowable Minimum ASHRAE Recommended Maximum ASHRAE Recommended Minimum

13. Temperature guidelines – at the inlet to IT equipment ASHRAE Allowable Maximum ASHRAE Allowable Minimum ASHRAE Recommended Maximum ASHRAE Recommended Minimum

14. Air management best scenario – isolate cold and hot 70-75º 95-100º

15. Another isolation scheme

16. Measured fan energy savings – 75% If mixing of cold supply air with hot return air can be eliminated- fan speed can be reduced

17. Better temperature control can allow raising the temperature in the entire data center! ASHRAE Recommended Range Ranges during demonstration

18. Best practices – Free cooling with air economizers

19. Encouraging outside air economizers Issue: Issue: –Many are reluctant to use air economizers –Outdoor pollutants and humidity control considered equipment risk Goal: Goal: –Encourage use of outside air economizers where climate is appropriate Strategy: Strategy: –Address concerns: contamination/humidity control –Quantify energy savings benefits

20. Outdoor measurements IBM Standard EPA Annual Health Standard EPA 24-Hour Health Standard and ASHRAE Standard

21. Measurements inside the centers IBM Standard EPA Annual Health Standard EPA 24-Hour Health Standard and ASHRAE Standard

22. Indoor measurements

23. Data center w/economizer

24. Humidity measurements ASHRAE Recommended Upper Limit ASHRAE Recommended Lower Limit ASHRAE Allowable Lower Limit ASHRAE Allowable Upper Limit

25. Findings Water soluble salts in combination with high humidity can cause failures Water soluble salts in combination with high humidity can cause failures Static electricity can occur with very low humidity Static electricity can occur with very low humidity New ASHRAE particle limits drastically lower than manufacturer standard New ASHRAE particle limits drastically lower than manufacturer standard Particle concentration typically (no economizer) an order of magnitude lower than new ASHRAE limits Particle concentration typically (no economizer) an order of magnitude lower than new ASHRAE limits Economizers, without other mitigation, can cause particle concentration to approach new ASHRAE limits but filtration can mitigate this Economizers, without other mitigation, can cause particle concentration to approach new ASHRAE limits but filtration can mitigate this

26. Best practices – power conversion

27. Inverter InOut Bypass Battery/Charger Rectifier Internal Drive External Drive I/O Memory Controller  Processor SDRAM Graphics Controller DC/DC AC/DC DC/DC AC/DC Multi output PS Voltage Regulator Modules 5V 12V 3.3V 12V 1.5/2. 5V 1.1V- 1.85V 3.3V 12V PWM/PFC Switcher Unregulated DC To Multi Output Regulated DC Voltages Data center power conversions AC voltage conversions

28. Prior research illustrated large losses in power conversion Uninterruptible Power Supplies (UPS) Power Supplies in IT equipment

29. With over 25 industry partners direct DC powering of servers was demonstrated

30. Typical AC distribution today 480 Volt AC

31. Facility-level DC distribution 380V.DC 480 Volt AC

32. AC system loss compared to DC AC system loss compared to DC 7-7.3% measured improvement 2-5% measured improvement Rotary UPS

33. What’s new Infrared thermography can be used as a visualization tool

34. websites:http://hightech.lbl.gov/datacenters/ www.energystar.gov/datacenters www.eere.energy.gov/datacenters/

35. Discussion/Questions??