1. 1 Data Centers for the 21st Century Working with Industry to Improve Energy Efficiency in Data Centers Dale Sartor, P.E. LBNL Applications Team June 7, 2007

2. 2 Crying Uncle!

3. 3 LBNL Computer Systems Power

4. 4 Why Data Centers

5. 5 Highly energy-intensive and rapidly growing Consume 10 to 100 times more energy per square foot than a typical office building A single rack of servers can be 20 kW –$17k per year per rack (at $.10/kWh) –Hundreds of racks per center can have significant impact on electricity supply and distribution Used about 45 billion kWh in 2005—about 1.2% of all retail U.S. electricity sales. At current growth, power requirements could double in less than 10 years. Why Data Centers

6. 6 Typical Data Center Energy End Use Server Load /Computing Operations Cooling Equipment Power Conversions & Distribution 100 Units 33 Units Delivered 35 Units Where Does it Go? Data Center Energy Use

7. 7 Save 20 billion kWh per year by 2015 –Worth $2 billion and roughly equal to annual electricity use in 1.8 million American homes Potentially defer need to build 2,300 MW of new generating capacity and avoid 3.4 million metric tons of carbon emissions (like taking 675,000 cars off the road) Extend life and capacity of existing data center infrastructures Potential Benefits of Improved Data Center Energy Efficiency

8. 8 Courtesy of Michael Patterson, Intel Corporation Overall power use in Data Centers

9. 9 Computational Energy Efficiency While my presentation focuses on the data center infrastructure, significant opportunities are available to increase computational energy efficiencies: Improve Utilization of Resources: –Consolidation and virtualization –Grid computing Enable and improve power management (dynamic loading) Improve software efficiency (and software instructions to hardware) Any reduction in IT equipment energy use has a corresponding savings in infrastructure –Potential first cost savings are often missed Very challenging to set standards

10. 10 Computational Energy Efficiency Standards ENERGY STAR for Servers EPA will release strawman proposal this year EPA considering power supply efficiency & system energy efficiency performance –Need support from industry stakeholders EPA also interested in other IT equipment -- storage, networking, etc. Efforts to Develop Server Performance Benchmark: SPEC No metric available to compare server energy efficiency SPEC Committee developing energy efficiency benchmark Working prototype developed, more info on progress: www.spec.org/specpower www.spec.org/specpower

11. 11 With funding from PG&E and CEC, LBNL conducted benchmark studies of 22 data centers: –Found wide variation in performance –Identified best practices CEC and PG&E continue R&D and demonstrations New DOE program will expand current knowledge base Efficient Data Centers - Building Knowledge Base

12. 12 Benchmarking – How do I Stack Up?

13. 13 Energy Intensity Growth Growing but not a good performance metric

14. 14 Data Center Cooling and Power Conversion Performance Varies Typical PracticeBetter Practice Server Load /Computing Operations Cooling & Power Conversions Server Load /Computing Operations Cooling & Power Conversions Infrastructure Efficiency

15. 15 Best Practices: Lessons from Benchmarking Total Data Center Power/ IT power

16. 16 The ratio of IT equipment to the total power is an indicator of relative overall efficiency. Examination of individual systems and components in the centers that performed well helped to identify best practices: Air management Right-sizing Central plant optimization Efficient air handling Free cooling Humidity control Liquid cooling Improving power chain UPSs and equipment power supplies On-site generation Design and M&O processes Using benchmark results to find best practices

17. 17 Enforce hot aisle/cold aisle arrangement Eliminate bypasses and short circuits Reduce air flow restrictions Proper floor tile arrangement Proper locations of air handlers Optimize Air Management

18. 18 Data Center HVAC often under-loaded Ultimate load uncertain Design for efficient part-load operation –modularity –variable-speed fans, pumps, compressors Upsize fixed elements (pipes, ducts) Upsize cooling towers Right-Size the Design

19. 19 Have one (vs. distributed cooling) Medium temperature chilled water Aggressive temperature resets Primary-only CHW with variable flow Thermal storage Monitor plant efficiency Optimize the Central Plant

20. 20 Fewer, larger fans and motors VAV easier Central controls eliminate fighting Outside-air economizers easier Design for Efficient Central Air Handling

21. 21 Outside-Air Economizers –Can be very effective (24/7 load) –Controversial re: contamination –Must consider humidity Water-side Economizers –No contamination question –Can be in series with chiller Use Free Cooling

22. 22 Eliminate inadvertent dehumidification –Computer load is sensible only –Medium-temperature chilled water –Humidity control at make-up air handler only Use ASHRAE allowable RH and temperature Eliminate equipment fighting –Coordinated controls on distributed AHUs Improve Humidity Control

23. 23 Water is 3500x more effective than air on a volume basis Cooling distribution is more energy efficient Water-cooled racks available now; liquid-cooled computers are coming Heat rejection at a higher temperature –Chiller plant more efficient –Water-side economizer more effective Use Liquid Cooling of Racks and Computers

24. 24

25. 25 Improving the Power Chain Increase distribution voltage DC distribution Improve equipment power supplies Improve UPS

26. 26 Power supplies in IT equipment generate much of the heat. Highly efficient supplies can reduce IT equipment load by 15% or more. UPS efficiency also varies a lot. (Do they need the same environment as the IT equipment or can their conditions be relaxed?) Specify Efficient Power Supplies and UPSs

27. 27 Can use waste heat for cooling –sorption cycles –typically required for cost effectiveness Swaps role with utility for back-up Air-quality issues Sell-back options –complex controls required Consider On-Site Generation

28. 28 Get IT and Facilities people to work together Use life-cycle total cost of ownership analysis Document design intent Introduce energy optimization early Benchmark existing facilities Re-commission as a regular part of maintenance Improve Design and Operations Processes

29. 29 Top best practices identified through benchmarking

30. 30 Design Guides were developed based upon the observed best practices Guides are available through PG&E and LBNL websites Self benchmarking protocol also available Design guidelines are available http://hightech.lbl.gov/datacenters.html

31. 31 A “research roadmap” was developed for the California Energy Commission and outlined key areas for energy efficiency research, development, and demonstration What is in the R&D Pipeline

32. 32  “Air management” improvements (PG&E) –Dramatic savings in fan energy –Improved cooling efficiency and output –Significant reduction in hot spots  Outside air economizers (PG&E) –Field test of air quality –Contamination and humidity control concerns generally unfounded  DC powering (CEC-PIER) –Facility level savings 10 to 20% LBNL Data Center Demonstrations

33. 33 Energy efficiency can slow expected growth in electricity use: Current trends will lead to ~10% reduction Simple management improvements can reduce consumption by an additional 20% Best practices can lead to ~50% reduction Impact of Energy Efficiency

34. 34 Comparison of Projected Electricity Use, All Scenarios, 2007 to 2011 Annual Energy Use (Billion kWh/year) 2007 2008 2009 2010 2011 2006 Baseline 58.7 0 140 120 100 80 60 40 20 Business as usual Current trends Improved operational management Best practice State of the art U.S. Opportunity Potential

35. 35 CEC is providing the technical foundation to improve data centers PG&E is the leading U.S. utility providing support to data centers DOE is initiating a national program to provide assistance in identifying and evaluating savings opportunities Where can I get help?

36. 36 Program Objectives –Provide systems approach –Build tools, expertise, and strategies –Raise awareness and recognize industry leaders Program Strategies (2007) –Build on successful Save Energy Now model –With industry input, develop appropriate tools, training, and qualified experts –Implement assessment program (solicitation in Fall) –Screen for industrial demonstrations –Federal government procurement specifications DOE Data Center Program www1.eere.energy.gov/industry/saveenergynow/partnering_data_centers.html

37. 37 Sponsors: –California Energy Commission (CEC) http://www.energy.ca.gov/pier/ –U.S. Department of Energy (DOE) http://www1.eere.energy.gov/industry/saveenergynow/partnering_data_centers.html –U.S. Environmental Protection Agency http://www.energystar.gov/datacenters Pacific Gas and Electric Company (PG&E) http://www.pge.com/docs/pdfs/biz/rebates/hightech/06_DataCenters-PGE.pdf Stakeholders: –Industry Organizations e.g., Green Grid, ASHRAE, AFCOM, 7x24, SVLG –Equipment suppliers –Research organizations –Consultants Sponsors and Stakeholders

38. 38 http://hightech.lbl.gov/datacenters.html Good starting point for those seeking efficiency measures Best Practices Case Studies Design Guidance Self-benchmarking Guide Benchmark data Other Reports (demonstrations) Web-based Resources

39. 39 Dale Sartor, P.E. Lawrence Berkeley National Laboratory Applications Team MS 90-3011 University of California Berkeley, CA 94720 DASartor@LBL.gov (510) 486-5988 http://Ateam.LBL.gov Contact Information: