2. Data Center Research Roadmap
A “research roadmap” developed for the California Energy Commission outlines key areas for energy efficiency research, development, and demonstration –
This includes strategies that can be implemented today.

3. Data Center research activities
Benchmarking and 22 data center case studies
Best practices identified
Self-benchmarking protocol
Power supply efficiency study
UPS systems efficiency study
Standby generation losses
Performance metrics – Computation/watt

4. LBNL data center demonstration projects
“Air management”
Outside air economizer
Contamination concerns
Humidity control concerns
DC powering
Facility level
Rack level

5. LBNL data center Federal projects
Case studies
Technical assistance
Emerging technology
Investigating use of infrared thermography as a visualization tool

6. Benchmarking energy end use

7. IT equipment load density

8. Overall power use in Data Centers
Courtesy of Michael Patterson, Intel Corporation

9. Performance varies
The relative percentages of the energy actually doing computing varies considerably.

10. Percentage of power delivered to IT equipment
Average 0.49

11. Benchmark results helped to find best practices
The ratio of IT equipment power to the total is an indicator of relative overall efficiency. Examination of individual systems and components in the centers that performed well helped to identify best practices.
Lets talk about a few….

12. Best practices topics identified through benchmarking

13. A word about appropriate environmental conditions…
ASHRAE published thermal guidelines
Majority of IT suppliers participated
Guidelines allow most centers to relax setpoints over standard practice
Recommended and allowable ranges of temperature and humidity are provided – at the inlet to the IT equipment
High temperatures in the “hot aisles” and return to air conditioners is desirable.

14. Temperature guidelines – at the inlet to IT equipment
ASHRAE Allowable Maximum
ASHRAE Recommended Maximum
ASHRAE Recommended Minimum
ASHRAE Allowable Minimum

15. Humidity guidelines – at the inlet to IT equipment
ASHRAE Allowable Maximum
ASHRAE Recommended Maximum
ASHRAE Recommended Minimum
ASHRAE Allowable Minimum

16. Best scenario – isolate cold and hot
95-100ºF
70-75ºF

17. Another isolation scheme

18. Fan energy savings – 75%
If mixing of cold supply air with hot return air can be eliminated-
fan speed can be reduced

19. Better temperature control can allow raising the temperature in the entire data center
ASHRAE Recommended Range
Ranges during demonstration

20. Best practices – free cooling with air economizers

21. Encouraging outside air economizers
Issue:
Many are reluctant to use air economizers
Outdoor pollutants and humidity control considered equipment risk
Goal:
Encourage use of outside air economizers where climate is appropriate
Strategy:
Address concerns: contamination/humidity control
Quantify energy savings benefits

22. Project objectives Identify potential failure mechanisms
Measure contamination levels inside and outside of data centers
Observe humidity control
Evaluate economizer effect on cumulative particulate exposure
Compare particle concentrations to guidelines

23. Measurements inside the centers
IBM Standard
EPA 24-Hour Health Standard
EPA Annual Health Standard
and ASHRAE Standard

24. Outdoor measurements IBM Standard EPA 24-Hour Health Standard
EPA Annual Health Standard
and ASHRAE Standard

25. Indoor measurements (note scale)

26. Data center w/economizer

27. Humidity measurements
ASHRAE Allowable Upper Limit
ASHRAE Recommended Upper Limit
ASHRAE Recommended Lower Limit
ASHRAE Allowable Lower Limit

28. Findings
Water soluble salts in combination with high humidity can cause current leakage
Static electricity (caused by humans) can occur with very low humidity
Particle concentration typically is an order of magnitude lower than new ASHRAE limits (without economizer)
Filtration and humidity control on make-up air can provide environments similar to those in closed data centers

29. Best practices – Power conversion

30. Data Center power conversions
AC voltage conversions
Internal Drive
External Drive
I/O
Memory Controller
m Processor
SDRAM
Graphics Controller
DC/DC
AC/DC
AC/DC Multi output PS
Voltage Regulator Modules 5V
12V
3.3V
1.5/2.5V
1.1V-1.85V
PWM/PFC
Switcher
Unregulated DC
To Multi Output
Regulated DC
Voltages
Inverter In
Out
Bypass
Battery/Charger
Rectifier

31. Power Supplies in IT equipment
Research illustrated large losses in power conversion
Power Supplies in IT equipment
Uninterruptible Power Supplies (UPS)

32. DC powering data centers
Goal:
Show that a DC system could be assembled with commercially available components and measure actual energy savings – a proof of concept demonstration.

33. Included in the demonstration
Side-by-side comparison of traditional AC system with new DC system
Facility level distribution
Rack level distribution
Power measurements at conversion points
Servers modified to accept
380 V. DC
Artificial loads to more fully simulate data center

34. Typical AC distribution today
380 V DC after first stage conversion
480 Volt AC

35. Facility-level DC distribution
480 Volt AC
380V.DC
380 V DC is delivered directly into the server to the same point as in an AC powered server. This eliminates the DC-AC conversion at the UPS and the AC-DC conversion in the server. Also, less equipment is needed.

36. Rack-level DC distribution
480 Volt AC

37. AC system loss compared to DC
7-7.3% measured improvement
2-5% measured improvement
Rotary UPS

38. Energy savings for a typical data center
20% or more facility level energy savings because:
Redundant UPS and server power supplies operate at reduced efficiency
Cooling loads would be reduced.
The demonstration comparisons were against “best in class” systems which performed better than typical systems we benchmarked.
Further optimization of conversion devices/voltages is possible

39. Demonstration set-up –
see website for more detail

40. DC power – next steps DC power pilot installation(s)
Standardize distribution voltage
Standardize DC connector and power strips
Server manufacturers develop power supply specification
Power supply manufacturers develop prototype
UL and communications certification

41. Design and Training Resources
A web-based training resource available on LBL’s website
Design guidelines available through PG&E’s Energy Design Resources website

42. website: http://hightech.lbl.gov/datacenters/

43. Discussion/Questions??