1. Trends and Effects of Energy Proportionality on Server Provisioning in Data Centers
Georgios Varsamopoulos, Zahra Abbasi, and Sandeep Gupta
Impact Laboratory
School of Computing, Informatics and Decision Systems Engineering
Arizona State University
Funded in parts by NSF CNS grants and by Intel Corp

2. Introduction-Motivation
The magnitude of data center energy consumption keeps growing
Addressing energy saving for internet Data Center
Energy proportionality (power performance) awareness
Thermal awareness
Projected Electricity Use of data centers\, 2007 to 2011
Historical energy use
Future energy use projection
- current efficiency trend
[Source: EPA]
Typical data center energy end use
[Source: Department of energy]

3. Talk outline Energy saving in DCs through workload management
Characterizing Energy Proportionality of systems
Introducing IPR and LDR to measure energy proportionality of systems
Effect of energy proportionality on thermal aware workload management (TASP and TAWD)
Simulation Study
Under real data profile, current trend of energy proportionality, and truly hypothetic trend of energy proportionality
Conclusion

4. Energy Consumption in Datacenter
Total energy consumption in a datacenter = Computing energy + Cooling energy[ref]
Techniques used to reduce total energy consumed by datacenters:
Server provisioning + Thermal awareness
[Chase et al. SOSP ’01] , [Chen et al. NSDI ’08], [Kusic et al. CCJ ’09] ,
[Abbasi et al. HPDC’10], [Moore et al. ATEC ’05], and [Tang et al. T-PDS ’08]
Workload distribution +Thermal awareness
[Abbasi et al. HPDC’10]

5. Thermal aware server provisioning (TASP) and workload distribution (TAWD)
Heat recirculation contribution
Computing capabilities of machines
Computing power efficiency
λ request/sec
Number of requests
TASP Tier (Epochs)
Time index (every 5 second)
TAWD Tier
(Slots)
HTTP requests over time, 1998 FIFA World Cup
{λi}
Load Dispatcher
Traffic flow
Parameters
Control data
∑λi= λ
λ1=0 λ2
λ3=0 λN
λN-1=0
On/Off Control
Server 1
Server 1
Server 2
Server 3
Server 3 ……
Server N-1
Server N-1
Server N

6. Fully Energy Proportional Systems
Energy consumption should be proportional to the system workload.
At 0% utilization level system should consume no power and power consumption should linearly increase with increasing utilization
PPeak
Power
Consumption (Watts)
PIdle
100
% utilization

7. Project objective Characterizing energy proportionality of systems
Introducing metrics to measure energy proportionality of systems
Will the ideal energy proportionality render server provisioning unnecessary technique?

8. Power performance behavior of modern systems
SPECpower_ssj2008 observations:
-Systems consume power at idle
-The power-utilization curve is not linear

9. Energy Proportionality Metrics
Linear power curve
System utilization 0%
100%
Power
0 W
Pidle
Ppeak
Actual power curve
Proportional power curve
Excessive power consumption
under low utilization
Idle to peak ratio
Measure of Energy Proportionality:
How close to origin power curve
starts
Linear deviation ratio
How close to linear it is
Measuring through a hypothetical line

10. Metric-based examples
Low IPR, Low LDR
High IPR, Low LDR
Low IPR, high LDR
high IPR, high LDR

11. Trend of computing systems LDR IPR for over past 3 yrs
By 2008
By 2009
By Dec. 2010

12. Why characterizing/exploiting energy proportionality is important?
Ideal systems
Old systems
No workload management is required. (If thermal impact of servers is negligible)
higher power efficiency at higher utilization levels.
Recent systems
Recent systems
Higher power efficiency at lower utilization levels
Significantly better energy efficiency by utilization at the maximum power efficiency

13. The region of interests…
Two plot show the diverging trend from LDR =0.

14. Design of experiments -Investigating the effect of EP on the performance of TASP and TAWD
How does the current trend of energy proportionality (LDR and IPR value) affect the performance of TASP and TAWD?
Nonlinearity of power-utilization curve/non zero idle power
How does the (true) power proportionality (IPR trend) affect the performance of TASP and TAWD?
Case 1
Case 2
Observed trends of LDR IPR values in the current systems
Hypothetical progress towards true energy proportionality

15. IPR-LDR Scatter plot Case 1 : 0≤IPR ≤ 1, 0 ≤ LDR ≤ 1
Synthesized power curves for case 1.
Case 1 : 0≤IPR ≤ 1, 0 ≤ LDR ≤ 1
Current trend of Energy Proportional systems
Case 2: 0 ≤ IPR ≤ 1, LDR =0
Hypothetical true Energy Proportional Systems
Synthesized power curves for case 2.

16. Simulation Setup Setup: Thermal profile of ASU HPCI Datacenter
(Dell 1855,1955 Servers)
Workload:
Synthesizing SPECweb FIFA World CUP 1998
Study
Constitution
Energy Proportionality
Server Provisioning over no server provisioning
Homogeneous Constitution (Dell 1855)
Case1,case2
Workload Management over Load Balancing
Heterogeneous Constitution
(Dell Dell 1955)
Case 1, case2
Model consist of 50 chassis of server blade
Homogeneous: All Dell 1855’
Heterogeneous: 20 chassis of Dell Dell 1955

17. Server Provisioning over no server provisioning
Homogenous Case 1
Heterogenous Case1
IPR = 0
Heterogenous
What we are trying to achieve?
Will Server provisioning give energy savings under current and ideal trend of energy proportionality for computing systems?
Homogenous Case 2
Heterogenous Case2
Homogeneous Case 1: Due to nonlinear curve saving between 30-60%, at IPR =0, Homogeneous Case 2. Savings face a larger reduction for Case 2.
Heterogonous Case 1:Energy saving is observed much more as server with less efficiency are shut off with provisioning.
IPR = 0

18. Server Provisioning over no server provisioning
Homogenous Case 1
Heterogeneous Case1
IPR = 0
Significant energy saving even for IPR=0
Heterogenous
Homogenous Case 2
Heterogenous Case2
Case 1: (current trend: IPR →0, i.e. zero power at idle, LDR →1, i.e. non linear power-utilization curve)
Heterogeneous case
50% energy saving at IPR=0 due to nonlinearity of power curve and heterogeneity
around 7-10% energy saving at IPR=0 due to thermal-awareness under different power density
Homogenous case
30% energy saving at IPR=0 due to nonlinearity of power curve
around 5-10% energy saving at IPR=0 due to thermal-awareness under different power density
Significant energy savings through server provisioning even when IPR=0
Non linearity of power-utilization curve makes system to be very low energy efficient when they are underutilized.
Power density has no effect on computing power aware server provisioning, it affect thermal aware server provisioning though
IPR = 0

19. Server Provisioning over no server provisioning
Homogenous Case 1
Heterogeneous Case1
Case 2: (Ideal trend: IPR →0, i.e. zero power at idle and LDR=0, i.e. linear power-utilization curve)
IPR = 0
Homogenous case
around 5-20% energy saving at IPR=0 due to thermal-awareness under different power density
Heterogeneous case
30% energy saving at IPR=0 due to heterogeneity
around 7-20% energy saving at IPR=0 due to thermal-awareness under different power density
Heterogenous
Homogenous Case 2
Heterogeneous Case2
Significant energy saving even for IPR=0:
Thermal awareness,
Heterogeneity
energy saving for IPR=0 :
Thermal awareness
Homogeneous Case 2. Savings face a larger reduction for Case 2.
Heterogeneous Case 2:Energy saving is observed much more as server with less efficiency are shut off with provisioning.

20. Work load distribution over Load Balancing
Homogenous Case 1
Heterogeneous Case1
What we are trying to achieve?
Will workload management (fine scale) give energy savings under current and ideal trend of energy proportionality for computing systems?
Significant energy saving
Homogenous Case 2
Heterogeneous Case2
Question: Whether load balancing under good energy proportionality renders “Thermal Aware” solution unnecessary?
Server Provisioning makes active server set to be homogeneous most of the time. If heterogeneous set of active servers is forced, the energy savings increase to 50%

21. Work load distribution over Load Balancing
Homogenous Case 1
Heterogeneous Case1
Significant energy savings
Case 1: (current trend: IPR →0, i.e. zero power at idle, LDR →1, i.e. non linear power-utilization curve)
TAWD saves energy up to 45% for both homogenous and heterogeneous case
The saving increases by increasing the nonlinearity of power-util curve
TAWD savings come from utilizing servers at their high energy efficiency level through short time granularity workload management
TAWD can be envisioned as a technique to compensate energy wasting due to nonlinear power-util curve of systems
Homogenous Case 2
Heterogeneous Case2
Question: Whether load balancing under good energy proportionality renders “Thermal Aware” solution unnecessary?
-TAWD make significant difference on energy consumption for both homogenous and heterogeneous case
-Server Provisioning makes active server set to be homogeneous most of the time. If heterogeneous set of active servers is forced, the energy savings of TAWD under heterogeneity increase to 50%

22. Work load distribution over Load Balancing
Homogenous Case 1
Heterogeneous Case1
Case 2: (Ideal trend: IPR →0, i.e. zero power at idle and LDR=0, i.e. linear power-utilization curve)
TAWD’s performance come from thermal awareness only
Energy savings of TAWD increases under higher power density, heterogeneity and higher proportionality
The fraction of computing energy decreases at IPR=0
Homogenous Case 2
Heterogeneous Case2
Question: Whether load balancing under good energy proportionality renders “Thermal Aware” solution unnecessary?
Server Provisioning makes active server set to be homogeneous most of the time. If heterogeneous set of active servers is forced, the energy savings increase to 50%
Energy savings through thermal awareness

23. Conclusion
Server provisioning scheme even for ideal energy proportional systems saves energy through thermal awareness
The performance of server provisioning is higher for the heterogeneous case
Current trends of energy proportionality motivates to revise the power aware workload management schemes
Trend: non-linear power-utilization curve
Solution: utilizing servers at their high energy efficiency level through fine granularity workload management
The savings of TASP and TAWD for being thermal aware is important for future high power dense data centers

24. Questions?

25. References
[Moore et al. ATEC ’05] J. Moore, J. Chase, P. Ranganathan, and R. Sharma, “Making scheduling "cool": temperature-aware workload placement in data centers,” in ATEC ’05: Proceedings of the annual conference on USENIX Annual Technical Conference.
[Tang et al. T-PDS ’08] Q. Tang, S. K. S. Gupta, and G. Varsamopoulos, “Energy-ecient thermal-aware task scheduling for homogeneous high-performance computing data centers: A cyber-physical approach,” IEEE Trans. Parallel Distrib. Syst., vol. 19, no. 11, pp. 1458–1472, 2008.
[Chase et al. SOSP ’01] J. Chase, D. Anderson, P. Thakar, A. Vahdat, and R. Doyle, “Managing energy and server resources in hosting centers,” in SOSP ’01: Proceedings of the eighteenth ACM symposium on Operating systems principles. New York, NY, USA: ACM, 2001, pp. 103–116.
[Chen et al. NSDI ’08] Y. Chen, A. Das, W. Qin, A. Sivasubramaniam, Q. Wang, and N. Gautam, “Managing server energy and operational costs in hosting centers,” SIGMETRICS Performance Evaluation Review, vol. 33, no. 1, pp. 303–314, 2005.
[Ranganathan et al. ISCA ’06] P. Ranganathan, P. Leech, D. Irwin, and J. Chase, “Ensemble-level power management for dense blade servers,”. ISCA ’06. 33rd International Symposium in Computer Architecture, 2006, pp. 66–77.
[Kusic et al. CCJ ’09] D. Kusic, J. O. Kephart, J. E. Hanson, N. Kandasamy, and G. Jiang, “Power and performance management of virtualized computing environments via lookahead control,” Cluster Computing, vol. 12, pp. 1–15, 2009.
[Abbasi et al.HPDC’10] Abbasi, Z., Varsamopoulos, G., and Gupta, S. K. S Thermal aware server provisioning and workload distribution for internet data centers. In ACM International Symposium on High Performance Distributed Computing (HPDC10). Chicago, IL.

26. Energy efficiency for various proportionality cases
Non Energy Proportional System
Low energy efficiency at lower utilization Level
Energy Proportional System
Energy Efficiency same for all utilization levels.

27. Energy efficiency for various proportionality cases
LDR >0
Efficiency higher at higher utilization levels
LDR <0
Efficiency higher at lower utilization levels