1. Anshul Gandhi (Carnegie Mellon University) Varun Gupta (CMU), Mor Harchol-Balter (CMU) Michael Kozuch (Intel, Pittsburgh)

2.  Server farms are important for today’s IT infrastructure (Amazon, Google, IBM, HP, …)  However, server farms cost a lot of money to power ($4 billion in 2006) Server Farm Requests

3.  How many servers, given request rate ?  Don’t want to waste power Requests Server Farm

4. 1. Server farm model 2. Provisioning for fixed arrival rate 3. Provisioning for unpredictable, time-varying arrival rate 4. Future work

5. 5 IDLE servers consume a lot of power ~ 60 % of BUSY BUSY IDLE OFF

6. 6 Turn IDLE servers OFF to save power BUSY OFF HOWEVER

7. 7 To turn on an OFF server.. BUSY OFFSETUP Time delay (setup time) 1 min – 5 mins and Power penalty peak power during setup time

8. 8 To turn on an OFF server.. BUSY OFFSETUP Should we ever turn servers OFF ?

9. 9  Server states: BUSY P BUSY 240 W IDLE P IDLE 150 W OFF P OFF 0 W SETUP P SETUP 240 W  Setup times: T OFF→ON 200 s T ON→OFF 0 s Intel Xeon E5320 2 X 1.86 GHz quad-core 4GB memory ON

10. 10  Poisson arrival process: λ(t) requests/sec  Exponentially distributed job sizes: E[S] secs  Load: ρ(t) = λ(t) ∙ E[S] Minimum # servers to handle incoming load Requests FCFS Server Farm

11. 11  Interested in response time and power conumption  Perf/W = 1/(Mean RT X Mean Power)  Maximize Perf/W

12. 1. Server farm model 2. Provisioning for fixed arrival rate 3. Provisioning for unpredictable, time-varying arrival rate 4. Future work

13. 13  Existing solutions: prediction based, reactive controllers.  Is there a simple, yet, near-optimal solution ? Poisson arrivals Server Farm Max. Perf/W

14. 14  Keep n servers always ON (M/M/n)  Servers are BUSY or IDLE

15. 15

16. 16  Turn servers OFF when IDLE  Servers are BUSY, OFF or in SETUP Auto-scales if n is high

17. 17

18. 18 T ON→OFF < γ E[S]/√ρ

19. 19  Best of {NEVEROFF, INSTANTOFF} is optimal for single-server  Multi-server ? For ρ > 10, we are within 20% of OPT

20. 1. Server farm model 2. Provisioning for fixed arrival rate 3. Provisioning for unpredictable, time-varying arrival rate 4. Future work

21. 21  Data center demand has daily variations  INSTANTOFF can auto-scale

22. 22  NEVEROFF requires continual updates based on predicted load  Predictions are not always accurate  Can we find a simple traffic-oblivious policy?  Auto-scaling in nature

23. 23  Like INSTANTOFF, except we wait for t wait seconds before turning IDLE servers OFF  Routing ? MRB routing is crucial !

24. 24  Rule of thumb: t wait ∙ P IDLE = T OFF→ON ∙ P ON

25. 25 Worse at higher frequencies

26. 26  1998 World Cup Soccer trace (ITA)

27. 1. Server farm model 2. Provisioning for fixed arrival rate 3. Provisioning for unpredictable, time-varying arrival rate 4. Future work

28. 28  Experimental evaluation of proposed schemes  Preliminary experiments on 15-server testbed using CPU-bound workload and sinusoidal arrival pattern  Experimental results agree with analysis  Web workloads: ▪ What does the experimental setup look like ?  Try out various arrival traces and workloads

29. 29  Anshul Gandhi, Varun Gupta, Mor Harchol-Balter, Michael Kozuch Optimality analysis of energy-performance trade-off for server farm management, PERFORMANCE 2010  Anshul Gandhi, Mor Harchol-Balter, Ivo Adan Server farms with setup costs, PERFORMANCE 2010  Anshul Gandhi, Varun Gupta, Mor Harchol-Balter, Michael Kozuch Energy-efficient dynamic capacity provisioning in server farms, CMU technical report CMU-CS-10-108