1. How Useful is Old Information? On the Management and Efficiency of Cloud Based Services seminar MICHAEL MITZENMACHER November 2010 Computer Science faculty Technion Alexander Zlotnik IEEE Transactions on Parallel and Distributed Systems, 1997

2. Amazon Elastic Load Balancing Share Incoming Traffic Scale Up Scale Down Detect & shutdown unhealthy EC2 instances Balance single/multiple Availability ZonesAvailability Zones Reports $0.025 per hour [$18/month] + Traffic

3. Background What is this paper not – Real system – Scale up/down system – Groups/AZ balancing

4. Background Supermarket model & benefits 1.On incoming task, poll d queues for their load 2.Send the task to the shorter queue -For d = 2 the service time is exponentially shorter -Further d = 3, 4, … the improvement is linear

5. Background Load Balancing Types – Centralized/Distributed – Static/Dynamic(adaptive)

6. Setting n nodes Tasks Arrival: Poisson (λn) Task is forwarded to a node for execution Execution rate is distributed Exp(µ) – Normalized to µ=1 Every T time units a bulletin board is updated with the current loads of all nodes

7. Policy Look at d random entries on the board, send the task to the shortest queue d=1 – M/M/1 d=n – Shortest queue  Static

8. Periodic updates 0T2T3TTtTt NodePosted LoadTrue Load 044 100 233 … n-122 True Load 3 1 2 3 2 2 3 1 3 1 2 0 1 1 2 3 Posted Load 1 1 2 3

9. Definitions P i,j (t) – Fraction of queues with posted load i, but have true load j q i (t) - Rate of arrivals at a queue of size i NodePosted LoadTrue Load 044 100 233 … n-122 01234…J 0 1 2 3 4 … I Posted Load True Load P i,j (t)

10. Between board updates 01234…J 0 1 2 3 4 … I Posted Load True Load µ qiqi µ qiqi P i,j (t)

11. On board update 01234…J 0 1 2 3 4 … I Posted Load True Load P i,j (t) 0 0 πiπi bibi

12. 01234…J 0 1 2 3 4 … I Posted Load True Load P i,j (t) bibi Σ b i (t) – Fraction of queues with load i posted

13. b i (t) – Fraction of queues with posted load i Arrival rate to a queue with posted load i: Tasks arrival Chance that d selected queues have load i or more Chance that a queue of size i is selected Same chance for all queues of size i

14. Fixed Cycle Hope: Convergence to a fixed point – State that there is no “motivation” to exit from – Periodic Updates – “Jump” on bulletin board updates, t=kT – Fixed cycle: b i do not change Or, for k≥k 0, P(kT) = P(k 0 T)

15. Fixed Cycle 01234…J 0 1 2 3 4 … I Posted Load True Load P i,j (t) bibi Σ πiπi Σ If π((k-1)T)= π(kT), the next phase will be the same

16. Fixed Cycle – finding vector π Method A Run the system until changes in π are small Method B m i,j (T) – Probability for M/M/1 queue to start with i tasks and after time T to have j tasks Solving a system of equations: Method A on truncated system of differential equations

17. Fixed Cycle – finding vector π Method B m i,j (T) – Probability for M/M/1 queue to start with i tasks and after time T to have j tasks – Solving a system of equations: Iterating 012…J 0 1 2 … I π – Iteration start = x π – Iteration end Until small changes in π

18. Fixed Cycle – finding vector π Method B m i,j (T) – Probability for M/M/1 queue to start with i tasks and after time T to have j tasks – Iterating a system of equations: Bessel function of the first kind

19. Fixed Cycle – finding vector π Method A on truncated system of differential equations – Bound I, J – Iterate on – Until small changes in π

20. Results

23. More complex Centralized Strategies Time based – Split T into subintervals – At subinterval [t k,t k+1 ) tasks sent to random server with load at most k

24. More complex Centralized Strategies Record-insert – Upon task arrival: Sent to random server of servers with lowest load Server’s load is incremented on bulleting board Task ending not updated (until end of phase) – Uses real loads, not expectation Always better than time-based

25. More complex Centralized Strategies

26. Continuous Update

27. Continuous Updates Tasks benefit on occasional accurate info – Supported by other graphs too Partial accurate info – not helpful – If not known which data is up to date

28. Individual Updates

29. Conclusions Load Balancing is useful even with stale information Choosing least loaded of 2 nodes is better than shortest or 3 or more More complex policies can further help load balancing (like time-based or record-insert)

30. Open Questions Additional metrics Different theoretical frameworks More realistic arrival patterns (heavy-tailed)

31. Personal review Powerful theoretical framework Interesting and relevant results Cons – Too few loads scenarios – No comparison between the amount of needed bandwidth to load balancing benefit

32. Relevant Information More references – Collection of works: “The power of Two Random Choices: A Survey of Techniques and Results” M. Mitzenmacher et al 2001 Not covered in this presentation: – Deviation of simulating differential equations instead of full system [Sections 3.4, 3.5] – Theory of some systems, but their results are – Competitive Scenarios [Section 6] – Small number of servers