1. 1 Randomized Load Balancing with & without Memory EE384Y – Packet Switch Architecture – II Rajan Goyal/Jianying Luo {rgoyal, jyluo}@stanford.edu

2. 2 Outline Introduction Motivation Simulation Study Acknowledgment

3. 3 Part - I

4. 4 Outline Introduction Motivation Simulation Study Acknowledgment

5. 5 Super Market Model 123n Allocator μ1μ1 poisson (nλ), λ < 1 μ2μ2 μ3μ3 μnμn Symmetric Service Rate: μ 1 = μ 2 = μ 3 = … = μ n = 1 Asymmetric Service Rate: Σ i μ i = n

6. 6 Assignment Policy Different policies for assigning load –Join shortest queue, SQ –Join random queue, SQ(1) –Join shortest of d random queues, SQ(d) –Join shortest of (d+m) queues, SQ(d,m) where d – fresh random samples, m – shortest of the (d+m) choices in last iteration.

7. 7 Outline Introduction Motivation Simulation Study Acknowledgment

8. 8 Motivation Given this background, we tried to study following three questions: –How many samples? i.e. d = ? –How much memory? i.e. m = ? –Given a choice of (d,m) and d+m=k, where k is constant, what is the right order of d and m?

9. 9 Outline Introduction Motivation Simulation Study Acknowledgment

10. 10 Simulation Setup 123n (d,m) policy μ1μ1 poisson (nλ), λ < 1 μkμk μ k+1 μnμn Asymmetric Service Rate: Σ i μ i = 1500,  1  i  1500, Σ i μ i = 1000,  1  i  25, Σ i μ i = 500,  26  i  1500, n=1500, k = 25 n μ n-1 System Unstable for λ  0.356 Symmetric Service Rate: μ 1 = μ 2 = μ 3 = … = μ n = 1

11. 11 Agenda –How many samples? i.e. d = ? –How much memory? i.e. m = ? –Given a choice of (d,m) and d+m=k, where k is constant, what is the right order of d and m?

12. 12 Effect of ‘d’ on Queue Size

13. 13 Effect of ‘d’ on Variance

14. 14 Effect of ‘d’ (Observations) For symmetric service rate, d>=2 gives exponential improvement over (d=1) policy in terms of maximum load. (d=2) achieves maximum gain for unit increase in d.

15. 15 Agenda –How many samples? i.e. d = ? –How much memory? i.e. m = ? –Given a choice of (d,m) and d+m=k, where k is constant, what is the right order of d and m?

16. 16 Effect of ‘m’ on Queue Size

17. 17 Effect of ‘m’ on Variance

18. 18 Effect of m (Observation) Memory keeps tail of the queue size very low, rather than average queue size. i.e. stops queues from growing. Memory reduces the variance of queue sizes among queues.

19. 19 Part - II

20. 20 Effect of memory Why memory helps? –Memory sort of gives (indirect) feedback to the allocator regarding server states, which is missing when just fresh samples are picked. –Memory is useful if variance of queue sizes (between all queues) is high. –So, memory is more effective for asymmetric service rate servers.

21. 21 Why memory helps?

22. 22 How much memory?

23. 23 How much memory?

24. 24 How much memory? (Observations) Increase of memory decreases the queue size and variance. However, when memory is large enough, the improvement due to increase in memory is not appreciable.

25. 25 Agenda –How many samples? i.e. d = ? –How much memory? i.e. m = ? –Given a choice of (d,m) and d+m=k, where k is constant, what is the right order of d and m?

26. 26 Exact Order d+m=k, where k is constant, what is right balance of d & m? –One extreme policy is (1,k-1) and other extreme is (k,0). Adv. of extra sample at the expense of memory? Adv. of extra memory at the expense of fresh sample? Simulation study for (d+m = 4, d+m = 8)

27. 27 Exact Order (d+m=4)

28. 28 Exact Order (d+m=4)

29. 29 Exact Order (d+m=4)

30. 30 Observations From (1, k-1) to (k, 0), ‘d’ is increasing and ‘m’ is decreasing. i.e. effects of ‘d’ and ‘m’ are competing with each other. Based on previous results, increase in ‘d’ has more effect as compared to increase in ‘m’. Also, (k,0) is unstable.

31. 31 Acknowledgment We are indebted to Balaji for suggesting this topic and insight. Thanks to Balaji & Devavrat for discussions.

32. 32 Questions

33. 33 Analysis SQ is ideal, but hard to implement when n is large. For symmetric service rates, it is clear that SQ(d) is stable. SQ(1) is stable for λ =i)= λ i. It’s well known 1 that: –For SQ(d), P(Q>=i)= λ (d i -1)/(d-1) Note: Our simulation is consistent with the analytical model

34. 34 Analysis (contd…) For asymmetric service rate, it is known 2 that: –SQ(d) is unstable even when d = O(n) †. –SQ(1,1) is stable, if λ < 1. 1 Load Balancing and Density Dependent Jump Markov Processes, Michael Mitzenmacher 2 The use of memory in randomized load balancing, Devavrat Shah & Balaji Prabhakar † sampling is done with replacement.

35. 35 Simulation Setup 123n (d,m) policy μ1μ1 poisson (nλ), λ < 1 μ2μ2 μ3μ3 μnμn Symmetric Service Rate: μ 1 = μ 2 = μ 3 = … = μ n = 1 n=1500, λ = 0.99