Load Balancing and Data centers ELEN6909 Network Algorithm and Dynamics —— project presentation Topic: Load Balancing and Data centers Reference Paper: "On choosing a task assignment policy for a distributed server system.” Harchol-Balter, Mor, Mark E. Crovella, and Cristina D. Murta. Lin Su UNI: ls3201 May 18, 2015

2. Task size distribution 3. Four task assignments approaches overview Topic: Task assignment policy for a distributed server system 1. Problem and model 2. Task size distribution 3. Four task assignments approaches 4. Simulation 5. Analysis and conclusion

1. Problem and model Problem: Model: In a distributed server system, requests for service arrive and must be assigned to one of the host machines for processing. —— task assignment policy. Which task assignment policy is better? (Random; Round-Robin; Size-based; Dynamic) Model: Model task size using Bounded Pareto distribution. Tasks arrive to the system according to a Poisson process. Dispatcher facility assigns arrived tasks to one of the hosts. Tasks assigned to each host are served in FCFS order.

How to determine which task assignment is better? 1. Problem and model How to determine which task assignment is better? Take task size as variable (Bounded Pareto distribution) Measure each policy’s performance by mean waiting time and mean slowdown. ——Minimize mean waiting time. ——Minimize mean slowdown. (slowdown = waiting time/service requirement)

The distribution of task size variable: Pareto distribution 2. Task size distribution The distribution of task size variable: Pareto distribution Exponential → Heavy-tailed → Pareto → Bounded Pareto Heavy-tailed distribution Bounded Pareto distribution

Heavy-tailed distribution Exponential distribution: Heavy-tailed distribution: A distribution with a “tail” that is “heavier” than an Exponential.

Properties of Heavy-tailed distribution There may be only one large size of task increases the mean waiting time a lot.

Pareto distribution (Power-tailed distribution) Power-tailed distribution is a particular case of Heavy-tailed distribution. Pareto distribution is a particular case of Power-tailed distribution.

Properties of Pareto distribution

Bounded Pareto distribution The lower α, the more variable the distribution. (Intuitively, you can think that more tasks of large size occur.)

3. Four task assignment approaches Four approaches: Random; Round-Robin; Size-based; Dynamic A new size-based task assignment: SITA-E Random: h hosts, each job gets assigned to a host with probability 1/h. Round-Robin: ith job assigned to host i mod h. Size-Based: Each host serves tasks whose service demand falls in a designated range. Dynamic: (Shortest-queue) (Least-work-remaining) job immediately dispatched to the host with the shortest queue.

Size-based task assignment: SITA-E SITA-E (Size Interval Task Assignment with Equal Load) Idea: Define the size range associated with each host such that the total work load directed to each host is the same. Motivation: Balance the load to minimizes mean waiting time. Use the task size distribution to define the cutoff points (range) so that the expected work directed to each host is the same.

4. Simulation Measurement parameters Mean waiting time. Mean slowdown. (slowdown = waiting time/service demand) Consider α in the range 1.1 to 1.9. (α value in the range 1.0 to 1.3 tend to be common in empirical measurements of computing system.) The lower α, the more variable the distribution. → the longer the mean waiting time/slowdown.

Simulation result observation Random and Round Robin policies are similar. As α declines, both of the performance metrics explode approximately exponentially. Dynamic policy performs quite well when α is larger (α→2), however when variability in task size decreases as α→1, Dynamic is unable to maintain good performance. SITA-E performs well over the entire range of α value studied. When task sizes are less variable, Dynamic task assignment exhibits better performance; When task sizes show more variability, SITA-E’s performance can be 100 times better than that of Dynamic.

5. Analysis and conclusion Notation/definitions used in Queuing Theory Pollaczek-Kinchin formula Analysis of four task assignment policies Conclusion

Notation/definitions used in Queuing Theory Little’s Law

Notation/definitions used in Queuing Theory Kendall Notation —— the standard system used to describe and classify a queuing node. M/M/1 queue represents the queue length in a system having a single server, where arrivals are determined by a Poisson process and job service times have an exponential distribution. M/D/1 queue represents the queue length in a system having a single server, where arrivals are determined by a Poisson process and job service times are fixed (deterministic). M/G/1 queue is a queue model where arrivals are Markovian (modulated by a Poisson process), service times have a General distribution and there is a single server.

Pollaczek-Kinchin formula

Analysis of four approaches

Analysis of four approaches Three reasons why SITA-E policy performs so well: By limiting the range of task sizes at each host, SITA-E greatly reduces the variance of the task size distribution, thereby improving performance at these host. Intensified by the heavy-tailed property, very few tasks are assigned to high numbered host. Small tasks observe proportionately lower waiting times.

Conclusion How the variability of the task size distribution influences which task assignment policy is best in a distributed system? When task sizes are not highly variable, the Dynamic policy is preferable. When task sizes show the degree of variability more characteristic of empirical measurements (α≈1), SITA-E is best. Random, Round-Robin and Dynamic policies show that their performance is directly proportional to the second moment of the task size distribution. SITA-E performs well for three main reasons: 1. By limiting the range of task sizes at each host, SITA-E greatly reduces the variance of the task size distribution, thereby improving performance at these host. 2. Intensified by the heavy-tailed property, very few tasks are assigned to high numbered host. 3. Small tasks observe proportionately lower waiting times.

ELEN6909 Network Algorithm and Dynamics —— project presentation Topic: Load Balancing and Data centers ——Task assignment policy for a distributed server system Reference Paper: "On choosing a task assignment policy for a distributed server system.” Harchol-Balter, Mor, Mark E. Crovella, and Cristina D. Murta. Thank you！ Lin Su UNI: ls3201 May 18, 2015