Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis 1  Load-balancing problem  migrate (large) jobs from a busy server to an underloaded  Depends on job sizes  which job to migrate? Q: What if job sizes are exponential?  Pareto: top 1% of jobs  50% of total load  Exp: top 1%  only 5% of total load

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Exponential  C 2 = 1 Q: How to capture C 2 < 1? A: Consider a sum of k random variables (IID) with exp(kμ)  Erlang-K 2

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis Q: How to capture C 2 > 1? A: Hyper-exponential Q: Expectation? C 2 ? Can combine both in many phases (Coxian distribution) to capture any distribution G with a Laplace Transform 3

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Poisson Arrivals  Service with C 2 = ½ (less variance than exponential)  Represent with erlang-2  service had two phases in tandem Plus: Can represent many complex problems Minus: Can only be solved numerically using matrix methods 4

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  M/G/1 system  M: stands for memoryless (poisson) arrivals  1: stands for a single server  G: stands for generic (any!) prob. distribution on service times Q: What is the expected queueing delay of an arriving customer E[T]? 5 Poisson (λ) arrivals Service time S: P(S ≤ s) = F(s) S1S1 S2S2 S3S3 S4S4 S0S0  S1, S2,.., SN : service times of queued jobs  S 0 : service time of job currently served  S e : remaining service time for current job arrival SeSe

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Time T between buses: random variable with mean E[T]  Residual time R = wait time for passenger  What is E[R]???  Case 1: Exponential with mean E[T] = 20min  Case 2: 90% chance to wait exactly 10min, 10% wait 110min  Answer?  A: 10min  B: 20min  C: 30min 6 t arriving at bus stop T R

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Why is this???  E[T]: average over all intervals T 1, T 2, T 3  E[R]: larger chance to fall (measure) inside a large interval!  The larger the higher  High variance => High E[R]  What about deterministic arrivals (every 20min)?  But what is E[R] exactly? Will need Renewal Theory for this 7 t T large T small T1T1 T2T2

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis (t1,t2)  Poisson: time between events is exponential  Renewal: time between events has general distribution F(t)  F(0) E(T) > 0 1. Independence: inter-arrival times are independent a) Poisson: b) Renewal (process renews after an arrival): 2. Stationary Increments: # of events in (t1,t2) = f(t2-t1) a) Poisson: at any (t1,t2), # of events in (t1,t2) independent of past b) Renewal: # events in T2 = (t1,t2) changes if known event at t1-ε 8 P{T < t} = F(t) √ √ √ X

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Rate of arrivals in the long run (t -> ∞)  Denote m(t) = E[N(t)]  Difficult to derive for small t!  For large t? 9

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Time between events distributed as F(T) with mean E[T]  At every event i, we get a reward (or cost) R i  R i is a random variable with mean E[R]  independent from all R j (j≠i)  How do we accumulate rewards over time? 10 R1R1 R2R2 R3R3 RNRN …

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  (time-) average wait a random customer sees is  Renewal-Reward Model  T i : cycle with mean E[T]  Renewal-Reward Theorem =>  Reward during a cycle of Ti => =>  Expected wait is R(t) t T1T1 T1T1 customer arrives at random t “sees” this wait time

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis Q: What is the expected queueing delay of an arriving customer E[T]? 12 Poisson (λ) arrivals Service time S: P(S ≤ s) = F(s) S1S1 S2S2 S3S3 S4S4 S0S0  S1, S2,.., SN : service times of queued jobs  S 0 : service time of job currently served  S e : remaining service time for current job arrival SeSe

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis (Pollaczek-Khinchin Formula)  M/M/1?  Service is exponential: E[S e ] = 1/μ  M/D/1?  Service is deterministic: E[S e ] = 1/(2μ)  D/D/1?  Arrival times and Service times deterministic:  M/G/1: potentially high delays even in low load!  (slide 14, case 2) E[S] = 20, E[S e ] = 32.5  3x worse than M/D/1 Variability causes queueing…and delays!

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  Can you think of some examples?  Example 1: Cognitive Network  A Primary User and a Secondary user  Primary user has a usage profile => Free Slots? -Equally sized silence periods every T seconds -Large variance in usage period (bursty)  Time (low traffic) SU has to wait until channel is available?  Example 2: Supercomputer  A high traffic user with large traffic variability  A low traffic user  Time to wait for job to complete (for low traffic user)  Example 3: An M/G/1/2 queue  Only one customer allowed in queue (+ one in service)  Customers finding already someone in queue leaves 14

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis  LCFS (Last Come First Serve)  Anyone coming after me, will get service before me  Seems like a really bad idea  Random  Whenever a job is finished, pick up any job in the queue with equal probability  Seems better than LCFS, and better than FCFS  Turns out that all have the same delay!!  Equal to that of M/G/1/FCFS 15

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis 16  CPUs  time-sharing  All processes currently running, share the CPU time equally  Web Servers  time-sharing  HTTP requests served in parallel (to give TCP the impression of continuity)  When k jobs in system  each job served at rate μ/k μ Poisson(λ)

Thrasyvoulos Spyropoulos / Eurecom, Sophia-Antipolis Q: M/M/1/PS performance? A: Exactly like M/M/1!!! Q: What about M/G/1/PS? A: Exactly like M/M/1 Key Result: Network of PS servers with general service has product form solution (like Jackson networks) 17