Effect of higher moments of job size distribution on the performance of an M/G/k system VARUN GUPTA Joint work with: Mor Harchol-Balter Carnegie Mellon University Jim Dai, Bert Zwart Georgia Institute of Technology

2 Multi-server/resource sharing systems are the norm today Multicore chips Call centers Server Farms

3 M/G/k: the classical multi-server model Poisson arrivals (rate ) J1J1 J i+1 JiJi J2J2 J i+2 GOAL : Analysis of mean delay (time spent in buffer)

4 M/G/k model assumptions and notation Poisson arrivals Service requirements (job sizes) are i.i.d. S ≡ random variable for job sizes Define Per server utilization or load: 0 <  < 1 Squared coefficient of variability (SCV) of job sizes: C 2  0

5 M/G/k mean delay analysis Lets take a step back: M/G/1

6 M/G/k mean delay analysis Lets take a step back: M/G/1

7 M/G/k mean delay analysis Lee and Longton (1959) –Simple and closed-form –Involves only first two moments of S –Exact for k=1 –Asymptotically exact in heavy traffic [Köllerström[74]] No exact analysis exists All closed-form approximations involve only the first two moments of S –Takahashi[77], Hokstad[78], Nozaki Ross[78], Boxma Cohen Huffels[79], Whitt [93], Kimura[94]

8 But… Q1: Are 2 moments of S enough to reasonably approximate E[Delay]? Does the third moment have no/negligible effect? Q2: How inaccurate can a 2-moment approximation be? Q3: Are 3 moments enough? 4 moments?

9 Outline Q1: Are 2 moments of S enough to reasonably approximate E[Delay]? Does the third moment have no/negligible effect? Q2: How inaccurate can a 2-moment approximation be? Q3: Are 3 moments enough? 4 moments?

10 H2H2 The H 2 class has three degrees of freedom Can vary E[ S 3 ] while keeping first two moments constant Can numerically evaluate M/H 2 /k using the matrix analytic method

11 E[Delay] vs. E[ S 3 ] k=10, E[ S ]=1, C 2 =19,  =0.9 E[Delay] E[S3]E[S3] X moment approx E[Delay] M/H 2 /k

12 X10 4 E[S3]E[S3] E[Delay] E[Delay] M/H 2 /k 2-moment approx E[ S 3 ] can have a huge impact on mean delay! The mean delay decreases as E[ S 3 ] increases!

13  x) Intuition for the effect of E[ S 3 ]  x) = load due to jobs smaller than x E[ S ]=1C 2 =19 x

14  x) E[ S 3 ]=600 Intuition for the effect of E[ S 3 ]  x) = load due to jobs smaller than x E[ S ]=1C 2 =19 x

15  x) E[ S 3 ]=600 E[ S 3 ]=700 Intuition for the effect of E[ S 3 ]  x) = load due to jobs smaller than x E[ S ]=1C 2 =19 x

16  x) E[ S 3 ]=600 E[ S 3 ]=700 E[ S 3 ]=1200 Intuition for the effect of E[ S 3 ]  x) = load due to jobs smaller than x E[ S ]=1C 2 =19 x

17 E[ S 3 ]=600 E[ S 3 ]=700 E[ S 3 ]=1200 E[ S 3 ]=15000  x) = load due to jobs smaller than x E[ S ]=1C 2 =19 Intuition for the effect of E[ S 3 ] x  x)

18 As E[ S 3 ] increases (with fixed E[ S ] and E[ S 2 ]): Load gets ‘concentrated’ on small jobs Load due to ‘big’ jobs vanishes Bigs become rarer, usually see small jobs only Causes drop in E[Delay] M/H 2 /k x  x) Increasing E[ S 3 ] Intuition for the effect of E[ S 3 ]

19 X10 4 E[S3]E[S3] E[Delay] E[Delay] vs. E[ S 3 ] k=10, E[ S ]=1, C 2 =19,  =0.9 E[Delay] M/H 2 /k 2-moment approx

20 Outline Q1: Are 2 moments of S enough to reasonably approximate E[Delay]? Does the third moment have no/negligible effect? Q2: How inaccurate can a 2-moment approximation be? Q3: Are 3 moments enough? 4 moments?

21 Outline Q1: Are 2 moments of S enough to reasonably approximate E[Delay]? Does the third moment have no/negligible effect? Q2: How inaccurate can a 2-moment approximation be? Q3: Are 3 moments enough? 4 moments?

22 {G|C 2 } ≡ positive distributions with mean 1 and SCV C 2 E[Delay] G1G1 G2G2 GAP  Error of 2-moment approx

23 Our Theorems Upper bound Lower bound –  <1-1/k –   1-1/k

24 E[Delay] GAP

25 E[Delay] Conjecture

26 Outline Q1: Are 2 moments of S enough to reasonably approximate E[Delay]? Does the third moment have no/negligible effect? Q2: How inaccurate can a 2-moment approximation be? Q3: Are 3 moments enough? 4 moments?

27 Outline Q1: Are 2 moments of S enough to reasonably approximate E[Delay]? Does the third moment have no/negligible effect? Q2: How inaccurate can a 2-moment approximation be? Q3: Are 3 moments enough? 4 moments?

28 What about higher moments? {G|C 2 } H2H2 H*3H*3 H * 3 class has four degrees of freedom Can vary E[ S 4 ] while keeping first three moments constant

29 X10 4 E[S3]E[S3] E[Delay] Increasing fourth moment E[Delay] vs. E[ S 4 ] k=10, E[S]=1, C 2 =19,  =0.9 E[Delay] M/H 2 /k 2-moment approx

30 X10 4 E[S3]E[S3] E[Delay] Increasing fourth moment E[Delay] M/H 2 /k 2-moment approx Even E[ S 4 ] can have a significant impact on mean delay! High E[ S 4 ] can nullify the effect of E[ S 3 ]!

31 E[Delay] The BIG picture (Conjecture) LB 1 =E[Delay] M/D/k UB 1,2 =(C 2 +1)E[Delay] M/D/k LB 1,2,3 UB 1,2,3,4 LB 1,2,3,4,5 Odd/Even moments refine the Lower/Upper bounds on mean delay

32 Outline Q1: Are 2 moments of S enough to reasonably approximate E[Delay]? Does the third moment have no/negligible effect? Q2: How inaccurate can a 2-moment approximation be? Q3: Are 3 moments enough? 4 moments?

33 Conclusions Shown that 2-moment approximations for M/G/k are insufficient Shown bounds on inaccuracy of 2-moment only approximations –(C 2 +1)/2 inaccuracy factor Observed alternating effects of odd and even moments

34 Thank you!

35 Open Questions Proof (or counter-example) of conjectures on bounds Are there other attributes of service distribution that characterize it better than moments? –For example, mean and variability of small and big jobs Where do real world service distributions sit with respect to these attributes?

36 {G|C 2 } ≡ positive distributions with mean 1 and SCV C 2 H2H2 The H 2 class has three degrees of freedom (  s,  p, p s ) Can vary E[ S 3 ] while holding first two moments constant

37 Look at the moments of H 2 … Load due to big jobs vanishes as E[ S 3 ] increases When k>1, a big job does not block small jobs This reduces the effect of variability (C 2 ) as third moment increases

38 Observations  < 1-1/k  UB/LB  (C 2 +1) –No 2-moment approximation can be accurate in this case [Kiefer Wolfowitz] [Scheller-Wolf]: When  > 1-1/k, E[Delay] is finite iff C 2 is finite. –Matches with the conjectured lower bound –Also popular as the “0 spare server” case

39 What about higher moments? {G|C 2 } H2H2 H*3H*3 H*3H*3 H * 3 allows control over fourth moment while holding first three moments fixed The fourth moment is minimized when p 0 =0 (H 2 distribution)

40 E[Delay]   0 H * has the smallest third moment in H n  {G|C 2 } third moment   as   0

41 Proof outline: Upper bound THEOREM: PROOF: Consider the following service distribution Intuition for conjecture: k>1 should mitigate the effect of variability; D * exposes it completely Note: D * has the smallest third moment in {G|C 2 }

42 Proof outline: Lower bound THEOREM: –  < 1-1/k –   1-1/k PROOF: Consider the following sequence of service distributions in {G|C 2 } as   0

43 E[Delay]   0 D * has the smallest third moment in {G|C 2 } third moment   as   0

44 E[Delay] Conjecture

45 E[Delay] GAP