Little’s Law & Operational Laws

Little’s Law Proportionality relation between the average number of jobs (E[N]) in a system and the average system time (E[T]) of those jobs (the job arrival rate is the proportionality constant) – No assumptions about the arrival or service processes Two similar versions: Open and closed systems Little’s Law for an ergodic open system: E[N] = λ  E[T] – λ is the average job arrival rate in the system Little’s law for an ergodic closed system: N = X  E[T] – N is constant and equal to the multi-programming level – X is the system throughput (rate of job completion) – E[T] = E[R] + E[Z], where R is response time, and Z think time – Alternative version (Response Time Law): E[R] = N/X – E[Z] 2

Applying Little’s Law – (1) Assume that the above system is ergodic with X = = 1 + 2, where 1 ( 2 ) is he arrival rate of red (green) jobs – Ergodicity requires <  Utilization Law: system = server – Let  (  i ) be fraction of time server is busy (with type i jobs) – Note: This is also the average number of (type i) jobs in service – Little’s Law implies  = /  (  i = i /  ), or  = E[S] Note: If E[S 1 ]  E[S 2 ], then  i = i /  i, where  i = E[S i ] Time in queue: system = queue – E[N Q i ] = i E[T Q i ] 3  1 2

Applying Little’s Law – (2) Assume N = 10, E[Z] = 5secs, and E[R] = 15secs What is the system throughput X? N = X  E[T] = X(E[Z] + E[R])  X = N/(E[R] + E[Z]) = 10/(5+15) = 0.5 jobs/sec 4 … in out N

Forced Flow Law Relates system throughput to throughput of individual device – X is system throughput, X i is throughput of device i – V i is number of visit to device i per job Forced Flow Law: X i = E[V i ]  X – Basically accounts for the fact that for every system completion, there must have been E[V i ] completions (visits) at device i, so the rate of completion at device i must be higher in that same proportion 5

Bottleneck Law D i is the per job service demand at device i, i.e., total demand across all visits to device i – D i = S i,1 + S i,2 + … + S i,V i – This implies E[D i ] = E[V i ]  E[S i ], if V i and S i are independent (a typical scenario) – Note that E[D i ] = B i /C, where B i is the busy time at device i and C is the number of service completion during that time period Bottleneck Law:  i = X  E[D i ] (  i = X i  E[S i ] = X  E[V i ]  E[S i ] = X  E[D i ]) – Rate of outside arrivals is X and each such arrival contributes E[D i ] worth of service time at device i, so that device i a fraction of time (  i )equal to X  E[D i ] 6

Summary Open systems – E[N] = λ  E[T] – E[N subsystem ] = subsytem E[T subsystem ] Closed systems – N = X  E[T] = X  (E[R] + E[Z]) – E[R] = N/X – E[Z] (Response Time Law) –  i = i /  i = i E[S i ] = X i E[S i ] (Utilization Law) – X i = E[V i ]  X (Forced Flow Law) –  i = X  E[D i ] (Bottleneck Law) 7

Building on Little’s Law Bounds on throughput and response time for closed systems where D max = max i {E[Di]} (system bottleneck) First expression holds for N small, while second expression holds for N large N * as the N value for which the two expressions are equal, i.e., N * = (D+E[Z])/D max – When N > N *, throughput is dominated by bottleneck component 8

Example (1) Which system has higher throughput? – First compute N * (N * = (D+E[Z])/D max ) for both systems D A = D CPU + D disk = 8.6, D A,max = 4.6  N A * = (8.6+5)/4.6 = 2.95 D B = D CPU + D disk = 6.8, D B,max = 4.9  N B * = (6.8+5)/4.9 = 2.41 N A * & N B * << N – So in both cases, the throughput is dominated by D max, and System A wins because it has a lower D max value 9 … Disk N = 10, E[Z] =5 CPU System A: D CPU = 4.6 and D disk = 4.0 System B: D CPU = 4.9 and D disk = 1.9

Example (2) Measurements for interactive system with N = 20 and E[Z] = 15 secs: – T = 650 secs (duration of measurements) – B CPU = 400 secs – B slowdisk = 100 secs – B fastdisk = 600 secs – C = C CPU = 200 jobs – C slowdisk = 2,000 jobs – C fastdisk = 20,000 jobs Improvements under consideration (are they worth it?) – Faster CPU – Rebalancing disks – Add a second fast disk, and split the load of the original fast disk using it (Note: This is sub-optimal – should rebalance across all three disks) – Combine all three improvements, including rebalancing all three disks 10 Summary – N = X  E[T] = X  (E[R] + E[Z]) – E[R] = N/X – E[Z] (Response Time Law) –  i = i /  i = i E[S i ] = X i E[S i ] (Utilization Law) – X i = E[V i ]  X (Forced Flow Law) –  i = X  E[D i ] (Bottleneck Law)

Example (2a) Measurements for interactive system with N = 20 and E[Z] = 15 secs: – T = 650 secs (duration of measurements) – B CPU = 400 secs, B slowdisk = 100 secs, B fastdisk = 600 secs – C = C CPU = 200 jobs. C slowdisk = 2,000 jobs, C fastdisk = 20,000 jobs Intermediate quantities – E[D CPU ] = B CPU /C = 400 secs/200 jobs = 2.0 secs/job – E[D slowdisk ] = B slowdisk /C = 100 secs/200 jobs = 0.5 secs/job – E[D fastdisk ] = B fastdisk /C = 600 secs/200 jobs = 3.0 secs/job – E[V CPU ] = C CPU /C = 200 visits/200 jobs = 1.0 visit/job – E[V slowdisk ] = C slowdisk /C = 2,000 visits/200 jobs = 10 visits/job – E[V fastdisk ] = C fastdisk /C = 20,000 visits/200 jobs = 100 visits/job – E[S CPU ] = B CPU /C CPU = 400 secs/200 visits = 2.0 secs/visit – E[S slowdisk ] = B slowdisk /C slowdisk = 100 secs/2,000 visits = 0.05 sec/visit – E[S fastdisk ] = B fastdisk /C fastdisk = 600 secs/20,000 visits =0.03 sec/visit Original system: D max = 3 secs/job, D = 5.5 secs/job, N * = 20.5/3  7 << N = 20 – So X  0.33 jobs/sec and E[R] = 45 secs 11 Summary – N = X  E[T] = X  (E[R] + E[Z]) – E[R] = N/X – E[Z] (Response Time Law) –  i = i /  i = i E[S i ] = X i E[S i ] (Utilization Law) – X i = E[V i ]  X (Forced Flow Law) –  i = X  E[D i ] (Bottleneck Law)

Example (2) – Faster CPU Metrics for interactive system with N = 20 and E[Z] = 15 secs: – E[D CPU ] = B CPU /C = 400 secs/200 jobs = 2.0 secs/job – E[D slowdisk ] = B slowdisk /C = 100 secs/200 jobs = 0.5 secs/job – E[D fastdisk ] = B fastdisk /C = 600 secs/200 jobs = 3.0 secs/job (bottleneck) – E[V CPU ] = C CPU /C = 200 visits/200 jobs = 1.0 visit/job – E[V slowdisk ] = C slowdisk /C = 2,000 visits/200 jobs = 10 visits/job – E[V fastdisk ] = C fastdisk /C = 20,000 visits/200 jobs = 100 visits/job – E[S CPU ] = B CPU /C CPU = 400 secs/200 visits = 2.0 secs/visit – E[S slowdisk ] = B slowdisk /C slowdisk = 100 secs/2,000 visits = 0.05 sec/visit – E[S fastdisk ] = B fastdisk /C fastdisk = 600 secs/20,000 visits =0.03 sec/visit Faster CPU (twice as fast): E[D CPU ] = 1.0 sec/job, but D max remains unchanged and N * stays approximately constant – Hardly any improvement as the fast disk is the bottleneck 12 Summary – N = X  E[T] = X  (E[R] + E[Z]) – E[R] = N/X – E[Z] (Response Time Law) –  i = i /  i = i E[S i ] = X i E[S i ] (Utilization Law) – X i = E[V i ]  X (Forced Flow Law) –  i = X  E[D i ] (Bottleneck Law)

Example (2) – Rebalance Disks Metrics for interactive system with N = 20 and E[Z] = 15 secs: – E[D CPU ] = B CPU /C = 400 secs/200 jobs = 2.0 secs/job – E[D slowdisk ] = B slowdisk /C = 100 secs/200 jobs = 0.5 secs/job – E[D fastdisk ] = B fastdisk /C = 600 secs/200 jobs = 3.0 secs/job – E[V CPU ] = C CPU /C = 200 visits/200 jobs = 1.0 visit/job – E[V slowdisk ] = C slowdisk /C = 2,000 visits/200 jobs = 10 visits/job – E[V fastdisk ] = C fastdisk /C = 20,000 visits/200 jobs = 100 visits/job – E[S CPU ] = B CPU /C CPU = 400 secs/200 visits = 2.0 secs/visit – E[S slowdisk ] = B slowdisk /C slowdisk = 100 secs/2,000 visits = 0.05 sec/visit – E[S fastdisk ] = B fastdisk /C fastdisk = 600 secs/20,000 visits =0.03 sec/visit Clearly, we have overloaded the fast disk. Optimal balancing is such that – E[V slowdisk ]  E[S slowdisk ] = E[V fastdisk ]  E[S fastdisk ] or E[V slowdisk ]  0.05 = E[V fastdisk ]  0.03 while keeping E[V slowdisk ] + E[V fastdisk ] = 110 – This gives E[V fastdisk ]  69 and E[V slowdisk ]  41 and consequently E[D slowdisk ] = E[D fastdisk ] = 2.06 secs, and therefore a new value of D max = 2.06 secs – D is now 6.12 secs and N * becomes 10, which remains smaller than N. Hence, throughput and response time are still dominated by D max, and the system improves to X  jobs/sec and E[R] = 26.2 secs 13 Summary – N = X  E[T] = X  (E[R] + E[Z]) – E[R] = N/X – E[Z] (Response Time Law) –  i = i /  i = i E[S i ] = X i E[S i ] (Utilization Law) – X i = E[V i ]  X (Forced Flow Law) –  i = X  E[D i ] (Bottleneck Law)

Example (2) – Add Another Fast Disk Metrics for interactive system with N = 20 and E[Z] = 15 secs: – E[D CPU ] = B CPU /C = 400 secs/200 jobs = 2.0 secs/job – E[D slowdisk ] = B slowdisk /C = 100 secs/200 jobs = 0.5 secs/job – E[D fastdisk ] = B fastdisk /C = 600 secs/200 jobs = 3.0 secs/job – E[V CPU ] = C CPU /C = 200 visits/200 jobs = 1.0 visit/job – E[V slowdisk ] = C slowdisk /C = 2,000 visits/200 jobs = 10 visits/job – E[V fastdisk ] = C fastdisk /C = 20,000 visits/200 jobs = 100 visits/job – E[S CPU ] = B CPU /C CPU = 400 secs/200 visits = 2.0 secs/visit – E[S slowdisk ] = B slowdisk /C slowdisk = 100 secs/2,000 visits = 0.05 sec/visit – E[S fastdisk ] = B fastdisk /C fastdisk = 600 secs/20,000 visits =0.03 sec/visit If we split the fast disk load across two fast disks, we get – E[D fastdisk1 ] = E[D fastdisk2 ] = 1.5 jobs/sec – The slow disk becomes the bottleneck which yieds a new value of D max = 2.0 secs – D does not change but since D max does, N* becomes  10. This remains smaller than N, so that the system improves to X = 0.5 jobs/sec and E[R] = 25 secs 14 Summary – N = X  E[T] = X  (E[R] + E[Z]) – E[R] = N/X – E[Z] (Response Time Law) –  i = i /  i = i E[S i ] = X i E[S i ] (Utilization Law) – X i = E[V i ]  X (Forced Flow Law) –  i = X  E[D i ] (Bottleneck Law)

Example (2) – Combine All Improvements Metrics for interactive system with N = 20 and E[Z] = 15 secs: – E[D CPU ] = B CPU /C = 400 secs/200 jobs = 2.0 secs/job – E[D slowdisk ] = B slowdisk /C = 100 secs/200 jobs = 0.5 secs/job – E[D fastdisk ] = B fastdisk /C = 600 secs/200 jobs = 3.0 secs/job – E[V CPU ] = C CPU /C = 200 visits/200 jobs = 1.0 visit/job – E[V slowdisk ] = C slowdisk /C = 2,000 visits/200 jobs = 10 visits/job – E[V fastdisk ] = C fastdisk /C = 20,000 visits/200 jobs = 100 visits/job – E[S CPU ] = B CPU /C CPU = 400 secs/200 visits = 2.0 secs/visit – E[S slowdisk ] = B slowdisk /C slowdisk = 100 secs/2,000 visits = 0.05 sec/visit – E[S fastdisk ] = B fastdisk /C fastdisk = 600 secs/20,000 visits =0.03 sec/visit With a faster CPU and two fast disks and rebalancing load across disks, we get – E[D CPU ] = 1 sec/job, E[D fastdisk1 ] = E[D fastdisk2 ] = E[D slowdisk ] = 1.27 jobs/sec  D = 4.8 secs Where we have used E[V slowdisk ]  0.05 = E[V fastdisk ]  0.03, and E[V slowdisk ] + 2E[V fastdisk ] = 110 – This gives D max = 1.27 secs and N*  16 that remains smaller than N – Hence, throughput and response time are still dominated by D max, and the system improves to X  jobs/sec and E[R] = 10.4 secs 15 Summary – N = X  E[T] = X  (E[R] + E[Z]) – E[R] = N/X – E[Z] (Response Time Law) –  i = i /  i = i E[S i ] = X i E[S i ] (Utilization Law) – X i = E[V i ]  X (Forced Flow Law) –  i = X  E[D i ] (Bottleneck Law)