1. CS 3150 Presentation 1 Ihsan Ayyub Qazi “Achievable sojourn times in M/M/1 and GI/GI/1 systems: A Comparison between SRPT and Blind Policies” A(t), Q(t)Q(t) D(t)D(t)

2. CS 3150 Presentation 2 Plan for the Presentation  Some basics  Motivation for minimizing sojourn times  Intuition for the Optimality of SRPT  Paper 1: “  Paper 1: “On the average sojourn time under M/M/1/SRPT” by Nikhil Bansal  Paper 2: “  Paper 2: “Achievable sojourn times by non- size based policies in a GI/GI/1 queue” by Nikhil Bansal

3. CS 3150 Presentation 3 Some Basics…  The sojourn time of a job is the time since a job arrives until the time it completes its service requirement.

4. CS 3150 Presentation 4 Why minimize sojourn time?  Sojourn time is one of the most useful measures of  user satisfaction and  performance in a scheduling system.  Keeping the average sojourn time low is often an important criteria in the choice of a scheduling policy.

5. CS 3150 Presentation 5 Optimizing the average sojourn time  SRPT policy that at any works on the job with the smallest remaining service requirement, achieves the optimum possible average sojourn time for all possible instances of the problem [1], [2]

6. CS 3150 Presentation 6 Optimality of SRPT   Shortest Job First (SJF) is a non- preemptive policy that schedules the jobs in increasing order of their sizes.   It is optimal for all non-preemptive polices   However, the SJF policy is optimal only when all the jobs are present at the server at once. Total Completion Time = 20+25 = 45 Average Completion Time = 45/2 = 22.5 Scheduler Total Completion Time = 5+25 = 30 Average Completion Time = 30/2 = 15 20 5 5

7. CS 3150 Presentation 7 Optimality of SRPT   One can view SRPT as a preemptive version of SJF.   Hence one can get an idea about the optimality of SRPT Scheduler 20 5 155 Total Completion Time = 5+20 = 25 Average Completion Time = 25/2 = 12.5 Total Completion Time = 15+20 = 35 Average Completion Time = 35/2 = 17.5 However, one can consider the remaining processing time (i.e. 15) as a separate job and hence swap it with 5

8. CS 3150 Presentation 8 Disadvantages of SRPT  SRPT requires exact knowledge of the service requirement for each job  It is unfair to Jobs with large service requirements and may even starve them.  Blind policies have many attractive properties such as  they are stateless and  easier to implement in a real system.

9. CS 3150 Presentation 9 “On the average sojourn time under M/M/1/SRPT” by Nikhil Bansal

10. CS 3150 Presentation 10 Plan  Problem  Results  Background  Analysis

11. CS 3150 Presentation 11 Problem “How much more can the sojourn time improve if the knowledge of job sizes is used while scheduling?”

12. CS 3150 Presentation 12 Classical Result  It is a well-known result that the average sojourn time in an M/M/1 system is  This holds for all scheduling policies that do not make use of the job sizes while scheduling [3].

13. CS 3150 Presentation 13 Problem Definition (revisited) “How much more can the sojourn time improve if the knowledge of job sizes is used while scheduling?” In essence, this question reduces to finding the expression for the average sojourn time in a M/M/1/SRPT system

14. CS 3150 Presentation 14 Plan  Problem  Results  Background  Analysis

15. CS 3150 Presentation 15 Main result of the Paper… The average sojourn time [under M/M/1/SRPT] with exponentially distributed job sizes varies as Thus SRPT offers a factor improvement over policies that ignore knowledge of job sizes while scheduling

16. CS 3150 Presentation 16 Improvement Factor

17. CS 3150 Presentation 17 Contribution of the paper  This result places a lower bound on the achievable average sojourn time under any scheduling policy in an M/M/1 system  The result provides an analytical justification for the empirically observed improvement under SRPT at high loads.

18. CS 3150 Presentation 18 Plan  Problem  Results  Background  Analysis

19. CS 3150 Presentation 19 Some Previous work…  Schrage and Miller first obtained the expression for the expected sojourn time for a job of size x under SRPT in the more general M/G/1 queuing system. They showed that, Mean Waiting Time Mean Residence Time

20. CS 3150 Presentation 20 Mean Waiting and Residence times for a job of size (Intuition)  Average time a job of size x takes from when it arrives to when it receives service for the first time  Average time it takes for a job of size x to complete once it begins execution. Expected Waiting Time of a job of size x corresponds to waiting for all jobs of sizes less than or equal to x to complete

21. CS 3150 Presentation 21 Plan  Problem  Results  Background  Analysis

22. CS 3150 Presentation 22 Analysis: Expressions with exponentially distributed job sizes

23. CS 3150 Presentation 23 Expression for the sojourn time under SRPT for exponential distributed job sizes  The average sojourn time under SRPT is given by

24. CS 3150 Presentation 24 Results  Theorem 1: For all p between 0 and 1  Theorem 2 (Heavy Traffic Case):

25. CS 3150 Presentation 25 Sketch of the proof  An upper bound on E[R] is derived.  Then an upper bound for E[W] is derived for the case when the utilization is more than 75%.  For other values of utilization, the result is shown to be true by considering the average sojourn time under FCFS as the upper bound.  It is shown that that the contribution of E[R] to E[T] is not significant.  Hence a good tight bound can be obtained for E[T] by lower bounding it by E[W]

26. CS 3150 Presentation 26 Upper Bounding the Sojourn Time  Lemma 1: For any load p, such that it is between 0 and 1  For any load p, such that it is between 2/3 and 1

27. CS 3150 Presentation 27 Comparison between the bounds on E[R] and E[W]

28. CS 3150 Presentation 28 Lower Bounding the Sojourn Time  Lemma 3:

29. CS 3150 Presentation 29 Sketch of the proof (contd)  For any load, the average sojourn time under SRPT is atleast equal to the average job size. Since SRPT is optimal E[T] is upper bounded by the average sojourn time under the policy FCFS. Thus  And hencefor p < 2/3. The theorem equation is seen to be true in this case.

30. CS 3150 Presentation 30 Upper Bound for Theorem 1 Lemma 3 gives the required lower bound, hence theorem 1 is proved

31. CS 3150 Presentation 31 Thankyou

32. CS 3150 Presentation 32 “Achievable sojourn times by non-size based policies in a GI/GI/1 queuing system” by Nikhil Bansal

33. CS 3150 Presentation 33 Plan  Problem  Results  Background  Analysis

34. CS 3150 Presentation 34 Problem How much worse can the average time under the best blind policy be as compared to SRPT (in a GI/GI/1 system)?

35. CS 3150 Presentation 35 Plan  Problem  Results  Background  Analysis

36. CS 3150 Presentation 36 Main result of the paper “For a GI/GI/1 system, the average sojourn time under the best blind policy is atmost time worse (upto a constant factors) then the best average sojourn time possible under any arbitrary policy”  Thus in a sense, the lack of knowledge of actual job-sizes does not pose a serious problem if the blind policy is chosen carefully.

37. CS 3150 Presentation 37 Improvement Factor of SRPT against the Best Blind Scheduling Policy

38. CS 3150 Presentation 38 Background and Motivation  A GI/GI/1 queuing system  Inter-arrival times are i.i.d rvs, A, taken from a General distribution G a  Job Sizes are i.i.d rvs, S, taken from a General distribution G s  Different from a G/G/1 system.  G s and G a specify a GI/GI/1 system completely. Therefore, the optimum blind policy A (which minimizes the average sojourn time) only depends on the respective distributions.  Opt(G a, G s ) = Average sojourn time under the optimum blind policy.

39. CS 3150 Presentation 39 Difference in the analyses of M/M/1 and G1/G1/1 queuing systems  In a M/M/1 system all blind policies are identical as far as the average sojourn time is concerned. In particular any blind scheduling policy has average sojourn equal to  So the question reduced to determining the average sojourn time under SRPT. Bansal showed that for exponential job sizes the average sojourn time under M/M/1/SRPT is

40. CS 3150 Presentation 40 So why is the analysis of a G1/G1/1 queuing system different and more difficult?  Unlike the case in M/M/1, all blind policies are no more identical !!!  No single blind policy is optimal for e.g.  In an M/G/1 system FCFS is optimum for job size distributions with increasing failure rates.  Foreground Background (FB) [that at time works on the job with the least attained service] is optimum for job size distributions with decreasing failure rates

41. CS 3150 Presentation 41 So why is the analysis of a G1/G1/1 queuing system different and more difficult?  Same blind policy can have very different behaviours for different job sizes.  While FB has average sojourn time when job sizes are exponentially distributed, the  Average sojourn time under FB varies as when the job sizes have the pareto distribution with

42. CS 3150 Presentation 42 What to do? So why is the analysis of a G1/G1/1 queuing system so difficult?  Furthermore, As the analytic expression for average sojourn time under an arbitrary GI/GI/1 system is not known and moreover no analytic expression for Opt(G a, G s ) for general G a, G s is known, therefore we cannot adopt the approach that we used in the case of M/M/1

43. CS 3150 Presentation 43 Hmmm…

44. CS 3150 Presentation 44 Plan  Problem  Results  Background  Analysis

45. CS 3150 Presentation 45 Competitive Analysis  Let A(I): Total sojourn time when the instance is executed according to the algorithm A.  We say that a deterministic algorithm has a competitive ratio c(n) if  The definition of competitive ratio is quite strict,  A more useful notion is that of a randomized algorithm. Worst Case Ratio over all input instances of size atmost n achieved by A and the optimum cost on that instance

46. CS 3150 Presentation 46 Competitive Analysis  The competitive ratio of a randomized algorithm is defined as  The crucial thing to observe is that there is no probabilistic assumption on the input instance.  The input instance is still chosen adversarially to maximize the ratio.

47. CS 3150 Presentation 47 Competitive Analysis  Randomized algorithms can have significantly better competitive ratios than deterministic algorithms.  NOTE: The notion of the performance of a randomized algorithm is dual to the notion of the average case performance of a system like GI/GI/1  While the former deals with the performance over a distribution over algorithms on a fixed input instance, the latter deals with the performance of a fixed algorithm on a distribution over input instances. Can we relate the two?

48. CS 3150 Presentation 48 Yao’s Minimax Theorem (Contrapositive) [Minimization Problem]  Suppose a cost minimization problem P has a c(R,n) competitive randomized online algorithm R for request sequences of length atmost n. Let distribution y(j) be any distribution over request sequences of length atmost n. Then,  In particular, for any distribution over the input instances, if we consider the algorithm A i that has the best average case performance on this distribution, then this performance is no worse than c(R,n) times the performance of the optimum algorithm.

49. CS 3150 Presentation 49 Plan  Problem  Results  Background  Analysis

50. CS 3150 Presentation 50 Some known results for competitive ratio of blind scheduling algorithm  For the problem of minimizing the average sojourn time:  Motwani, Phillips and Torng showed that no blind deterministic scheduling algorithm can have a competitive ratio better than  The same people showed that any randomized algorithm has a competitive ratio atleast

51. CS 3150 Presentation 51 BreakThrough….  In a significant breakthrough, Pruhs and Kalayanasundaram gave a non-trivial randomized algorithm that they called RMLF [5] and proved that it has a competitive ratio of  Later it was shown that RMLF is infact an competitive randomized algorithm and hence the best possible upto constant factors. In other words, competitive randomized algorithm and hence the best possible upto constant factors. In other words, “ There is a universal constant r, such that for any scheduling instance with atmost n jobs, the expected total sojourn time under RMLF is atmost rlogn times that under SRPT”

52. CS 3150 Presentation 52 Analysis  Theorem 1: GI/GI/1 system with inter-arrival distribution G a and service distribution G s, there exists a universal constant d such that  Where C is a bound on the sixth coefficient of variation of G a and G s, that is

53. CS 3150 Presentation 53 Analysis  Lemma 1: Given any probability distribution on the input instances with atmost n jobs, there is some deterministic algorithm the expected total sojourn time of which is atmost r(logn) times that under SRPT  If we take each busy period in a GI/GI/1 system as a separate instance then we can think of a GI/GI/1 system as defining a probability distribution on input instances.  Can we apply Lemma1 ?  No!  Because we cannot get a bound on the number of jobs a busy period might contain !!

54. CS 3150 Presentation 54 Analysis (Basic Idea)  For every n, there is a non-zero fraction of busy periods that contain more than n jobs.  Can we come up with a n such that most of the action happens with those many jobs in a busy period?  The analysis is based on this premise.  A busy period has about 1/1-p jobs on the average, hence the logn factor in lemma 1 should essentially be log(1/1-p)  Most of the action essentially happens in busy periods with atmost C 6 /(1-p) 6 jobs and hence lemma 1 can be replaced by this number.

55. CS 3150 Presentation 55 Analysis   The busy periods are independent of the actual scheduling policy involved.   Let B = set of all possible busy periods M = measure induced by GI/GI/1 system on B T A (B) = Total sojourn time incurred when A is executed during the busy period B n(B)= number of jobs in B   Average sojourn time of a job under an algorithm A can be expressed as

56. CS 3150 Presentation 56 Analysis   For any GI/GI/1 system, E[n(B)] is identical for every work-conserving scheduling policy. Thus it suffices to compare the expected total sojourn time of a busy period for two scheduling policies, in order to compare the average sojourn time under them.   A busy period is called bad if it contains more than N 0 =24C 6 /(1-p) 6.   Define a process P as follows: Whenever there is a bad busy period of length B, we replace it with another busy period with n(B) “dummy” jobs of length 0.   Let M’ be the measure induced by the new process on the busy periods. By construction: E M ’[n(B)] = E M [n(B)].   Now lemma 1 can be applied..

57. CS 3150 Presentation 57 Analysis  Key point: The only contribution to the expected total sojourn time under the measure M’ is due to the busy periods that contain atmost N 0 jobs.  Let Opt’ = blind algorithm A that minimizes  Let Opt’ = blind algorithm A that minimizes E M ’[T A (B)]

58. CS 3150 Presentation 58 Few Lemmas…  Lemma 2 (follows easily from lemma 1):  Lemma 3 [For any work-conserving algorithm A, uses lemma 4,5]  Lemma 4: Let S n =X 1 +X 2 +….+X n where X i are independent and identically distributed according to the random variable X. Then, for epsilon > 0,

59. CS 3150 Presentation 59 Few Lemmas…  Lemma 5: Given a GI/GI/1 system, let A and S be the random variables representing arrived time and service time respectively. Let and

60. CS 3150 Presentation 60 Proof of lemma 3  Since M and M’ differ only on bad busy periods, therefore for any work-conserving algorithm  To show the other side of the inequality we need to upper bound the contribution due to bad busy periods in  To show the other side of the inequality we need to upper bound the contribution due to bad busy periods in E M [T A (B)]   In a busy period of length l and consisting of n jobs, the total sojourn time of the jobs involved can be atmost nl, irrespective of the choice of the algorithm.   Thus we need to upper bound:

61. CS 3150 Presentation 61 Proof of Theorem 1  Consider the optimum blind policy Opt’ that minimizes E M’ [T Opt ’(B)]. It follows by lemma 3  By definition:  By lemma 2:  Which by lemma 3 implies  Combining the above we get

62. CS 3150 Presentation 62 Summary   The paper proves that for any GI/GI/1 queuing system, the performance of the best blind policy for that system achieves average sojourn close to the best possible by any other (non-blind) policy.   The result, however, does not give a construct way to produce the optimum (deterministic) blind scheduling policy for an arbitrary GI/GI/1 system.   The dependence on the sixth coefficient of variations of inter-arrival times and sizes is the artifact of the analysis. The finiteness of the sixth moments allows to show that the probability a busy period has more than n jobs decays at least as 1/n 3.

63. CS 3150 Presentation 63 References [1] On the average sojourn time under M/M/1/SRPT, Nikhil Bansal, to appear in Operations Research Letters, Volume 33, 2, March 2005, 195- 200.On the average sojourn time under M/M/1/SRPT [2] Achievable sojourn times by non-size based policies in a GI/GI/1 queue, Nikhil Bansal, submitted for publication.Achievable sojourn times by non-size based policies in a GI/GI/1 queue [3] D.R. Smith. A new proof of the optimality of the shortest remaining processing time discipline. Operations Research, 26:197–199, 1976. [4] L.E. Schrage. A proof of the optimality of the shortest processing remaining time discipline. Operations Research, 16:678–690, 1968. [5] R. W. Conway, W. L. Maxwell, and L. W. Miller. Theory of Scheduling. Addison-Wesley Publishing Company, 1967. [6] R. Motwani, S. Phillips, and E. Torng. Nonclairvoyant scheduling. Theoretical Computer Science,130(1):17–47, 1994. [7] B. Kalyanasundaram and K. Pruhs. Minimizing flow time nonclairvoyantly. In IEEE Symposium on Foundations of Computer Science, pages 345– 352, 1997. [8] A. C-C. Yao. Probabilistic computations: Toward a unified measure of complexity (extended abstract). In IEEE Symposium on Foundations of Computer Science (FOCS), 1977.

64. CS 3150 Presentation 64 Questions

65. CS 3150 Presentation 65 Thankyou