1. 1 Mor Harchol-Balter Carnegie Mellon University Computer Science Heavy Tails: Performance Models & Scheduling Disciplines

2. 2 “size” = service requirement load  < 1 Q: Which minimizes mean response time?

3. 3 “size” = service requirement jobs SRPT jobs load  < 1 jobs PS FCFS Q: Which best represents scheduling in web servers ?

4. 4 IDEA: Use SRPT instead of PS in Web servers client 1 client 2 client 3 “Get File 1” “Get File 2” “Get File 3” Request for file Requested file Linux 0.S. 1 2 3 1000 APACHE WEB SERVER Internet

5. 5 Objections to SRPT:   Need to know “size” of request  Unfairness to requests for big files

6. 6 Outline of Talk THEORY IMPLEMENT Papers are joint with Adam Wierman & Bianca Schroeder I: Investigating Unfairness in SRPT (M/G/1, c.f.m.f.v.) [Sigmetrics 01] II: Unfairness in All Scheduling Policies (M/G/1, c.f.m.f.v.) [Performance 02], [Sigmetrics 03] III: Implementation of SRPT in Web servers [Transactions on Computer Systems 03], [ITC03] THEORY

7. 7 Outline of Talk THEORY IMPLEMENT www.cs.cmu.edu/~harchol/ I: Investigating Unfairness in SRPT (M/G/1, c.f.m.f.v.) [Sigmetrics 01] II: Unfairness in All Scheduling Policies (M/G/1, c.f.m.f.v.) [Performance 02], [Sigmetrics 03] III: Implementation of SRPT in Web servers [Transactions on Computer Systems 03], [ITC03] THEORY

8. 8 SRPT has a long history... 1966 Schrage & Miller derive M/G/1/SRPT response time: 1968 Schrage proves optimality 1979 Pechinkin & Solovyev & Yashkov generalize 1990 Schassberger derives distribution on queue length BUT WHAT DOES IT ALL MEAN?

9. 9 SRPT has a long history (cont.) 1990 - 97 7-year long study at Univ. of Aachen under Schreiber SRPT WINS BIG ON MEAN! 1998, 1999 Slowdown for SRPT under adversary: Rajmohan, Gehrke, Muthukrishnan, Rajaraman, Shaheen, Bender, Chakrabarti, etc. SRPT STARVES BIG JOBS! Various o.s. books: Silberschatz, Stallings, Tannenbaum: Warn about starvation of big jobs... Kleinrock’s Conservation Law: “Preferential treatment given to one class of customers is afforded at the expense of other customers.”

10. 10 Real-world job sizes are Heavy Tailed Job size (x seconds) log-log plot [Sigmetrics 96] Heavy-tailed Property: “Largest 1% of jobs comprise half the load.”

11. 11 Unfairness Question SRPT PS ? ? Let  =0.9. Let G: Bounded Pareto(  = 1.1, max=10 10 ) Question: Which queue does biggest job prefer? M/G/1

12. 12 Results on Unfairness Let  =0.9. Let G: Bounded Pareto(  = 1.1, max=10 10 ) SRPT PS I SRPT

13. 13 Results on Unfairness Let G: Bounded Pareto(  = 1.1, max=10 10 ) PS SRPT

14. 14 Unfairness – General Distribution All-can-win-theorem: For all distributions, if   ½, E[T(x)] SRPT  E[T(x)] PS for all x.

15. 15 All-can-win-theorem: For all distributions, if   ½, E[T(x)] SRPT  E[T(x)] PS for all x. Proof:    x t dt 0 )1  )   x xFx 2 2 ))1(2 (  0  x dttft 2 )( E[Wait(x)] SRPT  E[Res(x)] SRPT E[T(x)] PS

16. 16 All-can-win-theorem: For all distributions, if   ½, E[T(x)] SRPT  E[T(x)] PS for all x. Proof cont.    x t dt 0 )1   Need sufficient condition s.t. )   x xFx 2 2 ))1(2 (  0  x dttft 2 )(

17. 17 All-can-win-theorem: For all distributions, if   ½, E[T(x)] SRPT  E[T(x)] PS for all x. Proof cont.  Need sufficient condition s.t. )   x xFx 2 2 ))1(2 (  0  x dttft 2 )(

18. 18 All-can-win-theorem: For all distributions, if   ½, E[T(x)] SRPT  E[T(x)] PS for all x. Proof cont.  Need sufficient condition s.t. Observe: )   x xFx 2 2 ))1(2 (  0  x dttft 2 )(

19. 19 Proof cont. All-can-win-theorem: For all distributions, if   ½, E[T(x)] SRPT  E[T(x)] PS for all x.   1  x 2 ))1(2  )  xFx 2 ( 0  x dttft 2 )( )  xFx 2 ( 0  x dttft 2 )( Suffices that: 2(1 - r (x)) 2 > 1 - r. Suffices that r (x) < 1 / 2

20. 20 Intuition E[Wait(x)]E[Res(x)]: small small 1) All-can-win-theorem: For all distributions, if   ½, E[T(x)] SRPT  E[T(x)] PS for all x. 2) Under Bounded Pareto distribution ( a = 1.1), r < 0.96, E[T(x)] SRPT < E[T(x)] PS for all x. *HT Property* 

21. 21 Outline of Talk THEORY IMPLEMENT www.cs.cmu.edu/~harchol/ I: Investigating Unfairness in SRPT (M/G/1, c.f.m.f.v.) [Sigmetrics 01] II: Unfairness in All Scheduling Policies (M/G/1, c.f.m.f.v.) [Performance 02], [Sigmetrics 03] III: Implementation of SRPT in Web servers [Transactions on Computer Systems 03], [ITC03] THEORY

22. 22 What is fair? Slowdown for job of size x Response time for job of size x PS does not bias towards any particular job size. Slowdown is independent of size Definition: A policy P is fair if E[S(x)] P ≤ E[S(x)] PS for all x. Otherwise, P is unfair.

23. 23 Always Fair Always Unfair Fair for all loads and distributions Unfair for all loads and distributions Sometimes Unfair Fair for some loads, and unfair for other loads* Classification of Scheduling Policies * and distributions

24. 24 Testing your intuition: * and distributions Always FairAlways Unfair Fair for all loads and distributions Unfair for all loads and distributions Sometimes Unfair Fair for some loads, and unfair for other loads* Where does SRPT lie? __________ What about other policies that prioritize based on remaining size? ____ What about preemptive policies that prioritize based on age? _____ What about preemptive policies that prioritize based on size? _____

25. 25 Classification of Scheduling Policies Lots of open problems… FSP Always FAIR Always Unfair Sometimes Unfair Age- Based Policies Preemptive Remaining-size based Policies Preemptive Size-based Policies Non- preemptive PS PLCFS FB PSJF LRPT FCFS LJFSJF SRPT

26. 26 Always Unfair Unfair for all loads and distributions Theorem: Any preemptive, size based policy, P, is Always Unfair. Always Unfair Case1: A finite size, y, receives lowest priority Case 2: No finite size receives lowest priority (2a) Priorities decrease monotonically -- PSJF (2b) Priorities decrease non-monotonically.

27. 27 Always Unfair Unfair for all loads and distributions Theorem: Any preemptive, size based policy, P, is Always Unfair. Always Unfair Case1: A finite size, y, receives lowest priority y V = Work in System

28. 28 Always Unfair Unfair for all loads and distributions Theorem: Any preemptive, size based policy, P, is Always Unfair. Always Unfair Case2a: Priorities decrease monotonically (PSJF) Infinite sized job has lowest priority... … but that job is treated fairly?

29. 29 Always Unfair Unfair for all loads and distributions Theorem: Any preemptive, size based policy, P, is Always Unfair. Always Unfair Case2a: Priorities decrease monotonically (PSJF) E[S(x)] 0 PS PSJF x Finding a hump shows PSJF is Always Unfair

30. 30 Always Unfair Unfair for all loads and distributions Theorem: Any preemptive, size based policy, P, is Always Unfair. Always Unfair Case2b: Priorities decrease non-monotonically E[S(x)] 0 PS PSJF x Find y beyond which PSJF treats all jobs unfairly. Find x > y, where x has lower priority than y. => x is treated unfairly.

31. 31 The mysterious hump 0 x 0 PS PSJF E[S(x)] x This hump appears in many common policies, 1. PSJF 2. FB 3. SRPT 4. SJF

32. 32 Outline of Talk THEORY IMPLEMENT www.cs.cmu.edu/~harchol/ I: Investigating Unfairness in SRPT (M/G/1, c.f.m.f.v.) [Sigmetrics 01] II: Unfairness in All Scheduling Policies (M/G/1, c.f.m.f.v.) [Performance 02], [Sigmetrics 03] III: Implementation of SRPT in Web servers [Transactions on Computer Systems 03], [ITC03] THEORY

33. 33 What does SRPT mean within a Web server? Many devices: Where to do the scheduling? Many jobs at once. From theory to practice:

34. 34 Server’s Performance Bottleneck 5 Linux 0.S. WEB SERVER (Apache) client 1 client 2 client 3 “Get File 1” “Get File 2” “Get File 3” Rest of Internet ISP Site buys limited fraction of ISP’s bandwidth We schedule bandwidth at server’s uplink.

35. 35 Network/O.S. insides of traditional Web server Sockets take turns draining --- FAIR = PS. Web Server Socket 1 Socket 3 Socket 2 Network Card Client1 Client3 Client2 BOTTLENECK

36. 36 Network/O.S. insides of our improved Web server Socket corresponding to file with smallest remaining data gets to feed first. Web Server Socket 1 Socket 3 Socket 2 Network Card Client1 Client3 Client2 priority queues. 1 st 2 nd 3 rd S M L BOTTLENECK

37. 37 Experimental Setup Implementation SRPT-based scheduling: 1) Modifications to Linux O.S.: 6 priority Levels 2) Modifications to Apache Web server 3) Priority algorithm design. Linux 0.S. 1 2 3 APACHE WEB SERVER Linux 1 2 3 200 Linux 1 2 3 200 Linux 1 2 3 200 switch WAN EMU

38. 38 Experimental Setup APACHE WEB SERVER Linux 0.S. 1 2 3 Linux 1 2 3 200 Linux 1 2 3 200 Linux 1 2 3 200 switch WAN EMU Trace-based workload: Number requests made: 1,000,000 Size of file requested: 41B -- 2 MB Distribution of file sizes requested has HT property. Flash Apache WAN EMU Geographically- dispersed clients 10Mbps uplink 100Mbps uplink Surge Trace-based Open system Partly-open Load < 1 Transient overload + Other effects: initial RTO; user abort/reload; persistent connections, etc.

39. 39 Load FAIR SRPT Mean Response Time (sec) Results: Mean Response Time......

40. 40 Percentile of Request Size Mean Response time (  s) FAIR SRPT Load =0.8 Mean Response Time vs. Size Percentile

41. 41 Transient Overload -- Mean response time SRPT FAIR

42. 42 Transient overload Response time as function of job size small jobs win big! big jobs aren’t hurt! FAIR SRPT

43. 43 New project: Scheduling dynamic web requests “$$$buy$$$” “buy” Web Server (eg: Apache/Linux) Internet Database Disks Low-Priority Client Locks High-Priority Client CPU(s) Need to schedule the database...

44. 44 Conclusion Misconceptions about unfairness Discrimination against high-performance scheduling policies Good news: Many high-performing policies are also fair in practice! Classifying policies with respect to unfairness is counter-intuitive.