1 Mor Harchol-Balter Carnegie Mellon University Computer Science Heavy Tails: Performance Models & Scheduling Disciplines

2 “size” = service requirement load  < 1 Q: Which minimizes mean response time?

3 “size” = service requirement jobs SRPT jobs load  < 1 jobs PS FCFS Q: Which best represents scheduling in web servers ?

4 IDEA: Use SRPT instead of PS in Web servers client 1 client 2 client 3 “Get File 1” “Get File 2” “Get File 3” Request for file Requested file Linux 0.S APACHE WEB SERVER Internet

5 Objections to SRPT:   Need to know “size” of request  Unfairness to requests for big files

6 Outline of Talk THEORY IMPLEMENT Papers are joint with Adam Wierman & Bianca Schroeder I: Investigating Unfairness in SRPT (M/G/1, c.f.m.f.v.) [Sigmetrics 01] II: Unfairness in All Scheduling Policies (M/G/1, c.f.m.f.v.) [Performance 02], [Sigmetrics 03] III: Implementation of SRPT in Web servers [Transactions on Computer Systems 03], [ITC03] THEORY

7 Outline of Talk THEORY IMPLEMENT I: Investigating Unfairness in SRPT (M/G/1, c.f.m.f.v.) [Sigmetrics 01] II: Unfairness in All Scheduling Policies (M/G/1, c.f.m.f.v.) [Performance 02], [Sigmetrics 03] III: Implementation of SRPT in Web servers [Transactions on Computer Systems 03], [ITC03] THEORY

8 SRPT has a long history Schrage & Miller derive M/G/1/SRPT response time: 1968 Schrage proves optimality 1979 Pechinkin & Solovyev & Yashkov generalize 1990 Schassberger derives distribution on queue length BUT WHAT DOES IT ALL MEAN?

9 SRPT has a long history (cont.) year long study at Univ. of Aachen under Schreiber SRPT WINS BIG ON MEAN! 1998, 1999 Slowdown for SRPT under adversary: Rajmohan, Gehrke, Muthukrishnan, Rajaraman, Shaheen, Bender, Chakrabarti, etc. SRPT STARVES BIG JOBS! Various o.s. books: Silberschatz, Stallings, Tannenbaum: Warn about starvation of big jobs... Kleinrock’s Conservation Law: “Preferential treatment given to one class of customers is afforded at the expense of other customers.”

10 Real-world job sizes are Heavy Tailed Job size (x seconds) log-log plot [Sigmetrics 96] Heavy-tailed Property: “Largest 1% of jobs comprise half the load.”

11 Unfairness Question SRPT PS ? ? Let  =0.9. Let G: Bounded Pareto(  = 1.1, max=10 10 ) Question: Which queue does biggest job prefer? M/G/1

12 Results on Unfairness Let  =0.9. Let G: Bounded Pareto(  = 1.1, max=10 10 ) SRPT PS I SRPT

13 Results on Unfairness Let G: Bounded Pareto(  = 1.1, max=10 10 ) PS SRPT

14 Unfairness – General Distribution All-can-win-theorem: For all distributions, if   ½, E[T(x)] SRPT  E[T(x)] PS for all x.

15 All-can-win-theorem: For all distributions, if   ½, E[T(x)] SRPT  E[T(x)] PS for all x. Proof:    x t dt 0 )1  )   x xFx 2 2 ))1(2 (  0  x dttft 2 )( E[Wait(x)] SRPT  E[Res(x)] SRPT E[T(x)] PS

16 All-can-win-theorem: For all distributions, if   ½, E[T(x)] SRPT  E[T(x)] PS for all x. Proof cont.    x t dt 0 )1   Need sufficient condition s.t. )   x xFx 2 2 ))1(2 (  0  x dttft 2 )(

17 All-can-win-theorem: For all distributions, if   ½, E[T(x)] SRPT  E[T(x)] PS for all x. Proof cont.  Need sufficient condition s.t. )   x xFx 2 2 ))1(2 (  0  x dttft 2 )(

18 All-can-win-theorem: For all distributions, if   ½, E[T(x)] SRPT  E[T(x)] PS for all x. Proof cont.  Need sufficient condition s.t. Observe: )   x xFx 2 2 ))1(2 (  0  x dttft 2 )(

19 Proof cont. All-can-win-theorem: For all distributions, if   ½, E[T(x)] SRPT  E[T(x)] PS for all x.   1  x 2 ))1(2  )  xFx 2 ( 0  x dttft 2 )( )  xFx 2 ( 0  x dttft 2 )( Suffices that: 2(1 - r (x)) 2 > 1 - r. Suffices that r (x) < 1 / 2

20 Intuition E[Wait(x)]E[Res(x)]: small small 1) All-can-win-theorem: For all distributions, if   ½, E[T(x)] SRPT  E[T(x)] PS for all x. 2) Under Bounded Pareto distribution ( a = 1.1), r < 0.96, E[T(x)] SRPT < E[T(x)] PS for all x. *HT Property* 

21 Outline of Talk THEORY IMPLEMENT I: Investigating Unfairness in SRPT (M/G/1, c.f.m.f.v.) [Sigmetrics 01] II: Unfairness in All Scheduling Policies (M/G/1, c.f.m.f.v.) [Performance 02], [Sigmetrics 03] III: Implementation of SRPT in Web servers [Transactions on Computer Systems 03], [ITC03] THEORY

22 What is fair? Slowdown for job of size x Response time for job of size x PS does not bias towards any particular job size. Slowdown is independent of size Definition: A policy P is fair if E[S(x)] P ≤ E[S(x)] PS for all x. Otherwise, P is unfair.

23 Always Fair Always Unfair Fair for all loads and distributions Unfair for all loads and distributions Sometimes Unfair Fair for some loads, and unfair for other loads* Classification of Scheduling Policies * and distributions

24 Testing your intuition: * and distributions Always FairAlways Unfair Fair for all loads and distributions Unfair for all loads and distributions Sometimes Unfair Fair for some loads, and unfair for other loads* Where does SRPT lie? __________ What about other policies that prioritize based on remaining size? ____ What about preemptive policies that prioritize based on age? _____ What about preemptive policies that prioritize based on size? _____

25 Classification of Scheduling Policies Lots of open problems… FSP Always FAIR Always Unfair Sometimes Unfair Age- Based Policies Preemptive Remaining-size based Policies Preemptive Size-based Policies Non- preemptive PS PLCFS FB PSJF LRPT FCFS LJFSJF SRPT

26 Always Unfair Unfair for all loads and distributions Theorem: Any preemptive, size based policy, P, is Always Unfair. Always Unfair Case1: A finite size, y, receives lowest priority Case 2: No finite size receives lowest priority (2a) Priorities decrease monotonically -- PSJF (2b) Priorities decrease non-monotonically.

27 Always Unfair Unfair for all loads and distributions Theorem: Any preemptive, size based policy, P, is Always Unfair. Always Unfair Case1: A finite size, y, receives lowest priority y V = Work in System

28 Always Unfair Unfair for all loads and distributions Theorem: Any preemptive, size based policy, P, is Always Unfair. Always Unfair Case2a: Priorities decrease monotonically (PSJF) Infinite sized job has lowest priority... … but that job is treated fairly?

29 Always Unfair Unfair for all loads and distributions Theorem: Any preemptive, size based policy, P, is Always Unfair. Always Unfair Case2a: Priorities decrease monotonically (PSJF) E[S(x)] 0 PS PSJF x Finding a hump shows PSJF is Always Unfair

30 Always Unfair Unfair for all loads and distributions Theorem: Any preemptive, size based policy, P, is Always Unfair. Always Unfair Case2b: Priorities decrease non-monotonically E[S(x)] 0 PS PSJF x Find y beyond which PSJF treats all jobs unfairly. Find x > y, where x has lower priority than y. => x is treated unfairly.

31 The mysterious hump 0 x 0 PS PSJF E[S(x)] x This hump appears in many common policies, 1. PSJF 2. FB 3. SRPT 4. SJF

32 Outline of Talk THEORY IMPLEMENT I: Investigating Unfairness in SRPT (M/G/1, c.f.m.f.v.) [Sigmetrics 01] II: Unfairness in All Scheduling Policies (M/G/1, c.f.m.f.v.) [Performance 02], [Sigmetrics 03] III: Implementation of SRPT in Web servers [Transactions on Computer Systems 03], [ITC03] THEORY

33 What does SRPT mean within a Web server? Many devices: Where to do the scheduling? Many jobs at once. From theory to practice:

34 Server’s Performance Bottleneck 5 Linux 0.S. WEB SERVER (Apache) client 1 client 2 client 3 “Get File 1” “Get File 2” “Get File 3” Rest of Internet ISP Site buys limited fraction of ISP’s bandwidth We schedule bandwidth at server’s uplink.

35 Network/O.S. insides of traditional Web server Sockets take turns draining --- FAIR = PS. Web Server Socket 1 Socket 3 Socket 2 Network Card Client1 Client3 Client2 BOTTLENECK

36 Network/O.S. insides of our improved Web server Socket corresponding to file with smallest remaining data gets to feed first. Web Server Socket 1 Socket 3 Socket 2 Network Card Client1 Client3 Client2 priority queues. 1 st 2 nd 3 rd S M L BOTTLENECK

37 Experimental Setup Implementation SRPT-based scheduling: 1) Modifications to Linux O.S.: 6 priority Levels 2) Modifications to Apache Web server 3) Priority algorithm design. Linux 0.S APACHE WEB SERVER Linux Linux Linux switch WAN EMU

38 Experimental Setup APACHE WEB SERVER Linux 0.S Linux Linux Linux switch WAN EMU Trace-based workload: Number requests made: 1,000,000 Size of file requested: 41B -- 2 MB Distribution of file sizes requested has HT property. Flash Apache WAN EMU Geographically- dispersed clients 10Mbps uplink 100Mbps uplink Surge Trace-based Open system Partly-open Load < 1 Transient overload + Other effects: initial RTO; user abort/reload; persistent connections, etc.

39 Load FAIR SRPT Mean Response Time (sec) Results: Mean Response Time......

40 Percentile of Request Size Mean Response time (  s) FAIR SRPT Load =0.8 Mean Response Time vs. Size Percentile

41 Transient Overload -- Mean response time SRPT FAIR

42 Transient overload Response time as function of job size small jobs win big! big jobs aren’t hurt! FAIR SRPT

43 New project: Scheduling dynamic web requests “$$$buy$$$” “buy” Web Server (eg: Apache/Linux) Internet Database Disks Low-Priority Client Locks High-Priority Client CPU(s) Need to schedule the database...

44 Conclusion Misconceptions about unfairness Discrimination against high-performance scheduling policies Good news: Many high-performing policies are also fair in practice! Classifying policies with respect to unfairness is counter-intuitive.