1. 1 Connection Scheduling in Web Servers Mor Harchol-Balter School of Computer Science Carnegie Mellon harchol@cs.cmu.edu

2. 2 “size” = service requirement load  < 1 Q: Which minimizes mean response time?

3. 3 “size” = service requirement jobs SRPT jobs load  < 1 jobs PS FCFS Q: Which best represents scheduling in web servers ?

4. 4 How about using SRPT in Web servers as opposed to the traditional processor-sharing (PS) type scheduling ?

5. 5 Many servers receive mostly static web requests. “GET FILE” For static web requests, know file size Approx. know service requirement of request. Immediate Objections 1) Can’t assume known job size 2) But the big jobs will starve...

6. 6 Outline of Talk 1) “Analysis of SRPT Scheduling: Investigating Unfairness” Bansal and Harchol-Balter 2) “Implementation of SRPT Scheduling in Web Servers” Harchol-Balter and Bansal and Schroeder THEORY IMPLEMENT

7. 7 THEORY SRPT has a long history... 1966 Schrage & Miller derive M/G/1/SRPT response time: 1968 Schrage proves optimality 1979 Pechinkin & Solovyev & Yashkov generalize 1990 Schassberger derives distribution on queue length BUT WHAT DOES IT ALL MEAN?

8. 8 THEORY SRPT has a long history (cont.) 1990 - 97 7-year long study at Univ. of Aachen under Schreiber Evaluate mean response time under SRPT for specific distributions ……….. SRPT WINS BIG ON MEAN! 1997, 1998 Variations on SRPT: Evaluation via simulation only: Mudiano, Cherkasova, etc. 1998, 1999 Slowdown for SRPT under adversary: Rajmohan, Gehrke, Muthukrishnan, Rajaraman, Shaheen 1998 Bender & Chakrabarti & Muthukrishnan: Theoretical results under adversarial arrival sequence: SRPT STARVES BIG JOBS! Various o.s. books: Silberschatz, Stallings, Tannenbaum: Warn about starvation of big jobs... Kleinrock’s Conservation Law: “Preferential treatment given to one class of customers is afforded at the expense of other customers.”

9. 9 THEORY Our Analytical Results (M/G/1): SRPT PS All-Can-Win Theorem: Under heavy-tailed workloads, ALL jobs, including the very biggest, prefer SRPT to PS, provided load not too close to 1. Almost-All-Win-Big Theorem: Under heavy-tailed workloads, 99% of all jobs perform significantly better under SRPT. ISRPT Counter-intuitive!

10. 10 THEORY Our Analytical Results (M/G/1): Moderate-Load Theorem: If load <.5, for every job size distribution, ALL jobs prefer SRPT to PS. Bounding-the-damage Theorem: For any load, for every job size distribution, for every size x, PSSRPT xTExTE)]([1 ([              For more theorems, see: www.cs.cmu.edu/~harchol

11. 11 What does SRPT mean within a Web server? Many devices: Where to do the scheduling? No longer one job at a time. IMPLEMENT From theory to practice:

12. 12 IMPLEMENT Previous work on implementation: 1999 Crovella, Frangioso, Harchol-Balter: SRPT scheduling of requests. User-level scheduling of reads and writes. 1998 Almeida, Dabu, Manikutty, Cao: Prioritizing HTTP requests at Web servers. “Nice” the low-priority process. What is the problem with both of these?

13. 13 Network/O.S. insides of traditional Web server Sockets take turns draining --- FAIR = PS. Web Server Socket 1 Socket 3 Socket 2 Network Card Client1 Client3 Client2 BOTTLENECK IMPLEMENT

14. 14 Network/O.S. insides of our improved Web server Socket corresponding to file with smallest remaining data gets to drain first. Web Server Socket 1 Socket 3 Socket 2 Network Card Client1 Client3 Client2 priority queues. 1 st 2 nd 3 rd S M L BOTTLENECK IMPLEMENT

15. 15 Experimental Setup Linux 0.S. Direct Connection 10Mb/s CLIENTS 1 2 3 1 2 3 200 FLASH WEB SERVER 200 Implement SRPT-based scheduling: 1) Modifications to Linux O.S.: 5 priority Levels 2) Modifications to Flash Web server 3) Priority algorithm design. IMPLEMENT

16. 16 Experimental Setup Linux 0.S. CLIENTS 1 2 3 1 2 3 200 FLASH WEB SERVER 200 Comparison of SRPT implementation vs. FAIR scheduling standard. Comparisons done under trace-based workload and under Web workload generator. Under trace-based workload: -- Number requests made: 90,000 -- Size of file requested: 41B -- 2 MB -- File sizes requested have heavy-tailed distribution. Experiment under range of loads. IMPLEMENT

17. 17 Preliminary Comments to Results: Linux 0.S. CLIENTS 1 2 3 1 2 3 200 FLASH WEB SERVER 200 Measured job throughput, byte throughput, and bandwidth utilization were same under SRPT and FAIR scheduling. Same set of requests complete. No additional CPU overhead under SRPT scheduling. CPU utilization always 1% - 5%. Network was bottleneck in all experiments. IMPLEMENT

18. 18 Load FAIR SRPT Mean Response Time (ms) Results: Mean Response Time

19. 19 FAIR SRPT Load Mean Slowdown Results: Mean Slowdown

20. 20 Requested File Size (bytes) FAIR SRPT Load = 0.8 Mean Response Time (  s) Results: Mean Response Time vs. Size

21. 21 Percentile of Request Size Mean Response time (  s) FAIR SRPT Load =0.8 Mean Response Time vs. Size Percentile

22. 22 Percentile of Request Size Variance in Response Time Percentile of Request Size FAIR SRPT Load =0.8 Variance in Response Time Variance in Response Time vs. Size Percentile

23. 23 Details, Optimizations, and Shortcuts Diffserv implementation headaches. The problem of high startup overhead: Scheduling Solutions! All the benefit with much less work: QUICKFIX.

24. 24 Percentile of Request Size (bytes) FAIR QUICKFIX Load = 0.8 Mean Response Time (  s) Results: Mean Response Time vs. Size

25. FAIR QUICKFIX Load = 0.8 Mean Response Time (  s) Results: Mean Response Time vs. Size Requested File Size (bytes)

26. 26 SRPT scheduling yields significant improvements in Mean Response Time at the server. Negligible starvation. No CPU overhead. No drop in throughput. Conclusions

27. 27 Big Future Work Handling propagation delay and slow clients. Non-static requests. Smart routers. Overload.