Simulation Evaluation of Hybrid SRPT Policies 4/16/2017 Simulation Evaluation of Hybrid SRPT Policies Mingwei Gong and Carey Williamson Department of Computer Science University of Calgary April 19, 2004

Introduction Web: large-scale, client-server system 4/16/2017 Introduction Web: large-scale, client-server system WWW: World Wide Wait! User-perceived Web response time involves: Transmission time, propagation delay in network Queueing delays at busy routers in the Internet Delays caused by TCP protocol effects (e.g., handshake, slow start, packet loss, retransmits) Queueing delays at the Web server itself, which may be servicing 100’s or 1000’s of concurrent requests Our focus in this work: Web request scheduling WORMS 2004

Example Scheduling Policies 4/16/2017 Example Scheduling Policies FCFS: First Come First Serve typical policy for single shared resource (“unfair”) e.g., drive-thru restaurant; playoff tickets PS: Processor Sharing time-sharing a resource amongst J jobs each job gets 1/J of the resources (equal, “fair”) e.g., CPU; VM; multi-tasking; Apache Web server SRPT: Shortest Remaining Processing Time pre-emptive version of Shortest Job First (SJF) give full resources to job that will complete quickest e.g., ??? (express lanes in grocery store)(almost) WORMS 2004

Research Methodology Trace-driven simulation Web server simulator 4/16/2017 Research Methodology Trace-driven simulation Input workload is empirical/synthetic trace Web server simulator Empirical trace (1 million requests, World Cup 1998) Synthetic traces (WebTraff) Probe-based sampling methodology Based on PASTA: Poisson Arrivals See Time Averages Any scheduling policy, any arrival process, any service time distribution. The research on SRPT began in the 1960’s. Previous work can be classified into three groups: First, theoretical work proved that SRPT is optimal in terms of mean response time and mean slowdown compared with other scheduling policies. Secondly, in practical work, SRPT was implemented in the Apache Web server The experimental results are consistent with theoretical work. Both works show that SRPT out-performs PS with respect to mean performance However, The SRPT policy is rarely used in practice, the main concern is the fear of penalizing the large jobs. WORMS 2004

Simulation Assumptions 4/16/2017 Simulation Assumptions User requests are for static Web content Server knows response size in advance Network bandwidth is the bottleneck All clients are in the same LAN environment Ignores variations in network bandwidth and propagation delay Fluid flow approximation: service time = response size Ignores packetization issues Ignores TCP protocol effects Ignores network effects (These are consistent with SRPT literature) WORMS 2004

Performance Metrics Slowdown: 4/16/2017 Performance Metrics Slowdown: The slowdown of a job is its observed response time divided by the ideal response time if it were the only job in the system Lower is better We consider mean slowdown as well as the variance of slowdown (complete distribution) WORMS 2004

Empirical Web Server Workload 4/16/2017 Empirical Web Server Workload 1998 WorldCup: Internet Traffic Archive: http://ita.ee.lbl.gov/ Item Value Trace Duration 861 sec Total Requests 1,000,000 Unique Documents 5,549 Total Transferred Bytes 3.3 GB Smallest Transfer Size (bytes) 4 Largest Transfer Size (bytes) 2,891,887 Median Transfer Size (bytes) 889 Mean Transfer Size (bytes) 3,498 Standard Deviation (bytes) 18,815 WORMS 2004

Probe-based Sampling Algorithm 4/16/2017 Probe-based Sampling Algorithm The algorithm is based on PASTA (Poisson Arrivals See Time Average) principle. S Slowdown (1 sample) S Repeat N times S WORMS 2004

Probe-based Sampling Algorithm 4/16/2017 Probe-based Sampling Algorithm For scheduling policy S =(PS, SRPT, FCFS, LRPT, …) do For load level U = (0.50, 0.80, 0.95) do For probe job size J = (1B, 1KB, 10KB, 1MB...) do For trial I = (1,2,3… N) do Insert probe job at randomly chosen point; Simulate Web server scheduling policy; Compute and record slowdown value observed; end of I; Plot marginal distribution of slowdown results; end of J; end of U; end of S; WORMS 2004

Slowdown Profile Plot “asymptotic convergence” “crossover region” (mystery hump) 8 PS x 1 1-p y Slowdown SRPT 1 8 Job Size WORMS 2004

Notation Details Number of jobs in the system: J Number of threads for a single server: K Number of servers in the system: M Probe jobs: 1KB, 10KB, 100KB, 1MB... Number of probes: 3000 All simulation results are for 95% load WORMS 2004

Single Server Scenario (M = 1) PS: Processor Sharing SRPT: Shortest Remaining Processing Time FSP: Fair Sojourn Protocol FSP computes the times at which jobs would complete under PS and then orders the jobs in terms of earliest PS completion times. FSP then devotes full service to the uncompleted job with the earliest PS completion time. “FSP response time dominates PS” (i.e., is never worse) E. Friedman and S. Henderson: “Fairness and Efficiency in Web Server Protocols”, Proc. ACM SIGMETRICS 2003. WORMS 2004

Mean Slowdown (M = 1) WORMS 2004

Variance of Slowdown (M = 1) WORMS 2004

A Hybrid SRPT/PS Policy for a Single Server (M = 1) Threshold-based policy, with threshold T T-SRPT Determining whether the system is "busy" or not depends on number of jobs (J) in the system. If J <= T Then use PS Else use SRPT Special cases: T = 0 is SRPT, T = is PS 8 WORMS 2004

Mean Slowdown for T-SRPT WORMS 2004

Variance of Slowdown for T-SRPT WORMS 2004

A Generalized SRPT Policy for a Multi-threaded Single Server K-SRPT Multi-threaded version of SRPT that allows up to K jobs (the K smallest RPT ones) to be in service concurrently (like PS), though with the same fixed aggregate service rate. Additional jobs (if any) in the system wait in the queue. Also preemptive, like SRPT. Let s = min (J, K) If J <= K Then J jobs each receive 1/s Else K jobs each receive 1/s (while J-K wait) Special cases: K = 1 is SRPT, K = is PS 8 WORMS 2004

Mean Slowdown for K-SRPT WORMS 2004

Variance of Slowdown for K-SRPT WORMS 2004

Multi-Server Scenario M-SRPT: Let s = M If J <= M Then J jobs each receive 1/s (M-J idle servers) Else M jobs each receive 1/s (while J-M wait) M-PS: Let s = max(M, J) Each job receives a service rate of 1/s M-FSP: Let s = M If J <= M Then J jobs (under PS) each receive 1/s Else M jobs (under PS) each receive 1/s WORMS 2004

Mean Slowdown for M-SRPT WORMS 2004

Variance of Slowdown for M-SRPT WORMS 2004

Mean Slowdown for M-PS WORMS 2004

Variance of Slowdown for M-PS WORMS 2004

Mean Slowdown for M-FSP WORMS 2004

Variance of Slowdown for M-FSP WORMS 2004

Summary Slowdown profile plots for several policies For the largest jobs, FSP better than SRPT and PS For small jobs, FSP is sometimes worse than SRPT Multi-threaded server results T-SRPT and K-SRPT provide a smooth transition between SRPT and PS, implying smoother tradeoff in fairness between small jobs and large jobs Multi-server results With more servers, mean slowdown worsens, but variance of slowdown often improves FSP does not response time dominate PS for M > 1 WORMS 2004

Thank You! Questions? For more information 4/16/2017 Thank You! Questions? For more information Email: {gongm,carey}@cpsc.ucalgary.ca WORMS 2004