1 Admission Control and Request Scheduling in E-Commerce Web Sites Sameh Elnikety, EPFL Erich Nahum, IBM Watson John Tracey, IBM Watson Willy Zwaenepoel, EPFL

2 Dynamic Content Is Important 1 3 2

3 Generating Dynamic Content http Database Server Web Server Dynamic Content Generator

4 Objective: Stable Performance Prevent overload (admission control) Sustain peak throughput in overload region Load Throughput Actual Ideal UnderloadoverloadSaturation

5 Objective: Improve Response Time Excess requests are queued Improve response time (request scheduling)

6 Main Idea: Gatekeeper Proxy Transparent Intercept requests to the bottleneck Maintains measurement-based estimates Performs: –Admission control –Request scheduling http Web Server Dynamic Content Generator Gate Keeper Database Server Transparently control how requests are admitted (non-invasive)

7 Admission Control Sustain peak throughput during overload Amount of work required by each request Capacity of the system

8 Estimating Work by Request Type Key Observations: –Finite number of request types –Requests of same type take same execution time –Different requests differ greatly in execution time –Online measurements Gatekeeper maintains per-request estimates http Web Server Dynamic Content Generator Gate Keeper Database Server

9 Service Time Distributions Request type is key! Online measurement

10 Estimating System Capacity Request load = –Execution time (work units of required) Database capacity = –max # work units before overload To determine system capacity –Binary search –Offline

11 req typeunits R15 R2500 R3300 …… R1 R2R R2R R1R R1 R2 R R2 Admission Control - Example 6 Table: maintain for each type, online estimates of service time (moving average)

12 Request Scheduling - Example (0+500) + (500+10) = 1010  505 (0+10) + (10+500) = 520  260

13 Request Scheduling Reduce average response time Use shortest job first (SJF) policy Reorder requests in admission queue No preemption

14 Large Variability: TPC-W Requests 95% require < 1000 ms Scheduling has high impact

15 Request Scheduling: Aging Prevent starvation Limit the delay due to scheduling Limit is X times “expected service time” Big delay due to sched.

16 Outline Motivation & Background The Gatekeeper Proxy Experimental Environment –Software & Hardware –Metrics & Methodology Results Summary and Conclusions

17 TPC-W Benchmark Transaction Processing Council (TPC-W) –Workload generator Models a large e-commerce site: –Online bookstore –Searching, browsing, buying, registration, … Persistent data –Static images on web server –All others on back-end database

18 TPC-W Benchmark - Snapshot Promotion (ad) Shopping Cart Next Interaction Image

19 TPC-W: Interactions 14 interactions, e.g.: –Home (read-only query) –Best sellers (complex) –Secure payment (ssl) –Shopping cart (update query) Scale –10,000 items –288,000 clients –350 MB database (fits in main memory) –183 MB images (in file system of web server)

20 Software GateKeeper Implemented in Java (JDBC driver) Web Server Apache App Server Jakarta Tomcat Database MySQL OS RedHat 7.2 Linux http Web Server Dynamic Content Generator Gate Keeper Database Server

21 Hardware CPU AMD Athlon 1.33 GHz Memory 768 MB Disk 60 GB, 12 msec, 5400 rpm Network 100 Mbps Ethernet http Apache Tomcat GateKeeper MySQL sql

22 http Apache Tomcat GateKeeper MySQL sql Emulated Clients Emulated Clients Client emulator – Session duration – Think time – Markov model Load is a function of the number of clients

23 Experiments Performance Metrics –Throughput (interactions/minute) –Response time (msec, submission to completion) –Examine each as a function of load (# of clients) Methodology –Average of 5 runs –100 second warm-up –600 second measurement

24 Admission Control - Throughput db processes: 345 used mem: 509 MB db processes: 233 used mem: 450 MB db processes: 49 used mem: 275 MB 3 1 2

25 Request Scheduling - Response Time

26 Request Scheduling - Analysis Waiting timeExecution (service) timeResponse time = Waiting time + Execution (service) time Large variability –Many short requests –Few very large requests

27 Request Scheduling - Explanation Average job

28 Request Scheduling - Explanation Short jobAverage jobLong job

29 Request Scheduling - Fairness Waiting timeExecution (service) timeResponse time = Waiting time + Execution (service) time Fairness trade-off –FIFO Fair: all wait for same amount of time –SJF Unfair: favors short requests Better average response time

30 Aging: Prevent Starvation ∞

31 In The Paper More results –Different bottleneck (database lock contention) –Online vs. offline measurements –DB2 Related work –Most other methods are invasive

32 Summary Presented the Gatekeeper proxy – Transparent (non-invasive) – Intercept requests – Online measurements Admission control – Consistent performance during overload – Improves throughput 10 % Request scheduling using SJF – Improves response time 14 times – Penalizes long jobs only 13 % – Aging controls penalty

33 Thank You!