1. Fair Scheduling in Web Servers CS 213 Lecture 17 L.N. Bhuyan

2. Objective Create an arbitrary number of service quality classes and assign a priority weight for each class. Provide service differentiation for different use classes in terms of the allocation of CPU and disk I/O capacities

22. Differentiated Service in a Web Cluster: Objective Create an arbitrary number of service quality classes and assign a priority weight for each class. Provide service differentiation for different use classes in terms of the allocation of CPU and disk I/O capacities Ref: Demand Driven Service Differentiation in Cluster-based Network Servers, Infocom 2001, by Zhu, Tang and Yang

23. Target System

24. Service Differentiation Requests of higher classes receive better services than lower classes, especially when the system is heavily loaded Request from lower classes should not be sacrificed for requests from higher classes when the system load is light

25. Definitions Requests: C 1, C 2, …, C N Corresponding Weights: W 1, W 2, …, W N Stretch factors: S 1, S 2, …, S N stretch factor: the ratio of the response time of a request to the service demand of that request Average arrive rate: λ i Average processing rate: μ i Minimum resource requirement of class I, ρ i = λ i /μ i

26. Optimal Problem Minimize: F= W 1 S 1 + W 2 S 2 + … + W N S N such that: S 1 ≤ S 2 ≤ … ≤ S N S 1, S 2, …, S N ≤ K K is a stretch factor bound, where K > 1 is a predefined threshold

27. Optimal Solution

28. Scheduling optimization for Resource-Intensive Web requests In SPAA’99 By Zhu, Smith and Yang

29. Request Classification Static Data web pages, images, etc. does not consume much system resource

30. Request Classification (Cont.) Dynamic Data e-commerce, database searching, personalized information generated dynamically, place greater I/O and CPU demands 1 to 2 orders of magnitude longer processing time than static requests based on IBM Olympics and Alexandria Digital Library data

31. Flat Architecture Server nodes can process both static and dynamic requests

32. Master/Slave Architecture Server nodes are divided in two groups: Slave group only processes dynamic requests Master group can handles both requests

33. How to partition a cluster Questions: 1: Given p nodes, what is the optimal number of master nodes? 2: What percentage of dynamic requests should be processed on masters? Goal: ensure the master/slave architecture outperforms flat architecture

34. M/S and Flat Models

35. Performance Metric Stretch factor: the ratio of response time at a particular load to that at no load Average stretch factor is more suited than average response time for systems with highly variable task sizes. Average stretch factor indicates load of a system.

36. Evaluation results M/S: up to 69% performance improvement over flat Separation of dynamic and static content Resource reservation: up to 68% improvement Resource requirement sampling: up to 23% improvement

37. Performance Guarantees for Internet Services (Gage) Environment: Web hosting services multiple logical web servers (service subscriber) on a single physical web server cluster. Gage: guarantee each web server with a pre specific performance a distinct number of URL requests to service per second

38. Components Each service subscriber maintain a queue Request classification determines the queue for each input request Request scheduling determines which queue to serve next to meet the QoS requirement for each subscriber. Resource usage accounting capture detailed resource usage associated with each subscriber’s service requests.

40. The Gage System QoS guarantee QoS is in terms of a fixed number of generic URL request which represents an average web site access Currently, assuming it is 10msec of CPU time, 10msec of disk I/O and 2000 bytes of network bandwidth Each subscribe is given a fixed number of generic requests. Other possible QoS metrics: response time, delay jitter etc. Using TCP splicing

42. Request Scheduling Two decisions: Which request should be serviced next according to each subscriber’s static resource reservation and dynamic resource usage Which RPN should service this request according to the load information on each RPN and also exploit access locality