Multiple-resource Request Scheduling. for Differentiated QoS Multiple-resource Request Scheduling for Differentiated QoS at Website Gateway Student: Ruo-Hua Feng Advisor: Dr. Ying-Dar Lin and Dr. Yuan-Cheng Lai Date: 2005/06/10 Department of Computer and Information Science National Chiao Tung University

High Speed Network Laboratory @ NCTU Agenda Server Resource Management for Web QoS mQoS Problem and Objectives mQoS Gateway Architecture mQoS Scheduling Algorithm Implementation and Evaluation Conclusions and Future Work References High Speed Network Laboratory @ NCTU

Server Resource Management for Web QoS When to consider resource management? Light load: unused resources Heavy load: processing delay How to manage the limited server resources? Request scheduling: Supports service differentiation How many resources to be managed? Single-resource management: Bottlenecks from the other resources Multiple-resource management: A request actually consumes multiple server resources High Speed Network Laboratory @ NCTU

Various Types of Workload Different proportion on # of resource-intensive requests Possible cases: Auction website CPU, Disk Album website Disk, Bandwidth CPU Disk BW Disk BW CPU High Speed Network Laboratory @ NCTU

Single-Resource (CPU) Request Scheduling >> = Server CPU Disk BW Bandwidth Waste Disk I/O CPU Req5 Req4 Req3 Req2 Req1 Toward Server CPU Disk Bandwidth >> = Server Disk BW CPU Bandwidth Overwhelm Disk I/O CPU Req5 Req4 Req3 Req2 Req1 Toward Server CPU Disk Bandwidth High Speed Network Laboratory @ NCTU

Server Resource Management Schemes No scheduling (nQoS): Single-resource scheduling (sQoS): Multiple-resource scheduling (mQoS): No resource differentiation Depends on workload Resource wasting Resource overloading Utilization maximization Resource differentiation High Speed Network Laboratory @ NCTU

mQoS Problem and Objectives Problem statement: To provide service differentiation by scheduling requests for managing “multiple resources” of the server at a website gateway Objectives: To differentiate the server resource utilization To balance and to maximize the resource utilization To improve the server throughput High Speed Network Laboratory @ NCTU

mQoS Gateway Architecture High Speed Network Laboratory @ NCTU

High Speed Network Laboratory @ NCTU Server Prober Request profiling: Sends once a request to the server Measures the resource consumption of each request Example: query page: (CPU:15, Disk I/O: 5, Bandwidth: 8 ) Server profiling: Sends huge amount of a specific resource-intensive request Measures the maximum number of concurrent requests when the resource is fully utilized Example: 100 concurrent CPU-intensive requests = 1500 units Web page table: Example: request CPU consumption = 15 / 1500= 1% Stores the resource consumption of a request in percentage High Speed Network Laboratory @ NCTU

High Speed Network Laboratory @ NCTU Request Classifier Determines the resource requirements tendency Web page table Determines the service class QoS policy table Puts a request into an appropriate class queue n resources * m classes = n*m class queues 3 2 class1 1 resouce1 class2 m classes class3 m*n queues resouce2 n resources resouce3 High Speed Network Laboratory @ NCTU

mQoS Scheduler Component I: Main Scheduler Goal: Resource Utilization Balance Main Scheduler CPU intensive CPU Sub-Scheduler 100 CPU Disk I/O Sub-Scheduler Disk I/O intensive 100 RAC: Availability of a server resource Disk I/O 100 Bandwidth Sub-Scheduler Bandwidth intensive Bandwidth Resource Availability Counter (RAC) High Speed Network Laboratory @ NCTU

mQoS Scheduler Component II: Sub Scheduler Goal: Resource Differentiation CPU sub-scheduler CPU intensive queues Class 1 60 Class 1 8,2,5 6,5,3 Class 2 30 Class 2 8,1,4 7,3,4 Main Scheduler 100 Class 3 7,3,2 9,5,6 Class 3 10 CPU Deficit Counter (DC) DC: Unused service quantum of a class Disk I/O intensive 60 100 Resource requirement for one request [CPU, Disk I/O, BW] 30 Disk I/O 10 100 Bandwidth intensive 60 Bandwidth 30 Resource Availability Counter (RAC) 10 High Speed Network Laboratory @ NCTU

High Speed Network Laboratory @ NCTU Scheduling Example 54 60 8,2,5 6,5,3 30 8,1,4 7,3,4 100 94 10 7,3,2 9,5,6 60 55 2,8,3 3,5,4 95 100 30 2,7,5 6,8,5 10 1,6,2 3,9,2 60 57 97 100 4,1,8 7,6,9 30 1,3,7 3,2,7 10 5,4,9 1,3,9 High Speed Network Laboratory @ NCTU

High Speed Network Laboratory @ NCTU Scheduling Example 47 54 8,2,5 30 8,1,4 7,3,4 94 87 10 7,3,2 9,5,6 55 49 2,8,3 3,5,4 95 89 30 2,7,5 6,8,5 10 1,6,2 3,9,2 57 48 88 97 4,1,8 7,6,9 30 1,3,7 3,2,7 10 5,4,9 1,3,9 High Speed Network Laboratory @ NCTU

High Speed Network Laboratory @ NCTU Scheduling Example 44 47 8,2,5 30 8,1,4 7,3,4 87 84 90 10 7,3,2 9,5,6 44 49 2,8,3 3,5,4 89 89 84 30 2,7,5 6,8,5 10 1,6,2 3,9,2 6,5,3 44 48 87 88 84 4,1,8 30 1,3,7 3,2,7 10 5,4,9 1,3,9 High Speed Network Laboratory @ NCTU

Design of mQoS Scheduler Issue Solution Component 1 Balancing server resource utilization Main scheduler triggers the sub-scheduler of the most available resource Main scheduler with Resource Availability Counter (RAC) 2 Differentiating server resources Sub-scheduler is derived from DRR scheduling Sub-scheduler with Deficit Counters (DC) Select request by ARC and pointer Once output a request High Speed Network Laboratory @ NCTU

Implementation of mQoS Gateway Based on Squid and Linux Read and parse a request Check the cache Prepare to forward the request Send the request to the server Classify the request Request processing module Added Bypass Response processing module Schedule the request Update the RAC Send the response to the client Prepare to forward the response Store in the cache Read and parse a response High Speed Network Laboratory @ NCTU

High Speed Network Laboratory @ NCTU Evaluation Testbed: Workload: 3 service classes of clients Service ratio: 6:3:1 3 types of resource-intensive requests CPU, Disk I/O, Bandwidth CPU-intensive requests are more than other types of requests Observation: nQoS, sQoS, and mQoS Resource utilization, server throughput, resource sharing, and user-perceived latency CPU-intensive 9%, 2%, 1% Disk I/O-intensive 1%, 9%, 2% Bandwidth-intensive 2%, 1%, 9% High Speed Network Laboratory @ NCTU

Server Resource Utilization nQoS: No differentiation mQoS: Resource differentiation Maximized utilization sQoS: Resource wasting High Speed Network Laboratory @ NCTU

High Speed Network Laboratory @ NCTU Server Throughput mQoS is improved by 21% High Speed Network Laboratory @ NCTU

Types of Outstanding Requests sQoS mQoS mQoS scheduling differentiates the utilization of every server resource High Speed Network Laboratory @ NCTU

High Speed Network Laboratory @ NCTU Resources can be shared by the active classes Resource Sharing nQoS Scheduling mQoS Scheduling sQoS Scheduling High Speed Network Laboratory @ NCTU

User-perceived Latency Differentiated the shortest High Speed Network Laboratory @ NCTU

Decomposition of the User-perceived Latency in the mQoS Scheduling User-perceived latency is differentiated due to queuing time High Speed Network Laboratory @ NCTU

Conclusions and Future Work The mQoS gateway is presented: Server resources can be differentiated among classes Resource utilization can be maximized Server throughput can be improved Evaluation summary: The three classes get 60%, 30%, and 10% utilization of every server resource as the pre-defined QoS policies The total server throughput of the mQoS is improved by 21% The user-perceived latency is also differentiated among classes Future work: Multiple-resource management for A server cluster or Web Services High Speed Network Laboratory @ NCTU

High Speed Network Laboratory @ NCTU References L. Eggert and J. Heidemann, “Application-Level Differentiated Services for Web Servers,” World Wide Web Journal, vol. 2, no. 3, pp. 133-142, Aug. 1999. R. Pandey, J. F. Barnes, and R. Olsson, “Supporting Quality of Service in HTTP Servers,” Proceedings of the 7th Annul ACM Symposium on Principles of Distributed Computing, pp. 247-256, Jun. 1998. V. Cardellini, E. Casalicchio, M. Colajanni, and M. Mambelli, “Web Switch Support for Differentiated Services,” ACM Performance Evaluation Review, vol. 29, no. 2, pp. 14-19, Sep. 2001. W. Leinberger, G. Karypis, and V. Kumar, “Job Scheduling in the presence of Multiple Resource Requirements”, Proceedings of the 7th International Conference on High Performance Networking and Computing, Apr.1999. M. Shreedhar and G. Varghese, “Efficient Fair Queuing Using Deficit Round- Robin,” IEEE/ACM Transaction on Networking, vol. 4, issue 3, pp. 375-385, Jun. 1996. K. Li, and S. Jamin, “A Measure-Based Admission Control Web server,“ Proceedings of the 9th Annual Joint Conference of the IEEE Computer and Communications Societies, Vol. 2, pp. 651-659, Mar. 2002. High Speed Network Laboratory @ NCTU

High Speed Network Laboratory @ NCTU References (cont.) L. Cherkasova and P. Phaal, “Session-Based Admission Control: A Mechanism for Peak Load Management of Commercial Web Sites,” IEEE Transactions on Computers, Vol. 51, Issue 6, pp. 669-685, Jun. 2002. S. Elnikety, J. Treacy, E. Nahum, and W. Zwaenepol, “A Method for Transparent Admission Control and Request Scheduling in E-Commerce Web Sites,” Proceedings of the 13th International World Wide Web Conference, pp. 276-286, May 2004. High Speed Network Laboratory @ NCTU