1. Multiple-resource Request Scheduling. for Differentiated QoS
Multiple-resource Request Scheduling for Differentiated QoS at Website Gateway
Student: Ruo-Hua Feng
Advisor: Dr. Ying-Dar Lin and Dr. Yuan-Cheng Lai
Date: 2005/06/10
Department of Computer and Information Science
National Chiao Tung University

2. High Speed Network Laboratory @ NCTU
Agenda
Server Resource Management for Web QoS
mQoS Problem and Objectives
mQoS Gateway Architecture
mQoS Scheduling Algorithm
Implementation and Evaluation
Conclusions and Future Work
References
High Speed Network NCTU

3. Server Resource Management for Web QoS
When to consider resource management?
Light load: unused resources
Heavy load: processing delay
How to manage the limited server resources?
Request scheduling: Supports service differentiation
How many resources to be managed?
Single-resource management:
Bottlenecks from the other resources
Multiple-resource management:
A request actually consumes multiple server resources
High Speed Network NCTU

4. Various Types of Workload
Different proportion on # of resource-intensive requests
Possible cases:
Auction website
CPU, Disk
Album website
Disk, Bandwidth
CPU
Disk BW
Disk BW
CPU
High Speed Network NCTU

5. Single-Resource (CPU) Request Scheduling
>> =
Server
CPU
Disk BW
Bandwidth
Waste
Disk I/O
CPU
Req5
Req4
Req3
Req2
Req1
Toward Server
CPU
Disk
Bandwidth
>> =
Server
Disk BW
CPU
Bandwidth
Overwhelm
Disk I/O
CPU
Req5
Req4
Req3
Req2
Req1
Toward Server
CPU
Disk
Bandwidth
High Speed Network NCTU

6. Server Resource Management Schemes
No scheduling
(nQoS):
Single-resource scheduling
(sQoS):
Multiple-resource scheduling
(mQoS):
No resource differentiation
Depends on workload
Resource wasting
Resource overloading
Utilization maximization
Resource differentiation
High Speed Network NCTU

7. mQoS Problem and Objectives
Problem statement:
To provide service differentiation by scheduling requests
for managing “multiple resources” of the server
at a website gateway
Objectives:
To differentiate the server resource utilization
To balance and to maximize the resource utilization
To improve the server throughput
High Speed Network NCTU

8. mQoS Gateway Architecture
High Speed Network NCTU

9. High Speed Network Laboratory @ NCTU
Server Prober
Request profiling:
Sends once a request to the server
Measures the resource consumption of each request
Example: query page: (CPU:15, Disk I/O: 5, Bandwidth: 8 )
Server profiling:
Sends huge amount of a specific resource-intensive request
Measures the maximum number of concurrent requests
when the resource is fully utilized
Example: 100 concurrent CPU-intensive requests = 1500 units
Web page table:
Example: request CPU consumption = 15 / 1500= 1%
Stores the resource consumption of a request in percentage
High Speed Network NCTU

10. High Speed Network Laboratory @ NCTU
Request Classifier
Determines the resource requirements tendency
Web page table
Determines the service class
QoS policy table
Puts a request into an appropriate class queue
n resources * m classes = n*m class queues 3 2
class1 1
resouce1
class2
m classes
class3
m*n queues
resouce2
n resources
resouce3
High Speed Network NCTU

11. mQoS Scheduler Component I: Main Scheduler
Goal: Resource Utilization Balance
Main Scheduler
CPU intensive
CPU
Sub-Scheduler
100
CPU
Disk I/O
Sub-Scheduler
Disk I/O intensive
100
RAC:
Availability of a server resource
Disk I/O
100
Bandwidth
Sub-Scheduler
Bandwidth intensive
Bandwidth
Resource Availability Counter
(RAC)
High Speed Network NCTU

12. mQoS Scheduler Component II: Sub Scheduler
Goal: Resource Differentiation
CPU sub-scheduler
CPU intensive queues
Class 1 60
Class 1
8,2,5
6,5,3
Class 2 30
Class 2
8,1,4
7,3,4
Main Scheduler
100
Class 3
7,3,2
9,5,6
Class 3 10
CPU
Deficit Counter (DC)
DC:
Unused service quantum of a class
Disk I/O intensive 60
100
Resource requirement for one request
[CPU, Disk I/O, BW] 30
Disk I/O 10
100
Bandwidth intensive 60
Bandwidth 30
Resource Availability Counter
(RAC) 10
High Speed Network NCTU

13. High Speed Network Laboratory @ NCTU
Scheduling Example 54 60
8,2,5
6,5,3 30
8,1,4
7,3,4
100 94 10
7,3,2
9,5,6 60 55
2,8,3
3,5,4 95
100 30
2,7,5
6,8,5 10
1,6,2
3,9,2 60 57 97
100
4,1,8
7,6,9 30
1,3,7
3,2,7 10
5,4,9
1,3,9
High Speed Network NCTU

14. High Speed Network Laboratory @ NCTU
Scheduling Example 47 54
8,2,5 30
8,1,4
7,3,4 94 87 10
7,3,2
9,5,6 55 49
2,8,3
3,5,4 95 89 30
2,7,5
6,8,5 10
1,6,2
3,9,2 57 48 88 97
4,1,8
7,6,9 30
1,3,7
3,2,7 10
5,4,9
1,3,9
High Speed Network NCTU

15. High Speed Network Laboratory @ NCTU
Scheduling Example 44 47
8,2,5 30
8,1,4
7,3,4 87 84 90 10
7,3,2
9,5,6 44 49
2,8,3
3,5,4 89 89 84 30
2,7,5
6,8,5 10
1,6,2
3,9,2
6,5,3 44 48 87 88 84
4,1,8 30
1,3,7
3,2,7 10
5,4,9
1,3,9
High Speed Network NCTU

16. Design of mQoS Scheduler
Issue
Solution
Component 1
Balancing
server resource utilization
Main scheduler triggers
the sub-scheduler
of the most available resource
Main scheduler with
Resource Availability Counter (RAC) 2
Differentiating
server resources
Sub-scheduler is derived from DRR scheduling
Sub-scheduler with
Deficit Counters (DC)
Select request by ARC and pointer
Once output a request
High Speed Network NCTU

17. Implementation of mQoS Gateway
Based on Squid and Linux
Read and parse
a request
Check
the cache
Prepare to forward
the request
Send the request
to the server
Classify
the request
Request processing module
Added
Bypass
Response processing module
Schedule
the request
Update
the RAC
Send the response
to the client
Prepare to forward
the response
Store in
the cache
Read and parse
a response
High Speed Network NCTU

18. High Speed Network Laboratory @ NCTU
Evaluation
Testbed:
Workload:
3 service classes of clients
Service ratio: 6:3:1
3 types of resource-intensive requests
CPU, Disk I/O, Bandwidth
CPU-intensive requests
are more than other types of requests
Observation:
nQoS, sQoS, and mQoS
Resource utilization, server throughput, resource sharing, and
user-perceived latency
CPU-intensive
9%, 2%, 1%
Disk I/O-intensive
1%, 9%, 2%
Bandwidth-intensive
2%, 1%, 9%
High Speed Network NCTU

19. Server Resource Utilization
nQoS: No differentiation
mQoS: Resource differentiation
Maximized utilization
sQoS: Resource wasting
High Speed Network NCTU

20. High Speed Network Laboratory @ NCTU
Server Throughput
mQoS is improved by 21%
High Speed Network NCTU

21. Types of Outstanding Requests
sQoS
mQoS
mQoS scheduling differentiates the utilization of every server resource
High Speed Network NCTU

22. High Speed Network Laboratory @ NCTU
Resources can be shared by the active classes
Resource Sharing
nQoS Scheduling
mQoS Scheduling
sQoS Scheduling
High Speed Network NCTU

23. User-perceived Latency
Differentiated
the shortest
High Speed Network NCTU

24. Decomposition of the User-perceived Latency in the mQoS Scheduling
User-perceived latency is differentiated due to queuing time
High Speed Network NCTU

25. Conclusions and Future Work
The mQoS gateway is presented:
Server resources can be differentiated among classes
Resource utilization can be maximized
Server throughput can be improved
Evaluation summary:
The three classes get 60%, 30%, and 10% utilization of every server resource as the pre-defined QoS policies
The total server throughput of the mQoS is improved by 21%
The user-perceived latency is also differentiated among classes
Future work:
Multiple-resource management for
A server cluster or Web Services
High Speed Network NCTU

26. High Speed Network Laboratory @ NCTU
References
L. Eggert and J. Heidemann, “Application-Level Differentiated Services for Web Servers,” World Wide Web Journal, vol. 2, no. 3, pp , Aug
R. Pandey, J. F. Barnes, and R. Olsson, “Supporting Quality of Service in HTTP Servers,” Proceedings of the 7th Annul ACM Symposium on Principles of Distributed Computing, pp , Jun
V. Cardellini, E. Casalicchio, M. Colajanni, and M. Mambelli, “Web Switch Support for Differentiated Services,” ACM Performance Evaluation Review, vol. 29, no. 2, pp , Sep
W. Leinberger, G. Karypis, and V. Kumar, “Job Scheduling in the presence of Multiple Resource Requirements”, Proceedings of the 7th International Conference on High Performance Networking and Computing, Apr.1999.
M. Shreedhar and G. Varghese, “Efficient Fair Queuing Using Deficit Round- Robin,” IEEE/ACM Transaction on Networking, vol. 4, issue 3, pp , Jun
K. Li, and S. Jamin, “A Measure-Based Admission Control Web server,“ Proceedings of the 9th Annual Joint Conference of the IEEE Computer and Communications Societies, Vol. 2, pp , Mar
High Speed Network NCTU

27. High Speed Network Laboratory @ NCTU
References (cont.)
L. Cherkasova and P. Phaal, “Session-Based Admission Control: A Mechanism for Peak Load Management of Commercial Web Sites,” IEEE Transactions on Computers, Vol. 51, Issue 6, pp , Jun
S. Elnikety, J. Treacy, E. Nahum, and W. Zwaenepol, “A Method for Transparent Admission Control and Request Scheduling in E-Commerce Web Sites,” Proceedings of the 13th International World Wide Web Conference, pp , May 2004.
High Speed Network NCTU