1 Routing and Scheduling in Web Server Clusters

2 Reference The State of the Art in Locally Distributed Web-server Systems Valeria Cardellini, Emiliano Casalicchio, Michele Colajanni and Philip S. Yu

3 Objectives  Routing in Distributed Web-based Servers  DNS-based routing mechanisms  Web server routing mechanisms  Dispatching Algorithms  Content-blind algorithms  Static  Client aware  Server aware  Client and Server aware  Content-aware algorithms  Client aware  Client and Server aware

4 Routing in Distributed Web-based Servers  DNS-based routing mechanisms  Handles multiple web servers hosting a Web site  The idea is conceived for locally distributed web servers but it is also applicable for geographically distributed web servers  Web server routing mechanisms  Routing mechanisms are implemented by the web server  Web servers can (re)direct a client request to another node

5 DNS-based routing  DNS-based routing intervenes at lookup phase  The authoritative DNS (A-DNS) server can select a different server for every address resolution  The A-DNS replies with (IP address, TTL)

6 Web server routing  Triangulation  HTTP redirection  URL rewriting

7 Web server routing: Triangulation  The client sends packets to the first contacted server  The first node routes packets to a second server  Routing is done at the TCP/IP level  Triangulation is based on tunneling!

8 Web server routing: HTTP redirection  HTTP protocol allows a Web server to respond to a client request with some redirection status code  HTTP has a built-in redirection mechanism  HTTP redirection is good for  Medium granularity down to individual Web pages  Allowing content-aware routing  Drawbacks  Extra round-trip time

9 Web server routing: HTTP redirection

10 Web server routing: HTTP rewriting  The first contacted server dynamically changes the links for the embedded objects within the Web page  This approach is used by Akamai  This approach introduces additional load on the redirecting Web server  It also cause a considerable DNS overhead

11 Dispatching Algorithms Strategies to select the target server of the web clusters  Static: Fastest solution to prevent web switch bottleneck, but do not consider the current state of the servers  Dynamic: Outperform static algorithms by using intelligent decisions, but collecting state information and analyzing them cause expensive overheads Requirements: (1) Low computational complexity (2) Full compatibility with web standards (3) state information must be readily available without much overhead

12 Content blind approach  Static Policies: Random distributes the incoming requests uniformly with equal probability of reaching any server Round Robin (RR) use a circular list and a pointer to the last selected server to make the decision Static Weighted RR (For heterogeneous severs) A variation of RR, where each server is assigned a weight Wi depending on its capacity

13 Content blind approach (Cont.)  Dynamic Client state aware static partitioning the server nodes and to assign group of clients identified through the clients information, such as source IP address Server State Aware Least Loaded, the server with the lowest load. Issue: Which is the server load index? Least Connection fewest active connection first

14 Content blind approach (Cont.)  Server State Aware Contd. Fastest Response responding fastest Weighted Round Robin Variation of static RR, associates each server with a dynamically evaluated weight that is proportional to the server load  Client and server state aware Client affinity instead of assigning each new connection to a server only on the basis of the server state regardless of any past assignment, consecutive connections from the same client can be assigned to the same server

15 Considerations of content blind  Static approach is the fastest, easy to implement, but may make poor assignment decision  Dynamic approach has the potential to make better decisions, but it needs to collect and analyze state information, may cause high overhead  Overall, simple server state aware algorithm is the best choice, least loaded algorithm is commonly used in commercial products

16

17 Content aware approach  Client state aware Cache Affinity The file space is partitioned among the server nodes. Load Sharing. SITEA (Size Interval Task Assignment with Equal Load) The web switch determines the size of the requested file and select the target server based on this information. CAP (Client-Aware Policy) The web switch requests are classified based on their impact on system resources: such as I/O bound, CPU bound

18 Content aware approach (Cont.)  Client state aware (Cont.) Service Partitioning Employs specialized servers for certain type of requests Client Affinity Using session identifier to assign all web transactions from the same client to the same server

19 Content aware approach (Cont.)  Client and server state aware LARD (Locality aware request distribution) Direct all requests of the same web object to the same server node as long as its utilization is below a given threshold. Cache Manager A cache manager that is aware of the cache content of all web servers.

20

21 Fair Scheduling in Web Servers

22 Objectives  Create an arbitrary number of service quality classes and assign a priority weight for each class.  Provide service differentiation for different use classes in terms of the allocation of CPU and disk I/O capacities

23

24

25

26

27

28

29

30 Fair Scheduling in a Web Cluster: Objective  Provide service differentiation (or QoS guarantee) for different user classes in terms of the allocation of CPU and disk I/O capacities => Scheduling  Balance the Load among various nodes in the cluster to ensure maximum utilization and minimum execution time => Load Balancing

31 Request Scheduling Two decisions:  Which request should be serviced next (Scheduling) according to each subscriber’s static resource reservation and dynamic resource usage  Which RPN should service this request (Load Balancing) according to the load information on each RPN (Least Load First) and also exploit access locality