Scalable Content-aware Request Distribution in Cluster-based Network Servers Jianbin Wei 10/4/2001

Jianbin Wei 2 Introduction $ Content-based request distribution can improve the throughput of the whole cluster server by cache affinity and get good balancing simultaneously. $ A single front-end quickly becomes the bottleneck of whole performance and limits the scalability of the distributed servers. $ TCP splicing strategy forwards the request based on the workload of back-end servers. Use weighted round-robin to get good load balancing. $ Decouple the role of the front-end into dispatcher and distributor. $ The dispatcher decides the distribution policy. $ The distributors perform the policy result by distributing the request.

Jianbin Wei 3 Performance of LARD and TCP splicing

Jianbin Wei 4 Solution $ Decouple the role of the front-end into dispatcher and distributor. $ The dispatcher decides the distribution policy. $ The distributors perform the policy result by distributing the request. $ The key insight is that the function of distributor can be executed parallel while the overhead of dispatcher accounts very small. $ With multiple front-end nodes, it must choose which distributor to process the request, which is difficult to achieve good load balancing with simple strategy.

Jianbin Wei 5 Models

Jianbin Wei 6 Prototype Implementation $ A persistent TCP control connection exists between any two nodes of the cluster and multiplexes the messages exchanged between any two components. $ The layer 4 switch maintains the load of each cluster node and decides the first part of client request. $ After receiving update message from the dispatcher that indicates which cluster server node is chose to service the request, it redistributes these requests. $ Upon connection termination, a close message is sent to the switch to discard the connection state.

Jianbin Wei 7 Cluster Operation

Jianbin Wei 8 Operation (cont.) $ The client contacts the switch, which decided which distributor to service the request; $ The distributor contacts the dispatcher to get the assignment of this request; $ The distributor distributes the request to the selected server node and sends this information to the switch; $ The server response goes to the client directly. $ Any TCP acknowledgments sent by the client to the distributor are forwarded to the server node using a forwarding module at the distributor node. (DNS round-robin)

Jianbin Wei 9 Technologies When using layer 4 switch $ The choice of distributor is made by the switch, using WRR strategy $ After a connection is handed off by the distributor, the switch is notified and the subsequent forwarding of TCP acknowledgments to the corresponding server node is handled by the switch. To decrease the communication between the dispatcher and the distributor, $ it hashes the URL to a 32 bit integer $ Sends several messages at the same time with dynamic degree of batching.

Jianbin Wei 10 Experimental Results

Jianbin Wei 11 Experimental Results (cont.) $ It shows the throughput results as the number of server nodes are increased. $ It indicates that the dispatcher is far from being a bottleneck. $ Due to the resource share between distributor and server component, the performance of N nodes is achieved by N-1 server nodes in LARD strategy. However, it affords greater scalability of the cluster server. $ When it runs the real workload, it shows the similar results.

Jianbin Wei 12 Conclusion $ This strategy is a scalable architecture for content-aware request distribution in Web server clusters. $ Drawback: because of the high overhead of handoff, it is necessary to decrease this cost to improve the performance and scalability of cluster server. (Scalable Web Server Cluster Design with Workload-Aware Request Distribution Strategy WARD, L. Cherkasova and M. Karlsson, HP Lab)