Web Server Load Balancing/Scheduling Asima Silva Tim Sutherland

Outline Web Server Introduction Information Management Basics Load Sharing Policies FLEX WARD EquiLoad AdaptLoad Summary Conclusions Future Work

Introduction to Web Server Load Balancing Internet Request enters a router Load balancing server determines which web server should serve the request Sends the request to the appropriate web server Router Load-Balancing Server Web Servers Request Response Traditional Web Cluster

How do we split up information? Content Server Farm ? Content– all the files on the servers, each square is a subcontent, ex. A file, or a group of files Server Farm – collection/cluster of servers, which can be grouped using different characteristics to efficiently process the requests

Information Strategies Partition Replication

Load Balancing Approaches File Distribution Routing Content/Locality Aware DNS Server Size Aware Centralized Router Workload Aware Distributed Dispatcher Content/Locality aware – grouping files by a common characteristic: ex. Location of the file, types of files, access rate of files Size Aware – grouped by the size of files Workload Aware – grouped by workload expected on the machine, frequency of the file accessed Load Balancing Router – Router that uses an algorithm which determines which server will process a request Dispatcher – transfers the request to the server to be processed, and any server can be a dispatcher

Issues Efficiently processing requests with optimizations for load balancing Send and process requests to a web server that has files in cache Send and process requests to a web server with the least amount of requests Send and process requests to a web server determined by the size of the request

FLEX Locality aware load-balancing strategy based on two factors: File Distribution Routing Content/Locality Aware DNS Server Size Aware Centralized Router Workload Aware Distributed Dispatcher Locality aware load-balancing strategy based on two factors: Accessed files, memory requirements Access rates (working set), load requirements Partitions all servers into equally balanced groups Each server transfers the response to the browser to reduce bottleneck through the router (TCP Handoff)

Flex Diagram DNS Server S1 S2 Requests Forwards Request S3 S4 To Client Browser S5 S6 W(S1) ≈ W(S2) ≈ W(S3) ≈ … ≈ W(S6) Ar(S1) ≈ Ar(S2) ≈ Ar(S3) ≈ … ≈ Ar(S6)

FLEX Cont. Advantages: Highly scalable Reduces bottleneck by the load balancer No software is required Reduces number of cache misses

FLEX Cont. II Disadvantages: Not dynamic, routing tale must be recreated Only compared to RR Number of access logs required on each server could be tremendous Responsibility of load-balancing and transferring response is given to web servers – unorganized responsibility How often to update access rates and working sets? Monitor?

WARD Workload-Aware Request Distribution Strategy File Distribution Routing Content/Locality Aware DNS Server Size Aware Centralized Router Workload Aware Distributed Dispatcher WARD Workload-Aware Request Distribution Strategy Server core are essential files that represent majority of expected requests Server core is replicated at every server Ward-analysis computes the nearly optimal core size determined by workload access patterns Number of nodes Node RAM TCP handoff overhead Disk access overhead

WARD Cont. Three components: dispatcher (load balancer), distributor (router), web server Three progressive architectures: CARD WARD LARD Dispatcher Dispatcher Distributor Front End LAN Server Dispatcher Distributor Front End LAN Server Switch Distributor Server Switch Front End Dispatcher Distributor Server LAN Single front-end distributor, centralized dispatcher Co-located distributor and server Co-located distributor, server, and dispatcher

WARD Diagram Switch S1 S2 S3 Requests S4 S5 S6 Queue: Queue: Requests Queue: Switch S4 Queue: S5 S6 Each computer is a distributor and a dispatcher Queue: Queue:

WARD Cont. II Similar to FLEX, sends response directly to client Minimizes forwarding overhead from handoffs for the most frequent files Optimizes the overall cluster RAM usage “by mapping a small set of most frequent files to be served by multiple number of nodes, we can improve both locality of accesses and the cluster performance significantly”

WARD Cont. III Advantages: No decision making, core files are replicated on every server Minimizes transfer of requests and disk reads, both are “equally bad” Outperforms Round Robin Efficient use of RAM Performance gain with increased number of nodes

WARD Cont. IV Disadvantages: Core files are created on past day’s data, could decrease performance up to 15% Distributed dispatcher increases the number of TCP requests transfers If core files not selected correctly, higher cache miss rate and increased disk accesses

WARD Results

EquiLoad File Distribution Routing Content/Locality Aware DNS Server Size Aware Centralized Router Workload Aware Distributed Dispatcher Determines which server will process a request determined by the size of the requested file Splits the content on each server by file size, forcing the queues sizes to be consistent.

EquiLoad Solves Queue Length Problems This is bad Queue This is better 1k 2k 100k 1k 1000k 1k 2k 100k

EquiLoad Diagram Dispatcher Distributor S1 S2 Requests 1k-2k 2k-3k (periodically calculates partitions) S1 S2 Requests 1k-2k 2k-3k Forwards Request S3 S4 Distributor 3k-10k 10k-20k To Client Browser S5 S6 20k-100k >100k

EquiLoad Advantages Dynamic repartitioning Can be implemented at various levels DNS Dispatcher Server Minimum queue buildup Performs well under variable workload and high system load

EquiLoad Disadvantages Cache affinity is neglected Requires a front end dispatcher Distributor must communicate with servers Thresholds of parameter adjustment

EquiLoad  AdaptLoad AdaptLoad improves upon EquiLoad using “fuzzy boundaries” Allows for multiple servers to process a request Behaves better in situations where server partitions are very close in size

AdaptLoad Diagram Distributor Dispatcher S1 S2 Requests 1k-3k 2k-4k (periodically calculates partitions) S1 S2 Requests 1k-3k 2k-4k Forwards Request S3 S4 Dispatcher 3k-10k 8k-20k To Client Browser S5 S6 15k-100k >80k

AdaptLoad Results

Summary File Distribution Routing Content/Locality Aware DNS Server Size Aware Centralized Router Workload Aware Distributed Dispatcher FLEX EquiLoad, AdaptLoad WARD

Conclusions There is no “best” way to distribute content among servers. There is no optimal policy for all website applications. Certain strategies are geared towards a particular website application.

Future Work Compare and contrast the three policies Figure out how often nodes should be repartitioned Compare each policy to a standard benchmark Figure out which policy works in a particular environment

Questions? Anyone have one?