1. Web Server Load Balancing/Scheduling
Asima Silva
Tim Sutherland

2. Outline Web Server Introduction Information Management Basics
Load Sharing Policies
FLEX
WARD
EquiLoad
AdaptLoad
Summary
Conclusions
Future Work

3. Introduction to Web Server Load Balancing
Internet
Request enters a router
Load balancing server determines which web server should serve the request
Sends the request to the appropriate web server
Router
Load-Balancing Server
Web Servers
Request
Response
Traditional Web Cluster

4. How do we split up information?
Content
Server Farm ?
Content– all the files on the servers, each square is a subcontent, ex. A file, or a group of files
Server Farm – collection/cluster of servers, which can be grouped using different characteristics to efficiently process the requests

5. Information Strategies
Partition
Replication

6. Load Balancing Approaches
File Distribution
Routing
Content/Locality Aware
DNS Server
Size Aware
Centralized Router
Workload Aware
Distributed Dispatcher
Content/Locality aware – grouping files by a common characteristic: ex. Location of the file, types of files, access rate of files
Size Aware – grouped by the size of files
Workload Aware – grouped by workload expected on the machine, frequency of the file accessed
Load Balancing Router – Router that uses an algorithm which determines which server will process a request
Dispatcher – transfers the request to the server to be processed, and any server can be a dispatcher

7. Issues
Efficiently processing requests with optimizations for load balancing
Send and process requests to a web server that has files in cache
Send and process requests to a web server with the least amount of requests
Send and process requests to a web server determined by the size of the request

8. FLEX Locality aware load-balancing strategy based on two factors:
File Distribution
Routing
Content/Locality Aware
DNS Server
Size Aware
Centralized Router
Workload Aware
Distributed Dispatcher
Locality aware load-balancing strategy based on two factors:
Accessed files, memory requirements
Access rates (working set), load requirements
Partitions all servers into equally balanced groups
Each server transfers the response to the browser to reduce bottleneck through the router (TCP Handoff)

9. Flex Diagram DNS Server S1 S2 Requests Forwards Request S3 S4
To Client Browser S5 S6
W(S1) ≈ W(S2) ≈ W(S3) ≈ … ≈ W(S6)
Ar(S1) ≈ Ar(S2) ≈ Ar(S3) ≈ … ≈ Ar(S6)

10. FLEX Cont. Advantages: Highly scalable
Reduces bottleneck by the load balancer
No software is required
Reduces number of cache misses

11. FLEX Cont. II Disadvantages:
Not dynamic, routing tale must be recreated
Only compared to RR
Number of access logs required on each server could be tremendous
Responsibility of load-balancing and transferring response is given to web servers – unorganized responsibility
How often to update access rates and working sets? Monitor?

12. WARD Workload-Aware Request Distribution Strategy
File Distribution
Routing
Content/Locality Aware
DNS Server
Size Aware
Centralized Router
Workload Aware
Distributed Dispatcher
WARD
Workload-Aware Request Distribution Strategy
Server core are essential files that represent majority of expected requests
Server core is replicated at every server
Ward-analysis computes the nearly optimal core size determined by workload access patterns
Number of nodes
Node RAM
TCP handoff overhead
Disk access overhead

13. WARD Cont.
Three components: dispatcher (load balancer), distributor (router), web server
Three progressive architectures:
CARD
WARD
LARD
Dispatcher
Dispatcher
Distributor
Front End
LAN
Server
Dispatcher
Distributor
Front End
LAN
Server
Switch
Distributor
Server
Switch
Front End
Dispatcher
Distributor
Server
LAN
Single front-end distributor, centralized dispatcher
Co-located distributor and server
Co-located distributor, server, and dispatcher

14. WARD Diagram Switch S1 S2 S3 Requests S4 S5 S6
Queue:
Queue:
Requests
Queue:
Switch S4
Queue: S5 S6
Each computer is a distributor
and a dispatcher
Queue:
Queue:

15. WARD Cont. II Similar to FLEX, sends response directly to client
Minimizes forwarding overhead from handoffs for the most frequent files
Optimizes the overall cluster RAM usage
“by mapping a small set of most frequent files to be served by multiple number of nodes, we can improve both locality of accesses and the cluster performance significantly”

16. WARD Cont. III Advantages:
No decision making, core files are replicated on every server
Minimizes transfer of requests and disk reads, both are “equally bad”
Outperforms Round Robin
Efficient use of RAM
Performance gain with increased number of nodes

17. WARD Cont. IV Disadvantages:
Core files are created on past day’s data, could decrease performance up to 15%
Distributed dispatcher increases the number of TCP requests transfers
If core files not selected correctly, higher cache miss rate and increased disk accesses

18. WARD Results

19. EquiLoad
File Distribution
Routing
Content/Locality Aware
DNS Server
Size Aware
Centralized Router
Workload Aware
Distributed Dispatcher
Determines which server will process a request determined by the size of the requested file
Splits the content on each server by file size, forcing the queues sizes to be consistent.

20. EquiLoad Solves Queue Length Problems
This is bad
Queue
This is better 1k 2k
100k 1k
1000k 1k 2k
100k

21. EquiLoad Diagram Dispatcher Distributor S1 S2 Requests 1k-2k 2k-3k
(periodically calculates partitions) S1 S2
Requests
1k-2k
2k-3k
Forwards
Request S3 S4
Distributor
3k-10k
10k-20k
To Client Browser S5 S6
20k-100k
>100k

22. EquiLoad Advantages Dynamic repartitioning
Can be implemented at various levels
DNS
Dispatcher
Server
Minimum queue buildup
Performs well under variable workload and high system load

23. EquiLoad Disadvantages Cache affinity is neglected
Requires a front end dispatcher
Distributor must communicate with servers
Thresholds of parameter adjustment

24. EquiLoad  AdaptLoad
AdaptLoad improves upon EquiLoad using “fuzzy boundaries”
Allows for multiple servers to process a request
Behaves better in situations where server partitions are very close in size

25. AdaptLoad Diagram Distributor Dispatcher S1 S2 Requests 1k-3k 2k-4k
(periodically calculates partitions) S1 S2
Requests
1k-3k
2k-4k
Forwards
Request S3 S4
Dispatcher
3k-10k
8k-20k
To Client Browser S5 S6
15k-100k
>80k

26. AdaptLoad Results

27. Summary File Distribution Routing Content/Locality Aware DNS Server
Size Aware
Centralized Router
Workload Aware
Distributed Dispatcher
FLEX
EquiLoad, AdaptLoad
WARD

28. Conclusions
There is no “best” way to distribute content among servers.
There is no optimal policy for all website applications.
Certain strategies are geared towards a particular website application.

29. Future Work Compare and contrast the three policies
Figure out how often nodes should be repartitioned
Compare each policy to a standard benchmark
Figure out which policy works in a particular environment

30. Questions?
Anyone have one?