1. Load Balancing in Distributed Systems
Nalini Venkatasubramanian

2. Global Distributed Systems and Multimedia
Motivation
A given computer is overloaded
Must decrease load and maintain certain characteristics
Scalability
Performance
Throughput
Ideally, this should be done transparently
Solution: Load Balancing
Global Distributed Systems and Multimedia

3. Introduction to Load Balancing
Distributed resource allocation
Can be thought of as “distributed scheduling”
Deals with distribution of processes among processors connected by a network
Handles issues such as deciding which process should be handled by a given processor
Can also be influenced by “distributed placement”
Important in data intensive environments and applications
Data placement may force process placement
Global Distributed Systems and Multimedia

4. Load Balancing Relationships
Load Balancer
Manages resources
Resource assignment depends on policy or policies in effect
Client
Requests resources
Requests services
Global Distributed Systems and Multimedia

5. Load Balancing Overhead
Satisfy client resource access without imposing large amounts of overhead
Performance
How well resources are managed
Efficiency
Cost of accessing and using a resource obtained through a load balancer
Global Distributed Systems and Multimedia

6. Global Distributed Systems and Multimedia
Load Balancing Issues
When to migrate processes or forward requests
Which processor should be chosen to handle a given process or request
Should processes be moved off a computer
How should searching for lightly loaded computer be performed
Global Distributed Systems and Multimedia

7. Load Balancing Issues (cont.)
When should load balancing decisions be made
What should be taken into account when making the above decisions
How should old data be handled
Should load balancing data be stored centrally, or in a distributed manner
Global Distributed Systems and Multimedia

8. Load Balancing Issues (cont.)
Should computers make decisions together
What is the performance/overhead tradeoff incurred by load balancing
Prevention of overloading a lightly loaded computer
Global Distributed Systems and Multimedia

9. Load Balancing Techniques
Basically two ways to perform load balancing
Statically
Resource is allocated once
Dynamically
Resource is allocated and managed (possibly dynamically reallocated) to ensure balanced load
Global Distributed Systems and Multimedia

10. Global Distributed Systems and Multimedia
Static Load Balancing
Resource allocation is performed once
Once resource is allocated it remains allocated (for what duration??)
Scheduling decisions are made
Deterministically
Probabilistically
Global Distributed Systems and Multimedia

11. Static Load Balancing – cont’d
Advantages
State generally need not be stored
Simplifies implementation
Less network traffic due to load balancing related messages
Disadvantages
Poor resource utilization
A given resource may be used much more than others
Does not adjust to fluctuations in the load
Possible for resource to become overloaded
Global Distributed Systems and Multimedia

12. Static Load Balancing – cont’d
Example of Static Load Balancing
Forwarding processes/requests to a given computer based on dynamically assigned addresses (e.g. via DNS)
Web servers (e.g. CNN)
UCI host ea.uci.edu is load balanced
Global Distributed Systems and Multimedia

13. Dynamic Load Balancing
Attempts to maintain a balanced load by managing resources while a resource is in use
May involve the following
Process migration
Disabling further access to a resource until a later time
Adding new resources “on-the-fly”
Global Distributed Systems and Multimedia

14. Dynamic Load Balancing Strategies
Distributed versus non-distributed
Should load information be stored centrally or across several hosts
Simplicity versus overhead and reliability
Cooperative versus non-cooperative
Should decisions be made by a single load balancer or several
Globally managed resources versus locally managed resources
Global Distributed Systems and Multimedia

15. Dynamic Load Balancing Strategies - cont’d
Adaptive versus non-adaptive
Should previous data effect scheduling decisions
Preemptive versus non-preemptive
Should a running process be preempted in favor of another process, or for migration to another resource
Global Distributed Systems and Multimedia

16. Global Distributed Systems and Multimedia
Case Studies
Load Balancing for Web Servers
Load Balancing for Parallel Computers
Load Balancing for Multimedia Applications
Global Distributed Systems and Multimedia

17. Multimedia Applications
Electronic
Commerce
Video
Servers
                                                
Global
Entertainment
Network
Web Servers
Distance Learning
Graphics Processing
Tele-medicine
Global Distributed Systems and Multimedia
Requirements - Availability, Reliability, Quality-of-Service, Cost-effectiveness, Security

18. Multimedia Load Management
Primary focus
resource optimization across streams and resource management across servers.
Quality of Service
continuous delivery requirement
minor violations of performance requirements
Admission Control - resource reservation/negotiation
Media Delivery - Resource scheduling (CPU,Disk)
Resource Mgmt. Implies Admission Control
Caching, VCR Control, Server Selection, Data Placement
Global Distributed Systems and Multimedia

19. Distributed MM Servers
Video Server Topology
Partitioned Server
Externally Switched
Fully Switched
Video File Placement
Heterogeneous Workload - large/small, hot/cold
Online Placement
Good placement is important
dynamic replication time-consuming, dynamic load-bal complementary
Global Distributed Systems and Multimedia

20. Dynamic Load Balancing
Adapts to statistical fluctuations and changing access patterns
Dynamic Migration
Deals with poor initial placement
Replication
Dynamic Segment Replication
partial replication (quick response, less expensive)
Total Replication
on-demand vs. predictive
Global Distributed Systems and Multimedia

21. Load Management of Distributed MM Servers
Adaptive Scheduling
Assigns requests to servers based on demand and load factors.
Invokes replication-on-demand, request migration
Predictive Placement
Invokes dereplication
Optimizations
Eager Replication
Lazy Dereplication
Global Distributed Systems and Multimedia

22. A Scalable Video Server Architecture
Distribution Network
requests
data
Distribution
Controller
Data
Source
Data
Source
Data
Source
Tertiery
Storage
...
control
Global Distributed Systems and Multimedia

23. Architectural View of a Networked MM System
Qos Broker and Load Management System
Node
Manager
Node
Manager
Node
Manager
Node
Manager
...
Local Data Streaming
Global Distributed Systems and Multimedia

24. Resources in a Video Server
Client
Client
Network
Processing
Module
Communication
Modules
Data Manipulation
Modules
Storage
Modules
Global Distributed Systems and Multimedia

25. Load Placement Scenario
Data
Source S2
Data
Source S1
Storage:
8 objects
Bandwidth:
3 requests
Storage:
2 objects
Bandwidth:
8 requests
Access Network
...
Clients
Global Distributed Systems and Multimedia

26. Characterizing Server Resource Usage
Ability to service a request on a server depends on:
resource available
characteristics of a request
Load factor(LF) for a request:
represents how far a server is from request admission threshold.
LF (Ri, Sj) = max (Dbi/DBj , Mi/Mj , CPUi/CPUj , Xi/Xj)
Global Distributed Systems and Multimedia

27. Global Distributed Systems and Multimedia
Adaptive Scheduling
When the broker receives a request Ri for a video object Vi :
Consider only data sources that have a copy of Vi.
Consider only data sources tha have sufficient resources to support Ri.
Chooser server for which LF (Ri, Sj) is a minimum.
If no such server exists
Reject request.
Perform replication-on-demand.
Perform request migration.
Global Distributed Systems and Multimedia

28. Predictive Data Placement
Determines when, where and how many replicas of a video object.
Initiated periodically.
Results in an assignment of replicas to data sources.
Greedy algorithm that uses revenue generated as a metric.
Global Distributed Systems and Multimedia

29. The Greedy Cost Placement Matrix
PM(Vi, Sj) is the maximum revenue that can accrue from allocating Vi to Sj.
Greedy heuristic: Map(Vi,Sj) = 1 if PM(Vi,Sj) = a b max(PM(Va,Sb))
Global Distributed Systems and Multimedia

30. Global Distributed Systems and Multimedia
Optimizations
To minimize the overhead of replication
Eager replication
Replication of video object in anticipation
Performed when server resources are free
Lazy Dereplication
Critical nature of storage resources
Mark reusable resources, reclaim disk space later
If disk blocks are not overwritten, can be reclaimed
Global Distributed Systems and Multimedia

31. Global Distributed Systems and Multimedia
Life of a video object
Global Distributed Systems and Multimedia

32. Performance Evaluation Policies
Global Distributed Systems and Multimedia

33. Performance Evaluation - Startup Latencies
Global Distributed Systems and Multimedia

34. Performance of the basic configuration
Global Distributed Systems and Multimedia

35. Performance Evaluation - Varying Replication BW
Global Distributed Systems and Multimedia

36. Performance Evaluation Summary
P1: entails high startup latency, requires high storage and replication bandwidth.
P2: Unacceptably poor performance.
P3: Similar performance to P4 in many cases.
At low transfer bandwidths, P4 outperforms P3.
P4: Performs well in all cases.
Global Distributed Systems and Multimedia