1. Load Balancing in Distributed Systems Nalini Venkatasubramanian nalini@ics.uci.edu

2. Global Distributed Systems and Multimedia2 Motivation zA given computer is overloaded yMust decrease load and maintain certain characteristics xScalability xPerformance Throughput yIdeally, this should be done transparently ySolution: Load Balancing

3. Global Distributed Systems and Multimedia3 Introduction to Load Balancing zDistributed resource allocation yCan be thought of as “distributed scheduling” xDeals with distribution of processes among processors connected by a network xHandles issues such as deciding which process should be handled by a given processor yCan also be influenced by “distributed placement” xImportant in data intensive environments and applications xData placement may force process placement

4. Global Distributed Systems and Multimedia4 Load Balancing Relationships zLoad Balancer yManages resources yResource assignment depends on policy or policies in effect zClient yRequests resources yRequests services

5. Global Distributed Systems and Multimedia5 Load Balancing Overhead zSatisfy client resource access without imposing large amounts of overhead yPerformance xHow well resources are managed yEfficiency xCost of accessing and using a resource obtained through a load balancer

6. Global Distributed Systems and Multimedia6 Load Balancing Issues zWhen to migrate processes or forward requests zWhich processor should be chosen to handle a given process or request zShould processes be moved off a computer zHow should searching for lightly loaded computer be performed

7. Global Distributed Systems and Multimedia7 Load Balancing Issues (cont.) zWhen should load balancing decisions be made zWhat should be taken into account when making the above decisions zHow should old data be handled zShould load balancing data be stored centrally, or in a distributed manner

8. Global Distributed Systems and Multimedia8 Load Balancing Issues (cont.) zShould computers make decisions together zWhat is the performance/overhead tradeoff incurred by load balancing zPrevention of overloading a lightly loaded computer

9. Global Distributed Systems and Multimedia9 Load Balancing Techniques zBasically two ways to perform load balancing yStatically xResource is allocated once yDynamically xResource is allocated and managed (possibly dynamically reallocated) to ensure balanced load

10. Global Distributed Systems and Multimedia10 Static Load Balancing zResource allocation is performed once yOnce resource is allocated it remains allocated (for what duration??) zScheduling decisions are made yDeterministically yProbabilistically

11. Global Distributed Systems and Multimedia11 Static Load Balancing – cont’d zAdvantages yState generally need not be stored xSimplifies implementation yLess network traffic due to load balancing related messages zDisadvantages yPoor resource utilization xA given resource may be used much more than others yDoes not adjust to fluctuations in the load xPossible for resource to become overloaded

12. Global Distributed Systems and Multimedia12 Static Load Balancing – cont’d zExample of Static Load Balancing yForwarding processes/requests to a given computer based on dynamically assigned addresses (e.g. via DNS) xWeb servers (e.g. CNN) xUCI host ea.uci.edu is load balanced

13. Global Distributed Systems and Multimedia13 Dynamic Load Balancing zAttempts to maintain a balanced load by managing resources while a resource is in use yMay involve the following xProcess migration xDisabling further access to a resource until a later time xAdding new resources “on-the-fly”

14. Global Distributed Systems and Multimedia14 Dynamic Load Balancing Strategies zDistributed versus non-distributed yShould load information be stored centrally or across several hosts xSimplicity versus overhead and reliability zCooperative versus non-cooperative yShould decisions be made by a single load balancer or several xGlobally managed resources versus locally managed resources

15. Global Distributed Systems and Multimedia15 Dynamic Load Balancing Strategies - cont’d zAdaptive versus non-adaptive yShould previous data effect scheduling decisions zPreemptive versus non-preemptive yShould a running process be preempted in favor of another process, or for migration to another resource

16. Global Distributed Systems and Multimedia16 Case Studies zLoad Balancing for Web Servers zLoad Balancing for Parallel Computers zLoad Balancing for Multimedia Applications

17. Global Distributed Systems and Multimedia17 Multimedia Applications Electronic Commerce Entertainment Distance Learning Tele-medicine Video Servers Graphics Processing Web Servers Requirements - Availability, Reliability, Quality-of-Service, Cost-effectiveness, Security

18. Global Distributed Systems and Multimedia18 Multimedia Load Management zPrimary focus yresource optimization across streams and resource management across servers. xQuality of Service continuous delivery requirement minor violations of performance requirements xAdmission Control - resource reservation/negotiation xMedia Delivery - Resource scheduling (CPU,Disk) yResource Mgmt. Implies Admission Control xCaching, VCR Control, Server Selection, Data Placement

19. Global Distributed Systems and Multimedia19 Distributed MM Servers zVideo Server Topology xPartitioned Server xExternally Switched xFully Switched zVideo File Placement yHeterogeneous Workload - large/small, hot/cold yOnline Placement yGood placement is important xdynamic replication time-consuming, dynamic load-bal complementary

20. Global Distributed Systems and Multimedia20 Dynamic Load Balancing yAdapts to statistical fluctuations and changing access patterns yDynamic Migration xDeals with poor initial placement yReplication xDynamic Segment Replication partial replication (quick response, less expensive) xTotal Replication on-demand vs. predictive

21. Global Distributed Systems and Multimedia21 Load Management of Distributed MM Servers zAdaptive Scheduling xAssigns requests to servers based on demand and load factors. xInvokes replication-on-demand, request migration zPredictive Placement xInvokes dereplication zOptimizations xEager Replication xLazy Dereplication

22. Global Distributed Systems and Multimedia22 A Scalable Video Server Architecture Distribution Network Distribution Controller Distribution Controller Data Source Data Source Data Source Data Source Data Source Data Source Tertiery Storage Tertiery Storage... control requests data

23. Global Distributed Systems and Multimedia23 Architectural View of a Networked MM System Qos Broker and Load Management System Node Manager Node Manager Node Manager Node Manager... Local Data Streaming

24. Global Distributed Systems and Multimedia24 Resources in a Video Server Network Client Communication Modules Data Manipulation Modules Storage Modules Processing Module

25. Global Distributed Systems and Multimedia25 Load Placement Scenario Data Source S2 Data Source S1 Access Network Clients... Storage: 8 objects Bandwidth: 3 requests Storage: 2 objects Bandwidth: 8 requests

26. Global Distributed Systems and Multimedia26 Characterizing Server Resource Usage yAbility to service a request on a server depends on: xresource available xcharacteristics of a request yLoad factor(LF) for a request: xrepresents how far a server is from request admission threshold. LF (Ri, Sj) = max (Dbi/DBj, Mi/Mj, CPUi/CPUj, Xi/Xj)

27. Global Distributed Systems and Multimedia27 Adaptive Scheduling zWhen the broker receives a request Ri for a video object Vi : xConsider only data sources that have a copy of Vi. xConsider only data sources tha have sufficient resources to support Ri. xChooser server for which LF (Ri, Sj) is a minimum. xIf no such server exists Reject request. Perform replication-on-demand. Perform request migration.

28. Global Distributed Systems and Multimedia28 Predictive Data Placement zDetermines when, where and how many replicas of a video object. zInitiated periodically. zResults in an assignment of replicas to data sources. zGreedy algorithm that uses revenue generated as a metric.

29. Global Distributed Systems and Multimedia29 The Greedy Cost Placement Matrix PM(Vi, Sj) is the maximum revenue that can accrue from allocating Vi to Sj. Greedy heuristic: Map(Vi,Sj) = 1 if PM(Vi,Sj) =  a  b max(PM(Va,Sb))

30. Global Distributed Systems and Multimedia30 Optimizations zTo minimize the overhead of replication yEager replication xReplication of video object in anticipation xPerformed when server resources are free yLazy Dereplication xCritical nature of storage resources xMark reusable resources, reclaim disk space later xIf disk blocks are not overwritten, can be reclaimed

31. Global Distributed Systems and Multimedia31 Life of a video object

32. Global Distributed Systems and Multimedia32 Performance Evaluation Policies

33. Global Distributed Systems and Multimedia33 Performance Evaluation - Startup Latencies

34. Global Distributed Systems and Multimedia34 Performance of the basic configuration p1 p2 p4 p3

35. Global Distributed Systems and Multimedia35 Performance Evaluation - Varying Replication BW

36. Global Distributed Systems and Multimedia36 Performance Evaluation Summary zP1: entails high startup latency, requires high storage and replication bandwidth. zP2: Unacceptably poor performance. zP3: Similar performance to P4 in many cases. xAt low transfer bandwidths, P4 outperforms P3. zP4: Performs well in all cases.