1. Copyright © 2012, MBL@CS.NCTU An SLA-aware load balancing scheme for cloud datacenters 指導教授：王國禎 學生：黎中誠 國立交通大學資訊工程系 行動計算與寬頻網路實驗室 1

2. Copyright © 2012, MBL@CS.NCTU Outlines Introduction Related work Proposed two-level decentralized load balancer architecture Proposed SLA-aware load balancing scheme Experimental environment Experimental results Conclusion Future work References 2

3. Copyright © 2012, MBL@CS.NCTU Introduction Cloud computing is the delivery of computing as a service rather than a product, whereby shared resources, software, and information are provided to computers and other devices as a utility (like the electricity grid) over a network (typically the Internet) [1] The cloud scales: Amazon EC2 growth [2] 3

4. Copyright © 2012, MBL@CS.NCTU Introduction (cont.) The load of a cloud computing system is highly dynamic Different users may require different services, and it may lead to load unbalance between the virtual machines (VMs) in a datacenter User requests are sending to a load balancer and the load balancer then forward them to the appropriate VMs for processing in cloud datacenters 4

5. Copyright © 2012, MBL@CS.NCTU Introduction (cont.) A service-level agreement (SLA) is a part of a service contract where the level of service is formally defined [3] The SLA will typically have a technical definition in terms of response time, throughput, or similar measurable details [3] 5

6. Copyright © 2012, MBL@CS.NCTU Introduction (cont.) The following figure illustrates a classic load balancing architecture in a cloud computing environment [4] 6

7. Copyright © 2012, MBL@CS.NCTU Introduction (cont.) We focus on two issues –Load balancing control [5] Centralized or decentralized –Three-level centralized load balancer [6] –Two-level decentralized load balancer [5] –Two-level decentralized SLA-aware load balancer (tldlb, proposed) –Load balancing algorithm [5] Static or dynamic –Weighted round-robin (wrr) [4] –Dynamic weighted round-robin (dwrr, proposed) 7

8. Copyright © 2012, MBL@CS.NCTU Related work Wang et al. proposed a three-level load balancer architecture [6], as shown below –The first level has a Request Manager that is used to assign a task to a suitable Service Manager –The second level includes several Service Managers that are used to divide tasks into some logical independent subtasks –The third level includes Service Nodes that are used to execute subtasks 8

9. Copyright © 2012, MBL@CS.NCTU Related work Three-level centralized load balancer architecture [6] 9

10. Copyright © 2012, MBL@CS.NCTU Related work (cont.) Rajavel proposed a two-level decentralized load balancer [5] –The Meta-Scheduler are connected to each other by network connections and each Meta-Scheduler in turn is connected to a group of clusters (resources) 10 Structure of decentralized load balancer [5]

11. Copyright © 2012, MBL@CS.NCTU Related work (cont.) The structure of Meta-Scheduler works as follows: –User submits their jobs to a Meta-Scheduler, and the jobs are stored in the queue of a request handler –Dispatch Manager obtains the submitted job periodically from the queue 11 Structure of Meta-Scheduler [5]

12. Copyright © 2012, MBL@CS.NCTU Related work (cont.) –Load Balancer will perform the load balancing by exploiting the information gathered from Load Monitor and Information Manager –Information Manager will query Load Monitor and sends the host load information to the Load Balancer –Transfer Manager gives permission rights for the execution of a given job to a remote host –Execution Manager will keep updating the job status to the Dispatch Manager 12

13. Copyright © 2012, MBL@CS.NCTU Related work (cont.) Qualitative comparison of different load balancing architectures 13 Load balancing approach Wang [6] Rajavel [5]tldlb (proposed) ArchitectureCentralizedDecentralized ScalabilityLowHigh Spare VM pool No Yes

14. Copyright © 2012, MBL@CS.NCTU Related work (cont.) The weighted round-robin (wrr) scheduling algorithm will assign a fixed weight to each VM, an integer value that indicates the VM’s processing capacity [4] The capacity based scheduling algorithm [7] monitors the resources of each VM and distributes more requests to the VM which has more remaining resources Artificial neural network (ANN) based load predictor has the lowest prediction errors [8], and the proposed dwrr (Dynamic weighted round-robin) combine ANN and SLA requirements together to avoid SLA violation 14

15. Copyright © 2012, MBL@CS.NCTU Related work (cont.) Qualitative comparison of different load balancing algorithm 15 Load balancing algorithm wrr [4] Capacity based [7] dwrr (proposed) Static or dynamic StaticDynamic Load predictionNo Yes SLA-awareNo Yes

16. Copyright © 2012, MBL@CS.NCTU Proposed two-level decentralized load balancer architecture We propose a new architecture, called SLA-aware two-level decentralized load balancer (tldlb), to support dynamic load balancing in cloud data center There are two levels in our load balancer design: global load balancer and local load balancer Each global load balancer connected to a SLA- aware local load balancer that forms a virtual zone 16

17. Copyright © 2012, MBL@CS.NCTU Proposed two-level decentralized load balancer architecture (cont.) The proposed load balancer architecture is shown as follows: 17

18. Copyright © 2012, MBL@CS.NCTU Proposed two-level decentralized load balancer architecture (cont.) Local load balancer –Monitoring local VMs load which are in the same virtual zone –Choosing a proper VM to handle a request by Request Scheduler Global load balancer –Global balancers are connected to each other via P2P connections –When the responsible zone of a global load balancer is overloaded, it will select another virtual zone to share the load by forwarding requests to the zone 18

19. Copyright © 2012, MBL@CS.NCTU Proposed two-level decentralized load balancer architecture (cont.) The local load balancer architecture is shown as follows: 19

20. Copyright © 2012, MBL@CS.NCTU Proposed two-level decentralized load balancer architecture (cont.) Request Handler –This module receives user requests and forward them to the Request Scheduler module Request Scheduler –This module assigns user requests from Request Handler to correspond VMs based on the weights from the Weight Adjustment module Load Monitor –It will monitor current four load metrics (CPU, MEM, NET, Disk I/O utilization) of each VM 20

21. Copyright © 2012, MBL@CS.NCTU Proposed two-level decentralized load balancer architecture (cont.) History Storage –The load data collected by Load Monitor will be stored in this module –These load data can support the Load Prediction to predict the load at next time slot Load Prediction –This module uses load history of each VM, which is stored in the History Storage module to predict the future load of each VM –The load prediction results (NI i ) will send to Weight Adjustment module 21

22. Copyright © 2012, MBL@CS.NCTU Proposed two-level decentralized load balancer architecture (cont.) SLA Engine –This module will record the response time of each request and check of the request satisfies its SLA requirements Weight Adjustment –This module will adjust the weight of each virtual machine according to the capacity index (CI i ) from Load Monitor and the neural index (NI i ) from Load Prediction 22

23. Copyright © 2012, MBL@CS.NCTU Proposed two-level decentralized load balancer architecture (cont.) Spare VM pool –There are some suspended VMs in the spare VM pool –When active VMs can’t handle incoming requests to meet the SLA requirements, Request Handler will wake up some VMs to service requests 23

24. Copyright © 2012, MBL@CS.NCTU Proposed SLA-aware load balancing scheme We propose a novel load balancing algorithm, called dwrr (dynamic weighted round-robin), to dispatch requests to different VMs We give each active VM a weight according to the capacity index (CI i ) from Load Monitor and the neural index (NI i ) from Load Prediction The Request Scheduler module distributes the requests to active VMs by these weights 24

25. Copyright © 2012, MBL@CS.NCTU Proposed SLA-aware load balancing scheme (cont.) Capacity index (CI) –The critical resource is different when cloud datacenter provides varies services –The load Monitor module collects four load metrics, utilizations of CPU, memory, network bandwidth, and disk I/O 25

26. Copyright © 2012, MBL@CS.NCTU Proposed SLA-aware load balancing scheme (cont.) Neural index (NI) –Artificial Neural Network (ANN) has the ability to do optimization and prediction –We use the delta learning rule in our ANN design –In our ANN design, we consider avoiding SLA violations, such as the response time violation 26

27. Copyright © 2012, MBL@CS.NCTU Proposed SLA-aware load balancing scheme (cont.) We use following formula to determine the weight of each VM 27

28. Copyright © 2012, MBL@CS.NCTU Proposed SLA-aware load balancing scheme (cont.) 28 –The following figure shows schematic representation of the artificial neural network model for deriving the neural index

29. Copyright © 2012, MBL@CS.NCTU Proposed SLA-aware load balancing scheme (cont.) – We train the weights in the hidden layer via delta learning rule 29

30. Copyright © 2012, MBL@CS.NCTU Proposed SLA-aware load balancing scheme (cont.) 30

31. Copyright © 2012, MBL@CS.NCTU Proposed SLA-aware load balancing scheme (cont.) Capacity index Neural index Weight 31

32. Copyright © 2012, MBL@CS.NCTU Proposed SLA-aware load balancing scheme (cont.) 32

33. Copyright © 2012, MBL@CS.NCTU Experimental environment We built a small test bed that include three active VMs (VM 1, VM 2, VM 3 ) with different capability and two spare VM (VMs 1, VMs 2 ), which running in an apache web server in a virtual zone for our experiment Requests are based on a real web service We use the modified apache benchmark (ab) to collect real web traffic 33

34. Copyright © 2012, MBL@CS.NCTU Experimental environment (cont.) We set up an experimental environment as shown below 34

35. Copyright © 2012, MBL@CS.NCTU Experimental environment (cont.) OSCentOS 5.5 Virtual machine hypervisorXen Number of VMs3 Number of spare VMs2 ApplicationWeb service Duration (time limit)60 sec Response time specified in the SLA2000, 1000, 432 ms Pre-reaction rate80% Transfer functionLog-sigmoid Learning rate0.5 35 Our experimental environment setup and related parameters are shown in the following table

36. Copyright © 2012, MBL@CS.NCTU Experimental environment (cont.) VM 1 VM 2 VM 3 VM s1 VM s2 CPU (core)12322 Memory (MB)512102420481024 Virtual disk (GB)10 Weight of wrr124-- 36 VM configuration

37. Copyright © 2012, MBL@CS.NCTU Experimental results 37 We compare four scheduling algorithms –Response time specified in the SLA: 2000 ms

38. Copyright © 2012, MBL@CS.NCTU Experimental results (cont.) 38 The proposed dwrr is 1.86 times faster than wrr, 1.49 times faster than capacity based, and 1.21 times faster than ANN scheduling algorithms in terms of average response time –Response time specified in the SLA: 2000 ms

39. Copyright © 2012, MBL@CS.NCTU Experimental results (cont.) 39 We compare four scheduling algorithms –Response time specified in the SLA: 1000 ms

40. Copyright © 2012, MBL@CS.NCTU Experimental results (cont.) 40 We compare four scheduling algorithms –Response time specified in the SLA: 432 ms

41. Copyright © 2012, MBL@CS.NCTU Experimental results (cont.) 41 The proposed tldlb can reduce the SLA violation rate by activating VMs in the spare VM pool module in time

42. Copyright © 2012, MBL@CS.NCTU Conclusions We have presented SLA-aware decentralized load balancer architecture We combine of VM load metrics monitoring (CPU, memory, network, disk I/O utilization) and neural network to adjust the weight of each VM Experimental results support that dwrr is 1.86 times faster than wrr, 1.49 times faster than capacity based, and 1.21 times faster than ANN scheduling algorithms in terms of average response time in the limited resources 42

43. Copyright © 2012, MBL@CS.NCTU Conclusions (cont.) The experiment results support that our proposed dwrr algorithm has faster response time, which means we can handle more requests per second This load balancing algorithm can avoid SLA violations via activating VMs in the spare VM pool Since our scheduling algorithm is simple and efficient, it is well-suited for cloud computing environments to service a large number of requests with short response time 43

44. Copyright © 2012, MBL@CS.NCTU Future work Implementing the global balancer and do some experiments about user requests redirection performance Deploying our load balancer to the real cloud datacenter 44

45. Copyright © 2012, MBL@CS.NCTU References [1] Cloud computing – Wiki http://en.wikipedia.org/wiki/Cloud_computing.http://en.wikipedia.org/wiki/Cloud_computing [2] Average daily instance launch counts http://jackofallclouds.com/wp- content/uploads/2010/12/cloudkick_yearly_ec2_launch_counts.png.http://jackofallclouds.com/wp- content/uploads/2010/12/cloudkick_yearly_ec2_launch_counts.png [3] Service-level agreement – Wiki http://en.wikipedia.org/wiki/Service- level_agreement.http://en.wikipedia.org/wiki/Service- level_agreement [4] Linux Virtual Server http://www.linuxvirtualserver.org.http://www.linuxvirtualserver.org [5] R. Rajavel, “De-Centralized Load Balancing for the Computational Grid environment,” in Proceeding of International Conference on Communication and Computational Intelligence (INCOCCI), pp. 419-424, Dec. 2010. [6] S. C. Wang, K. Q. Yan, W. P. Liao, and S. S. Wang, “Towards a Load Balancing in a three-level cloud computing network,” in Proceeding of IEEE International Conference on Computer Science and Information Technology (ICCSIT), vol. 1, pp. 108 - 113, Jul. 2010. [7] C. C. Li, and K. C. Wang, “SLA-aware Load Balancing for Cloud Data Centers,” report, 2012 [8] V. Nae, A. Iosup, R. Prodan, “Dynamic Resource Provisioning in Massively Multiplayer Online Games,” IEEE Transactions on Parallel and Distributed Systems, vol. 22, no. 3, pp. 380-395, Mar. 2011. 45