1. Dynamic Load Balancing in Distributed Content-based Publish/Subscribe Alex K. Y. Cheung & Hans-Arno Jacobsen University of Toronto November 30 th, 2006 Middleware 2006 MIDDLEWARE SYSTEMS RESEARCH GROUP

2. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 2 Overview Introduction  Background  Motivation Contributions  Load Balancing Framework  Load Estimation Experiments  Setup Conclusions  Future Work  Goals  Related Work  3 Offload Algorithms  Results

3. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 3 Background on Distributed Pub/Sub subscriber stock = ‘YHOO’ price > $35.00 subscriber stock = ‘YHOO’ price > $30.00 publisher stock = ‘YHOO’ price = $40.00 publisher stock = ‘GM’ price = $20.00 broker multicast

4. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 4 Motivation A distributed content-based publish/subscribe system offers: 1.Loosely coupled data sources and sinks 2.Flexible routing based on message content 3.Comprised of a distributed broker overlay network Main focus: online games, decentralized workflow execution, and real-time monitoring systems Real-world usage patterns are never evenly distributed:  Performance degradation  Instability and unavailability  Under-utilized resources: Reduced scalability Poor investment in resources

5. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 5 Research Goal Develop a load balancing algorithm for distributed content-based pub/sub having these strong properties: 1.Dynamic – load balancing can be invoked anytime when overload or uneven load distribution is detected 2.Adaptive – a distinct offload algorithm designed to load balance on each type of resource 3.Heterogeneous – account for brokers with different resource capacities and subscriptions subscribing to different publication sets 4.Distributed – scalability of load balancing algorithm and the pub/sub infrastructure 5.Transparent – publishers and subscribers have no idea about load balancing

6. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 6 Related Work Compared to Tersptra et al. [5], Meghdoot [2], and Opportunistic Overlays [1]:  Applied to dedicated broker pub/sub architecture  Accounts for heterogeneity  Unlike load sharing, the proposed solution is a dynamic load balancing algorithm that can distribute load in the absence of new node joins  Load estimation prevents overloading load-accepting brokers in load balancing  Balances 3 performance metrics simultaneously on a best-effort approach using adaptive offload algorithms input utilization ratio, matching delay and output utilization ratio  Load balancing is done by migrating subscribers

7. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 7 Overview Introduction Contributions  Load Balancing Framework  Load Estimation  3 Offload Algorithms Experiments Conclusions

8. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 8 Load Balancing Framework Pub Sub PEER (Padres Efficient Event Routing) clusters Promotes 2 levels of load balancing: 1.Local-level 2.Global-level cluster-head brokers edge-brokers

9. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 9 Sub Load Balancing Framework Pub

10. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 10 Sub Load Balancing Framework Local Load Balancing Global Load Balancing Pub offloading broker load-accepting broker

11. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 11 Load Estimation Padres Real-time Event-to-Subscription Spectrum (PRESS)  Bit vector approach to record, compare, and estimate subscription load reduced/gained at each broker How much additional traffic does the subscription [class,=,*] introduce onto a broker already subscribed to [class,=,’SPORTS’] ?

12. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 12 [class,=,’STOCK’], [value,>,40] [class,=,’STOCK’], [value,>,45] [class,=,’SPORTS’] [class,=,*] Load Estimation with PRESS SubscriptionsPRESS bit vector 1. [class,’STOCK’], [value,60] 2. [class,’SPORTS’] 3. [class,’STOCK’], [value,99] 4. [class,’SPORTS’], [year,04] 5. [class,’STOCK’], [value,42] Publication arrival order: 11011101 10011001 00110011 11011101 00110011 00000000 00000000 00000000 00000000 00000000 To answer the question: “How much additional traffic does the subscription [class,=,*] introduce onto another broker already subscribing to [class,=,’SPORTS’] ? Answer: 3/5 of the measured total incoming traffic Assuming an incoming publication rate of 5 msg/s, then the additional publication rate is: (5 msg/s) x (3/5) = 3 msg/s

13. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 13 Offload Algorithms Input Offload Algorithm Load balances on the input utilization ratio: input utilization ratio = incoming pub rate / max match rate Offload subscriptions that reduce the incoming publication rate of the broker Increase matching rate by reducing the matching delay through the match offload algorithm only if the broker is overloaded input queue Incoming pub/sub messages Matching Engine & Routing Tables … Load Balancer output queues Outgoing pub/sub messages PADRES broker

14. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 14 Offload Algorithms Match Offload Algorithm Load balances on the matching delay Offload subscriptions with the lowest traffic input queue Incoming pub/sub messages Matching Engine & Routing Tables … Load Balancer output queues Outgoing pub/sub messages PADRES broker

15. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 15 Offload Algorithms Output Offload Algorithm Load balances on the output utilization ratio output utilization ratio = output bandwidth used / total output bandwidth Offload subscriptions with the highest traffic possible. Favor subscriptions that do not increase the incoming publication rate onto the load- accepting broker input queue Incoming pub/sub messages Matching Engine & Routing Tables … Load Balancer output queues Outgoing pub/sub messages PADRES broker

16. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 16 Load Balancing Components DETECTOR PIE MEDIATOR PIE messages OFFLOAD ALGORITHMS InputMatchOutput LOAD ESTIMATION PRESS Mediation and migration protocols Subscribers to offload Target broker Load Balancer Establish and teardown load balancing sessions between two brokers/clusters Coordinate transparent subscriber migrations Calculates the set of subscribers to offload for balancing the performance metric in question based on load information about the subscriptions and load-accepting broker Padres Real-time Event-to- Subscription Spectrum (PRESS) uses a bit vector approach to estimate load of subscriptions Triggers load balancing if overload or uneven load distribution is detected by examining the 3 performance metrics Padres Information Exchange (PIE) for exchanging load information with other brokers

17. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 17 Overview Introduction Contributions Experiments  Setup  Results Conclusions

18. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 18 Experiments Integrated the load balancing algorithm into PADRES [3] with 20,000 LOC Results obtained through emulation  Bandwidth, queuing, and matching delays Dataset consists of real stock quote publications obtained from Yahoo! Finance [6]  [class,'STOCK'],[symbol,'YHOO'],[open,30.25],[high,31.75],[low,30.00],[close,31.62], [volume,203400],[date,'1-May-96'] Subscription distribution: 0.20 = [class,eq,'STOCK'],[symbol,eq,'REPLACE_SYMBOL'],[high,>,REPLACE_HIGH] 0.20 = [class,eq,'STOCK'],[symbol,eq,'REPLACE_SYMBOL'],[low,<,REPLACE_LOW] 0.20 = [class,eq,'STOCK'],[symbol,eq,'REPLACE_SYMBOL'],[volume,>,REPLACE_VOLUME] 0.34 = [class,eq,'STOCK'],[symbol,eq,'REPLACE_SYMBOL'] 0.05 = [class,eq,'STOCK'],[volume,>,REPLACE_VOLUME] 0.01 = [class,eq,'STOCK']

19. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 19 Macro Experiment Setup – Local LB B3B2 B1B4 B0 Pub 40 unique publishers Sub 2000 subscribers, 20% sink zero traffic Broker ID CPU (MHz) Memory (MB) Bandwidth (Mbps) B020003210 B1100160.5 B2200161 B3400322 B41000325

20. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 20 Local LB – Input Utilization Ratio new subscribers join converged 50% of the publishers increased their publication rate by 100%

21. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 21 Local LB – Matching Delay converged 50% of the publishers increased their publication rate by 100% new subscribers join

22. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 22 Local LB – Output Utilization Ratio

23. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 23 Local LB – Subscriber Distribution

24. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 24 Local LB – Client Perceived Delivery Delay

25. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 25 Local LB – Input Utilization Ratio Estimation Accuracy

26. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 26 Local LB – Press Samples on Input Utilization Ratio Estimation Accuracy

27. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 27 Scalability on Number of Edge Brokers

28. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 28 Scalability on Number of Clusters

29. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 29 Summary of Other Experimental Results Load balancing effectiveness is independent of:  Number of subscribers  Distribution of zero-traffic subscribers  Number of brokers/clusters Load balancing on the input and output utilization ratios also balances their queuing delay counterparts Decreasing the detection threshold can reduce load difference between brokers at the expense of higher overhead Increasing the PIE publication frequency can decrease the convergence time in local load balancing at the expense of higher overhead

30. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 30 Conclusions Load balancing solution for distributed content- based pub/sub that is dynamic, adaptive, distributed, transparent, and accommodates for heterogeneity Consisting of 3 key contributions:  Load balancing framework  Load estimation methodologies  3 offload algorithms Promote scalability by evenly distributing load onto the available resources

31. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 31 Future Work Evaluation on a real-world test bed, such as PlanetLab [4] Support load balancing for cluster-head brokers Integrate load balancing with other on-going research projects in PADRES, such as composite subscriptions, historical data access, and failure recovery Self-tuning capability

32. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 32 References [1] Y. Chen and K. Schwan, “Opportunistic overlays: Efficient content delivery in mobile ad hoc networks,” in ACM Middleware ’05 Grenoble, 2005. [2] A. Gupta, O. D. Sahin, D. Agrawal, and A. E. Abbadi, “Meghdoot: content- based publish/subscribe over p2p networks,” in Proceedings of the 5 th ACM/IFIP/USENIX international conference on Middleware. New York, NY, USA: Springer-Verlag New York, Inc., 2004, pp. 254-273. [3] G. Li and H.-A. Jacobsen, “Composite subscriptions in content-based publish/subscribe systems,” in ACM Middleware ’05 Grenoble, 2005. [4] PlanetLab. [Online]. Available: http://www.planet-lab.org/ [5] W. W. Terpstra, S. Behnel, L. Fiege, A. Zeidler, and A. P. Buchmann, “A peer-to-peer approach to content-based publish/subscribe,” in DEBS ’03: Proceedings of the 2 nd international workshop on distributed event-based systems. New York, NY, USA: ACM Press, 2003, pp. 1-8. [6] Yahoo! Finance. [Online]. Available: http://finance.yahoo.com

33. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 33 The End Thank you

34. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 34 Offload Strategies Poset [Carzaniga et al.01] [class,eq,*] [class,eq,’stock’][class,eq,’jobs’] [class,eq,’stock’], [name,eq,’cyber’] [class,eq,’stock’], [value,>,30] [class,eq,’jobs’], [type,eq,’engineering’]

35. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 35 Handoff 1.Tell selected subscribers to connect to new broker 2.Subscribers issue subscription 3.Broker 2 counts down from 10 x (n-1) seconds after receiving first subscription before assuming migration is complete 2 1 3 = subscriber [class,’SUB_CONTROL’], [command,’MIGRATE’], [IP,123.45.67.89], [port,1234] payload [offloadID,’e8sU27’], [population,2]

36. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 36 Handoff 2 1 3 7.Broker 2 updates its timer. 8.Broker 2 sends OK message to broker 1 when t = 0 or all subscribers are connected 9.Broker 1 tells all remaining migrating subscribers to unsubscribe from broker 1 after its processing delay 10.Subscribers unsubscribe from broker 1 Done! = subscriber OK unsubscribe

37. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 37 Hand-off Goal:  Achieve 100% delivery ratio in migration process Solution:  Subscribers issue subscription to remote broker  Remote broker sends to original broker that it received all subscribers (guarantees subscriptions are processed by the cluster-head already)  Original broker tells the subscribers to disconnect from it after a delay: delay = (input queuing delay) + (output queuing delay) + (matching delay) To ensure publications still in waiting in the queues get delivered to the migrated subscriber

38. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 38 Local LB – Output Utilization Ratio Estimation Accuracy

39. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 39 Local LB – Overhead

40. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 40 Local LB – Press Samples on Output Utilization Ratio Estimation Accuracy

41. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 41 Macro Experiment Setup – Global LB B12 B11 B10 B22 B21 B20 B32 B31 B30 B42 B41 B40 Pub 40 Sub 2000

42. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 42 Global LB – Input Utilization Ratio

43. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 43 Global LB – Matching Delay

44. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 44 Global LB – Output Utilization Ratio

45. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 45 Global LB – Subscriber Distribution

46. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 46 Global LB – Overhead

47. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 47 Related Work Tersptra et al. [1]  Load sharing scheme for distributed hash table (DHT) based peer-to-peer (P2P) pub/sub involving a unique dissemination tree from each node Meghdoot [2]  Load sharing technique for DHT based P2P pub/sub  Strategically have new incoming nodes to share the matching or bandwidth load of existing node Opportunistic Overlays [3]  Load sharing as a secondary requirement to optimizing delivery path in a mobile P2P publish/subscribe environment

48. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 48 Load Balancing Framework = Local PIE Message

49. MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware 2006 49 Load Balancing Framework = Global PIE Message