1. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Hans-Arno Jacobsen June 23, 2011 Resource Allocation Algorithms for Publish/Subscribe Systems http://padres.msrg.org Joint work with Alex King Yeung Cheung

2. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Green Resource Allocation Algorithms for Publish/Subscribe Systems http://padres.msrg.org

3. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Publish/Subscribe in Practice GooPS ▫ Google’s internal pub/sub messaging middleware to integrate applications across data centers ▫ Hundreds of brokers with tens of thousands of pub/sub clients Yahoo Message Broker ▫ Yahoo’s pub/sub middleware ▫ Used for example in PNUTS key/value-store (cf. VLDB’08) SuperMontage ▫ Tibco’s pub/sub distribution network for NASDAQ’s quote and order-processing GDSN (Global Data Synchronization Network) ▫ A global pub/sub network that allows retailers and suppliers (i.e., Walmart, Target, Metro, etc.) to exchange timely and accurate supply chain data 3ICDCS 2011 (Distributed and brokered publish/subscribe)

4. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Problem 4 P P S S S S Input Brokers Subscribers Publishers P P P P P P S S S S Output S S S S S S S S P P P P P P P P Overload! Deployment strategy that uses the least number of brokers? ICDCS 2011

5. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Challenges Brokers have limited and heterogeneous resource capacities – Computational – I/O or bandwidth – Memory and storage Publishers publish at different message rates Subscribers have unique interests that sink zero or more publications from zero or more publishers 5ICDCS 2011

6. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Challenges When Scaling Up 6 S S S S S S S S P P P P P P P P How to connect the brokers to minimize traffic while avoiding overload? S S S S S S S S How to allocate subscribers to brokers? P P P P P P P P How to connect the publishers if subscribers sink traffic from >2 publishers? How to allocate subscribers to brokers? This is an NP-complete problem! ICDCS 2011

7. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Additional Requirements Minimize – Amount of processing – Amount of messages forwarded Work effectively under any workload distribution (defined or undefined) Readily adaptable to any pub/sub system by being language independent – Content-based (XPath, regex, ranged, SQL, composite subscriptions, etc.) – Topic-based pub/sub 7ICDCS 2011

8. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Summary of Our Approach Phase 1: Subscription profiling (& publisher) – Record publications delivered to each subscription Phase 2: Subscription to broker allocation – Allocate subscriptions to brokers depending on the load induced by each subscription Phase 3: Broker overlay construction – Construct and configure broker overlay Apply publisher re-allocation (GRAPE, cf. ICDCS’2010) ICDCS 20118 (A customizable framework )

9. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Phase 1: Subscription Profiling ICDCS 20119 000000000 Message ID of first index Start of bit vector 1 Publications delivered to subscription B34-M213 B34-M215 B34-M216 B34-M217 B34-M220 B34-M222 B34-M225 B34-M226 B34-M213 01010101010101 Profile of each subscription per advertisement maintained at the subscriber’s first broker Message ID Cardinality of bit vector approximates bandwidth requirement of subscription Used to compute “closeness” between any two subscriptions in the allocation phase based on clustering algorithm. E.g, closeness = |s i ∩ s j | Fixed vector size; shift left if next publication is out of bit vector range

10. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Phase 2: Subscription Allocation Algorithms MANUAL & AUTOMATIC as baseline – Tree with fanout of 2; random placement of clients (manual) – Random allocation (automatic) Fastest Broker First (FBF) – Assign subscriptions randomly to the next most powerful broker Bin Packing – Like FBF, but assigns the next highest traffic subscription PAIRWISE-N, PAIRWISE-K (Riabov et al. ICDCS’02) – Pairwise subscription clustering where the number of clusters is specified beforehand CRAM (Clustering with Resource Awareness and Minimization) – Dynamically determines the number of clusters – Utilizes a novel one-to-many clustering scheme – Evaluated with 4 different subscription closeness metrics, with one derived from Banavar et al. ICDCS '99 10ICDCS 2011

11. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Allocation with Bin Packing 11 S S S S S S S S S S S S ICDCS 2011

12. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Allocation Result (Bin Packing) 12 S S S S S S S S S S S S ICDCS 2011

13. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Allocation with CRAM 1.Find and cluster a pair of subscriptions having next highest non-zero “closeness” 2.Run BIN PACKING algorithm with new pairing 3.Allocation fails, if: – More brokers are allocated than without this pairing – Not all subscriptions can be allocated to brokers 4.On failure, undo and remember incompatible pairing 5.Repeat loop until no more pairings can be found Initially BIN PACKING is run to determine initial allocation Pairings found are combined and re-inserted in sub pool Final subscription clustering is last successful allocation ICDCS 201113 (Basic version)

14. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Summary of Optimizations Grouping of subscriptions with equal profiles – Apply CRAM an groups – In our experiments, reductions of up to 61% Limit closeness computations among groups – Exploit covering relationships among subscriptions – Disregard groups with small closeness – In our experiments, a 20x improvement, roughly One-to-many clustering – Cluster groups of subscriptions & covered subs ICDCS 201114

15. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Closeness Metrics ICDCS 201115 XOR: |s i XOR s j | -1 IOU:| s i ∩ s j | 2 / |s i U s j | IOS:|s i ∩ s j | 2 / |s i | + |s j | Intersect: |s i ∩ s j | Ideally, find subscriptions sharing highest overlap in traffic, while introducing least amount of non-overlapping traffic. XOR is derived from Banavar et al. ICDCS '99) Good for highest overlap Good for least non-overlapping traffic Good for both conditions, yield 0 for empty relationships, favour clustering higher traffic subs } (Intersection over sum & … over union) (If value is 0, defined as MAXVAL)

16. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Traditional One-to-One Clustering 16 Bit Vector of S 1 Bit Vector of S 2 C = 8 2 /(36+24) = 1.07 Closeness, C = |s i ∩ s j | 2 |s i | + |s j | C = 4 2 /(36+4) = 0.4 C = 1 2 /(24+1) = 0.04 S 1a S 1b S 1c S 2a S 2b S 2c S 2d S 2e S 2f S 2g S 2h ICDCS 2011

17. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG New One-to-Many Clustering 17 Bit Vector of S 1 Bit Vector of S 2 C = 8 2 /(36+24) = 1.07 C = |s i ∩ s j | 2 |s i | + |s j | C = 4 2 /(36+4) = 0.4 C = 1 2 /(24+1) = 0.04 C = 12 2 /(36+12) = 3 C = 8 2 /(24+8) = 2 S 1a S 1b S 1c S 2a S 2b S 2c S 2d S 2e S 2f S 2g S 2h ICDCS 2011

18. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG S S S S Phase 3: Broker Overlay Construction 18 S S S S S S S S S S S S S S ICDCS 2011

19. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Bin Packing’s Final Overlay 19 S S S S S S S S S S S S S S S S S S P P P P ( ( GRAPE ) ) ( ( GRAPE ) ) ICDCS 2011

20. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Greedy Relocation Algorithm for Publishers of Events (GRAPE) Distributed algorithm that dynamically relocates publishers to minimize – Broker message rates, and/or – Delivery Delay Similar three phased design: 1.Profile load of subscriptions matching each publisher 2.Determine the placement strategy that minimizes the specified metric 3.Transparently migrate the publisher Cf. GRAPE paper from ICDCS 2010 20ICDCS 2011

21. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Evaluation Implemented on the PADRES open source content-based publish/subscribe system Evaluated on a cluster testbed using 80 brokers Evaluated on SciNet using 1000 brokers Comparison against two related approaches (Riabov et al. ICDCS’02, Banavar et al. ICDCS’99) Homogeneous and heterogeneous scenarios Workload saturates the initial deployment (MANUAL) ICDCS 201121 http://padres.msrg.org

22. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Output Utilization Ratio 22 Resource aware algorithms make full use of allocated resources ICDCS 2011

23. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Broker Message Rate 23 CRAM reduced message rate by up to 92% Clustering significantly reduces message rate ICDCS 2011 Allocating fewer brokers does not help

24. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Number of Allocated Brokers 24 Reduces number of allocated brokers by up to 91% ICDCS 2011 Uses all resources

25. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Computation Time 25 91% improvement at only 30% higher computation time ICDCS 2011

26. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Impact of Publisher Relocation & Subscription Clustering 26 50% reduction in broker message rate ICDCS 2011

27. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Broker Message Rates Using Various Closeness Metrics 27 XOR closeness metric cannot identify empty- relations ICDCS 2011

28. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Conclusions CRAM combines the benefits of ▫ Subscription clustering from PAIRWISE-N/K ▫ Resource awareness from Bin Packing by simultaneously reducing both ▫ Broker message rate (up to 92%) ▫ Number of allocated brokers (up to 91%) to meet green IT objectives! By using bit vectors, CRAM is ▫ Language independent (XPath, regex, topics) ▫ Effective for any workload distribution 28ICDCS 2011

29. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Q & A 29ICDCS 2011

30. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG ICDCS 201130

31. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Future Work React dynamically by growing and shrinking the network in incremental steps Improve runtime of the CRAM algorithm by parallelization or reducing its computational complexity Model workload with more sophisticated methods, such as stochastic processes, to improve accuracy of load estimation Address fault resiliency 31ICDCS 2011

32. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Related Works - Clustering Riabov et al. (ICDCS’02) ▫ The number of clusters K is pre-specified ▫ Each cluster is a multicast address, thus there is no upper limit on its size ▫ Event space is divided into grids ▫ Supports only ranged subscriptions ▫ Their pairwise clustering considers each subscription individually Gryphon (ICDCS'99) ▫ Supports only equal and * subscriptions ▫ Each cluster is stored in memory, the upper bound limit is not a major concern SUB-2-SUB (IPTPS'06) ▫ Supports only ranged subscriptions ▫ Each cluster is a p2p network, thus there is no upper limit on the cluster size 32ICDCS 2011

33. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Related Works – Broker Overlay Construction, Publisher and Subscriber Placement Algorithms Baldoni et al. (The Computer Journal), Jaeger et al. (SAC'07) Migliavacca et al. (DEBS’07) – Reconfigure broker overlay to reduce delivery delay and broker processing load Cheung et al. (Middleware’06, ICDCS’10) – Load balancing by relocating subscriber clients – Reduce delivery delay and broker processing load by relocating publisher clients 33ICDCS 2011

34. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Hop Count Using Various Closeness Metrics 34ICDCS 2011

35. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Computation Time vs. Bit Vector Size 35ICDCS 2011

36. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Allocated Brokers vs. Bit Vector Size 36ICDCS 2011

37. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Average Hop Count 37ICDCS 2011

38. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Computation Time Using Various Closeness Metrics 38 108% higher computation time using Gryphon-derived closeness metric (XOR). ICDCS 2011

39. MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Delivery Delay 39 Overload with Pairwise-K ICDCS 2011