1 Towards Scalable Pub/Sub Systems Shuping Ji 1, Chunyang Ye 2, Jun Wei 1 and Arno Jacobsen 3 1 Chinese Academy of Sciences 2 Hainan University 3 Middleware.

Slides:

Advertisements

Similar presentations

Opportunistic Multipath Forwarding in Publish/Subscribe Systems Reza Sherafat Kazemzadeh AND Hans-Arno Jacobsen Middleware Systems Research Group University.

Advertisements

PODC 2007 © 2007 IBM Corporation Constructing Scalable Overlays for Pub/Sub With Many Topics Problems, Algorithms, and Evaluation G. Chockler, R. Melamed,

Efficient Event-based Resource Discovery Wei Yan*, Songlin Hu*, Vinod Muthusamy +, Hans-Arno Jacobsen +, Li Zha* * Chinese Academy of Sciences, Beijing.

Alex Cheung and Hans-Arno Jacobsen August, 14 th 2009 MIDDLEWARE SYSTEMS RESEARCH GROUP.

1 Cycle Detection in Publish/Subscribe Overlay Networks Reza Sherafat Alex Cheung Prof. Cristiana Amza ECE1747 – Course Project University of Toronto.

Multicast in Wireless Mesh Network Xuan (William) Zhang Xun Shi.

Partition-Tolerant Distributed Publish/Subscribe Systems

Topologically-Aware Overlay Construction and Server Selection Sylvia Ratnasamy, Mark Handly, Richard Karp and Scott Shenker Presented by Shreeram Sahasrabudhe.

Cognitive Publish/Subscribe for Heterogeneous Clouds Šarūnas Girdzijauskas, Swedish Institute of Computer Science (SICS) Joint work with:

Transactional Mobility in Distributed Content-Based Publish/Subscribe Systems Songlin Hu*, Vinod Muthusamy +, Guoli Li +, Hans-Arno Jacobsen + * Chinese.

A New Approach for the Construction of ALM Trees using Layered Coding Yohei Okada, Masato Oguro, Jiro Katto Sakae Okubo International Conference on Autonomic.

Applications over P2P Structured Overlays Antonino Virgillito.

©NEC Laboratories America 1 Hui Zhang Samrat Ganguly Sudeept Bhatnagar Rauf Izmailov NEC Labs America Abhishek Sharma University of Southern California.

Carnegie Mellon University Complex queries in distributed publish- subscribe systems Ashwin R. Bharambe, Justin Weisz and Srinivasan Seshan.

1 An Overlay Scheme for Streaming Media Distribution Using Minimum Spanning Tree Properties Journal of Internet Technology Volume 5(2004) No.4 Reporter.

Hermes: A Distributed Event- Based Middleware Architecture Peter Pietzuch and Jean Bacon 1st DEBS Workshop, Vienna,

An Evaluation of Scalable Application-level Multicast Using Peer-to-peer Overlays Miguel Castro, Michael B. Jones, Anne-Marie Kermarrec, Antony Rowstron,

The Delta Routing Project Low-loss Routing for Hybrid Private Networks George Porter (UCB) Minwen Ji, Ph.D. (SRC - HP Labs)

Distributed Publish/Subscribe Network Presented by: Yu-Ling Chang.

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) Application-Level Multicast.

Daibo Liu 1 Daibo Liu 1, Zhichao Cao 2, Xiaopei Wu 2, Yuan He 2, Xiaoyu Ji 3 and Mengshu Hou 1 ICDCS, 2015, Columbus TeleAdjusting: Using Path Coding and.

ENHANCING AND EVALUATION OF AD-HOC ROUTING PROTOCOLS IN VANET.

Alex King Yeung Cheung and Hans-Arno Jacobsen University of Toronto June, 24 th 2010 ICDCS 2010 MIDDLEWARE SYSTEMS RESEARCH GROUP.

Distributed Quality-of-Service Routing of Best Constrained Shortest Paths. Abdelhamid MELLOUK, Said HOCEINI, Farid BAGUENINE, Mustapha CHEURFA Computers.

Effects of Routing Computations in Content-Based Routing Networks with Mobile Data Sources Vinod Muthusamy, Milenko Petrovic, Hans-Arno Jacobsen University.

SensIT PI Meeting, January 15-17, Self-Organizing Sensor Networks: Efficient Distributed Mechanisms Alvin S. Lim Computer Science and Software Engineering.

Publisher Mobility in Distributed Publish/Subscribe Systems Vinod Muthusamy, Milenko Petrovic, Dapeng Gao, Hans-Arno Jacobsen University of Toronto June.

Supporting Disconnected Operations in Publish/Subscribe Systems Vinod Muthusamy Joint work with Milenko Petrovic, Ioana Burcea, H.-Arno Jacobsen, Eyal.

Content-Based Routing in Mobile Ad Hoc Networks Milenko Petrovic, Vinod Muthusamy, Hans-Arno Jacobsen University of Toronto July 18, 2005 MobiQuitous 2005.

BitTorrent enabled Ad Hoc Group 1  Garvit Singh( )  Nitin Sharma( )  Aashna Goyal( )  Radhika Medury( )

Network and Communications Ju Wang Chapter 5 Routing Algorithm Adopted from Choi’s notes Virginia Commonwealth University.

MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware A Policy Management Framework for Content-based Publish/Subscribe Middleware Hans-Arno Jacobsen Department.

TOMA: A Viable Solution for Large- Scale Multicast Service Support Li Lao, Jun-Hong Cui, and Mario Gerla UCLA and University of Connecticut Networking.

Impact of Topology on Overlay Multicast Suat Mercan.

1 Internet Routing. 2 Terminology Forwarding –Refers to datagram transfer –Performed by host or router –Uses routing table Routing –Refers to propagation.

Dynamic Load Balancing in Distributed Content-based Publish/Subscribe Alex K. Y. Cheung & Hans-Arno Jacobsen University of Toronto November 30 th, 2006.

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Total Order in Content-based Publish/Subscribe Systems Joint work with: Vinod Muthusamy, Hans-Arno Jacobsen.

Designing Routing Protocol For Mobile Ad Hoc Networks Navid NIKAEIN Christian BONNET EURECOM Institute Sophia-Antipolis France.

Distributed Automatic Service Composition in Large-Scale Systems Songlin Hu*, Vinod Muthusamy +, Guoli Li +, Hans-Arno Jacobsen + * Chinese Academy of.

Historic Data Access in Publish/Subscribe Middleware System Research Group University of Toronto.

Network Layer Support for Service Discovery in MANETs Ulas Kozat and Leandros Tassiulas University of Maryland, College Park Presented by Wei Gao.

MIDDLEWARE SYSTEMS RESEARCH GROUP Modelling Performance Optimizations for Content-based Publish/Subscribe Alex Wun and Hans-Arno Jacobsen Department of.

MIDDLEWARE SYSTEMS RESEARCH GROUP Adaptive Content-based Routing In General Overlay Topologies Guoli Li, Vinod Muthusamy Hans-Arno Jacobsen Middleware.

Minimal Broker Overlay Design for Content-Based Publish/Subscribe Systems Naweed Tajuddin Balasubramaneyam Maniymaran Hans-Arno Jacobsen University of.

ICDCS Beijing China Routing of XML and XPath Queries in Data Dissemination Networks Guoli Li, Shuang Hou Hans-Arno Jacobsen Middleware Systems Research.

Analysis and algorithms of the construction of the minimum cost content-based publish/subscribe overlay Yaxiong Zhao and Jie Wu

NUS.SOC.CS5248 Ooi Wei Tsang Application-Level Multicast.

Information-Centric Networks10b-1 Week 10 / Paper 2 Hermes: a distributed event-based middleware architecture –P.R. Pietzuch, J.M. Bacon –ICDCS 2002 Workshops.

Information-Centric Networks Section # 10.2: Publish/Subscribe Instructor: George Xylomenos Department: Informatics.

Peter R Pietzuch and Jean Bacon Peer-to-Peer Overlay Networks in an Event-Based Middleware DEBS’03, San Diego, CA, USA,

1 State-of-the-art in Publish/Subscribe Middleware for Supporting Mobility Sumant Tambe EECS Preliminary Examination December 11, 2007 Vanderbilt University,

Distributed Automatic Service Composition in Large-Scale Systems Songlin Hu*, Vinod Muthusamy +, Guoli Li +, Hans-Arno Jacobsen + * Chinese Academy of.

Design and implementation of an intentional naming system William Adjie-WinotoElliot Schwartz Hari BalakrishnanJeremy Lilley MIT Laboratory for Computer.

Community Clustering in Distributed Publish/Subscribe System Wei Li 1,2,Songlin Hu 1, Jintao Li 1, Hans-Arno Jacobsen 3 1 Institute of Computing Technology,

Author Utility-Based Scheduling for Bulk Data Transfers between Distributed Computing Facilities Xin Wang, Wei Tang, Raj Kettimuthu,

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) Application-Level Multicast.

Internet Traffic Engineering Motivation: –The Fish problem, congested links. –Two properties of IP routing Destination based Local optimization TE: optimizing.

Multicast with Network Coding in Application-Layer Overlay Networks Y. Zhu, B. Li, and J. Guo University of Toronto Present by Cheng Huang

Congestion Avoidance with Incremental Filter Aggregation in Content-Based Routing Networks Mingwen Chen 1, Songlin Hu 1, Vinod Muthusamy 2, Hans-Arno Jacobsen.

Supporting On-Demand Elasticity in Distributed Graph Processing Mayank Pundir*, Manoj Kumar, Luke M. Leslie, Indranil Gupta, Roy H. Campbell University.

Optimizing Distributed Actor Systems for Dynamic Interactive Services

Advanced Computer Networks

Application-Level Multicast

A Study of Group-Tree Matching in Large Scale Group Communications

Hyunchul Park, Kevin Fan, Manjunath Kudlur,Scott Mahlke

Distributed Publish/Subscribe Network

Composite Subscriptions in Content-based Pub/Sub Systems

Foundations for Highly-Available Content-based Publish/Subscribe Overlays Young Yoon, Vinod Muthusamy and Hans-Arno Jacobsen.

COMPUTER NETWORKS CS610 Lecture-16 Hammad Khalid Khan.

Presentation transcript:

1 Towards Scalable Pub/Sub Systems Shuping Ji 1, Chunyang Ye 2, Jun Wei 1 and Arno Jacobsen 3 1 Chinese Academy of Sciences 2 Hainan University 3 Middleware Systems Research Group, University of Toronto

2 Existing routing algorithms for Pub/Sub 1.Event Flooding: Events are flooded to brokers, heavy bandwidth use. 2.Multicast-based Routing (MBR): Event space is partitioned into disjoint multicast groups (the right graph), hard to partition, bandwidth waste. 3.Filter-based Routing (FBR): Widely used method, events are only routed to interested brokers, redundant matching, forwarding by non-interested brokers.

3 Limitations of the FBR algorithm 1.Difficulty in supporting general overlay topologies. 2.Subscription duplication: As shown in right graph, subscriptions are duplicated at B1, B2, B3, and B4. 3.Redundant and repeated event matching. 4.Lack of flexibility in supporting overlay reconfiguration: hard to change the topology in left graph to the one in right graph.

4 D-DBR and MERC algorithm Dynamic Destination-based Routing algorithm  Event matching and event routing are decoupled.  Subscriptions from local and remote brokers are separated.  Topology can be dynamically configured for optimization.  Events are attached with destinations of interested brokers. Match at Edge and Routing inter-Cluster algorithm  Overlay is divided into interconnected clusters.  The destination list size is limited.  Better scalability than D-DBR algorithm.  A reference model to construct large-scale pub/sub systems.

5 Two layers of D-DBR algorithm Matching Layer:  Only interested events are inserted into Match Queue.  Routing information divided into four tables. Multicasting Layer:  In charge of transferring events to interested brokers.  Topology changes do not affect upper matching layer.

6 Matching and routing layer Local/remote events/subs processing at the matching layer Event routing at the multicasting layer

7 MERC routing example

8 Evaluation Experiments on Computing Facility  We implemented both D-DBR and MERC algorithm in PADRES, a representative, open-source, content-based pub/sub system based on the FBR algorithm.  Experiments are executed on both acyclic linear topology and general topology.  Evaluation metrics: System Throughput, Event Routing Latency, CPU & RAM Utilization, and Subscription Duplication. Experiments Based on Simulations  Evaluation metrics: Destination List Overhead, Routing Accuracy, System Robustness, Topology Maintenance Overhead.  Routing accuracy is defined as the ratio between the number of brokers interested in an event over the number of brokers receiving that event when it is routed towards the interested brokers.

9 Performance Latency in acyclic linear topologyCPU utilization in acyclic linear topology Latency in general topology Sub duplication in general topology

10 Routing accuracy and robustness Routing accuracy improvement for D-DBR Robustness under connection failures Robustness under broker failures

11 Destination list overhead Average destination list size Destination list size distributionDestination list size at source brokers