Content-Based Routing

The Need for a Distributed Implementation Disadvantages of a centralized notification service Single point of failure Bottleneck wrt. bandwidth and computing power Does not exploit locality although most interaction will be local Neither reliable nor scalable to wide-area scenarios Service must somehow be distributed and replicated Multiple cooperating service instances necessary! Asynchronous Middleware © Gero Mühl

Broker Network B4 B2 B3 B1 Service instances called brokers Each broker manages a set of exclusive local clients to which it forwards matching notifications it processes Notifications are stepwise forwarded through broker network Cyclic forwarding and delivery of duplicates must be avoided! Easy in acyclic topologies but these provide no fault tolerance Complex in topologies with cycles B4 B2 B3 B1 Local clients Asynchronous Middleware © Gero Mühl

Flooding Each notification is delivered to all brokers Advantages A broker forwards an incoming notification to all neighbors if it was received from a local client and to all other neighbors if it was received from a neighbor Advantages No routing tables necessary Subscriptions become active immediately Implementations can easily exploit IP-Multicast Disadvantage Many notifications may be processed and forwarded unnecessarily wasting computing power and network bandwidth Asynchronous Middleware © Gero Mühl

Efficiency of Flooding Efficient if most notifications need to be delivered to a majority of brokers or subscriptions are highly transient Not efficient if filters are highly selective or locality among subscriptions exists Asynchronous Middleware © Gero Mühl

Content-Based Routing Each broker manages a filter-based routing tables A routing table TB consists of a set of routing entries A routing entry is a pair (F, D) consisting of a filter F and a destination D which can be a local client or a neighbor broker Local and remote routing entries sub(G) n B4 sub(F) n (G,B4) B2 B3 n n Routing tables (F,B2) (G,B3) B1 n n ∈ N(F) n ∈ N(G) Asynchronous Middleware © Gero Mühl

Content-Based Routing (contd.) Forwarding Engine Routing Table (F, B3) (G, B4) (H, B3) n ∈ N(F) n ∉ N(G) n ∉ N(H) Broker B1 1. n 2. n B2 B3 B4 Asynchronous Middleware © Gero Mühl

Notification Forwarding in Acyclic Topologies Each broker forwards an incoming notification to all neighbor brokers and to local clients for which there is a matching routing entry in the routing table. If the notification was received from a neighbor broker, it is not forwarded to that neighbor. Asynchronous Middleware © Gero Mühl

Routing Algorithms Routing tables must be updated if subscriptions are issued or revoked Update processes triggered Many update processes might be in progress currently Routing algorithm Subscriptions become active gradually Race conditions and latency Fully active after corresponding update process terminated Forwarding of unsubscriptions only done for efficiency Delivery of non-matching notifications can be prevented locally Asynchronous Middleware © Gero Mühl

Routing Algorithms (contd.) A correct routing algorithm ensures that Each update process terminates within finite time After an update process terminated, all published notifications matching this subscription are delivered to the subscribing client No duplicates and no non-matching notifications are delivered Asynchronous Middleware © Gero Mühl

Routing Algorithms (contd.) Trade-Off Flooding notifications vs. filtering at intermediate brokers and updating of routing tables Optimal approach heavily depends on current setting! In dynamic environments adaptive solutions necessary Asynchronous Middleware © Gero Mühl

Simple Routing Not scalable! B1 B3 B4 B2 Only assumption: Filters can be uniquely identified Each subscription stored in every routing table Size of routing tables grows linearly in the number of active subscriptions and the number of brokers Each routing table affected by a new/cancelled subscription #brokers-1update messages necessary if update information is not batched Not scalable! (G,B3) (F,B3) (F,B1) (G,B2) (F,B1) B1 1. sub(F) G B3 B4 G B2 2. sub(G) (F,B3) (F,B3) Asynchronous Middleware © Gero Mühl

Simple Routing (contd.) New subscription flooded into the broker network If a broker receives a subscription F from a neighbor B, it adds an entry (F, B) to its routing table and forwards the subscription F to all its other neighbors Unsubscription flooded into the broker network If a broker receives an unsubscription F from a neighbor B, it removes the entry (F, B) from its routing table and forwards the unsubscription F to all its other neighbors Asynchronous Middleware © Gero Mühl

Advanced Content-Based Routing Goals Smaller routing tables Reduced filter forwarding overhead Possible Solutions Routing algorithms that exploit similarities and merge subscriptions Integration of advertisements (announcements of producers) Trade accuracy vs. efficiency perfect routing Notifications are forwarded only if a matching subscription exists in respective subnet imperfect routing Notifications may be forwarded without any matching subscription Asynchronous Middleware © Gero Mühl

Identity-Based Routing Uses identity tests for routing decisions F≡G ⇔ N(F) = N(G) Avoids Forwarding of identical filters Routing entries with identical filters for the same neighbor broker n1 F≡G n2 geometrical visualization (F,B3) (F,B1) (F,B2) (F,B1) sub(F) B1 F B4 B3 F sub(F) B2 (F,B3) (F,B3) Asynchronous Middleware © Gero Mühl

Covering-Based Routing Uses covering tests for routing decisions F covers G ⇔ N(F) ⊇ N(G) Avoids Forwarding of covered filters Routing entries with covered filters for the same neighbor broker Forwarding of subscriptions similar to identity-based routing Forwarding of unsubscriptions more complex due to uncovered subscriptions F G Asynchronous Middleware © Gero Mühl

Covering-based Routing (contd.) Processing of a new subscription F from a neighbor B Routing entries regarding B whose filter is covered by F are dropped F is forwarded only to those other neighbors to which no covering subscription was forwarded before which is still active (G,B3) (F,B1) (G,B2) (F,B1) sub(F) B1 F G G B4 B3 B2 G sub(G) (F,B3) (F,B3) Asynchronous Middleware © Gero Mühl

Covering-Based Routing (contd.) Processing of an unsubscription F from a neighbor B F is forwarded only to those other neighbors to which no covering subscription was forwarded before which is still active Along with F, uncovered subscriptions must be forwarded (G,B3) (H,B3) (F,B1) (G,B2) (H,B2) F sub(F) B1 G F F G H B3 B4 H B2 F sub(G) sub(H) (G,B3) (H,B3) (F,B3) (F,B3) Asynchronous Middleware © Gero Mühl

Merging-Based Routing Filters can be merged perfectly N(F) = N(G) ∪ N(H) imperfectly N(F) = N(G) ∪ N(H) Merging generates new covers F H G (H,B3) (F,B3) (G,B3) F G B1 H 1. sub(G) B3 F B4 H H (G,B1) (H,B2) (G,B1) 2. sub(H) B2 (G,B3) Asynchronous Middleware © Gero Mühl

Merging-Based Routing (contd.) Merging-based routing is perfect iff only perfect mergers are generated and mergers are forwarded and cancelled in a way such that no unnecessary notifications are received later on … otherwise it is imperfect Asynchronous Middleware © Gero Mühl

Perfect Merging-Based Routing Possible implementation on top of covering-based routing Brokers merge suitable routing entries (F1, D),…, (Fn, D) with the same destination D to a single entry (F, D) such that F is a perfect merger of {F1, …, Fn} Merged entries are removed from routing table Merger is added to routing table and forwarded like a normal subscription received from D Asynchronous Middleware © Gero Mühl

Hierarchical Routing Algorithms One broker is labeled as root broker R Any notification and filter is forwarded “upwards” to R Notifications are only forwarded “downwards” if a matching filter exists Leads to smaller routing tables but increases the load imposed on brokers on higher levels such as the root broker (F,B1) R n n (F,B3) B1 B2 n n B3 B4 B5 B6 1. sub(F) 2. pub(n) n ∈ N(F) Asynchronous Middleware © Gero Mühl

Hybrid Routing Algorithms Peer-to-Peer Routing B7 B0 (F,B1) (F,B1) n n Combine hierarchical with peer-to-peer routing (F,B3) B1 B2 (F,B0) n n B3 B4 B5 B6 n n Hierarchical Routing 1. sub(F) 2. pub(n) n ∈ N(F) Asynchronous Middleware © Gero Mühl

Advertisements Producers specify the notifications they will publish Advertisements are filters that are forwarded like subscriptions by the help of a second routing table Subscriptions are only forwarded to those destinations for which there is an overlapping advertisement Test for overlapping: N(F) ∩ N(G) ≠ ∅ G F Asynchronous Middleware © Gero Mühl

Advertisements (contd.) B3 B4 H F 1. 2. F F ... ... B1 B5 G (G,B4) (H,B5) Subscription Advertisements Routing Table Asynchronous Middleware © Gero Mühl

Advertisements (contd.) New advertisement If B receives a new advertisement from H, B forwards those overlapping subscriptions to H which can newly be served Cancelled advertisement If B receives a cancelled advertisement from H, B drops those overlapping subscriptions which can no longer be serviced Asynchronous Middleware © Gero Mühl

Advertisements (contd.) Race condition Published notifications may be dropped if they match only a newly servable subscription that has not yet arrived! Potential solutions Keep notifications that match only the new advertisement until all newly servable subscriptions have arrived  corrupts FIFO producer ordering Block a producer that publishes a notification that matches only the new advertisement until all newly servable subscriptions have arrived  introduces some synchronicity Asynchronous Middleware © Gero Mühl

Combining Advertisements with Hierarchical Routing Advertisements are distributed using peer-to-peer routing Subscriptions are distributed using hierarchical routing but they are only forwarded “upwards” if an overlapping advertisement exists A flag indicates whether for an advertisements an overlapping subscription exists in an upper sub-tree R false (B,B4) B1 B2 G B3 B4 B5 B6 1. adv(F) 2. sub(G) N(F) ∩ N(G) ≠ ∅ Asynchronous Middleware © Gero Mühl

Topology Changes Easy approach preserving an acyclic topology A new broker B is connected to exactly one broker B’ B’ sends all interesting subscriptions to B Simple Routing: all subscriptions Identity-based Routing: identical subscriptions are suppressed Covering-based Routing: covered subscriptions are suppressed If advertisements are used, they are exchanged instead of subscriptions Only leaf brokers can be removed Asynchronous Middleware © Gero Mühl

Supporting Routing Optimizations Complexity of routing optimizations depends on the underlying data/filter model In the most general case not computable E.g., covering test among relational expressions is NP-complete Scalability vs. Expressiveness Expressiveness must be limited to a sensible level Asynchronous Middleware © Gero Mühl

Identity among Filters Two filters F1 and F2 are identical, written F1 ≡ F2, iff they contain the same number of attribute filters and if for each attribute filter in F1 there is an attribute filter in F2 such that these attribute filters are identical Example of identical filters F1 = {x ≥ 2} ∧ {y > 5} F2 = {x ≥ 2} ∧ {y > 5} ≡ ≡ ≡ Asynchronous Middleware © Gero Mühl

Overlapping among Filters Two filters F1 and F2 are disjoint iff for an attribute filter in F1 there exists an attribute filter in F2 that constrains the same attribute such that these attribute filters are disjoint Two filters are overlapping iff they are not disjoint Example for disjoint filters F1 = {x ≥ 2} ∧ {y > 5} F2 = {x < 1} ∧ {y = 7} ⊓ ⊓ ⊓ Asynchronous Middleware © Gero Mühl

Covering among Filters A filter F1 covers a filter F2, written F1 ⊒ F2, iff for each attribute filter in F1 there is a covered attribute filter in F2 Example of covering filters F1 = {x ≥ 2} ∧ {y > 5} F2 = {x = 4} ∧ {y = 7} ∧ {z ∈ [3,5]} ⊒ ⊒ ⊒ Asynchronous Middleware © Gero Mühl

Filter Lattice Filters are arranged in a lattice according to covering relation Lattice is updated according to new and cancelled subscriptions Lattice is used by covering-based routing algorithm F1:={x>5} F2:={x>8} F4:={x=6} F5:={x>6 ∧ y>7} F3:={x=9} F6:={x>7 ∧ y=9} F7:={x=9 ∧ y=9} Asynchronous Middleware © Gero Mühl

Merging of Filters Two filters F1 and F2 can be merged perfectly iff they are identical in all but a single attribute filter Merging Example F1 = {x ≥ 2} ∧ {y < 4} F2 = {x ≥ 2} ∧ {y > 6} F3 = {x ≥ 2} ∧ {y ∉ [4,6]} (Merger) ≡ Asynchronous Middleware © Gero Mühl

Example: Geographic Information Systems (GIS) F = {(Type = TrafficInformation) ∧ (Location around(Frankfurt,50km))} G = {(Type = TrafficJam) ∧ (Length ≥ 5km?) ∧ (Location around(Darmstadt,20km))} F covers G H = {(Type = TrafficJam) ∧ (Location around(Frankfurt,40km))} I = {(Type = TrafficJam) ∧ (Location around(Wiesbaden,40km))} H and I can be merged imperfectly Frankfurt Darmstadt X X Frankfurt X X X Wiesbaden Asynchronous Middleware © Gero Mühl

Bibliography A. Carzaniga. Architectures for an Event Notification Service Scalable to Wide-area Networks. PhD thesis, Politecnico di Milano, Milano, Italy, Dec. 1998. G. Mühl. Large-Scale Content-Based Publish/Subscribe Systems. PhD thesis, Darmstadt University of Technology, 2002. Chapter 3 A. Carzaniga, D. S. Rosenblum, and A. L. Wolf. Design and Evaluation of a Wide-Area Event Notification Service. ACM Transactions on Computer Systems, 19(3):332--383, 2001. Asynchronous Middleware © Gero Mühl