Content-based Publish-Subscribe Over Structured P2P Networks Peter Triantafillou and Ioannis Aekaterinidis Presented by Jesse Chen 4/10/07

Outline of Talk What is a publish/subscribe system? What is a publish/subscribe system? Why do we build publish/subscribe systems over p2p networks? Why do we build publish/subscribe systems over p2p networks? What is an event/subscription? What is an event/subscription? How to store subscriptions? How to store subscriptions? How to update subscriptions? How to update subscriptions? The matching algorithm The matching algorithm Delivering of events to the subscribers Delivering of events to the subscribers

What is a publish/subscribe system? A communication (information dissemination) system that delivers published notifications from every producer (publishers) to all interested consumers (subscribers). A communication (information dissemination) system that delivers published notifications from every producer (publishers) to all interested consumers (subscribers). Two types of publish/subscribe systems: Two types of publish/subscribe systems: Topic based Topic based Users express interest by joining a group (topic) Users express interest by joining a group (topic) Content based Content based Users express interest by specifying subscriptions (a collection of predicates over the values of some attributes). Users express interest by specifying subscriptions (a collection of predicates over the values of some attributes).

Why Build Publish/Subscribe Systems Over P2P Networks? To avoid the lack of scalability and fault- tolerance of centralized approaches. To avoid the lack of scalability and fault- tolerance of centralized approaches. To take advantages of Chord: decentralized, scalable, self-organizing. To take advantages of Chord: decentralized, scalable, self-organizing. Chord’s structured network: using DHT for easy resource lookup/routing. Chord’s structured network: using DHT for easy resource lookup/routing.

What is a publish-subscribe system? Publisher 1 Publisher 2 … Matching Algorithm Subscriber 1 Subscriber 2 … Event Subscription - Publishers and subscribers are decoupled. - Publishers are the information producers and the subscribers are the information consumers. - Subscriptions act as filters that filter out unrelated events and the system only delivers matched events to the subscribers.

What is a publish-subscribe system? An example of a publish/subscribe system: An example of a publish/subscribe system: Online stock quote Online stock quote Publishers: stock exchanges - NYSE, NASDAQ Publishers: stock exchanges - NYSE, NASDAQ Events might contain the following information: Events might contain the following information: Opening price, closing price, high/low price, volume… Opening price, closing price, high/low price, volume… Subscribers: stock traders Subscribers: stock traders Subscription: A stock’s price at a particular moment. Subscription: A stock’s price at a particular moment.

What is an event? A set of typed attributes A set of typed attributes Each attribute consists of a type, a name and a value Each attribute consists of a type, a name and a value

What is a subscription? Subscription 1 Subscription 2 An event matches a subscription if and only if all the subscription’s attribute predicate/constraints are satisfied.

The subscription identifier Each subscription is identified by an id (subID) Each subscription is identified by an id (subID) C1: The id of the node receiving the subscription C1: The id of the node receiving the subscription m bits for a m-bit Chord identifier address space m bits for a m-bit Chord identifier address space C2: The id of the subscription itself C2: The id of the subscription itself Size is the rounded-up log of the maximum number of outstanding subscriptions a node can have. Size is the rounded-up log of the maximum number of outstanding subscriptions a node can have. C3: The number of attributes declared C3: The number of attributes declared

How to store subscriptions?

For an attribute a with value v(a): For an attribute a with value v(a): Generate the key k: k = h(v(a)) Generate the key k: k = h(v(a)) Place subID at the node = successor(k) Place subID at the node = successor(k) Equality constraint attribute Equality constraint attribute Place subID at Place subID at node = successor(h(“NYSE”)) Range value attribute Range value attribute Place subID at the following nodes: Place subID at the following nodes: Successor(h(8.31)), successor(h(3.32)), …, successor(h(8.69)).

How to store subscriptions? Subscription generator Successor(h(NYSE)) SubID list: SubID 1 Successor(h(OTE)) SubID list: SubID1 Successor(h(8.69)) SubID list: SubID1 … Successor(h(8.31)) SubID list: SubID 1 The average number of hops needed to store a subID: Equality constraint: Range attribute:

Updating subscriptions The old node is forced to delete the SubID for the attribute that belongs to the subscription. The old node is forced to delete the SubID for the attribute that belongs to the subscription. The new node is going to store the subID The new node is going to store the subID Subscription generator Successor(h(NYSE)) SubID list: SubID 1 Successor(h(NASDAQ)) SubID list: SubID 1 An update is performed by contacting:

Updating subscriptions Range attribute Range attribute Updating depends on the new values of the range bounds(V low- new(a),V high-new(a)) compare to the old range bounds. Updating depends on the new values of the range bounds(V low- new(a),V high-new(a)) compare to the old range bounds. V low-new(a) < V low(a) V low-new(a) < V low(a) Store SubID at the following nodes: [V low-new(a), V low(a)) Store SubID at the following nodes: [V low-new(a), V low(a)) V high-new(a) > V high(a) V high-new(a) > V high(a) Store SubID at the following nodes: (V high(a), V high-new(a)] Store SubID at the following nodes: (V high(a), V high-new(a)] V low-new(a) > V low(a) or V high-new(a) V low(a) or V high-new(a) < V high(a) Delete SubID at the following nodes: [V low(a), V low-new(a)) or (V high-new(a), Vhigh(a)] Delete SubID at the following nodes: [V low(a), V low-new(a)) or (V high-new(a), Vhigh(a)] The average number of nodes need to be contacted: K(logN). The average number of nodes need to be contacted: K(logN). K depends on whether new range is smaller or wider than old one. K depends on whether new range is smaller or wider than old one.

The Matching Algorithm – Matching Events with Subscriptions

Successor(h(NYSE)) SubID list: SubID 1 Successor(h(OTE)) SubID list: SubID1, SubID 2 Successor(h(8.40)) SubID list: SubID1 Successor(h(8.80)) SubID list: Matching algorithm will retrieve the SubID lists for each attribute in the event by going to the successor nodes. Count the number of lists containing SubID1: 3 Number of attributes declared in C3 field of SubID1: 3

How to deliver the event to the subscriber? Informing the node which generated the subscription with the matched event. Informing the node which generated the subscription with the matched event. The node stores the subscription and it also holds metadata information about the subscription (the IP address of the subscription generator). The node stores the subscription and it also holds metadata information about the subscription (the IP address of the subscription generator). Finally, the event is delivered to the subscriber. Finally, the event is delivered to the subscriber.

Thank You!