1. Foundations for Highly-Available Content-based Publish/Subscribe Overlays
Young Yoon, Vinod Muthusamy and Hans-Arno Jacobsen

2. PADRES: an Overlay of Content-based Pub/Sub Brokers
1. Advertise
Publisher
3. Publish
Subscriber
This is a project on PADRES, an enterprise service bus as an overlay of content-based pub/sub brokers. See how a notifcation path gets constructed with advertisement and subscription routing.
2. Subscribe
2/22/2019

3. Inside a PADRES Broker Matching Engine + Routing Table input queue
subscription
dest
Matching Engine
Routing Table +
input queue
output queue dest1
output queue dest2
service time < 3s dest2
Inside the broker, there are matching engine, routing table, input/output queues.
service time < 2s dest3
2/22/2019

4. Content-based Publish/Subscribe as an Application Integration Fabric
Stock-market monitoring
Network management and monitoring
Algorithmic trading
Distributed business process execution
Business activity monitoring
Enterprise service bus
Content-based Publish/Subscribe as an Application Integration Fabric, as pointed out in the paper. A number of examples can be found. These application can be in a large scale, thus a publish/subscribe overlay is needed, like PADRES.
2/22/2019

5. Obstacles for the Pub/Sub Overlay
Publisher
Congestion due to I/O overload or matching overhead
Crash failure
Periodic broker maintenance
Subscriber
Congestion can occur for various reasons, such as I/O overhead and matching overhead at a broker. This can significantly affect message delivery.
Messages can get
delayed or even get lost
2/22/2019

6. Approaches for High-Availability
Overlay transformation
by link re-wiring (planned) P2
Planned or on-demand approaches to transforming an overlay to achieve high-availability. The terms can be debatable, but the key difference whether you deploy a new broker from a pool of standby brokers or not.
Planned transformation is about changing a topology, on-demand does not change increases a broker capacity
Explain implication on the message delivery re-wiring is not enough
Overlay transformation
by adaptive broker replication (on-demand)
2/22/2019

7. Outline High-availability for pub/sub overlays
Primitive operations for overlay transformations
Fundamental message resumption protocols
2/22/2019

8. What Does High-Availability Mean in Pub/Sub Overlays?
Avoiding the violation of functional and non-functional properties on pub/sub messaging
2/22/2019

9. Functional Properties of Pub/Sub Messaging
Complete delivery of publications to all interested subscribers
In-order delivery of publications to all interested subscribers
More formally defined in the paper. A stronger functional properties require to be satisfied even with a broker failure.
2/22/2019

10. Non-Functional Properties of Pub/Sub Messaging
Publication delivery resumption delay
How long does it take to resume the message delivery?
Last message processed time
How long does it take to process all the pending messages due to failures in the overlay?
2/22/2019

11. High-availability through planned overlay transformation:
Primitive operations and protocols
First, let’s go through the primitive operations and protocols for a planned overlay transformation
2/22/2019

12. Primitive Operations for Planned Overlay Transformation
SHIFT(Bi, Bj , Bk ) Bi Bi Bj Bj Bk Bk
2/22/2019

13. Theorem
There is always a plan to transform any valid* overlay G to another valid overlay G’ using only the SHIFT primitives. * acyclic and connected graph
2/22/2019

14. Disruption during Planned Transformation (1)
Routing table entry: subscription id | last hop B3 B1
S1 | B1
S1 | B2
S1 | B1
S1 | B2 B2 B3 B2 B3 B1 B1
SHIFT(B1, B2, B3) S1 S1
Routing state changes at the brokers
2/22/2019

15. Disruption during Planned Transformation (2)
Publication drop or looping
Not updated yet
Updated
S1 | B3
S1 | B2 P P1 B2 B3 B1 S1
2/22/2019

16. Disruption during Planned Transformation (3)
Messages can be delivered out of order
S1 | B1
S1 | B2
S1 | B3
S1 | B1
S1 | B3
S1 | B1 B2 B3 B2 B3 B2 B3 P2 P1 P1 P2 B1 B1 B1 S1 S1 S1
P1 09:00 (propagates slowly)
P2 09:01 (propagates quickly to B1)
P2 arrives at 09:03
P1 arrives at 09:04
2/22/2019

17. Synchronous Shift Operation
Routing table update
Make a new connection and Buffer user messages
S1 | B1
S1 | B2
S1 | B3
S1 | B2 B2 B3 B2 B3 B2 B3
Shift request
Shift request B1 B1 B1
Routing table update
S1 | B3
S1 | B1
S1 | B3
S1 | B1
S1 | B3
S1 | B1 B2 B3 B2 B3 B2 B3
ACK
FIN
ACK B1 B1 B1
Routing table update
2/22/2019

18. Planned Transformation ExampleB1 B2 B3 B4 B5 B6 B7 B8 B1 B2 B3 B4 B5 B6 B7 B8
Suppose the following extreme case of a chained broker topology. Assume that replacing the link betwee B1 and B2 with the link between B1 and B8 turned is a better plan.
2/22/2019

19. Our Implementation of Planned Transformation: Global v. Incremental
Involve all brokers at once B1 B2 B3 B4 B5 B6 B7 B8
Involve 3 brokers at a time
Suppose the following extreme case of a chained broker topology. Assume that replacing the link betwee B1 and B2 with the link between B1 and B8 turned is a better plan. B1 B2 B3 B4 B5 B6 B7 B8
2/22/2019

20. Disruption due to Global and Incremental Transformation (1)
Incremental approach shows less disruption
2/22/2019

21. Disruption due to Global and Incremental Transformation (2)
Incremental approach exhibits constant publication delivery resumption delay
2/22/2019

22. High-availability through on-demand overlay transformation:
Primitive operations and protocols
2/22/2019

23. Primitive Operations for On-demand Overlay Transformation
REPLICATE(Bi, Bj)
CONSOLIDATE() N N
Neighbor Broker F
Faulty Broker F R R R R
Replica N
2/22/2019

24. Purpose of the On-Demand Transformation
Replace a permanently
failed broker
Exclusively re-route through the replica N N
Faulty Broker
Replica
Faulty Broker
Replica
Offload temporarily failed broker
Load-balance between the replicas N N S S
2/22/2019

25. P P S S Replication N N N N Adv_Request Sub_forward Adv_Request
Faulty Broker
Replica
Faulty Broker
Replica
Adv_Request
Sub_forward N N
Established overlay S S
Physical connection
2/22/2019

26. Disruptions During On-demand Transformation (1)
Message can be dropped P1 N1
Not delivered to S2
09:04 F R
09:00 N3 N2
11:30 S1 S2
2/22/2019

27. Disruptions During On-demand Transformation (2)
Messages can be delivered out of order P
09:03
09:01 N1 F R N2
09:10
09:05 S
2/22/2019

28. Two Basic Message Resumption Protocols
Synchronous resumption
Wait until replication is
completed on the new broker
Incremental resumption
Resume publication as soon as subscription path is established through the replica
2/22/2019

29. Incremental Message Resumption Protocol
Stop retrieving incoming user messages R
Notify when replication is done N
Copy over outbound messages
Stop forwarding messages F
2/22/2019

30. Advanced Queuing Method for Incremental Resumption
Priority Queue
High-priority
control messages
Wait
Ready S1 P1 P1
Round –robin de-queue
User-generated messages S2 P1 P1 S3 P1 P1 .. ..
2/22/2019

31. Incremental Resumption at the Replica (1): Incomplete Replication of Routing States
Sub
Lasthop S1 B1
S’ = {S1}
Matching subs. of P1: S = {S1,S2,S3} R B1 B0 B2 F B3
2/22/2019

32. Incremental Resumption at the Replica (2): Caching of Publications
Sub
Lasthop S1 B1 S2 B2
Pub
sentTo
matches P1
{B1}
{S2, S3} S2 B1 R B0 B2 F B3
2/22/2019

33. Synchronous vs. Incremental Resumption Policies
Incremental approach resumes message delivery quicker
Synchronous approach takes less replication time
2/22/2019

34. Ordering and Performance
Faulty Broker
Replica N
Not much delay caused by the re-ordering
Ordered by source using sequence ID S
2/22/2019

35. Conclusions
Defined primitives for content-based pub/sub overlay transformation
Classified disruptions in terms of violation of functional and non-functional messaging properties
Quantified the trade-offs of the overlay transformation protocols
Demonstrated small overhead of our protocols
2/22/2019

36. Thank You!