Replicated LevelDB on JBoss Fuse Jakub Knetl jknetl@redhat.com Red Hat 2018/5/14

Messaging systems Mechanisms for exchanging data (messages) Advantages: Clients can communicate asynchronously Wide set of (platform independent) clients Reliability Scalability What is messaging system Why to use it instead of direct data exchange

Basic concepts Message Header Body (content) Destination Clients Producer Consumer

Point to point comunication Queue destination One of consumers gets message Load is distributed across consumers Message is stored until some consumer receives it

Point to point communication Consumers Producer Queue

Publish subscribe communication Message is delivered to all subscribers When no subscriber connected message is thrown away Special case – durable subscriber

Publish/subscribe Subscribers Publisher Topic

JBoss Fuse Open-source ESB Messaging system based on Apache ActiveMQ Runs in OSGI container Integration framework Webservices Easy cluster deployment Lots of componetns Integrated in OSGI container WebConsole Fabric

JBoss Fuse

High availability Messaging systems processes business critical data Server must be accessible 24/7 Various mechanisms to ensure HA

Introduction to HA in ActiveMQ Group of brokers forms logically one broker Master broker Communicates with clients Slave brokers Passive Clients cannot connects to it Election mechanism Client reconnects in case of failure Message acknowledgment after the message is stored safely Important note: Multiple brokers forms logically one broker

Different topologies Shared JDBC master/slave Shared file system master/slave Replicated levelDB master/slave (shared nothing)

Shared JDBC You need shared database Locking on database Reliable, but rather slow Single point of failure

Shared JDBC Client Master Master Brokers Database

Shared file system Convenient when having SAN Faster than JDBC Needs file system with reliable locking! Single point of failure

Shared filesystem Client Master Master Brokers Shared FS

Replicated LevelDB Shared nothing No single point of failure! Servers coordinates themselves by exchanging messages (replication protocol) Needs Zookeeper server

Choosing master That's why ZooKeeper is important Brokers register themselves at zookeeper Zookeeper decides who will become master

Replication protocol Slave brokers connects to the master Messages are replicated to slaves Message acknowledgment after replicating at least to quorum of brokers (N/2 - 1).

Replicated LevelDB A B Master C Master

Demo