Messaging Systems and Technology

Introduction Synchronous middleware doesn’t suit all applications Message Oriented Middleware (MOM) provides features like: asynchronous communications between processes store-and-forward capabilities transactional messaging

Basic Asynchronous Operations Send (dest, message) Receive ( target, message) receivesend queue

MOM usage Applications often need: deferred processing of some ‘slow’ transaction (eg printing an invoice) support efficient 1-to-many and many-to-many communications to send messages whether the server is available or not to provide event notification to a dynanic user community loosely coupled client-server systems

Example - Transactions

MOM Technologies Broadly two (somewhat intersecting) categories Message brokers (IBM, BEA, Microsoft, etc) Broadcast-based publish-subscribe (TIBCO, IONA) Both categories differ greatly in their features and capabilities

Some Application Examples Stock price update notification General workflow systems Application integration System management

Some Example Products TIBCO/Rendezvous IONA’s OrbixTalk IBM’s MQ Series (Publish-Subscribe)

TIB/Rendezvous Pioneers of broadcast/multicast publish- subscribe technology Publishers and subscribers communicate using subjects Pub Subject Sub Create/ Publish Register/ Subscribe

Subjects Hierarchical names identify a subject of interest /CSIRO /CSIRO/gossip /CSIRO/gossip/ADSaT /CSIRO/work/ADSaT Wildcards can be used /CSIRO/** /CSIRO/*/ADSaT

Multicast/Broadcast A published message is sent across the network only once All subscribers receive the same message IP Multicast or broadcast is used on LANs Important scalability issue as bandwidth/processor usage is low

LAN Architecture

WAN Architecture Filtered, Point-to-Point

Quality-of-Service (QOS) Reliable publisher informed after time-out period if message cannot be delivered to a subscriber Certified guaranteed delivery or both parties informed of failure to survive process failure messages can be logged to disk until specified time-out period expires

Programming Options C/C++/Java Self-describing message format TIB/ObjectBus layers a standard CORBA- compliant ORB upon the TIBCO protocols. TIBIOP provides broadcast messages in a CORBA environment.

Security Security is a problem in publish-subscribe multicast technology TIBCO Data Security (DS) product: publisher controls secure key distribution to subscribers using digital signatures subscribers use key to encrypt messages on a subject choice of encryption algorithm available

OrbixTalk Uses IP multicast in CORBA environment

Topics Communication based on topics (read subjects, wildcards, etc) 3 QOS levels: –otmcp: UDP IP multicast. Limits packet size to 1280 bytes and no guarantee of receipt. –otrmp: Augments IP multicast to ensure message delivery to all subscribers until configurable time-out expires –otsfp: OrbixTalk MessageStore daemon uses a store- and-forward protocol to provide guaranteed message delivery.

OrbixTalk Architecture

Architecture Issues Daemons can have hot backups to support failure of primary daemon Several MessageStore daemons can be used to facilitate load balancing: each OrbixTalk process only use one MessageStore Compaction utility must be scheduled to remove old messages

Security/WANs No specific security or WAN support security can be programmed using Orbix Transformers WANs traffic needs appropriately configured routers, or an IIOP bridge between LANs to be written

MQSeries PubSub MQSeries is probably most widely deployed MOM product Basic technology provides a message queue architecture: communication via shared queues managed by Queue Managers persistent/non-persistent messages transactional queue access with XA broad platform support

MQSeries PubSub 1 PubSub Broker per QM Topics interchanged by Brokers if needed LAN/WAN support

Architecture Brokers can be organized to communicate hierarchically USA WestEast HQ Asia HK

Features Typical hierachical topic names, wildcards, etc Streams can be used as higher level topic partitioning scheme DEFAULT stream per broker optional additional streams for specified topics broker allocates a thread to handle each stream

Quality-of-Service Persistent messages written to log file survive broker/QM failure Non-persistent messages lost if broker fails fast Message priority supported in queues

Other features MQ can act as an XA transaction manager Basic security provided, integrated with native operating system Security exits support integration with 3rd party products Other products, eg MQ Integrator

Performance Test Publishers and subscribers run on different machines Dual pentium 500Mhz NT 4.0 boxes, 0.5GB memory MQ QM/broker and Orbix daemons run on same machine as publishers Reliable, non-presistent protocols used

256 Byte Message

512 Byte Message

Interpretation Multicast technologies are fast! TIBCO rvd becomes bottleneck ‘Publisher too fast’ exceptions OrbixTalk generates large number of interrupts one per message per listener MQ scales well, but queue/broker architecture is inherently slower

Summary MOM provides excellent solution to many business problems Range of technologies available, with very different: architectures/technologies features performance Important to understand issues and select right one for your business