Download presentation
Presentation is loading. Please wait.
1
CIS007-3 Comparative Integrated Systems
Communication including Message Oriented Middleware (Remote Procedure Call, Message and Stream-Oriented Communication) Sue Brandreth
2
Outline Introduction Fundamentals: Layered Protocols, Middleware, Types of Communication (Remote Procedure Call (RPC)) Message-Oriented Communication Transient communication Persistent communication (Stream-Oriented Communication)
3
Introduction
4
Review In a distributed system, processes
run on different machines exchange information through message passing Successful distributed systems depend on communication models that hide or simplify message passing
5
Interprocess Communication
Modern distributed systems consist of thousands or even millions of processes scattered across a network such as the Internet To study distributed systems we need to examine how processes on different machines exchange information Challenge: the underlying network is unreliable! Interprocess communication is at the heart of all distributed systems
6
Interprocess Communication
How can processes on different remote machines exchange information? No primitives based on shared memory (like in non-distributed parallel systems) Communication in distributed systems Based on low-level message passing Offered by the underlying network Harder than using shared memory
7
Three Models for Communication
Three widely used models for communication: Remote Procedure Call (RPC) Hide most of the intricacies of message passing Ideal for client/server applications Message-Oriented Middleware (MOM) High-level message queuing model, similar to Communication does not follow the rather strict pattern of client/server interaction.
8
Three Models for Communication
Data Streaming (Stream Oriented Communication) Used in multimedia distributed systems Provides support for communication of continuous media, such as audio and video Stream: continuous flow of messages Subject to various timing constraints
9
Layered Protocols; Middleware; Types of Communication
Fundamentals
10
Layered Protocols: OSI Model
Physical: transferring bits Data link: Detect and correct errors, checksum, parity bit, CRC etc, group bits into frames Network: Routing, IP protocol (connectionless), IP packet Transport: Reliable connection, handle package losses, TCP (Connection-oriented) and UDP (connectionless) Session: Synchronisation, dialog control Presentation: Semantics Application: All application protocols (FTP, HTTP -> these are special purpose protocols, but HTTP became a more general purpose one) TCP/IP: All above Transport is Application Layers, interfaces, and protocols in the OSI model.
11
Layered Protocols: Headers
A typical message as it appears on the network.
12
What is Middleware? An application that logically lives (mostly) in the application layer Contains many middleware protocols General purpose Application independent “Glue” between actual applications and lower-level layers Middleware is an application that logically lives (mostly) in the application layer, but contains many general-purpose protocols that warrant their own layers, independent of other, more specific applications. Middleware protocols are used for establishing various middleware services, e.g., authentication protocols that are not closely tied to any specific application, but can be integrated into a middleware system as a general service. Authorisation protocols tend to have a general, application-independent nature. The distributed locking protocol by which a shared resource can be protected against simultaneous access by a collection of processes being distributed across multiple machines.
13
What is Middleware? Provides a communication infrastructure for application On top of a distributed system Provides an API Data transformation Error handling Localization and identification of client and server Middleware is an application that logically lives (mostly) in the application layer, but contains many general-purpose protocols that warrant their own layers, independent of other, more specific applications. Middleware protocols are used for establishing various middleware services, e.g., authentication protocols that are not closely tied to any specific application, but can be integrated into a middleware system as a general service. Authorisation protocols tend to have a general, application-independent nature. The distributed locking protocol by which a shared resource can be protected against simultaneous access by a collection of processes being distributed across multiple machines.
14
Types of Middleware Remote Procedure Call (RPC)
Historic interest Object-Oriented Middleware (OOM) Java RMI CORBA Message-Oriented Middleware (MOM) Java Message Service IBM MQSeries Web Services Event-Based Middleware Transaction Processing Monitors (TPM) Object Request Brokers (ORB)
15
Message Oriented Middleware
Message Oriented Middleware (or “MOM”) is one particular form of middleware, which is capable of facilitating the transportation of asynchronous messages from one component to another.
16
Middleware Protocols Used for establishing various middleware services
Example Middleware protocols: Authentication and authorisation protocols Not closely tied to any specific application, but a general service Distributed commit protocol All processes carry out an operation or the operation is not carried out at all Distributed locking protocol Shared resource can be protected against simultaneous access by a collection of processes being distributed across multiple machines Middleware is an application that logically lives (mostly) in the application layer, but contains many general-purpose protocols that warrant their own layers, independent of other, more specific applications. Middleware protocols are used for establishing various middleware services, e.g., authentication protocols that are not closely tied to any specific application, but can be integrated into a middleware system as a general service. Authorisation protocols tend to have a general, application-independent nature. The distributed locking protocol by which a shared resource can be protected against simultaneous access by a collection of processes being distributed across multiple machines.
17
Middleware Communication Protocols
Support high-level communication services, ie. Call a procedure or invoke an object on a remote machine (in a highly transparent way) Message-passing services (comparable to those offered by the transport layer). Reliable multicast-services Scale to thousands of receivers across a wide-area network Must be implemented with application requirements in mind (hence in Middleware rather than in transport layer) Middleware communication protocols support high-level communication services, e.g., Protocols that allow a process to call a procedure or invoke an object on a remote machine in a highly transparent way. Protocols that offer message-passing services (comparable to those offered by the transport layer). Protocols that help offer reliable multicast services that scale to thousands of receivers across a wide-area network. The reason not to put them into transport layer and instead to keep them in a higher (middleware) layer is due to the fact that reliable multicasting services that guarantee scalability can be implemented only if application requirements are taken into account. The original transport services may also be offered as a middleware service, without being modified.
18
Adapted Communication Reference Model
Slightly adapted reference model. Session and Presentation have been replaced An adapted reference model for networked communication.
19
COMMUNICATION MODELS
24
Classification based on type of content:
Events Commands Data Streams
30
QUIZ TIME
31
QUIZ TIME
32
QUIZ TIME
33
QUIZ TIME
34
QUIZ TIME
35
QUIZ TIME
36
QUIZ TIME
37
QUIZ TIME
38
Types of Communications
Middleware offers the various alternatives in communication to applications Types of communication can be classified along several dimensions: Persistent vs. transient Synchronous vs. asynchronous Discrete vs. streaming Various combinations are possible
39
Persistent vs. Transient Communication
Persistent communication Message stored by the communication middleware as long as it takes to deliver it Sender can terminate after executing send. Sending application does not need to continue execution after submitting the message (allows for asynchronous communication) Receiving application need not be executing when the message is submitted. Receiver will get message next time it runs. Example: systems The middleware offers the various alternatives in communication to applications (persistent vs transient; asynchronous vs synchronous): Persistent communication A message that has been submitted for transmission is stored by the communication middleware (at one or several of its storage facilities) as long as it takes to deliver it to the receiver. Consequently, the sending application does not need to continue execution after submitting the message. Likewise, the receiving application need not be executing when the message is submitted. Example: systems Transient communication A message is stored by the communication system only as long as the sending and receiving applications are executing. Consequently, if the middleware cannot deliver a message due to a transmission interrupt, or because the recipient is currently not active, the message will simply be discarded. Example: transport-level communication services (being built upon traditional store-and-forward routers – if the router can’t deliver, it drops the message)
40
Persistent vs. Transient Communication
Message is stored by the communication system only as long as the sending and receiving applications are executing. Communication errors or inactive receiver cause the message to be discarded Transport-level communication is transient If the middleware cannot deliver a message due to a transmission interrupt, or because the recipient is currently not active, the message will simply be discarded Example: transport-level communication services (being built upon traditional store-and-forward routers – if the router can’t deliver, it drops the message) The middleware offers the various alternatives in communication to applications (persistent vs transient; asynchronous vs synchronous): Persistent communication A message that has been submitted for transmission is stored by the communication middleware (at one or several of its storage facilities) as long as it takes to deliver it to the receiver. Consequently, the sending application does not need to continue execution after submitting the message. Likewise, the receiving application need not be executing when the message is submitted. Example: systems Transient communication A message is stored by the communication system only as long as the sending and receiving applications are executing. Consequently, if the middleware cannot deliver a message due to a transmission interrupt, or because the recipient is currently not active, the message will simply be discarded. Example: transport-level communication services (being built upon traditional store-and-forward routers – if the router can’t deliver, it drops the message)
41
Persistent vs. Transient Communication
Persistent communication: A message is stored at a communication server as long as it takes to deliver it to the receiver. Transient communication: A message is discarded by a communication server as soon as it cannot be delivered at the next server, or at the receiver.
42
Asynchronous vs. Synchronous Communication
Asynchronous communication Sender continues execution immediately after it has submitted its message to the middleware software Message is (temporarily) stored immediately by the middleware upon submission Non-blocking Synchronous communication Sender is blocked until its request is known to be accepted ie. The OS or middleware notifies acceptance of the message, or The message has been delivered to the receiver, or The receiver processes it and returns a response Asynchronous communication A sender continues immediately after it has submitted its message for transmission. That is, the message is (temporarily) stored immediately by the middleware upon submission. Synchronous communication The sender is blocked until its request is known to be accepted Essentially, three points where synchronisation can take place. The sender may be blocked until the middleware notices that it will take over transmission of the request. its request has been delivered to the intended recipient its request has been fully processed, up to the time that the recipient returns a response.
43
Synchronous Communication
Some observations: Client/Server computing is generally based on a model of synchronous communication: Client and server have to be active at the time of communication Client issues request and blocks until it receives reply Server essentially waits only for incoming requests, and subsequently processes them Drawbacks of synchronous communication: Client cannot do any other work while waiting for reply Failures have to be dealt with immediately (the client is waiting) In many cases the model is simply not appropriate (mail, news)
44
Synchronous Communication: Three Synchronization Points
Three possible points of synchronisation: The sender may be blocked until the middleware notices that it will take over transmission of the request; its request has been delivered to the intended recipient; its request has been fully processed, up to the time that the recipient returns a response
45
Asynchronous Communication: Messaging
Message-oriented middleware: Aims at high-level asynchronous communication: Processes send each other messages, which are queued Sender need not wait for immediate reply, but can do other things Middleware often ensures fault tolerance
46
Discrete vs. Streaming Communication
All communicating parties communicate by messages Each message forms a complete unit of information Streaming Messages are related to each other By the order they are sent By their temporal relationship One-way communication; a “session” consists of multiple messages from the sender that are related either by send order (TCP streams), temporal proximity (multimedia streams), etc. Discrete: all communicating parties communicate by messages, each message forming a complete unit of information. Streaming: involves sending multiple messages, one after another, which are related to each other by the order they are sent and by their temporal relationship.
47
Combinations Various combinations of persistent/transient with asynchronous/synchronous communications exist in practice, ie. Persistence combined with synchronisation at request submission a common scheme for many message-queuing systems Transient communication with synchronisation after the request has been fully processed a scheme for Remote Procedure Calls
48
Messaging Combinations
49
Important….. RPC supports access transparency, but is not always appropriate RPC is usually synchronous, and is transient Message-oriented communication is more flexible
50
Message-oriented Communication
51
Message-Oriented Middleware (MOM)
Communication using messages Messages stored in message queues Message servers decouple client and server Various assumptions made about message content Client App. Server App. Message Servers local message queues message queues local message queues Network Network Network
52
Forms of MOM Forms of MOM: Message Queuing Publish-Subscribe
53
Message-Oriented Persistent Communication
Message Queuing Systems or Message-Oriented Middleware (MOM) Important class of message-oriented middleware services Persistent asynchronous communication Offer intermediate-term storage capacity for messages No need for either the sender or receiver to be active during message transmission Support message transfers that are allowed to take minutes.
54
Properties of MOM Asynchronous interaction
Client and server are only loosely coupled Messages are queued Good for application integration Support for reliable delivery service Keep queues in persistent storage Processing of messages by intermediate message server(s) May do filtering, transforming, logging, … Networks of message servers Natural for database integration
55
Message-Queuing Model
Basic idea: applications communicate by inserting messages in specific queues Messages forwarded over a series of communication servers before being finally being delivered to the destination Receiver can be down when message was sent No need for the sender to be executing when a message is picked up by the receiver Sender is offered the guarantee that its message will eventually be inserted into the recipient’s queue, but no guarantee about when Basic idea: applications communicate by inserting messages in specific queues These messages could be forwarded over a series of communication servers before being finally delivered to the destination, even if the destination was down when the message was sent The sender is offered the guarantee that its message will eventually be inserted into the recipient’s queue, but no guarantee about when. No need for the receiver to be executing when a message is being sent to its queue. Likewise, no need for the sender to be executing when a message is picked up by the receiver
56
Message-Oriented Middleware
Persistent. Processes communicate through message queues Queues are maintained by the message-queuing system Sender appends to queue, receiver removes from queue Neither the sender nor receiver needs to be on-line when the message is transmitted
57
Synchronous Communication
58
Synchronous Communication
Also known as ‘Rendevouz’ Tightly coupled Pros: Client is informed about message delivery No buffering necessary Cons: Sender and receiver have to run on the same time Direct connection necessary Client is blocked Lower degree of parallelity
59
Asynchronous (Persistent) Communication
60
Asynchronous (Persistent) Communication
Loosely coupled Pros: Higher degree of parallelity Enables faster sending than transmitting May compensate speed deviation between sender and receiver (but not speed differences) Sender and receiver have not to run on the same time Cons: Complicates synchronisation Sender has no information about message delivery Possible buffer overflows
61
Asynchronous (Persistent) Communication
Asynchronous data transport between processes Data is transmitted in messages Based on queues Sender puts messages into a queues Message is transmitted to receiver Receiver reads message from queues Usually FIFO
62
Queues Are named message destination that serve as an intermediate between sender and receiver Allow processes to execute and fail independently Can mask process failures and communication failures
63
Message Priorities – Highest Priority First
64
Message Priorities – Weighted Fair Scheduling
65
Message Oriented Middleware
Additional / Optionally Support of multiple Messaging models Queue management Connection Management Quality-of-Service (QoS) Data transformation
66
Queue Manager Is a specialized kind of database
Provides queuing functionality via an API to applications Provides means for administration Creation/deletion of queues Allows to start and to stop queues Alter properties of existing queues Allows monitoring of performance, failures, and recoveries Often queue managers can be configured to forward messages to other queue managers
67
Additional / Optional Queue Manager Functionality
Often called Message Broker Message Transformation Primary function = Reformatting data/information “Rosetta stone” of the system -> universal translation Understands the structure/formats of sources and targets Intelligent Routing Flow control See ‘Content based routing’ Message Dictionary Rules Engine Repository Filtering Access control
68
MOM – Message Oriented Middleware
69
Time Decoupling by Queues
70
Location Decoupling by Queues
71
Types of Message Based Communication
72
Asynchronous Persistent
Most important Quality of Service is guaranteed delivery Possible failures: Receiver is not running. No connection to receiver. Guaranteed delivery by persistent storage on intermediate nodes ie. as a file or in database If the receiver is up again or if the connection is available, message is delivered. Guaranteed delivery but not always in time.
73
Queuing Messaging Pattern
One-to-One One sender, one receiver Point-to-Point Message Passing (MP) One-to-Many One sender, many receivers Many-to-One Many senders, one receiver Many-to-Many Many senders, many receivers -> Publish and Subscribe
74
Message Passing / Point-to-Point
75
Publish and Subscribe
76
Publish/Subscribe Model
One important and common use of a Message Broker is to help form the Publish/Subscribe model (paradigm) In such a model, a broker is responsible not only for converting messages but also for matching applications based on the messages that are being exchanged Applications may publish a message on topic X by sending it to the broker Applications with interest in messages on topic X can subscribe to these messages, and will then receive them from the broker
77
Publish and Subscribe Example
78
Request/Reply with Queues
Correlation ID used to match request with corresponding reply at the client
79
Message Based Systems / MOM Products
Canonical example: IBM MQSeries (Sockets) – Berkeley Sockets OpenJMS Apache ActiveMQ (maintained by Fusesource) IBM WebSphere MQ (IBMs MQSeries) TIBCO TIBCO Rendezvous Oracle Advanced Queuing Microsoft Microsoft Message Queuing (MSMQ) SUN Sun ONE Message Queue (JMS) RabbitMQ JBoss HornetQ
80
What is IBM WebSphere? WebSphere provides software for SOA environments that enables dynamic, interconnected business processes, and delivers highly effective application infrastructures for all business situations. WebSphere is IBM's application and integration software platform, and includes the entire middleware infrastructure, including servers, services, and tools, needed to create, deploy, run, and monitor round-the-clock, enterprise-wide web applications and cross-platform, cross-product solutions.
81
WebSphere MQ (IBM) Cross Platform
Dominant Messaging software – 70% of market Messaging API same on all platforms Guaranteed one-time delivery Two-Phase Commit Wide EAI industry support
82
MQ Series API (Basic) Connect to a Queue Manager Open a queue
Put or get messages Close a queue Commit or roll back Disconnect
83
MQ Series API (Advanced Features)
Triggering – automatically starting an application to process a message IMS and CICS Bridges – reusing legacy transactions without modification Confirmation of message arrival, delivery Grouping of messages Load balancing
84
Local Queuing
85
Distributed Queuing
86
MQSeries
87
Distributed Queuing
88
MS Message Queue (MQ) Deployed in its Windows Server operating systems since Windows NT 4 and Windows 95 .NET: System.Messaging Create Queues via computer administration Queue name: „Computername\private$\Queuename“
89
What is MSMQ? Message Queuing (MSMQ) technology enables applications running at different times to communicate across heterogeneous networks and systems that may be temporarily offline. Applications send messages to queues and read messages from queues. The following illustration shows how a queue can hold messages that are generated by multiple sending applications and read by multiple receiving applications.
90
MSMQ Features Point-to-point messaging Persistent Transactional
Trigger Normal send/receive APIs Very similar to IBM MQ Series Supports IP multicast as well…
91
MSMQ Queue Types Application Queues Destination Queues Private Public
ComputerName\private$\QueueName\ private$\QueueName Public Response Queues Temporary / Local Queues
92
MSMQ Queue Types Application Queues Destination Queues Response Queues
Private ComputerName\private$\QueueName\ private$\QueueName Public Response Queues Temporary / Local Queues System Queues Mediation Queues Dead Letter Queues Journal Queues
93
Amazon SQS Part of the AWS Cloud Computing Infrastructure
Amazon Elastic Compute Cloud (Amazon EC2) Amazon Simple Storage Service (Amazon S3) Amazon Elastic BlockStore Amazon CloudFront Amazon SimpleDB … Amazon Simple Queue Service (Amazon SQS™)
94
Amazon SQS
95
Amazon SQS Features Multiple consumers and producers
Configurable Message Queues Unlimited number of queues and messages SOAP – APIs Language support Java, PHP, Ruby, .NET At-Least-Once delivery No guarantee of message order
96
Amazon SQS Operations CreateQueue ListQueues DeleteQueue SendMessage
ReceiveMessage DeleteMessage SetQueueAttributes GetQueueAttributes
97
Amazon SQS Queues and Message Identifier – Queue URL
– Message ID – Receipt Handler MbZj6wDWli+JvwwJaBV+3dcjk2YW2vA3+STFFljTM8tJJg6HRG6PYSasuWXPJB+Cw Lj1FjgXUv1uSj1gUPAWV66FU/WeR4mq2OKpEGYWbnLmpRCJVAyeMjeU5ZBdtcQ+QE auMZc8ZRv37sIW2iJKq3M9MFx1YvV11A2x/KSbkJ0=
98
Amazon SQS
99
Amazon SQS
100
Java Message Service (JMS)
Interface specification by Sun Microsystems to define access to MOM Defines syntax and semantics First published in 1988 Current specification: ??? JMS supports: Point-to-Point Publish-and-Subscribe Request/Reply
101
Java Message Service (JMS)
Two roles: JMS Provider = MOM Server = Queue Manger = Message Broker JMS Client = Sender = Receiver Messages Administrative Objects published by the JMS provider using JINI ConnectionFactory DestinationFactory
102
JMS Features Transacted sessions Persistent/nonpersistent delivery
Durable / non durable subscriber TTL Message Priorities Message selectors SQL-like syntax to access header fields: subscriber = session.createSubscriber(topic, “priority > 6 AND type = ‘alert’ ”); JMS can be used as a transport layer for SOAP Message Consumption Synchronously Asynchronously
103
JMS Example
104
JMS Example - HelloWorld
// Use JNDI to get ConnectionFactory and Queue QueueConnectionFactory connectionFactory = QueueConnectionFactory) ctx.lookup ("QueueConnectionFactory"); // Create connection and session QueueConnection queueConnection connectionFactory.createQueueConnection(); QueueSession queueSession queueConnection.createQueueSession(...); Queue queue = (Queue) ctx.lookup ("myQueue"); QueueSender sender = queueSession.createSender(queue); // Create and send a message Message message = queueSession.createTextMessage(); message.setText("Hello World"); queueSender.send(message);
105
Apache ActiveMQ Leading Open Source messaging platform
Multi-Language Support Clients and Protocols from C, C++, C#, Ruby, Perl, Python, PHP Supported Java Standards: JMS 1.1, J2EE 1.4, JCA 1.5 Multi-Protocol Support: TCP, NIO, UDP, SSL, HTTP/S, STOMP
106
Apache ActiveMQ Features
Failover High Availability Clustering Wide area deployment Management Security
107
Apache ActiveMQ: Failover
failover:(tcp://broker1:61616,tcp://broker2:6 1616,tcp://broker3:61616)?initialReconnectDelay=100 failover://(tcp://masterhost:61616,tcp://slavehost:61616)?randomize=false
108
Apache ActiveMQ: Synchronous Consumer
public class MyConsumer { ... public void doCreateConsumer() { Destination destination = consumer.getSession().createQueue("JOBS." + job); MessageConsumer messageConsumer = consumer.getSession().createConsumer(destination); while ((message = consumer.receive(timeout)) != null) { processMessage(message); }
109
Apache ActiveMQ: Asynchronous Consumer
public class MyConsumer { ... public void doCreateConsumer() { Destination destination = consumer.getSession().createQueue("JOBS." + job); MessageConsumer messageConsumer = consumer.getSession().createConsumer(destination); messageConsumer.setMessageListener(new MyMessageListener(job)); }
110
Appendix
111
IBM MQSeries (1/3) Basic concepts: Message transfer:
Application-specific messages are put into, and removed from queues Queues always reside under the regime of a queue manager Processes can put messages only in local queues, or through an RPC mechanism Message transfer: Messages are transferred between queues Message transfer between queues at different processes, requires a channel At each endpoint of channel is a message channel agent (MCA) Setting up channels using lower-level network communication facilities (e.g., TCP/IP) (Un)wrapping messages from/in transport-level packets Sending/receiving packets
112
IBM MQSeries (2/3) Channels are inherently unidirectional
MQSeries provides mechanisms to automatically start MCAs when messages arrive, or to have a receiver set up a channel Any network of queue managers can be created; routes are set up manually (system administration)
113
IBM MQSeries (3/3) Routing: By using logical names, in combination with name resolution to local queues, it is possible to put a message in a remote queue Question: What’s a major problem here?
114
Message Broker Observation: Message queuing systems assume a common messaging protocol: all applications agree on message format (i.e., structure and data representation) Message broker: Centralized component that takes care of application heterogeneity in a message-queuing system: Transforms incoming messages to target format, possibly using intermediate representation May provide subject-based routing capabilities Acts very much like an application gateway
115
IBM MQSeries One-to-one reliable message passing using queues
Persistent and non-persistent messages Message priorities, message notification Queue Managers Responsible for queues Transfer messages from input to output queues Keep routing tables Message Channels Reliable connections between queue managers Messaging API: MQopen Open a queue MQclose Close a queue MQput Put message into opened queue MQget Get message from local queue
116
Java Message Service (JMS)
API specification to access MOM implementations Two modes of operation *specified*: Point-to-point one-to-one communication using queues Publish/Subscribe cf. Event-Based Middleware JMS Server implements JMS API JMS Clients connect to JMS servers Java objects can be serialised to JMS messages A JMS interface has been provided for MQ pub/sub (one-to-many) - just a specification?
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.