© C Xu, 1998,2007 Message-Oriented Persistent Communication.

Slides:



Advertisements
Similar presentations
DISTRIBUTED COMPUTING PARADIGMS
Advertisements

Dr. Kalpakis CMSC621 Advanced Operating Systems Communication.
Last Class: RPCs and RMI
Communication Chapter 2. Layered Protocols (1) Layers, interfaces, and protocols in the OSI model. 2-1 Physical: voltage, transmission rate, mechanical,
EECS122 - UCB 1 CS 194: Distributed Systems Remote Object Invocation, Message- Oriented Communications (Based on textbook slides) Computer Science Division.
A Study in JMS (Java Messaging Service) Chad Beaudin CS 522 Fall Semester 2002.
Persistent and Transient Objects Persistent objects continue to exist even if they aren’t in the address space of a server process Transient objects existence.
1 Java Networking – Part I CS , Spring 2008/9.
CS 582 / CMPE 481 Distributed Systems Communications (cont.)
Chapter 2  Communication 1 Communication Chapter 2.
A Study in JMS (Java Messaging Service) Chad Beaudin CS 522 Fall Semester 2002.
Computer Science Lecture 2, page 1 CS677: Distributed OS Last Class: Introduction Distributed Systems – A collection of independent computers that appears.
Univ. of TehranDistributed Operating Systems1 Advanced Operating Systems University of Tehran Dept. of EE and Computer Engineering By: Dr. Nasser Yazdani.
ECSE Software Engineering 1I HO 7 © HY 2012 Lecture 7 Publish/Subscribe.
Click to add text Introduction to z/OS Basics © 2006 IBM Corporation Chapter 15: WebSphere MQ.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved DISTRIBUTED SYSTEMS.
EEC-681/781 Distributed Computing Systems Lecture 6 Wenbing Zhao Department of Electrical and Computer Engineering Cleveland State University
WXES2106 Network Technology Semester /2005 Chapter 8 Intermediate TCP CCNA2: Module 10.
An adapted reference model for networked communication.
Communication Part II Message-Oriented Communication Chapter 4.
Communication. Asynchronous RPC (1) a)The interconnection between client and server in a traditional RPC b)The interaction using asynchronous RPC 2-12.
Messaging Passing Communication AOS Dan Huang. Why Need Message RPC and ROI hiding communication for transparency. Hiding communication is not an appropriate.
Message-Oriented Communication Synchronous versus asynchronous communications Message-Queuing System Message Brokers Example: IBM MQSeries 02 – 26 Communication/2.4.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved DISTRIBUTED SYSTEMS.
Messaging Technologies Group: Yuzhou Xia Yi Tan Jianxiao Zhai.
Process-to-Process Delivery:
Chapter 17 Networking Dave Bremer Otago Polytechnic, N.Z. ©2008, Prentice Hall Operating Systems: Internals and Design Principles, 6/E William Stallings.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved DISTRIBUTED SYSTEMS.
Communication. 4.1 Fundamentals 4.2 Remote Procedure Call 4.3 Message-oriented Communication 4.4 Stream-oriented Communication 4.5 Multicast Communication.
Client Server Technologies Middleware Technologies Ganesh Panchanathan Alex Verstak.
OS2-Sem 831, Rasool Jalili Communication Chapter 2.
Asynchronous Communication Between Components Presented By: Sachin Singh.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved DISTRIBUTED SYSTEMS.
1 Java Message Service Манин П Enterprise messaging Key concept: 1. Messages are delivered asynchronously 2. Sender is not required to wait for.
DISTRIBUTED COMPUTING PARADIGMS. Paradigm? A MODEL 2for notes
Communication Chapter 2.
Communication in Distributed Systems Cheng-Zhong Xu.
Chapter 4: Interprocess Communication‏ Pages
Distributed Systems Concepts and Design Chapter 4.
Chapter 2 Applications and Layered Architectures Sockets.
The Socket Interface Chapter 21. Application Program Interface (API) Interface used between application programs and TCP/IP protocols Interface used between.
Message Oriented Communication Prepared by Himaja Achutha Instructor: Dr. Yanqing Zhang Georgia State University.
3.1 Silberschatz, Galvin and Gagne ©2009Operating System Concepts with Java – 8 th Edition Chapter 3: Processes.
Hwajung Lee.  Interprocess Communication (IPC) is at the heart of distributed computing.  Processes and Threads  Process is the execution of a program.
 Communication Distributed Systems IT332. Outline  Fundamentals  Layered network communication protocols  Types of communication  Remote Procedure.
Distributed Computing Systems
Distributed (Operating) Systems -Communication in Distributed Systems- Computer Engineering Department Distributed Systems Course Assoc. Prof. Dr. Ahmet.
Computer Engineering Department Distributed Systems Course
AMQP, Message Broker Babu Ram Dawadi. overview Why MOM architecture? Messaging broker like RabbitMQ in brief RabbitMQ AMQP – What is it ?
Communication Chapter 2.
OS2-Sem 831, Rasool Jalili Communication Chapter 2.
OS2-Sem 831, Rasool Jalili Communication Chapter 2.
OS2-SUT– Sem ; R. Jalili Communication Chapter 2.
Communication Chapter 4. Introduction  Interprocess Communication is at the heart of all distributed systems.  Communication in distributed systems.
September 28, 2010COMS W41561 COMS W4156: Advanced Software Engineering Prof. Gail Kaiser
Communication Chapter 2. Layered Protocols (1) Layers, interfaces, and protocols in the OSI model. 2-1.
OS2-Sharif University of Technology - Sem ; R. Jalili Communication Chapter 2.
Slide No. 1 of 111 JMS ( J AVA M ESSAGE S ERVICE ) -Dhananjay Singh.
1 Prof. Leonardo Mostarda University of Camerino Distributed Systems – Remote Procedure Calls Prof. Leonardo Mostarda-- Camerino,
MQ Series Cross Platform Dominant Messaging sw – 70% of market
Last Class: Introduction
Last Class: RPCs and RMI
Message Based Communication
Process-to-Process Delivery:
Harjutus 3: Aünkroonne hajussüsteemi päring
Message Queuing.
MQ Series Cross Platform Dominant Messaging sw – 70% of market
Process-to-Process Delivery: UDP, TCP
Chapter 6 – Distributed Processing and File Systems
Exceptions and networking
Presentation transcript:

© C Xu, 1998,2007 Message-Oriented Persistent Communication

© C Xu, 1998,2007 Organization of Comm System Examples: --- Client/Outbound Mail Server/Inbound Server MPI --- Message Passing Layer Difference??

© C Xu, 1998,2007 Another Example of Persistent Comm u Persistent communication of letters back in the days of the Pony Express. Although the means of transportation and the means by which letters are sorted have changed, the mail principle remains the same

© C Xu, 1998,2007 Persistence and Synchronicity u Persistent vs Transient Communication –Persistent: submitted messages are stored by the comm. system as long as it takes to deliver it the receiver –Transient: messages are stored by the comm system as long as the sending and receiving app. are executing. u Asynchronous vs synchronous –Sync: a client is blocked until its message is stored in a local buffer at the receiving host, or actually delivered to the receiver –Async: A sender continues immediately after it has submitted its message for transmission. (The message is stored in a local buffer at the sending host, or otherwise at the first communication server.

© C Xu, 1998,2007 a) Persistent asynchronous communication E.g. b) Persistent synchronous communication (??) ( a weaker form: when the sender is blocked until its message is stored at receiver-side comm server )

© C Xu, 1998,2007 c) Transient asynchronous communication (e.g. UDP, asynchronous RPC). Transmission fails if the receiver is not executing at the time the msg arrives. d) Receipt-based transient synchronous communication. Sender is blocked until the msg is stored in a local buffer at the receiving host. (Example?)

© C Xu, 1998,2007 e) Delivery-based transient synchronous communication at message delivery. (Sender is blocked until the msg is delivered to the receiver) f) Response-based transient synchronous communication (e.g. RPC and RMI) (Sender is blocked until it receives a reply msg from the receiver)

© C Xu, 1998,2007 Different Forms of Communication u Persistent Asynchronous u Persistent Synchronous u Transient Asynchronous u Transient Synchronous –Receipt-based (weakest, e.g. async. RPC) –Delivery-based –Response-based (strongest, e.g. RPC) u Need for persistent comm. in middleware, large- scale system –Components of a large scale distributed system may not always be immediately accessible –Failure should be masked with a recovery procedure –But, persistent comm. incurs long delays

© C Xu, 1998,2007 Message-Oriented Transient Comm. u Socket is a communication endpoint to which an appl can write data to and from which incoming data can be read u Socket primitives for TCP/IP. PrimitiveMeaning SocketCreate a new communication endpoint BindAttach a local address to a socket ListenAnnounce willingness to accept connections AcceptBlock caller until a connection request arrives ConnectActively attempt to establish a connection SendSend some data over the connection ReceiveReceive some data over the connection CloseRelease the connection

© C Xu, 1998,2007 Berkeley Sockets u Connection-oriented communication pattern using sockets.

© C Xu, 1998,2007 MPI: Another Transient Comm u Basic message-passing primitives of MPI. PrimitiveMeaning MPI_bsendAppend outgoing message to a local send buffer of MPI runtime MPI_sendSend a message and wait until copied to local or remote buffer MPI_ssendSend a message and wait until receipt starts MPI_sendrecvSend a message and wait for reply MPI_isendPass reference to outgoing message, and continue MPI_issendPass reference to outgoing message, and wait until receipt starts MPI_recvReceive a message; block if there are none MPI_irecvCheck if there is an incoming message, but do not block

© C Xu, 1998,2007 Synchrony in MPI u MPI_bsend  Transient synchronous u MPI_send (blocking send) –Block the caller until the msg be copied to MPI runtime system at the receiver’s side;  Receipt-based transient comm –or until the receiver has initiated a receive operation  delivery-based transient comm u MPI_ssend  delivery-based u MPI_sendrecv  response-based

© C Xu, 1998,2007 Message Queuing Model u MQ: a software-engineering component used for interprocess comm. It utilizes a queue for messaging—the passing of control or of content. [Wikipedia] u MQ provides a Persistent Asynchronous comm protocol –Sender and receiver do not need to connect to the msg queue at the same time; msgs placed on the queue are stored until the recipient retrieves them. –Most msg queues have set limits on the size of data that can be transmitted in a single msg. Those with no such limits are often referred to as mailboxes.

© C Xu, 1998,2007 u Four combinations for loosely-coupled comm. using queues (sender and receiver can execute completely independently of each other) 2-26 MQ Model

© C Xu, 1998,2007 MQ: Interface to a Queue PrimitiveMeaning PutAppend a message to a specified queue (non-blocking) GetBlock until the specified queue is nonempty, and remove the first message PollCheck a specified queue for messages, and remove the first. Never block. Notify Install a handler to be called when a message is put into the specified queue. One-way Communication Primitives: Addressing: System wide unique destination queue Active message: The handler to be executed on arrival at the dest. Mobile object: messages in principle contains any data. How about objects?

© C Xu, 1998,2007 Msg Oriented Middleware (MOM) u Many impl function internally in OS or appl –In UNIX: msgctrl(), msgget(), msgsnd() u Distributed impl across different systems for msg passing –Enhanced fault resilient: msgs don’t get lost even in the event of a system failure. u Examples: –IBM’s WebSphere MQ (MQ series), –Oracle Advaced Queueing (AQ) –Microsoft’s MSMQ –Sun’s Java API: Java Message Service (JMS) since 2005 –Amazon’s Simple Queue Service for building automated workflow (aws.amazon.com/sqs)

© C Xu, 1998,2007 MOM (Cont’) u Msgs are forwarded over comm servers and eventually delivered to dest, even if it was down when the msg was sent. u Each app has its own queue and a queue can be read only by its assoc. app. u Multiple apps can share a single queue u Guaranteed that a msg will eventually be inserted in the recipient’s queue, but no guarantee about when, or if the msg will actually be read. –Immediate-term storage capacity for messages u Messaging domain: –Point-to-point vs publish/subscribe

© C Xu, 1998,2007 General Architecture: u Source Queue, Destination Queue, or 3rd-party messaging service (Amazon) u Mapping of queue-level addressing to network-level addressing. (analogous to DNS in system) u Queue manager u Special manager working as routers: forward msg between queue managers

© C Xu, 1998,2007 Queue Manager, Router, and Overlay Network 2-29 Static vs dynamic routing with routers Multicasting with routers

© C Xu, 1998,2007 Message Brokers (e.g. Amazon’s SQS u Integration of legend applications with new ones to form a single coherent distributed information system u Message broker acts as an appl-level gateway to convert incoming msg to a format that can be understood by the dest app. u The general organization of a message broker in a msg-queuing system.

© C Xu, 1998,2007 Amazon Simple Queue Service u Amazon’s SQS offers a reliable, highly scalable, hosted queue for storing messages as they travel between computers. (aws.amazon.com/sqs) u By using Amazon SQS, developers can simply move data between distributed components of their applications that perform different tasks, without losing messages or requiring each component to be always available. u Make it easy to build an automated workflow u Pricing: $0.01 for 10K SQS requests.

© C Xu, 1998,2007 Basic Queue Requests u CreateQueue: Create queues for use with your AWS account. u ListQueues: List your existing queues. u DeleteQueue: Delete one of your queues. u SendMessage: Add any data entries to a specified queue. u ReceiveMessage: Return one or more messages from a specified queue. u DeleteMessage: Remove a previously received message from a specified queue. u SetQueueAttributes: Control queue settings like the amount of time that messages are locked after being read so they cannot be read again. u GetQueueAttributes: See information about a queue like the number of messages in it.

© C Xu, 1998,2007 Example: IBM WebSphere MQ u Launched in 1992, previously known as MQ series u De-facto standard for messaging across platforms u Queue Manager: handles storage, timing issues, triggering, and all other functions not directly actual movement of data u QM comm with outside via local Binding, or remote Client connection

© C Xu, 1998,2007 Message Channels u MC is a unidirectional, reliable connection between a sending and a receiving queue manager u Each end is managed by a Message Channel Agent (MCA) –check send queue, wrap messages into transport-level packets, send to associated receiving MCA –listening for incoming packets, unwrap them, store into queues AttributeDescription Transport typeDetermines the transport protocol to be used FIFO deliveryIndicates that messages are to be delivered in the order they are sent Message lengthMaximum length of a single message Setup retry count Specifies maximum number of retries to start up the remote MCA Delivery retriesMaximum times MCA will try to put received message into queue

© C Xu, 1998,2007 Message Transfer u Address: queue manager + destination queue; u Each queue manager has a system-wide unique id; Local aliases for queue manager names u Routing tables: (Dest, SendQueue) u Transfer along a channel can take place only if both MCAs are up and runing

© C Xu, 1998,2007 Message Queue Interface u Primitives available in an MQSeries MQI PrimitiveDescription MQopenOpen a (possibly remote) queue MQcloseClose a queue MQputPut a message into an opened queue MQgetGet a message from a (local) queue MQSerie also provides facilities to signal applications when msgs arrive. For more info, see

© C Xu, 1998,2007 Messaging Domain u Point-to-Point u Publish/Subscribe –Clients address messages to a topic

© C Xu, 1998,2007 Publish/Subscribe Messaging u Request/Reply (RPC, MRI) model has limitations in two types of applications –Update of state info that changes over time »Send requests regularly for update of the new info. NOT efficient –Action in response to event occurrence: »Send event info to ALL nodes who are interested in u Publish/Subscribe messaging –Any number of customers of info can receive messages that are provided by one or more producers of the info –Publisher: producer of the info –Subscriber: consumer of the info –Topic (or named logical channel) to be subscribed by consumers to register their interests –Publish/subscribe broker: track subscriptions to individual topics and provide facilitates for a publisher to publish msgs on a given topic u Topic-based vs content-based vs hybrid systems

© C Xu, 1998,2007 JMS Programming u Available in Java System Message Queue V3.6 u Interfaces: –ConnectionFactory »Administrated object used to create a connection to JMS provider »Defined by administrator, rather than programmer –Connection »A conn represents a comm link between app and msg server »Administrated object –Destination: where msg is delivered and consumed »Queue or topic –MessageConsumer –MessageProducer –Message –Session: Single threaded context for sending and receiving messages u

© C Xu, 1998,2007 JMS Programming Model

© C Xu, 1998,2007 Producer (1/3) injects resources for a connection factory, queue, and topic private static ConnectionFactory private static Queue private static Topic topic; Assigns either the queue or topic to a destination object, based on dest type : Destination dest = null; try { if (destType.equals("queue")) { dest = (Destination) queue; } else { dest = (Destination) topic; } } catch (Exception e) {}

© C Xu, 1998,2007 Producer (2/3) Creates a Connection and a Session: Connection connection = connectionFactory.createConnection(); Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE); Creates a MessageProducer and a TextMessage: MessageProducer producer = session.createProducer(dest); TextMessage message = session.createTextMessage(); Sends one or more messages to the destination: for (int i = 0; i < NUM_MSGS; i++) { message.setText("This is message " + (i + 1)); System.out.println("Sending message: " + message.getText()); producer.send(message); }

© C Xu, 1998,2007 Producer (3/3) Sends an empty control message to indicate the end of the message stream: producer.send(session.createMessage()); Sending an empty message of no specified type is a convenient way to indicate to the consumer that the final message has arrived. Closes the connection in a finally block, automatically closing the session and MessageProducer: } finally { if (connection != null) { try { connection.close(); } catch (JMSException e) { } } }

© C Xu, 1998,2007 Async Consumer 1. injects resources for a connection factory, queue, and topic. 2. Assigns either the queue or topic to a destination object, based on the specified dest type. 3. Creates a Connection and a Session. 4. Creates a MessageConsumer. 5. Creates an instance of the TextListener class and registers it as the message listener for the MessageConsumer: listener = new TextListener(); consumer.setMessageListener(listener); 6. Starts the connection, causing message delivery to begin. 7. Listens for the messages published to the destination, stopping when the user types the character q or Q: System.out.println("To end program, type Q or q, " + "then "); inputStreamReader = new InputStreamReader(System.in); while (!((answer == 'q') || (answer == 'Q'))) { try { answer = (char) inputStreamReader.read(); } catch (IOException e) { } } 8. Closes the connection, which automatically closes the session and MessageConsumer.

© C Xu, 1998,2007 Async Consumer (2/2) When a message arrives, the onMessage method is called automatically. The onMessage method converts the incoming message to a TextMessage and displays its content. If the message is not a text message, it reports this fact: public void onMessage(Message message) { TextMessage msg = null; try { if (message instanceof TextMessage) { msg = (TextMessage) message; System.out.println("Reading message: " + msg.getText()); } else { System.out.println("Message is not a " + "TextMessage"); } } catch (JMSException e) } catch (Throwable t) { } }

© C Xu, 1998,2007 vs Message Queue u is a special case of Message Queue – aims at providing direct support for end users – makes direct use of the underlying transport services (SMTP over TCP); routing is left out –General MQ can satisfy a different set of requirements: » guaranteed message delivery, »message priorities, »logging facilities, »efficient multicasting, »load balancing, »fault tolerance, etc.

© C Xu, 1998,2007 Msg-Queuing Model in Naplet ?