Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Chapter 4 Communication DISTRIBUTED SYSTEMS (dDist) 2014

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Plan Communication protocols –Lower-level and higher-level Communication services –Communication types –Remote Procedure Call –Message-oriented communication –Multicasting Concrete instances (next slide set) –Java sockets –Java Remote Method Invocation (RMI) –Java Message Service (JMS)

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved What is this? 1. Host name 2. Get IP Address 3. Create socket 4. Send byte stream 5. Receive byte stream Classic network API

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved What the Server Sent… HTTP/ OK Cache-Control: private, max-age=0 Date: Tue, 13 Jan :54:03 GMT Expires: -1 Content-Type: text/html; charset=ISO Set-Cookie: SS=Q0=RGlzdHJpYnV0ZWQgU3lzdGVtcw; path=/search Set-Cookie: PREF=ID=65e9063ff5abbe9b:TM= :LM= :S=8eXFF2qVWFnORQDj; expires=Thu, 13-Jan :54:03 GMT; path=/; domain=.google.com Set-Cookie: NID=18=uC_NCqha9TuaEScwxTiR-4tUbWBe0ZIacM7Y2gSPjKXsRafw0FEMd1- 9gKQ5ywD2ExWp5v3wBz5ofe_IiZbmHH_I8pVeDEeakm1lFpaVenZXBpHXUXLsJHvccoaLld2M; expires=Thu, 13-Jan :54:03 GMT; path=/; domain=.google.com Server: gws Transfer-Encoding: chunked 1dbe Distributed Systems - Google Search body{background:#fff;color:#000;margin:3px 8px}#gbar{height:22px;padding- left:2px}.gbh,.gbd{border-top:1px solid #c9d7f1;font- size:1px}.gbh{height:0;position:absolute;top:24px;width:100%}#gbi,#gbs{background:#fff;left:0;position:absolute;top:24px;visibility:hidden;z- index:1000}#gbi{border:1px solid;border-color:#c9d7f1 #36c #36c #a2bae7;z-index:1001}#guser{padding-bottom:7px !important}#gbar,#guser{font- size:13px;padding-top:1px all{.gb1,.gb3{height:22px;margin-right:.73em;vertical- align:top}#gbar{float:left}}.gb2{display:block;padding:.2em.5em}a.gb1,a.gb2,a.gb3{color:#00c !important}.gb2,.gb3{text- decoration:none}a.gb2:hover{background:#36c;color:#fff !important}.ts{border-collapse:collapse}.ts td{padding:0}.ti,.bl,form,#res h3{display:inline}.ti{display:inline-table}.fl:link,.gl a:link{color:#77c}a:link,.w,#prs a:visited,#prs a:active,.q:active,.q:visited{color:#00c}.mblink:visited,a:visited{color:#551a8b}a:active{color:red}.cur{color:#a90a08;font-weight:bold}.b{font- weight:bold}.j{width:42em;font-size:82%}.s{max-width:42em}.j font[size="-1"],.e.j{font-size:100%}.sl,.e.j font[size="-1"]{font-size:82%}#gb{text- align:right;padding:1px 0 7px;margin:0}.hd{position:absolute;width:1px;height:1px;top:-1000em;overflow:hidden}.f,.m,#tads h2,#mbEnd h2{color:#676767}.a,cite,.cite,.cite:link{color:green;font-style:normal}#mbEnd{float:right}h1,ol{margin:0;padding:0}li.g,body,html,.std,#tads h2,#mbEnd h2,h1{font-size:small;font-family:arial,sans-serif}#tads h2,#mbEnd h2,h1{font-weight:normal}#ssb,.clr{clear:both}#nav a,#nav a:visited,.blk a{color:#000}#nav.b a,#nav.b a:visited{color:#00c;font-size:medium}.ch{cursor:pointer}h3,.med{font-size:medium;font- weight:normal;padding:0;margin:0}.e{margin:.75em 0}.slk td{padding-left:40px;padding-top:5px;vertical-align:top}.slk div{padding-left:10px;text-indent:- 10px}#mbEnd cite{display:block;text-align:left}#mbEnd p{margin:-.5em 0 0.5em;text-align:center}#bsf,#ssb,.blk{border-top:1px solid #6b90da;background:#f0f7f9}#bsf,#ssb{margin:11px 0}#bsf{border-bottom:1px solid #6b90da}#ssb div{float:left;padding:4px 0 0;padding- left:4px;padding-right:.5em}#prs a,#prs b{margin-right:.6em}#ssb p{text-align:right;white-space:nowrap;margin:.1em 0;padding:.2em}#ssb{margin- top:0;padding:.1em}#mbEnd{background:#fff;padding:0;border-left:10px solid #fff;border-spacing:0;white-space:nowrap}#res{padding- right:1em}#tads{background:#fff8dd}#tads li{padding:0 3px 0 5px;margin:0}#tads.tam,#tads.tal{padding-top:12px}#mbEnd li{margin:1em 0;padding:0}.xsm{font-size:x-small}.sm{margin: px;padding:0}ol li{list-style:none}.sm li{margin:0}.gl,#bsf a,.nobr,#brs a{white- space:nowrap}#mbEnd.med{white-space:normal}.sl,.r{display:inline;font-weight:normal;margin:0}.r{font-size:medium}h4.r{font-size:small}.g{margin:1em 0}em{font-weight:bold;font-style:normal}em,b{text-decoration:inherit} window.google={kEI:"W_FsSY2lEsOe-gbNn- yNDQ",kEXPI:"17259",kHL:"en"};

Communication Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Goal: Establish a connection to service known to run on port 4 Servers (and clients) have a number of ports (2 16 ) Typically services run on known ports (http: 80) Service Application

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Layered Protocols Figure 4-1. Layers, interfaces, and protocols in the OSI model. The example

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Layered Protocols Physical layer –Concerned with transmitting bits in frames –Volts, bits/second, direction, … Data-link layer –Applies checksums to frames Network layer –Addressing enables routing Internet protocol suite examples Not specified Internet Procotol (IP), Internet Control Message Protocol (ICMP)

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Layered Protocols Transport layer –Reliability: handles lost packets and ordering –Connection-oriented/connection-less; –Communication between processes Session layer –Keeps track of who is communicating with whom –Synchronization facilities Presentation layer –Semantics –Transforms data from application layer to common format Application layer –Standard network applications Transmission Control Protocol (TCP), User Datagram Protocol (UDP) File Transfer Protocol (FTP), HyperText Transfer Protocol (HTTP) Internet protocol suite examples

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Layered Protocols Figure 4-2. A typical message as it appears on the network.

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Protocol layering example Browser process HTTPTCPLink1 IP Link1 IP Link2 Web server process HTTPTCPLink1 IP Physical Link 1Physical Link 2 Router

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved HTTP Protocol layering example Browser process TCP Link1 IP Link1 IP Link2 Web server process HTTP TCP Link1 IP Physical Link 1Physical Link 2 Router H Browser wants to request a page. Calls HTTP with the web address (URL). HTTP’s job is to convey the URL to the web server. HTTP learns the IP address of the web server, adds its header, and calls TCP.

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved HTTP Protocol layering example Browser process TCP Link1 IP Link1 IP Link2 Web server process HTTP TCP Link1 IP Physical Link 1Physical Link 2 H TCP’s job is to work with server to make sure bytes arrive reliably and in order. TCP adds its header and calls IP. (Before that, TCP establishes a connection with its peer.) T Router

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved HTTP Protocol layering example Browser process TCP Link1 IP Link1 IP Link2 Web server process HTTP TCP Link1 IP Physical Link 1Physical Link 2 H IP’s job is to get the packet routed to the peer through zero or more routers. IP determines the next hop from the destination IP address. IP adds its header and calls the link layer (e.g., Ethernet) with the next hop address. T Router I

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved HTTP Protocol layering example Browser process TCP Link1 IP Link1 IP Link2 Web server process HTTP TCP Link1 IP Physical Link 1Physical Link 2 H The link’s job is to get the packet to the next physical box (here a router). It adds its header and sends the resulting packet over the “wire”. T Router IL1

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved HTTP Protocol layering example Browser process TCP Link1 IP Link1 IP Link2 Web server process HTTP TCP Link1 IP Physical Link 1Physical Link 2 H The router’s link layer receives the packet, strips the link header, and hands the result to the IP forwarding process. T Router I

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved HTTP Protocol layering example Browser process TCP Link1 IP Link1 IP Link2 Web server process HTTP TCP Link1 IP Physical Link 1Physical Link 2 H The router’s IP forwarding process looks at the destination IP address, determines what the next hop is, and hands the packet to the appropriate link layer with the appropriate next hop link address. T Router I

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved HTTP Protocol layering example Browser process TCP Link1 IP Link1 IP Link2 Web server process HTTP TCP Link1 IP Physical Link 1Physical Link 2 H The packet goes over the link to the web server, after which each layer processes and strips its corresponding header. T Router IL2 HTI HT H

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Plan Communication protocols –Lower-level and higher-level Communication services –Communication types –Remote Procedure Call –Message-oriented communication –Multicasting Concrete instances (next time) –Java sockets –Java Remote Method Invocation (RMI) –Java Message Service (JMS)

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Middleware A service for applications –Not an end goal in itself No single “right way” of providing the service –Some applications use non-interchangeable approaches Application-independent enough that many applications can use the same approach Semi-generic services like –Authentication –Remote Method Invocation –Message Queuing Services –Mutual Exclusion –Database systems

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Middleware Protocols Figure 4-3. An adapted reference model for networked communication. Includes Session & Presentation

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Types of Communication (1/2) Persistent –Once scheduled for delivery a message is guaranteed to be delivered / be available in the future –Example: Transient –A message is only kept by the system as long as the session is running –Example: UDP/IP

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Types of Communication (2/2) Asynchronous –The sender continues right after scheduling a transmission –Allows the parties to be active a different times –Example: Synchronous –The sender is blocked until the message is known to be delivered Maybe even until a reply is returned –Requires all parties present at same time –Example: HTTP

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Remote Procedure Call (RPC) Pre-RPC: Most applications were built directly over the Internet primitives RPC introduced by Birrell and Nelson in 1985 –Their idea: mask distributed computing system using an access-transparent abstraction Looks like normal procedure call Hides all aspects of distributed interaction Supports an easy programming model Today, RPC is the core of many distributed systems –We will look at Java RMI as an example

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Conventional Procedure Call Figure 4-5. (a) Parameter passing in a local procedure call: the stack before the call to read. (b) The stack while the called procedure is active. count = read(fd, buf, nbytes)

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Remote Procedure Call The procedure call is executed on a remote server instead of locally –Maybe to computationally unload clients –Maybe for maintainability –Maybe because most of the data needed for the call is on the server Client stub: Same parameters as the conventional function, but request the execution on the server Server stub: Will do procedure calls for clients and return the result

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Client and Server Stubs Figure 4-6. Principle of RPC between a client and server program.

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Passing Value Parameters (1) Figure 4-7. The steps involved in a doing a remote computation through RPC.

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Parameter Passing? Call-by-value –Value is copied to stack –Changing actual parameter does not change original value Call-by-reference –Pointers to values are put on stack –Changing actual parameter changes original value Call-by-copy/restore –Value is copied to stack –Stack contents copied back to original value Local Java primitive types, Java RMI when arguments are Serializable Local Java reference types, Java RMI when arguments implement the tagging interface Remote

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Marshalling Different operating systems use different representations of even basic values –Example: Intel uses little endian and SPARC uses big endian This requires Marshalling: –Translation between the format of the operating systems of the client and the server, or –Translation to and from a common network format

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved A Marshalling Problem (1/3) Word Sending [ (int)5, ”Jill” ] from 32-bit Intel to 32-bit SPARC 5, 0, 0, 0, J, I, L, L 83,886,080 JILL 5

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved A Marshalling Problem (2/3) Figure 4-8. (c) The message after being inverted 5 LLIJ

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved A Marshalling Problem (3/3) This problem shows that Marshalling must occur at a level where the semantics of bits is still known In lack of a standard presentation level for the Internet protocol stack it must be done in the middleware level Java will automatically Marshall any object implementing the tagging interface Serializable

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Synchronous RPC Figure (a) The interaction between client and server in a traditional RPC.

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Asynchronous RPC (1/3) Figure (b) The interaction using asynchronous RPC.

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Asynchronous RPC (2/3) Figure A client and server interacting through two asynchronous RPCs.

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Asynchronous RPC (3/3) Just the distributed version of the software architectural style known as –Hollywood Principle –Don’t-call-us-we-will-call-you –Call-back style

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Message-Oriented Communication RPC is great, but what if –Client and server are not running on the same time? –We cannot wait for the server to acknowledge? Use messaging as communication type

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Transient/Persistent Messaging Transient –No support for persistence of messages sent –Sockets More next time –Message-Passing Interface (MPI) Specialized for high-speed highly connected communication like in server cluster Persistent –Sent messages stored in queue or on a topic, delivered upon request –Basically a distributed queue –Message queuing systems

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Message-Queuing Model (1) Figure Four combinations for loosely- coupled communications using queues.

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Message-Queuing Model (2) Figure Basic interface to a queue in a message-queuing system.

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved General Architecture of a Message- Queuing System (1/3) Messages are put into local source queues –Typically non-blocking Messages are routed between queues by queue manages Message are removed from destination queues when they reach them –Using polling or callback

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved General Architecture of a Message- Queuing System (2/3) Figure The relationship between queue- level addressing and network-level addressing.

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved General Architecture of a Message- Queuing System (3/3) Figure The general organization of a message- queuing system with routers.

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Message Brokers: Marshalling++ Figure The general organization of a message broker in a message-queuing system.

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Multicasting Multicasting = Same message to group of receiver Multicasting is often a convenient abstraction in communication –Supported by broadcast on local network –Little support for multicasting on Internet Application-level multicasting –Based on overlay networks –Often done along trees Epidemic dissemination –Probabilistic

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Overlay Construction Figure The relation between links in an overlay and actual network-level routes.

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Chord and Multicasting Generate identifier for group –mid (10) Find multicast root –root := lookup(mid) (12) Nodes join group (4, 1) –Find forwarders throughout lookup(mid) (7, 12) –Forwarder notes children (7, 4, 1) Multicast –lookup(mid) (12) –send data via root

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Epidemic Protocols (Eugster et al., 2004)

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Information Dissemination Models Anti-entropy propagation model –Node P picks another node Q at random –Subsequently exchanges updates with Q Approaches to exchanging updates –P only pushes its own updates to Q –P only pulls in new updates from Q –P and Q send updates to each other

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Drawbacks of Push-Only When many nodes are infected, a sending node is very likely to pick a partner which is already infected Waste of work as this gives no further infection Makes it relatively slow to infect the very last part of the network

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Drawbacks of Pull-Only Not applicable in a setting when it is not know when messages are sent –Say we want to send a virus alert When few nodes are infected, a non- infected node is very likely to pick an exchange partner which is not infected Waste of work as this gives no further infection Makes it relatively slow to get off the ground

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Gossiping (1/3) A specific epidemic model which works via pushing, but mitigates the bad sides of push-only by closing down when it is detected that most of the network is infected At this point another approach can be used to sweep up, like pull-only

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Gossiping (2/3) P received update to data –Contact arbitrary node Q –Push update data to Q –If Q already has update, stop spreading with possibility 1/k For large number of nodes, fraction s of susceptible nodes will satisfy

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Gossiping (3/3) Figure The relation between the fraction s of update-ignorant nodes and the parameter k in pure gossiping. The graph displays ln(s) as a function of k. 1% ign.

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved Summary Communication is fundamental in distributed systems –Low level –High level Different communication types beneficial in different contexts –RPC –Message-oriented –Multicast Remain to see practical implementation/API –More next time