Inter-process Communication

2 Berkeley Sockets (I) Socket primitives for TCP/IP. PrimitiveMeaning SocketCreate a new communication endpoint BindAttach a local address to a socket ListenAnnounce willingness to accept connections AcceptBlock caller until a connection request arrives ConnectActively attempt to establish a connection SendSend some data over the connection ReceiveReceive some data over the connection CloseRelease the connection

3 Berkeley Sockets (II) Connection-oriented communication pattern using sockets.

4 Connected vs Connectionless (I) IP  best-effort, unreliable, connectionless –Remembers nothing about a packet after it has sent it –Checksum computed on header only No assumptions about the underlying physical medium –Serial link, Ethernet, Token ring, X.25, ATM, wireless CDPD, … UDP: – (optional) checksum – notion of port

5 Connected vs Connectionless (II) TCP  reliable connection-oriented service –Segments are sent in IP datagrams –Checksum of data in each segment –Sequence # of the 1 st byte in the segment –Acknowledge-and-retransmit mechanism Each side maintains a receive window –Range of sequence # that this side is prepared to receive »Any arriving data with sequence # outsiode the receive window is discarded »Queuing of data arriving out-of-order »Window slides to the right, if the next expected sequence # has arrived »… and an ACK is sent back with the sequence # expected next Send window: –Bytes sent but not yet acknowledged »RTO timer (retransnmission timeout) »Timeout does not always mean that the data was lost !! –Bytes that can be sent but have not yet been sent

6 UDP Failure Model Omission failures –timeouts –duplicate messages –lost messages –Need to maintain history Last reply sent to each client –provided that a client can make only one request at a time »interprets each request as the ACK for the previous reply –periodic ‘’purge’’ of history »No ACK for the last response received before client terminates Fixed max. buffer size (8 KB) No message order guarantee Process crash failures

7 TCP Failure Model Reliable message delivery –checksums, sequence numbers, timeouts –no need for applications to deal with retransmissions duplicates reordering –no need for histories Flow control mechanism –large transfers without overwhelming the receiver … BUT not reliable sessions: –Connections may be severed or severely congested Processes cannot distinguish network from process failure Processes cannot tell if their recent messages were received

8 TCP is a stream protocol No inherent notion of “message boundary” –The amount of data in a packet is not directly related to the amount of data delivered to TCP in the send() call –No reliable for the receiver to determine how the data was packetized Several packets may have arrived between recv() calls –The amount of data returned in any given read() is unpredictable Fixed-length messages Variable-length messages –End-of-record marker –Fixed-length header (including record length) + variable data

9 TCP Failure Modes (I) “ TCP guarantees delivery of the data it sends ” –True or False ? Guarantee to whom ? False … How can we handle outages & crashes ? TCP NIC IP NIC IP NIC IP TCP NIC IP Application (A)Application (B) User-space kernel-space

10 TCP Failure Modes (II) IP is a best-effort, unreliable protocol –… so the TCP layer is the first place in the data path where it makes senses to even talk about guarantees The sender’s TCP layer can make no guarantee about segments that arrive at the receiver’s TCP layer –An arriving segment may be corrupted, or it may contain duplicate data, or it may be out of order … The receiver’s TCP layer guarantees to the sender’s TCP layer that any segment that it ACKs & all data that came before it have been correctly received –This does not mean that the data has been delivered to the application … ot that it will ever be delivered !! For example, the receiving host may crash after the ACK but before delivery …

11 TCP Failure Modes (III) It also makes sense to talk about guarantees at application B (receiver) –There can be no guarantee that all data sent by application A will arrive –However, all data that does arrive will be in order and uncorrupted Avoid the attitude that “TCP will take care of everything” TCP is an end-to-end protocol, providing a reliable transport mechanism between peers … The “peers” are the TCP layers of the sender & the receiver !!

12 TCP Failure Modes (IV) Explicit acknowledgements –What does the client do if the server does not ACK receipt ?? –It may not be safe to simply resend a request … Network outagePeer crashesPeer’s host crash When a problem occurs at an endpoint, there is generally no alternative path  The problem persists until it is repaired An intermediate router may send the originator an ICMP message indicating that the destination network or the host is unreachable OR: The sender eventually times-out & resends the segments not ACKed. This continues until the sender gives up & drops the connection (~9 minutes). Pending read  ETIMEDOUT Otherwise, the next write fails  SIGPIPE or EPIPE

13 TCP Failure Modes (V) Peer crash: –Indistinguishable from the case of the peer calling close() and then exit() –The peer’s TCP layer issues a FIN segment This does not necessarily imply that the peer has no more data to send, or even that it is not willing to receive more data … –Reception of the FIN may come at different execution states of the application If client is blocked, TCP has no way of notifying it –The next transmission generates a RST segment  ECONNRESET –If the RST is ignored & more data is transmitted  SIGIPE »This may occur if the client performs >=2 consecutive write() calls without an intervening read()  Notification takes place only after the 2 nd write() If client has a pending read(), it gets an immediate error indication (eg: read() returns EOF)

14 TCP Failure Modes (VI) Peer’s host crash: –The peer’s TCP cannot issue the FIN segment –Until recovery, this case cannot be distinguished from a network outage The peer’s TCP no longer responds, but the sender keeps retransmitting … Until either the host recovers, or the sender gives up the connection  ETIMEDOUT –If the host reboots before the sender gives up, a retransmitted segment may arrive at the TCP layer … without it having knowledge of the connection  RST If sender has a read() pending  ECONNRESET Else, the next write() results in a SIGPIPE signal

15 Behavior of Peers Checking for client termination –Heartbeats, timeouts for read operations, SO_KEEPALIVE option, … Checking for valid input –Buffer overflow errors

16 We rely on DNS …

17 The Message-Passing Interface Some of the most intuitive primitives of MPI. PrimitiveMeaning MPI_bsendAppend outgoing message to a local send buffer MPI_sendSend a message and wait until copied to local or remote buffer MPI_ssendSend a message and wait until receipt starts MPI_sendrecvSend a message and wait for reply MPI_isendPass reference to outgoing message, and continue MPI_issendPass reference to outgoing message, and wait until receipt starts MPI_recvReceive a message; block if there are none MPI_irecvCheck if there is an incoming message, but do not block

18 Group Communication Multicasting: 1-to-many comm. pattern –Applications: replicated services (better fault tolerance) discovery of services replicated data (better performance) propagation of event notifications –Failure model: depends on implementation: –IP multicast (UDP datagrams): omission failures »class-D Inet addresses: “1110” bit prefix »TTL –reliable multicast –ordered multicast »FIFO »Causal »Total

19 Conventional Procedure Call a)Parameter passing in a local procedure call: the stack before the call to read b)The stack while the called procedure is active

20 Software layers Applications and Services RPC and RMI request-reply protocol marshalling and external data representation UDP and TCP middleware RPC is more than a (transport) protocol: a structuring mechanism for distributed systems

21 Steps of a Remote Procedure Call 1.Client procedure calls client stub in normal way 2.Client stub builds message, calls local OS 3.Client's OS sends message to remote OS 4.Remote OS gives message to server stub 5.Server stub unpacks parameters, calls server 6.Server does work, returns result to the stub 7.Server stub packs it in message, calls local OS 8.Server's OS sends message to client's OS 9.Client's OS gives message to client stub 10.Stub unpacks result, returns to client

22 Client and Server Stubs Principle of RPC between a client & server program.

23 Example (Sun RPC - ONC) long square(long) example –client ren.eecis.udel.edu 11 –result: 121 Need RPC specification file (square.x): defines procedure name, arguments & results Run rpcgen square.x: generates square.h, square_clnt.c, square_xdr.c, square_svc.c square_clnt.c & square_svc.c: Stub routines for client & server square_xdr.c: XDR (External Data Representation) code - takes care of data type conversions

24 RPC Specification File (square.x) struct square_in { longarg1; }; struct square_out { long res1; }; program SQUARE_PROG { version SQUARE_VERS { square_out SQUAREPROC(square_in) = 1;// procedure # } = 1;// version # } = 0x ;// program # IDL – Interface Definition Language

25 Parameter Specification & Stub Generation procedureCorresponding message

26 Writing a Client & a Server The steps in writing a client & a server in DCE RPC.