1. CS 582 / CMPE 481 Distributed Systems Communications (cont.)

2. Class Overview What is a network? Protocols Layered architecture Reference models Middleware IPC –Why IPC? –Design Considerations –Client-Server Communication –Group Communication RPC

3. Middleware Middleware is invented to provide common services and protocols that can be used by many different applications: –A rich set of communication protocols, but which allow different applications to communicate –Marshaling and unmarshaling of data, necessary for integrated systems –Naming protocols, so that different applications can easily share resources –Security protocols, to allow different applications to communicate in a secure way –Scaling mechanisms, such as support for replication and caching What remain are truly application-specific protocols

4. Interaction Model in Distributed Systems A principal goal of distributed systems –a single system image => resource sharing concurrency Cooperation –explicit communication is required between components Common form of interaction –client-server interaction model interprocess communications (IPC)

5. Principal Functions Allows communication between components (processes) Shields one process from failure of another Provides modularity by a well defined interface mechanism hides distinction between local and remote communications

6. Abstraction of Communication Subsystem Regards a network as another I/O device –no abstraction beyond a transport protocol client and server are wholly responsible for message exchange over a given transport mechanism (e.g. TCP/IP) primitives: write (send) and read (receive) Request and reply –abstraction of message passing required to execute a procedure at a server primitives: DoOperation, GetRequest, and SendReply Remote procedure call –hides the separation between a client and a server makes invocation of a remote procedure at a server same as that of a local procedure

7. Design Considerations in IPC Data representation Marshalling Calling semantics Addressing Reliable delivery Versatility

8. Data Representation Due to a heterogeneity property of distributed systems Things to consider –data type representation –byte ordering –word boundary Policy –conversion to the common data type agreed by both a sender and a receiver –a receiver makes it right (tagging) –a sender negotiates with a receiver Parameter passing Examples: Sun XDR, ISO ASN.1, Xerox Courier

9. Marshalling marshal·ling 1.To arrange or place (troops, for example) in line for a parade, maneuver, or review. 2.To arrange, place, or set in methodical order. 3.To enlist and organize. 4.To guide ceremoniously; conduct or usher. Marshalling –linearization of an operation invocation –convert the type of data into a common type (if needed) –pack linearized data into a message Unmarshalling –reverse operation of marshalling at the receiving peer –upcalls

10. Calling Semantics Remote operation invocation may need a different behavior than a local one –sender may not need a result either at all or immediately –sender may want to do other operations in parallel (e.g. multithreading) Semantics –synchronous: same as local operation invocation (blocked) timeout is used to avoid indefinite wait –delayed synchronous: the sender gets a reply later –asynchronous: the sender does not need a reply Blocking vs. Non-blocking –R/RR/RRA [Spector 1982]

11. Addressing Identification of communication peers Location-independent identifiers –functional addressing: port in Amoeba –globally unique identifier: UUID in NCA Types of destination identifiers –Port –mailbox

12. Reliable Delivery End-to-end argument [Saltzer, Reed, Clark] –error recovery in lower levels of protocols is only useful for purposes of increasing efficiency protocols for application-level end-to-end checking are always required Possible causes for retransmission (client -> server) –server still working on it –server crashes –request gets lost IPC should provide some level of failure transparent but also failure visibility (accurate report of failures)

13. Reliable Delivery (cont.) Fault management based on execution semantics –at-least-once: idempotent –at-most-once: session –exactly-once Retransmission mechanisms –reply can be considered an acknowledgement –selective retransmission: sequence number –explicit acknowledgement –retransmission timer

14. Versatility Types of IPC –remote operation –bulk data transfer –group communication –continuous media Rate control –stop-and-wait –sliding window –Blasting Multipacket QoS –real-time: retry is better than retransmission –stream-oriented: steady and low delay but packet lost is ok

15. Examples: Unix IPC Socket provides a message passing mechanism –send/sendto and recv/recvfrom primitives –select allows non-blocking semantics (with timeout) –addressing: host id + port number –scatter-gather mechanism for large datagram –QoS TCP: reliable transmission UDP: unreliable but fast transmission

16. Group Communication Purposes of multicast messages –fault tolerance based on replicated services –locating objects in distributed services –better performance through replicated data –multiple update Atomicity –atomic multicast all or nothing synchronization among members (e.g. join or leave of members) –Reliable best effort to deliver messages but no guarantee computation or query

17. Ordering in Group Communication Why ordering is concerned? –concurrent execution of update requests at replicas may result in inconsistency among replicated data –serial equivalence of update requests is required expense of ordering should also be considered Ordering requirements –total ordering requests are processed in the same order at all replicas –causal ordering causally related requests are only ordered at all replicas –sync ordering requests are ordered in sync before or after a certain request at all replicas

18. Ordering in Group Communication (cont.) Request handling at replicas –every request is held-back until ordering constraints can be met –request is defined to be stable at a replica once no request from a client and bearing a lower unique identifier can be subsequently delivered to replica; that is, all prior requests have been processed Request ordering implementation –group communication –exchanging gossip messages among replicas