2. 3. Communication in distributed system

3. 3.1 Introduction Interprocess communication is at the heart of distributed systems Distributed Application Communication middleware Low-level message passing underlying network IP address port,... Name,...

4. Middleware Protocols An adapted reference model for networked communication. 2-5 P69

5. Distributed Middleware Communication middleware: RPC, RMI, MPI –name DCE, CORBA –message-oriented –various communication forms Authentication, authorization protocols Distributed commit protocols

6. Client-Server TCP Normal operation of TCP.Transactional TCP. 2-4 P65

7. Two issues for communication mechanism: How indicate destination? How many methods (forms)? S T Q P

8. Indicate sending and receiving processes 1) ex, 2) Unique name ex, www.nju.edu.cn;procedure name( ) S T Q P 1. Addressing

9. …... Domain name IP address Type www.nju.edu.cn 202.119.32.6 A cs.nju.edu.cn 202.119.36.5 A …… …... DNS FTPd web server ftp://nju.edu.cn www.nju.edu.cn 202.119.32.6 8021

10. 3) Mailbox (queue, indirect) multi-in, multi-out 2. Methods 1) Synchronous/asynchronous sender receiver mailbox

11. P1 P2 send receive waiting receive waiting send Synchronous: sender will be blocked if receiver is not ready send( ), recv( ); RPC

12. P1 P2 send receive Internet? Asynchronous UDP, Asynchronous RPC

13. 2) Transient/persistent Persistent communication: A message is stored by communication system as long as it takes to deliver it. Neither the sender nor the receiver need to be up. Ex,email Synchronous wait, no broadcast safe, simple Asynchronous buffer full,not safe parallelism, broadcast Defects merits Comparison P100-1

14. Persistence in Communication P100

15. Transient communication: A message is stored by communication system as long as the sending and receiving applications are executing Ex, send, recv in Socket Question: difference (relation) between 1) and 2) transient synchronous persistent asynchronous As to: 1) sender/receiver; 2) message send, recv; RPC UDP; asyn RPC email

16. Two successive messages have a temporal relationship. Ex, movie, audio stream 3) Stream-oriented communication

17. P1P2 Proc G(u,v) Call P2.G(x,y) node 1 node 2 ……….. 3.2 RPC P68 2003.7Worm.msBlast

18. 1. Conventional procedure call Node 1 process Procedure Q(u,box,v) { … { … Q(i,buf,j) … … } return addr. i buf j sp stack result

19. 2. How does RPC work? node 1 node 2 process Procedure Q(u,box,v) { … { … Q(i,buf,j) … … } } ? stack return sp return addr 2 i buf j sp result return addr.1 buf j i proc Q int i ? buf int j result

20. Original procedure call: Now push return addr.1 push i Q(i,buf,j) push buf push j goto Q goto c_stub take result

21. C_stub pack parameters & name send them to S_stub receive result unpack result return S_stub receive message unpack message push return addr. goto Q( ) take & pack result send it to C_stub Q( ) { } transform

22. Steps of a Remote Procedure Call Client procedure calls client stub in normal way Client stub builds message, calls local OS Client's OS sends message to remote OS Remote OS gives message to server stub Server stub unpacks parameters, calls server Server does work, returns result to the stub Server stub packs it in message, calls local OS Server's OS sends message to client's OS Client's OS gives message to client stub Stub unpacks result, returns to client P72

23. Passing Value Parameters Steps involved in doing remote computation through RPC 2-8 P74

24. Issues to be handled Machine may be not identical Execute in different spaces Machine may crash 3. Parameter passing Implementation should consider: Different types of computers Pointer or call by reference

25. 1) Deal with different types of computers 5Jill (a) Original message on the Pentium (b) The message after receipt on the SPARC (c) The message after being inverted. The little numbers in boxes indicate the address of each byte P75

26. How represent parameters? –In intermediate form –Indicating in type i type i type j Message should include parameters and their type 2) Deal with call by reference proc Q int i char buf int j

27. Node1Node2 Main( ) { int B[100] proc write(name,buf,len) … { int len, buf[ ]; write(f1,B,100)… … } proc read(name, buf, len) { int len, buf[ ]; read(f2,B,100 ) … } B 5200

28. read write

29. proc write(in char name, in int buf[], in int len); proc read(in char name, out int buf[], in int len); –Input parameter  deliver the array –Output parameter  needn’t 4. Where stubs come from? Write server specification (interfaces) including:specification Server name, version, list of proc s Stub compiler generates c_stub & s_stubgenerates

30. Specification of file server Specification of file_server, version 3.1: long read(in char name[MAX_PATH], out char buf[BUF_SIZE], in long bytes, in long position); long write(in char name[MAX_PATH], in char buf[BUF_SIZE], in long bytes, in long position); int create(in char name[MAX_PATH], in int mode); int delete(in char name[MAX_PATH]); end; return IDL

31. 5. Steps for using RPC write specification compile write server code and link with s_stub write client code and link with c_stub specification C_stub S_stub Stub compiler

32. The steps in writing a client and a server in DCE RPC. P82

33. {…… }

34. 6. Dynamic binding --- How client find server 1) Register: name, version, handle(IP,port), unique id 2) Look up: name, version (first time) 3) Reply: handle, unique id 4) Request: unique id, parameters 5) Deregister: name, version, unique id binder clientserver 1 2 3 4

35. Advantage: flexible –Server move won’t affect client –Binder can even load if multiple servers support the same interface Disadvantage –Overhead at running time –Binder is bottleneck

36. Addressing: name Method: synchronous, transient RPC NFS XDR Socket/TLI TCP|UDP IP ARP Driver ICMP FTP Telnet Addressing Method

37. 7. Extended RPC --- asynchronous RPC (a) The interconnection between client and server in a traditional RPC (b) The interaction using asynchronous RPC ABAB 100 P79

38. A client and server interacting through two asynchronous RPCs 2-13 A client wants to prefetch network addresses of a set of hosts P80

39. 3.3 RMI (or Remote Object Invocation) Common organization of a remote object with client-side proxy. P86

40. 1. Distributed object RMI Compares to RPC: Same: –the steps of call is similar –separating interface from implementation Difference –procedure is a segment of program. –object is a variable of a data type, method is in object.

41. Object reference compare to pointer. System provides 2. Object references Object references are similar to pointer, but not the same. object reference remote point local object distributed obj

42. … employee promote() {…} dismiss() {… } hire() {… } Corp server Sale server salesman bonus() {… } advertise cost() {… } personnel 1. IP address 2. 3. ? Object servers P152 personnel.hire( )

43. Information object reference contains network address of the machine server id or endpoint (i.e.port number) object id P89

44. May use location server location server clientserver 1 2 3 4 Server can move

45. Server id instead of server’s endpoint. A daemon listens to a well-known endpoint and knows every server. 3. Implementation of object references server1 server2 keep a table daemon

46. 4. Several remote object invocation 1) CORBA Open, heterogeneous environment 2) DCOM Microsoft world Object C ++ Java C func P( ) How introduce object? chapter 9

47. 3) Java RMI Java environment Distributed objects have been integrated into the language Summary Addressing: name Method: synchronous, transient

48. 3.4 Message-oriented communication 1. Message-oriented transient communication 1) RPC and RMI advantages: hide communication; message-oriented disadvantage: synchronous; both need to be up 2) Recall socket P105 addressing: IP address, port forms: synchronous/asynchronous, transient shortage: low-level, byte-oriented/TCP-IP P99

49. Process0 Process1 Process5 Process4 Process3 Process2 Process1 Process0 Process2 3) MPI P107 Jupiter World designed for parallel applications; offer efficient communication primitives; for high-speed nets. communicator

50. Addressing (groupID, processID) Forms support most transient communication forms ?A B;email Need for various communication forms Synchronous/asynchronous Transient/persistent 100

51. Various combinations:P103 Why list so many forms? Where to use? For efficiency. (a)email (c) A B (e) send recv (f) RPC 100 (persistent, asynchronous) (transient, asynchronous) (transient, synchronous) (transient, synchronous/asyn)

52. 2. Message-oriented persistent communication 1) Message-queuing model Addressing: name of destination queue Method: persistent, asynchronous 2) Basic idea Application inserts messages in specific queue the messages are forwarded over a series of communication servers eventually the messages are delivered to destination P109

53. Message-Queuing System The general organization of a message-queuing system with routers. 2-29 P112

54. Four combinations for loosely-coupled communications using queues. 2-26 P110

55. 3) Basic interface to a queue Install a handler to be called when a message is put into the specified queue. Notify Check a specified queue for messages, and remove the first. Never block. Poll Block until the specified queue is nonempty, and remove the first message Get Append a message to a specified queue Put Meaning Primitive callback P110

56. 4) Implementation All machines in message-queuing system should have a queue manager which manages: queues: source queue, destination queue (name) transmission of messages from S to D mapping queues to network locations

57. Two kinds of implementations a) Create its own forwarding servers (routers), like multicast gateway + Mbone b) with the aid of Internet routers, like email a) Architecture of IBM MQSeries For IBM mainframes, access large scale database Addressing: (destQM, destQ),

58. Example: IBM MQSeries General organization of IBM's MQSeries message-queuing system. 2-31 P116

59. Message Transfer The general organization of an MQSeries queuing network using routing tables and aliases. (QA,SQ1)

60. Why do message-queue systems send messages to queue, instead of directly to receivers? Some applications need this kind of communication form. For example, tasks are sent to a set of processes who perform these tasks. Two ways: a dispatcher + workers workers P143

61. 3.5 Stream-oriented communication 1. Requirements Real life –audio stream (simple stream) –movie (complex stream) Features –time-dependent information. –data stream Stream media? P119

62. Addressing: or destination name Method: continuous data stream

63. Requirement for transmission: –timing for continuous data stream –synchronization for complex stream Ex, –Samples in audio stream are played at intervals of 1/44100 sec. –Video and audio sub-streams need synchronize.

64. Data Stream Setting up a stream between two processes across a network. P122 MP3

65. An example of multicasting a stream to several receivers. Differentiated services

66. 4. Stream Synchronization Meaning Maintain temporal relation between streams which may be continuous or discrete data stream. Essence Synchronize at the level of data units. What is a data unit? a single audio stream a series of samples a video stream a series of frames

67. Data unit Ex1, synchronization for stereo audio stream takes place every 1/44100 sec a data unit = a sample Ex2, what about synchronization between an audio stream and a video stream? video frame30Hz audio sample44100Hz sample/frame44100/30=1470 an audio data unit = 1470 samples 1/30 sec

68. Synchronization mechanism A process executes read (write) operations on several simple streams, ensuring timing and synchronization. Ex, alternating between reading an image and reading a block of audio samples every 33ms sender receiver audio stream video stream

69. Does sender send two separate streams or a complex stream? A complex stream. Where synchronization should take place, sending or receiving side? Sender: multiplex different streams into a single stream containing all data units, including those for synchronization(timestamp) Receiver: demultiplexes the stream, using the timestamps of each packet

70. Synchronization Mechanisms (2) The principle of synchronization as supported by high- level interfaces. 2-41 Offer interface to specify the rates

71. Summary 1. Key issues for designing communication mechanism Addressing: how indicate sender and receiver, LAN, WAN Methods : synchronous/asynchronous persistent/transient stream-oriented communication Middleware Examples?

72. How does it work? Issues to be handle: type, reference (object?) IDL: is used to write server’ specification C_stub Steps for using RPC S_stub Binder 3. RMI Difference between RPC and RMI Object reference and its features 2. RPC

73. –address, serverID, objectID CORBA etc 4. Message-oriented communication Why need it? Transient communication: MPI Persistent communication: Message-queuing system

74. Requirement for transmission Features Synchronization 5. Stream-oriented communication