1. Univ. of TehranDistributed Operating Systems1 Advanced Operating Systems University of Tehran Dept. of EE and Computer Engineering By: Dr. Nasser Yazdani Interprocess Communication Lecture 5: Interprocess Communication

2. Univ. of TehranDistributed Operating Systems2 How Processes communicate Improving process communication to make systems more modular, scalable and flexable. References “improving IPC by Kernel Design”, by Jochen Liedtke.

3. Univ. of TehranDistributed Operating Systems3 Outline Why IPC? IPC mechanisms Pipe, Socket, Shared memory Message RPC (Remote Procedure call) Universal address space. How improve IPC through kernel.

4. Univ. of TehranDistributed Operating Systems4 IPC Fundamentals What is IPC? Mechanisms to transfer data between processes To share resources Client/server paradigms Inherently distributed applications Reusable software components Other good software engineering reasons Why is it needed? Not all important procedures can be easily built in a single process To make systems modular, flexible and scalable. Critical for micro-kernel OS and distributed systems.

5. Univ. of TehranDistributed Operating Systems5 The Basic Concept of IPC A sending process needs to communicate data to a receiving process Sender wants to avoid details of receiver’s condition Receiver wants to get the data in an organized way

6. Univ. of TehranDistributed Operating Systems6 IPC from the OS Point of View OS address space Process A Process B Private address space Each process has a private address space Normally, no process can write to another process’s space How to get important data from process A to process B?

7. Univ. of TehranDistributed Operating Systems7 Solutions to IPC Fundamentally, two options 1. Support some form of shared address space Shared memory 2. Use OS mechanisms to transport data from one address space to another Files, messages, pipes, RPC

8. Univ. of TehranDistributed Operating Systems8 Differences in Treatment of IPC Shared memory OS has job of setting it up And perhaps synchronizing But not transporting data Messages, etc OS involved in every IPC step OS transports data

9. Univ. of TehranDistributed Operating Systems9 Ideal IPC Characteristics Fast Easy to use Well defined synchronization model Versatile Easy to implement Works remotely

10. Univ. of TehranDistributed Operating Systems10 IPC and Synchronization Synchronization is a major concern for IPC Allowing sender to indicate when data is transmitted Allowing receiver to know when data is ready Allowing both to know when more IPC is possible

11. Univ. of TehranDistributed Operating Systems11 IPC and Connections IPC mechanisms can be connectionless or require connection Connectionless IPC mechanisms require no preliminary setup Connection IPC mechanisms require negotiation and setup before data flows Sometimes much is concealed from user, though

12. Univ. of TehranDistributed Operating Systems12 Connectionless IPC Data simply flows Typically, no permanent data structures shared in OS by sender and receiver + Good for quick, short communication + less long-term OS overhead - Less efficient for large, frequent communications - Each communication takes more OS resources per byte

13. Univ. of TehranDistributed Operating Systems13 Connection-Oriented IPC Sender and receiver pre-arrange IPC delivery details OS typically saves IPC-related information for them Advantages/disadvantages pretty much the opposites of connectionless IPC

14. Univ. of TehranDistributed Operating Systems14 IPC Through File System Sender writes to a file Receiver reads from it But when does the receiver do the read? Often synchronized with file locking or lock files Special types of files can make file-based IPC easier Process A Process B Data

15. Univ. of TehranDistributed Operating Systems15 Message-Based IPC Sender formats data into a formal message With some form of address for receiver OS delivers message to receiver’s message input queue (might signal too) Receiver (when ready) reads a message from the queue Sender might or might not block

16. Univ. of TehranDistributed Operating Systems16 Message-Based IPC Diagram Process A Process B Data sent from A to B OS B’s message queue

17. Univ. of TehranDistributed Operating Systems17 Procedure Call IPC Interprocess communication uses same procedure call interface as intraprocess Data passed as parameters Information returned via return values Complicated since destination procedure is in a different address space Generally, calling procedure blocks till call returns

18. Univ. of TehranDistributed Operating Systems18 Procedure IPC Diagram Process A Process B Data as parameters main () {. call();. }. server();. Data as return values

19. Univ. of TehranDistributed Operating Systems19 Shared Memory IPC Processes share a common piece of memory either physically or virtually Communications via normal reads/writes May need semaphores or locks In or associated with the shared memory Process A Process B write variable x main () {. x = 10. }. print(x);. read variable x x: 10

20. Univ. of TehranDistributed Operating Systems20 Synchronizing in IPC How do sending and receiving process synchronize their communications? Many possibilities, based on which process block when Blocking Send, Blocking Receive, Often called message rendezvous Non-Blocking Send, Blocking Receive, Essentially, receiver is message-driven Non-Blocking Send, Non-Blocking Receive, polling or interrupt for receiver

21. Univ. of TehranDistributed Operating Systems21 Addressing in IPC How does the sender specify where the data goes? The mechanism makes it explicit (shared memory or RPC) Or, there are options: Direct Addressing Sender specifies name of the receiving process using some form of unique process name Receiver can either specify name of expected sender or take stuff from anyone Indirect Addressing: Data is sent to queues, mailboxes, or some other form of shared data structure: Much more flexible than direct addressing

22. Univ. of TehranDistributed Operating Systems22 Duality in IPC Mechanisms Many aspects of IPC mechanisms are duals of each other Which implies that these mechanisms have the same power First recognized in context of messages vs. procedure calls At least, IPC mechanisms can be simulated by each other

23. Univ. of TehranDistributed Operating Systems23 Which IPC mechanism? Depends on model of computation And on philosophy of user In particular cases, hardware or existing software may make one perform better

24. Univ. of TehranDistributed Operating Systems24 Typical UNIX IPC Mechanisms Different versions of UNIX introduced different IPC mechanisms Pipes Message queues Semaphores Shared memory Sockets RPC

25. Univ. of TehranDistributed Operating Systems25 Pipes Only IPC mechanism in early UNIX systems (other than files) Uni-directional Unformatted Uninterpreted Interprocess byte streams Accessed in file-like way

26. Univ. of TehranDistributed Operating Systems26 Pipe Details One process feeds bytes into pipe A second process reads the bytes from it Potentially blocking communication mechanism Requires close cooperation between processes to set up Named pipes allow more flexibility

27. Univ. of TehranDistributed Operating Systems27 Pipes and Blocking Writing more bytes than pipe capacity blocks the sender Until the receiver reads some of them Reading bytes when none are available blocks the receiver Until the sender writes some Single pipe can’t cause deadlock

28. Univ. of TehranDistributed Operating Systems28 UNIX Message Queues Introduced in System V Release 3 UNIX Like pipes, but data organized into messages Message component include: Type identifier Length Data

29. Univ. of TehranDistributed Operating Systems29 Semaphores Also introduced in System V Release 3 UNIX Mostly for synchronization only Since they only communicate one bit of information Often used in conjunction with shared memory

30. Univ. of TehranDistributed Operating Systems30 UNIX Shared Memory Also introduced in System V Release 3 Allows two or more processes to share some memory segments With some control over read/write permissions Often used to implement threads packages for UNIX

31. Univ. of TehranDistributed Operating Systems31 Sockets Introduced in 4.3 BSD A socket is an IPC channel with generated endpoints Great flexibility in it characteristics Intended as building block for communication Endpoints established by the source and destination processes

32. Univ. of TehranDistributed Operating Systems32 UNIX Remote Procedure Calls Procedure calls from one address space to another On the same or different machines Requires cooperation from both processes In UNIX, often built on sockets Often used in client/server computing

33. Univ. of TehranDistributed Operating Systems33 Problems with Shared Memory Synchronization Protection Pointers

34. Univ. of TehranDistributed Operating Systems34 Synchronization Shared memory itself does not provide synchronization of communications Except at the single-word level Typically, some other synchronization mechanism is used E.g., semaphore in UNIX Events, semaphores, or hardware locks in Windows NT

35. Univ. of TehranDistributed Operating Systems35 Protection Who can access a segment? And in what ways? UNIX allows some read/write controls Windows NT has general security monitoring based on the object-status of shared memory

36. Univ. of TehranDistributed Operating Systems36 Pointers in Shared Memory Pointers in a shared memory segment can be troublesome For that matter, pointers in any IPC can be troublesome Process A Process B x: 10 y: 20 z: __ a: __b: __ w: 5

37. Univ. of TehranDistributed Operating Systems37 A Troublesome Pointer Process A Process B x: 10 y: 20 z: __ a: __b: __ w: 5

38. Univ. of TehranDistributed Operating Systems38 How share pointers? Copy-time translation Locate each pointer within old version of the data Then translate pointers are required Requires both sides to traverse entire structure Not really feasible for shared memory Reference-time translation Encode pointers in shared memory segment as pointer surrogates, typically, as offsets into some other segment in separate contexts. So each sharer can have its own copy of what is pointed to Slow, pointers in two formats Pointer Swizzling cache results in the memory location, only first reference is expensive But each sharer must have his own copy Must “unswizzle” pointers to transfer data outside of local context Stale swizzled pointers can cause problems All involve somehow translating pointers at some point before they are used

39. Univ. of TehranDistributed Operating Systems39 Shared Memory in a Wide Virtual Address Space When virtual memory was created, 16 or 32 bit addresses were available Reasonable size for one process But maybe not for all processes on a machine And certainly not for all processes ever on a machine

40. Univ. of TehranDistributed Operating Systems40 Wide Address Space Architectures Computer architects can now give us 64-bit virtual addresses A 64-bit address space, consumed at 100 MB/sec, lasts 5000 years Orders of magnitude beyond any process’s needs 40 bits can address a TB Should OS designers care about wide address space? what can we do with them? One possible answer: Put all processes in the same address space Maybe all processes for all time?

41. Univ. of TehranDistributed Operating Systems41 Implications of Single Shared Address Space IPC is trivial Shared memory, RPC Separation of concepts of address space and protection domain Uniform address space

42. Univ. of TehranDistributed Operating Systems42 Address Space and Protection Domain A process has a protection domain The data that cannot be touched by other processes And an address space The addresses it can generate and access In standard systems, these concepts are merged

43. Univ. of TehranDistributed Operating Systems43 Separating Concepts These concepts are potentially orthogonal Just because you can issue an address doesn’t mean you can access it. Though clearly to access an address you must be able to issue it. Existing hardware can support this separation

44. Univ. of TehranDistributed Operating Systems44 Context-Independent Addressing Addresses mean the same thing in any execution context So, a given address always refers to the same piece of data Key concept of uniform-address systems Allows many OS optimizations/improvements

45. Univ. of TehranDistributed Operating Systems45 Uniform-Addressing Allows Easy Sharing Any process can issue any address So any data can be shared All that’s required is changing protection to permit desired sharing Suggests programming methods that make wider use of sharing

46. Univ. of TehranDistributed Operating Systems46 To Opal System New OS using uniform-addressing Developed at University of Washington Not intended as slight alteration to existing UNIX system Coming material specific to Opal

47. Univ. of TehranDistributed Operating Systems47 Protection Mechanisms for Uniform-Addressing Protection domains are assigned portions of the address space They can allow other protection domains to access them Read-only Transferable access permissions System-enforced page-level locking

48. Univ. of TehranDistributed Operating Systems48 Program Execution in Uniform-Access Memory Executing a program creates a new protection domain The new domain is assigned an unused portion of the address space But it may also get access to used portions E.g., a segment containing the required executable image

49. Univ. of TehranDistributed Operating Systems49 Virtual Segments Global address space is divided into segments Each composed of variable number of contiguous virtual pages Domains can only access segments they attach to Attempting to access unattached segment causes a segment fault

50. Univ. of TehranDistributed Operating Systems50 Persistent Memory in Opal Persistent segments exist even when attached to no current domain Recoverable segments are permanently stored And can thus survive crashes All Opal segments can be persistent and recoverable Pointers can thus live forever on disk

51. Univ. of TehranDistributed Operating Systems51 Code Modules in Opal Executable code stored in modules Independent of protection domains Pure modules can be easily shared Because they are essentially static Can get benefit of dynamic loading without run-time linking

52. Univ. of TehranDistributed Operating Systems52 Address Space Reclamation Trivial in non-uniform-address systems Tricky in uniform-address systems Problem akin to reclaiming i_nodes in the presence of hard links But even if segments improperly reclaimed, only trusting domains can be hurt

53. Univ. of TehranDistributed Operating Systems53 Windows NT IPC Inter-thread communications Within a single process Local procedure calls Between processes on same machine Shared memory

54. Univ. of TehranDistributed Operating Systems54 Windows NT and Client/Server Computing Windows NT strongly supports the client/server model of computing Various OS services are built as servers, rather than part of the kernel Windows NT needs facilities to support client/server operations Which guide users to building client/server solution

55. Univ. of TehranDistributed Operating Systems55 Client/Server Computing and RPC In client/server computing, clients request services from servers Service can be requested in many ways But RPC is a typical way Windows NT uses a specialized service for single machine RPC

56. Univ. of TehranDistributed Operating Systems56 Local Procedure Call (LPC) Similar in many ways to RPC But optimized to only work on a single machine Primarily used to communicate with protected subsystems Windows NT also provides a true RPC facility for genuinely distributed computing

57. Univ. of TehranDistributed Operating Systems57 Basic Flow of Control in Windows NT LPC Application calls routine in an application programming interface Which is usually in a dynamically linked library Which sends a message to the server through a messaging mechanism

58. Univ. of TehranDistributed Operating Systems58 Windows NT LPC Messaging Mechanisms Messages between port objects Message pointers into shared memory Using dedicated shared memory segments

59. Univ. of TehranDistributed Operating Systems59 Port Objects Windows NT is generally object-oriented Port objects support communications Two types: Connection ports Used to establish connections between clients and servers Named, so they can be located Only used to set up communication ports Communication ports Used to actually pass data Created in pairs, between given client and given server Private to those two processes Destroyed when communications end

60. Univ. of TehranDistributed Operating Systems60 Windows NT Port Example Client process Server process Connection port

61. Univ. of TehranDistributed Operating Systems61 Windows NT Port Example Client process Server process Connection port

62. Univ. of TehranDistributed Operating Systems62 Communication ports Windows NT Port Example Client process Server process Connection port

63. Univ. of TehranDistributed Operating Systems63 Communication ports Windows NT Port Example Client process Server process Connection port Send request

64. Univ. of TehranDistributed Operating Systems64 Message Passing through Port Object Message Queues One of three methods in Windows NT to pass messages 1. Client submits message to OS 2. OS copies to receiver’s queue 3. Receiver copies from queue to its own address space

65. Univ. of TehranDistributed Operating Systems65 Characteristics of Message Passing via Queues Two message copies required Fixed-sized, fairly short message ~256 bytes Port objects stored in system memory So always accessible to OS Fixed number of entries in message queue

66. Univ. of TehranDistributed Operating Systems66 Message Passing Through Shared Memory Used for messages larger than 256 bytes Client must create section object Shared memory segment of arbitrary size Message goes into the section Pointer to message sent to receiver’s queue

67. Univ. of TehranDistributed Operating Systems67 Setting up Section Objects Pre-arranged through OS calls Using virtual memory to map segment into both sender and receiver’s address space If replies are large, need another segment for the receiver to store responses OS doesn’t format section objects

68. Univ. of TehranDistributed Operating Systems68 Characteristics of Message Passing via Shared Memory Capable of handling arbitrarily large transfers Sender and receiver can share a single copy of data i.e., data copied only once Requires pre-arrangement for section object

69. Univ. of TehranDistributed Operating Systems69 Server Handling of Requests Windows NT servers expect requests from multiple clients Typically, they have multiple threads to handle requests Must be sufficiently general to handle many different ports and section objects

70. Univ. of TehranDistributed Operating Systems70 Message Passing Through Quick LPC Third way to pass messages in Windows NT Used exclusively with Win32 subsystem Like shared memory, but with a key difference Dedicated resources

71. Univ. of TehranDistributed Operating Systems71 Use of Dedicated Resources in Quick LPC To avoid overhead of copying Notification messages to port queue And thread switching overhead Client sets up dedicated server thread only for its use Also dedicated 64KB section object And event pair object for synchronization

72. Univ. of TehranDistributed Operating Systems72 Characteristics of Message Passing via Quick LPC Transfers of limited size Very quick Minimal copying of anything Wasteful of OS resources

73. Univ. of TehranDistributed Operating Systems73 Shared Memory in Windows NT Similar in most ways to other shared memory services Windows NT runs on multiprocessors, which complicates things Done through virtual memory

74. Univ. of TehranDistributed Operating Systems74 Section Views Given process might not want to waste lots of its address space on big sections So a process can have a view into a shared memory section Different processes can have different views of same section Or multiple views for single process

75. Univ. of TehranDistributed Operating Systems75 Shared Memory View Diagram Process A Process B Physical memory Section view 1view 2 view 3

76. Univ. of TehranDistributed Operating Systems76 Next Lecture Scheduling References Surplus Fair Scheduling: A Proportional- Share CPU Scheduling Algorithm for Symmetric Multiprocessors Scheduler Activations: Effective Kernel Support for User-Level Management of Parallelism",