Carnegie-Mellon University, Pittsburgh Presented by Mehmet Belgin The Duality of Memory and Communication in the Implementation of a Multiprocessor Operating System Young, M. / Tevanian, A. / Rashid, R. / Golub, D. / Eppinger, J. / Chew, J. / Bolosky, W. / Black, D. / Baron, R. Carnegie-Mellon University, Pittsburgh ACM 1987 Presented by Mehmet Belgin October, 7th 2004

MACH Operating System MACH is a Microkernel object oriented OS History A ‘light weight’ kernel. Basic abstractions over the hardware. Design goals are multiprocessor support, binary compatibility with Berkeley UNIX, Network transparency, flexible memory management and IPC, High performance, simple programmer interface. History Project for DARPA by Carnegie-Mellon University (1985-1994) Based on Accent (by CMU) MACH  NeXT  NeXTSTEP  MacOS X (Darwin) Some principles are used in Windows NT/XP GNU Hurd on MACH Microkernel MacOS X 10 / 7 / 2004

Focus of the Paper External memory management facility of MACH Four basic abstractions are inherited from Accent: Task and Thread : To control program execution Port and Message : For InterProcess Communication MACH introduces a fifth one: ‘Memory Object’ The relationship (duality) between memory and communication: Flexibility and efficiency Multiprocessor support Performance 10 / 7 / 2004

Inter-Process Communication (IPC) Port: A communication channel (send/receive). May have any number of senders, but only one receiver. Message: Collection of data objects to be used in thread communications. Access to a port is granted by receiving a message containing a port capability. Tasks allocate/deallocate their ports to perform communication. Very clean and flexible interface, small number of abstractions. Analogy to TCP/IP communication: Instead of hosts, there are MACH tasks. Instead of firewall rules, there are port rights. Both provide reliability in transferring massages. 10 / 7 / 2004

Virtual Memory Management Each process has its own virtual address space. Maintenance of virtual address space is implemented by keeping a linearly ordered page table for each process in kernel. Only restriction: Regions (contiguous areas of address spaces) must be aligned on system page boundaries. Operations on a task can be performed by Virtual memory operation calls. MACH supports sharing of memory among tasks of ancestry through inheritance. 10 / 7 / 2004

External Memory Management Special-purpose memory managers, instead of kernel-supplied systems. MACH kernel acts like a cache manager Address space of a process Memory Objects External Memory Managers Mapped areas Destination Ref: Distributed Operating Systems: Concepts and Design, by Pradeep, K. Sinha 10 / 7 / 2004

…External Memory Management The interface between data manager and MACH kernel consists of three parts: Calls by an application to map a memory object into its address space Calls by the kernel to the data manager (generally as a respond to requests of data manager) Calls by the data manager on the MACH kernel to control the use of its memory object. 10 / 7 / 2004

Example: Consistent Network Shared Memory (1) Client A calls server to acquire X Client A Client B (5) Client B calls server to acquire X (4) Client A is resumed (2) Client A calls vm_allocate_with_pager to map X into its address space (6) Client B calls vm_allocate_with_pager to map X into its address space (8) Client B is resumed Shared Memory Server MACH Kernel A (3) Kernel A calls pager_init (…) (7) Kernel B calls pager_init (…) MACH Kernel B Initialization: Clients map object X into their address spaces 10 / 7 / 2004

Example: Consistent Network Shared Memory Client A Client B (1) Client A read faults (4) Client A is resumed (5) Client B read faults (8) Client B is resumed (6) Kernel B calls pager_data_request(…) Shared Memory Server (2) Kernel A calls pager_data_request(…) (3)Server calls pager_data_provided(…) (7) Server calls pager_data_provided(…) MACH Kernel A MACH Kernel B Clients read from the same shared memory data page 10 / 7 / 2004

Example: Consistent Network Shared Memory Client A Client B (1) Client A write faults on data being read by B (5) Client A is resumed Shared Memory Server (3) Server calls pager_flush_request(…) (2) Kernel A calls pager_data_unlock(…) (4) Server calls pager_data_lock(…) MACH Kernel A MACH Kernel B Client A writes a page being read on another host (copy_on_write) 10 / 7 / 2004

External Memory Management Implementation Address Maps: A doubly linked list to keep information of mappings from a range of addresses to a region of a memory object. Two levels: Top and Sharing. Virtual Memory Object Structures: A structure to represent the information (ports, size, etc.) about memory objects to conveniently release associated pages when it is destroyed. Resident Memory Structures: Each resident page corresponds to a page of physical memory. Records the memory object, its offset and permissions. A hashed table is kept for a fast lookup. Pageout Queues: For page replacements. Three kind of pageout queues are linked trough the resident page structures: active, inactive, free. Fault Handling: (!) Validity and protection, page lookup (cache), copy-on-write, hardware validation. 10 / 7 / 2004

Problems of External Memory Management Problems may occur if memory manager: does not return data fails to free flushed data floods the cache changes data backs its own data Most of these are can be prevented by various mechanisms like using some virtual memory calls, multiple threads, backing up data to safe memory. Memory Failures: (Analogous to communication failures) Specifying a timeout period: May require recovery (memory backed by ‘Default Pager’). Default Pager: Similar interface to other data managers, but a trusted component. 10 / 7 / 2004

Benefits & Applications Multiprocessor UMA, NUMA and NORMA systems can be supported. Mach primitives enables a programmer to choose from either shared memory and message passing paradigms Emulating Operating Systems Generic Operating system calls can be implemented. Theoretically, two operating system emulations can run on MACH kernel at the same time. Successful implementations for task migration, database management (Camelot) and AI knowledge bases (Agora) exists. 10 / 7 / 2004

Analysis of the Paper Good Bad A significant contribution to the literature Two expressive examples Detailed information of implementation Bad Big amount of information, more than needed (especially in multiprocessor issues section) No test results for performance evaluation 10 / 7 / 2004

Questions and Comments THANK YOU! 10 / 7 / 2004