User-Level Interprocess Communication for Shared Memory Multiprocessors By Brian N. Bershad, Thomas E. Anderson, Edward D. Lazowska, and Henry M. Levy. Presented by Damon Tyman (all figures/tables from the paper)

The progression LRPC (Lightweight Remote Procedure Call; Feb 1990) Optimize RPC for performance in “common case” cross-domain (same-machine) communication. URPC (User-Level Remote Procedure Call; May 1991) Eliminate the involvement of the kernel in cross-address space communication and thread management, utilizing shared memory and user-level threads.

Introduction Efficient interprocess communication encourages system decomposition across address space boundaries. Advantages of decomposed systems: Failure isolation Extensibility Modularity Unfortunately, when cross-address space communication is slow, these advantages are usually traded for better system performance.

Kernel-level interprocess communication Traditionally, interprocess communication has been the responsibility of the kernel. Problems with kernel-based communication: Architectural performance barriers Performance limited by cost of invoking the kernel and reallocating processor to a different address space. In authors’ earlier work, LRPC, 70% of overhead can be attributed to kernel mediation. Interaction with high-performance user-level threads For satisfactory performance, medium- and fine-grained parallel applications need user-level thread management. Costs (performance and system complexity) high for partitioning strongly interdependent communication and thread management across protection boundaries.

User-level cross-address space communication With shared memory multiprocessor, kernel can be eliminated from cross- address space communication. User-level managed cross-address space communication: Shared memory can be utilized as the data transfer channel. Messages sent between address spaces directly—no kernel intervention. Frequency of processor reallocation can be reduced by preferring threads from the same address space as the processor is already active in. The inherent parallelism of sending and receiving messages can be exploited for improved call performance.

User-level Remote Procedure Call (URPC) URPC provides safe and efficient communication between address spaces on the same machine without kernel mediation. Three components of interprocess communication isolated: processor reallocation, thread management, and data transfer. Control transfer between address spaces handled by thread management and processor reallocation. Compared to traditional “small kernel systems” in which the kernel handles all three components, in URPC only processor reallocation involves the kernel. URPC implementation cross-address space procedure call latency 93 microseconds compared to LRPC’s (using kernel-managed communication) 157 microseconds.

Messages vs. Procedure Calls Commonly, in existing systems communication between applications achieved by sending messages over a narrow channel that supports just a few operations. While powerful construct, message passing may require thinking in a paradigm not supported by the programming language in use. RPC better supports the programming paradigm supported by the programming language. Message-passing may serve as the underlying technique, but programming approach more familiar. With many programming languages, communication within an address space has synchronous procedure call, data typing, and shared memory features. RPC reconciles the above with the untyped, asynchronous messages of cross- address space communication by using an abstraction that hides these characteristics of the cross-address space communication from the user.

Synchronization To the programmer, cross-address space procedure calls appear synchronous (calling thread blocks waiting for return). At the thread management level and below, however, the call is asynchronous. While blocked, another thread can be run. URPC favors threads in the same address space as the processor is already running. As soon as the reply arrives, the blocked thread can be scheduled to any processor assigned to its address space. All the involved scheduling operations can be handled by a user-level thread management system, avoiding the need to reallocate any processors to a new address space (if the callee address space has at least one processor assigned to it).

User view/system components Two software packages used by stub and application code: FastThreads (user-level threads scheduled on top of middleweight kernel threads) and URPC package

Processor reallocation overhead There is significantly less overhead in switching a processor over to a thread in the same address space (context switching) than reallocating to a thread in a different address space (processor reallocation). Processor reallocation high overhead: Requires changing the protected mapping registers that define the virtual address space context. Scheduling costs to choose address space for reallocation. Substantial long-term costs due to poor cache and TLB performance from constant locality switches (greater for processor reallocation than just context switching). Minimal latency same-address space context switch takes about 15 microseconds on the C-VAX while cross-address space processor reallocation takes 55 microseconds (doesn’t consider long-term costs!).

Processor Reallocation Policy An optimistic reallocation policy is employed: threads from the same address space are scheduled whenever possible. Assumes: The client has other work to do. The server will soon have a processor with which to service a request. Optimistic assumptions may not always hold (allowing for a processor reallocation to server) Single-threaded applications High-latency I/O operations Real-time applications (bounded call latency) Server may not have its own processor. Voluntary return of processors cannot be guaranteed. No way to enforce policies regarding return of processors. If client gives processor up to server, may never get back.

Sample Execution One client (an editor) and two servers (window manager and file manager) each have their own address space. T1 and T2 are threads in the editor. Two available processors.

Data transfer using shared memory Data flows between URPC packages in different address spaces over a bidirectional shared memory queue with non-spinning test-and-set locks on either end. Receiver should never have to delay (spin-wait) while sender holds the lock. “Abusability factor” not increased by shared memory message channels Denial of service, bogus results, communication protocol violation still possible. Up to higher-level protocols to filter abuses up to application layer. Communication safety responsibility of stubs Arguments passed in shared memory buffers during binding phase. No kernel copying necessary.

Cross-address space call and thread management Strong correlation between communications functions and thread management synchronization functions RPCs synchronous with respect to calling thread Each communication function has corresponding thread management function (send <-> start, receive <-> stop) Thread classification: Heavyweight: kernel makes no distinction between thread and address space Middleweight: kernel-managed threads but decoupled from address spaces (can be multiple threads per address space) Lightweight: threads managed by user-level libraries Fine-grained parallel applications demand high-performance thread management, only possible with user-level threads. Two-level scheduling: lightweight user-level threads are scheduled on top of weightier kernel-managed threads.

URPC Performance Processing times close for client and server Nearly half of time goes to thread management (dispatch)

URPC Performance C: client processors, S: server processors; T:runnable threads in client address space Latency is traded for throughput; T=2 from T=1 means ~20% latency increase, but ~75% throughput increase

And beyond… LRPC: improve RPC for cross-domain communication performance. URPC: move inter-process communication to user-level as much as possible. Threads managed at user-level but scheduled on top of middleweight threads. Scheduler Activations (Feb 1992): provide better abstraction (than kernel threads) for kernel support of user-level threads.