B. N. Bershad, T. E. Anderson, E. D. Lazowska and H. M B.N. Bershad, T.E. Anderson, E.D. Lazowska and H.M. Levy Lightweight Remote Procedure Call ACM Transactions on Computer Systems, Vol.8, No.1, February 1990, 37-55 Lanfranco Muzi PSU – April 21th, 2005

Presentation outline What is LRPC? O.S. communication – most common case LRPC execution model Performance of LRPC Conclusions

What is LRPC? A communication facility designed and optimized for communication between protection domains in the same machine Target: small-kernel operating systems separate components placed in disjoint domains messages used for interdomain communication

O.S. communication - the common case Cross-domain vs cross-machine Vast majority of calls do not cross machine boundaries

O.S. communication: the common case Size and complexity of calls ~70% 95% of calls to 10 procedures 75% to 3 procedures! Most frequent case: less than 200 bytes Vast majority of calls: small, simple arguments/results

O.S. communication: the common case Performance (I) “In existing RPC systems, cross- domain calls are implemented in terms of the facilities required by cross-machine ones”

O.S. communication: the common case Performance (II) “Theoretical minimum”: 2 traps and 2 context switches Overhead: stub, message buffer, access validation, message transfer (queues), scheduling (independent threads), context switches…

LRPC execution model Client binds to server before making first call Call causes kernel trap Kernel: validate caller, create linkage, dispatch client’s thread to server Thread executes in server Control (and thread) back to kernel and caller

Binding - 1 Different from RPC “Import” call from client via kernel Server provides Procedure Descriptor List, used by kernel to allocate A-stacks and create linkage record At completion, kernel returns to client a Binding object and A-stack list

Binding - 2 A-stack: Read-write and shared by both domains Linkage record: Records caller’s return address + current SP Binding object: Client’s key to interface (must be presented to kernel at every call)

Calling Client stub: Kernel: Pop A-stack off the queue, push arguments Put A-stack address, Binding Object and procedure identifier into registers Trap Kernel: Validation etc. Find E-stack in server domain and update user thread stack pointer Reload CPU virtual memory registers with those of the server domain

Optimization 1 Argument copying RPC A-stack LRPC A-stack: Read-write and shared by both domains

Optimization 2 Stub: choice between cross-domain or –machine in 1st line Validation: only on call (linkage is in thread’s control block on return) Scheduling in RPC: different threads in each domain signalling at each other (block client, dispatch in server…)

Optimization 3 Multiprocessors: Reduced lock contention: each A-queue has its own lock – no other locking required Reduced context-switch overhead: Domain caching on idle processors – move to idle CPU avoiding context switch

Performance of LRPC - 1 Average on 100,000 cross-domain calls LRPC/MP uses domain caching TAOS uses RPC Taos/LRPC ~= 3!

Performance of LRPC - 2 The “context switches” cost is what domain caching helps reduce Single processor: LRPC adds only 48ms

Call throughput on a multiprocessor Performance of LRPC - 3 Call throughput on a multiprocessor Domain caching disabled RPC scales poorly due to lock contention, which is very limited in LRPC

Conclusions LRPC adopts an optimized approach to the most common case in O.S. communication Performs well on the majority of PC between address spaces on the same machine Avoids unnecessary synchronization, kernel-level thread management and data copying LRPC performs better than RPC in the common case, and scales better on multiprocessors Authors conclude that it is a good communication alternative for small-kernel O.S. (further developments?)

References B.N Bershad et al. “Lightweight remote procedure call” ACM Trans. Comput. Syst. 8,1 (Feb.1990) 37-55 A.D. Birrel et al. “Implementing remote procedure calls” ACM Trans. Comput. Syst. 2,1 (Feb.1984) 39-59 B.N. Bershad et al. “User-level interprocess communication for shared memory multiprocessors” ACM Trans. Comput. Syst. 9,2 (May1991) 175-198 A.S. Tanenbaum “Modern Operating Systems” 2nd ed. Prentice Hall (RPC: 534-540) Silberschatz, Galvin, Gagne “Operating system concepts” 6th ed., John Wiley & Sons (RPC: 117-124) “Remote procedure calls” Pznan University of technology – Institute of computing Science http://skisr.cs.put.poznan.pl/download/rso/lec/w4_rpc.doc