Implementing Remote Procedure Calls ANDREW D. BIRRELL and BRUCE JAY NELSON Presented by Tony Bock

Remote Procedure Calls Bring Threaded Model to IPC Individual processes have their own address spaces providing a natural protection boundary relative to other processes Individual processes have their own address spaces providing a natural protection boundary relative to other processes Previously, coordination between address spaces required message passing and event-oriented style Previously, coordination between address spaces required message passing and event-oriented style RPC allows process-oriented, thread driven programming model for IPC whether local or remote RPC allows process-oriented, thread driven programming model for IPC whether local or remote

Advantages of RPC Simple – Straightforward semantics make it easier to build and maintain correct distributed programs Simple – Straightforward semantics make it easier to build and maintain correct distributed programs Efficient – Procedure call mechanism appears simple enough to enable rapid communication Efficient – Procedure call mechanism appears simple enough to enable rapid communication General – Procedures are the common way to communicate between portions of a program General – Procedures are the common way to communicate between portions of a program

Challenges Linking functions in a separate address space Linking functions in a separate address space Discovering/specifying RPC target Discovering/specifying RPC target Calling/returning across processes Calling/returning across processes Passing arguments by reference Passing arguments by reference Unreliability of networks Unreliability of networks Retransmits must not lead to multiple callsRetransmits must not lead to multiple calls Crashes can occur at either end at any timeCrashes can occur at either end at any time Deliver High Performance Deliver High Performance

Linking: Use Stubs to Import/Export Interfaces Application is linking against functions that reside in a different address space Application is linking against functions that reside in a different address space Server is returning results to separate address space Server is returning results to separate address space Server and client link to programmatically generated stubs of a common interface module at compile time Server and client link to programmatically generated stubs of a common interface module at compile time RPC runtime manages communication between machines/processes RPC runtime manages communication between machines/processes User writes interface, client, and server code but doesn’t need to write any code for communication mechanism User writes interface, client, and server code but doesn’t need to write any code for communication mechanism Client Stubs (Import) Server Stubs (Export) Interface Module bool Foo(int) int Bar(char) int Baz(int) Application Function Library RPC Runtime RPC Runtime NETWORK

Binding – Matching Callers to Callees Stubs are only place holders, client still needs to find the server Stubs are only place holders, client still needs to find the server Servers register their exported interfaces with secure database servers Servers register their exported interfaces with secure database servers Register time-based unique server IDRegister time-based unique server ID Clients can specify or select server from available list or bind statically by network address Clients can specify or select server from available list or bind statically by network address Get server’s ID and network addressGet server’s ID and network address Clients include server’s ID and unique sequence number with each call Clients include server’s ID and unique sequence number with each call Name Address Interface Server ID Interface? Name Address Server ID Interface Function Offset Server ID Sequence Number ServerClient Database

Crossing Process Boundaries Caller and Callee have separate address spaces Caller and Callee have separate address spaces Can’t dereference pointers directlyCan’t dereference pointers directly Caller needs to call functions in separate processCaller needs to call functions in separate process Callee needs to return to separate processCallee needs to return to separate process RPC runtime layers on server & client coordinate communication across network RPC runtime layers on server & client coordinate communication across network Client calls result in messages sent by RPC to server with index of desired function from client stubsClient calls result in messages sent by RPC to server with index of desired function from client stubs Server RPC uses index to find desired function in server stubs and switches to serverServer RPC uses index to find desired function in server stubs and switches to server Server “calls home” to ask client for values when dereferencingServer “calls home” to ask client for values when dereferencing Client Stubs (Import) Server Stubs (Export) Interface Module bool Foo(int) int Bar(char) int Baz(int) Application Function Library RPC Runtime RPC Runtime Foo() Index of Foo() Foo() Index of Foo() *value? CallerCallee

Reliability RPC must overcome network’s inherent unreliability RPC must overcome network’s inherent unreliability Network can drop any packetNetwork can drop any packet Either machine can crash at any timeEither machine can crash at any time Mustn’t crash others by so doing Mustn’t crash others by so doing Need to discern crash vs. running slowly Need to discern crash vs. running slowly Each application issues at most one RPC to a particular server/interface at time Each application issues at most one RPC to a particular server/interface at time Each request includes a monotonically increasing sequence number & the server IDEach request includes a monotonically increasing sequence number & the server ID Sequence ID includes “conversation ID”, a unique time-based client identifierSequence ID includes “conversation ID”, a unique time-based client identifier

Reliability Server checks ID’s when receiving calls and keeps track of current (highest) sequence number Server checks ID’s when receiving calls and keeps track of current (highest) sequence number If server ID doesn’t match, server has crashed since client connected: dropIf server ID doesn’t match, server has crashed since client connected: drop Keeps track of current (highest) sequence numberKeeps track of current (highest) sequence number < current, this is a retransmit of old request: drop < current, this is a retransmit of old request: drop = current, retransmit of current request: issue ACK if requested, then drop = current, retransmit of current request: issue ACK if requested, then drop > current, new request: client acknowledges receipt of RPC result > current, new request: client acknowledges receipt of RPC result Conversation IDs ensure sequence numbers don’t repeat if client restartsConversation IDs ensure sequence numbers don’t repeat if client restarts

Packet Protocol – Simple Case Existing protocols focused on one-way bulk data transfers Existing protocols focused on one-way bulk data transfers Even substantial per-connection overhead is insignificant to data transmit timeEven substantial per-connection overhead is insignificant to data transmit time RPC generates many small packets in each direction so it uses a minimal protocol RPC generates many small packets in each direction so it uses a minimal protocol RPC result implies acknowledgment of request – function has run exactly onceRPC result implies acknowledgment of request – function has run exactly once Each RPC request implies acknowledgment of previous result – Each application has at most one outstanding requestEach RPC request implies acknowledgment of previous result – Each application has at most one outstanding request Sender must resend until acknowledged, retransmits include request for ACKSender must resend until acknowledged, retransmits include request for ACK Advantages of RPC protocol Advantages of RPC protocol Minimal per-connection setup and teardown costsMinimal per-connection setup and teardown costs Minimal state when connection is idleMinimal state when connection is idle Server just keeps last sequence number Server just keeps last sequence number Client has a list of server addresses & IDs Client has a list of server addresses & IDs Minimize delay between RPC request and response – no handshaking phaseMinimize delay between RPC request and response – no handshaking phase No idle time communication, i.e.: keep-alive packetsNo idle time communication, i.e.: keep-alive packets Request Result

Multi-Packet Requests Need to make sure all packets arrive in order Need to make sure all packets arrive in order Ask for explicit ACK with each transmit except the lastAsk for explicit ACK with each transmit except the last Retransmits still ask for explicit ACKRetransmits still ask for explicit ACK Return result still implies receipt of last packetReturn result still implies receipt of last packet

Crashed or Slow? If server crashes, caller needs to move on otherwise wait for slow server If server crashes, caller needs to move on otherwise wait for slow server While waiting for response caller periodically pings server While waiting for response caller periodically pings server If responses keep coming, server is just slow: wait forever (just like local procedure call)If responses keep coming, server is just slow: wait forever (just like local procedure call) If responses stop coming, server is crashed: throw exceptionIf responses stop coming, server is crashed: throw exception You There? Yes. Exception: Comm_Failed

Performance Both machines maintain multiple idle server processes ready to handle RPCs Both machines maintain multiple idle server processes ready to handle RPCs Reduces cost of process creationReduces cost of process creation Process may be created/destroyed to adapt to trafficProcess may be created/destroyed to adapt to traffic Each packet includes source and destination process ID Each packet includes source and destination process ID When expecting a packet, a process registers with the Ethernet handlerWhen expecting a packet, a process registers with the Ethernet handler When the handler receives an expected packet, it passes it directly to the waiting process with only one context switchWhen the handler receives an expected packet, it passes it directly to the waiting process with only one context switch If no one’s waiting, packet passes to idle server process for decision makingIf no one’s waiting, packet passes to idle server process for decision making

Summary RPCs allow for procedure oriented programming across process boundaries RPCs allow for procedure oriented programming across process boundaries Programmatically generated stubs abstract transition between processesProgrammatically generated stubs abstract transition between processes RPC runtime translates calls to networkRPC runtime translates calls to network User only writes application and server codeUser only writes application and server code Lightweight protocol minimizes per-client server load, handshaking, and idle state maintenance Lightweight protocol minimizes per-client server load, handshaking, and idle state maintenance System of time-based ID’s along with active state pinging enhances reliability and aids in securing network System of time-based ID’s along with active state pinging enhances reliability and aids in securing network