Implementing Remote Procedure Calls

Andrew D. Birrell and Bruce Jay Nelson

Xerox Palo Alto Research Center

Published:

ACM Transactions on Computer Systems, Vol. 2, No. 1, February 1984, pages 39-59.

Cristopher Holm

Authors:

RPC

RPC (remote procedure call) is a mechanism for transfer of control and data, and subsequent resumption of control:

•It acts in the same manner as a local procedure call:

•Some process or thread is executing …

•A procedure call is made, suspending execution of the caller

•The procedure runs to completion and returns control to the caller

•The caller resumes execution

•The authors wanted their implementation of this concept to be relatively transparent to the programmer, so that an RPC call would look and feel semantically like a local procedure call.

Their development environment was the Cedar programming language, running the Mesa OS on Dorado hardware.

Issues

1. Call Semantics with the possibility of machine or communications failure.

2. Call arguments that include addresses when shared address space may or may not be utilized.

3. Integration of RPC into existing and future systems.

4. Binding procedure caller and procedure callee.

5. Transfer protocols for data and control.

6. Ensuring data integrity and security.

Goals

1. To make distributed computation (i.e. – RPC) easy.

2. Efficiency in RPC communication.

3. To make RPC semantics as powerful as possible, while still preserving simplicity and efficiency.

4. Ensuring secure and reliable communication.

5. Making the semantics of RPC as close as possible to that of local procedure calls.

RPC StructureCaller

RPCRuntime

Transmit packet(s)

Receive packet(s)

Send packet

NetworkNetwork

NetworkNetwork

Receive packet

User StubUser

Procedure call

procedure return

Packarguments

Unpackresult(s)

Callee

RPCRuntime

Receivepacket(s)

Transmitpacket(s)

Server Stub Server

Procedure call

Procedure return

Unpackarguments

Packresult(s)

Receive packet

Send packet

An interface is made up of two components:

1. Type – what it can do.This concept is similar to an object oriented programming interface:

• An entity provides an interface to the outside world, stating how it behaves, what data it expects for this behaviour and what data it provides (if any).

• The underlying functionality is hidden.

2. Instance – a particular provider of this type.The instance is similar to an object that implements the OOP interface.

BindingThe caller needs to bind to a callee that can perform the remote procedure. This is specified for the callee by the abstraction that the authors call an interface.

Authors’ interface example:Type: mail serverInstance: a specific mail server of type mail server

Binding, ContinuedThe caller binds to the callee by specifying something to uniquely identify the callee (Naming in this case) and the callee’s location.

But first the caller must find out what callees are available to handle the request at the time of the procedure call. This is accomplished by a database lookup:

•When a callee wishes to export an interface (make it available to callers), it stores information about its interface in a network accessible database.

Database

RPCRuntime

UpdateDatabase

Server Stub Server

ExportInterface

ExportInterface

Callee

Binding, ContinuedThe caller can then find the server callee in a database lookup:

•By specifying a particular instance of the desired interface type and receiving location information about that instance, or

•By specifying the type of the interface and receiving a list of instances that implement that type, and then iterating through them to find an available match.

Caller

RPCRuntimeUser StubUser

ImportInterface

ImportInterface

QueryDatabase

InterfaceInfo

InterfaceInfo

InterfaceInfo

Database

Who’s available?

AvailableInterfaces

Transport Protocol

The protocol used is intended for small, discrete chunks of data, each of which can contain:

•Identifiers specifying caller, callee and call.

•Requested procedure and procedure arguments.

•Procedure results.

•Acknowledgements of received packets.

•Exception information.

A caller may send requests for acknowledgement to the callee, and as long as the callee responds, the caller may wait indefinitely for results if the remote procedure deadlocks or loops (just like local procedure calls).

Protocol, Continued

Simple calls

•Retransmission of a packet (either from caller or callee) occurs until an acknowledgement is received.

•To the caller, a received packet containing the procedure results is viewed as an acknowledgement.

•To the callee, a received packet containing a new procedure call is viewed an an acknowledgement of the last procedure result sent.

•Each call by the caller carries a unique identifier so that subsequent calls to the same procedure may be processed, but duplicate packets (from retransmissions) for the same call will be discarded.

•Any given caller (process or thread on a given machine) will have at most one outstanding remote call.

Protocol, Continued

Complicated calls

•An acknowledgement is expected for each packet sent.

•The caller may send additional packets, called probes, if the callee is taking a long time to send results. After a certain threshold of probes sent without an acknowledgment, the caller may raise an exception to the user about a communication failure (again, a deadlocked callee can’t be detected).

•If the contents of a packet (procedure arguments or return results) are too large to fit in one packet, multiple packets are sent with all but the last requiring acknowledgement before transmission of the next. Each packet is sequentially marked.

Protocol, ContinuedCaller

RPCRuntime

Transmit first packet

Receiveresult

User

Procedure call

procedure return

Callee

Server

Procedure call

procedure return

RPCRuntime

Call[Ids, packet=0]

Ack[Ids, packet=0]

Call[Ids, packet=1]Transmit next packet

Transmit ack

Receivepacket 0

Receivepacket 1

Receive ack

Retransmit next packet

Call[Ids, packet=1, needAck] Receivepacket 1

Transmit ackAck[Ids, packet=1]

Receive ack

Transmitresult

Transmit ackrequest

Result[Ids]

Result[Ids, needAck]Receiveresult

Transmit ackAck[Ids]

Receive ack

Exceptions

•Exceptions for RPC are published in a server’s interface along with all of the normal procedure calls.

•In this way, the remote call acts just like a local call, propagating any exceptions back to the caller and any handlers that may be waiting there to catch them.

•One additional exception is available, the RPCRuntime call failed exception, which provides one of the few differences between a local and remote procedure call. This exception may be raised when there are communication difficulties.

ProcessesA server callee maintains a pool of available server processes to handle incoming requests:

•This saves the cost of creating a new process to handle each request.

•A new process is created to handle a new request when the available processes are busy.

•To save on the costs of context switches between processes, each packet contains Ids of calling and serving processes.

•A typical simple call will incur 4 process context switches, including context switching between incoming packet interrupt handler and the target process identified by the process ID in the packet.

Performance

Procedure

0 args / 0 results*

1 arg / 1 result*

2 args / 2 results*

4 args / 4 results*

10 args / 10 results*

1 word array**

4 word array**

10 word array**

40 word array**

100 word array**

Resume exception

Unwind exception

Minimum (s)

1059

1070

1077

1115

1222

1069

1106

1214

1643

2915

2555

3374

Median (s)

1097

1105

1127

1171

1278

1111

1153

1250

1695

2926

2637

3467

Transmission (s)

131

142

152

174

239

131

174

239

566

1219

284

284

Local-only (s)

9

10

11

12

17

10

13

16

51

98

134

196

* n word-size arguments passed to the callee, n word-size return values returned to the caller

** arguments and return values were packed in word arrays of size n

Listed below are times for remote procedures to complete in comparison to local procedure calls:

Conclusion

The authors concluded that their implementation and its performance was acceptable, and noted that programmers at Xerox had begun to use RPC with success.

They noted that there were a number of optimizations that could be made to improve performance, but viewed some of these as “extreme measures”.

Implementations were created in a number of other languages, including C and SmallTalk.