CS 582 / CMPE 481Distributed Systems

Communications (cont.)

Remote Procedure Call(RPC)

Motivation• Construction of distributed applications is a difficult task

– Heterogeneity– Separation– Partial failure

• Leverage a local procedure call mechanism [Birrell & Nelson]– procedure calls are a well-known and well-understood mechanism

for transfer of control and data within a program– Well engineered processes operate in isolation (black box) – the same principle can be extended to the client-server interaction

• clean and simple semantics• Efficiency• Generality

– Communication between caller & callee can be hidden by using procedure call mechanism

Local Procedure Call

before after

• E.g. read(int fd, char* buf, int bytes)1. push parameter values of the procedure on a stack2. call procedure3. use stack for local variables4. pop results (in parameters)

• “communication” with local procedure – copy data to/from stack

RPC Principle

LPC vs. RPC• Client-server fails independently

– extra exception handling– return value needs to be overloaded with failure codes

from the server• Global variables and pointers cannot be used across

the interface

Design Consideration• Abstraction of RPC semantics• Heterogeneity• Parameter passing• Binding• Failure Transparency• Support for concurrency• Orphan handling• Performance

Level of Abstraction of RPC Semantics• Integrate into the language

– extend language to provide constructs for RPC semantics– examples: Cedar, Argus, Mercury

• Describe in a special purpose interface definition language– interface definition language supports language

independence• limited representation of parameter types

– examples: Sun XDR, DCE IDL, Xerox Courier

Example: Argus• Provides programming language constructs for

remote operation invocation – Guardian

• a special kind of an object which implements a number of procedures that are run in response to remote requests

• interface is defined as a set of handlers– Action

• supports atomic transactions

• Ref– B. Liskov, “Distributed programming in Argus,”

CACM 31(3), pp. 300-312, March 1988.

Examples: Sun RPC• Provides XDR language for interface

definition– each procedure is numbered with a version

number– the type of a result and an input parameter is

either simple or complex (e.g. struct)– only one parameter is allowed

(when -N option is not used)• parameter is always input• multiple parameters -> a single structure

Heterogeneity• Marshalling policy

– conversion to the common data type agreed by both a sender and a receiver: Sun RPC

– a receiver makes it right (tagging): NCS RPC– sender negotiates with a receiver

• RPC mechanisms– define a veneer layer to encapsulate multiple RPC

implementations [HRPC]• control, data representation, and transport components

• Accommodate multiple transport protocols

Parameter Passing• Considerations for parameter passing in RPC

– direction of parameters• input: client -> server• output: server -> client• input & output: client <-> server

– call-by-reference• use call-by-value-result instead

– pointer parameters• no meaning to pass a pointer to memory location• linearize before passing it

– procedure parameter• procedure should be accessible across the network

Binding• Naming

– mapping from a name to a particular service• static linking: most RPCs• dynamic linking: Apollo NCS RPC• procedure variable: Argus, Mercury

– exportation and importation• server exports its service interface• client imports exported service

– interface consists of• name: uniquely identifiable

– version info and location info can be included

• signature: parameter name & type

Binding

• How to determine where server is? Which procedure to call?– “Resource discovery” problem

• Broadcast requests– broadcast request & process incoming replies

• Name service: advertises servers and services.– Example: Birrel et al. uses Grapevine.

• Early versus late binding.– Early: server address and procedure name hard-coded in

client.– Late: go to name service.

Binding (cont.)•Locating a server

– via a binder (like a directory service)• well-know address• run-time notification• broadcasting (recruiting)

Failure Transparency• Ordered delivery

– request and reply delivered in order (e.g. Argus)• RPC Execution semantics

– How many times will a remote procedure be executed by the callee per call?

– Semantics• at-least-once• at-most-once• exactly-once

Execution Semantics• At-least-once

– when a caller receives the expected result, it implies that the requested call has been executed one or more (at least once) at a callee

– the callee does not remember multiple executions due to call duplication or retry after failure

– acceptable if operations are idempotent – example: Sun RPC - NFS

Execution Semantics (cont.)• At-most-once

– when a caller receives the expected result, it implies that the requested call has been executed exactly once. If not completed, executed zero or one time

– the callee can detect duplicated requests at the absence of failure but persist through the abnormal termination (failure amnesia)

– example: NCS RPC

Execution Semantics (cont.)• Exactly-once

– when a caller receives the expected result, it implies that the requested call has been executed exactly once. If not completed, executed zero times

– the callee is failure-resilient: logging and two-phase commit

– example: Argus

Support for Concurrency• RPC behaves differently from LPC

– the client does not need to be blocked while the server executes the requested call

– the client may not need the result• Approaches

– Synchronous RPC• a process/thread per call (at a client or a server)

– Asynchronous RPC• two types

– no result– delayed result: delayed synchronous

• Examples: Argus, Mercury(Promises), X11 RPC

Synchronous & Asynchronous RPC

Synchronous Asynchronous

Client Server Client Server

Orphan Handling• A server becomes an orphan when its results are no longer

needed due to– duplicate call messages– client crashes or aborts in the middle of a call

• Orphans are bad because– resource waste– Interference– Inconsistency

• Orphan termination– server detects a duplicate message and aborts an orphan if exists – client keeps a list of servers and issues abort requests to them as

part of the crash recovery procedure

Performance• Performance consideration points

– marshalling and unmarshalling– message transmission– request execution semantics– execution time

• Approaches– Leverage highly tuned transport protocols– Allow an application to make a right choice based on its

needs• request-n-reply: high throughput -> RPC• large data transfer: low latency -> byte streams

RPC Implementation

work

Caller Callee

Callpacket

Result

UserClientstub

RPCruntime

RPCruntime

Serverstub Server

call pckargs

xmit rcv unpk call

returnpckresult

xmitrcvunpkresult

return

RPC Implementation• RPC runtime mechanism responsible for

retransmissions, acknowledgments.• Stubs responsible for data packaging and un-

packaging; – AKA marshalling and un-marshalling:

putting data in form suitable for transmission. Example: Sun’s XDR.

Case Study : Sun RPC• Defines format for messages, arguments, and

results.– Uses UDP or TCP.

• over UDP message length ≤ 64 kbytes ~ 8-9 kbytes– broadcast RPC is an option– Uses XDR (eXternal Data Representation) to represent

procedure arguments and header data.– Compiler system to automatically generate distributed

programs.– Remote execution environment: remote program

• mutually exclusive execution of procedure in remote program

Identifying Remote Programs and Procedures

• Conceptually, each procedure on a computer is identified by pair :– (prog, proc)

• prog: 32-bit integer identifying remote program• proc: integer identifying procedure

– Set of program numbers partitioned into 8 sets.• 0x00000000 - 0x1fffffff assigned by SUN• 0x20000000 - 0x3fffffff assigned by local system manager• 0x40000000 - 0x5fffffff temporary• 0x60000000 - 0xffffffff reserved

– Multiple remote program versions can be identified:(prog, version, proc)

Example RPC Program Numbersname assigned no descriptionportmap 100000 port mapperrstatd 100001 rstat, rup, perfmeterrusersd 100002 remote usersnfs 100003 file systemypserv 100004 yp (NIS)mountd 100005 mount, showmountdbxd 100006 DBXprog (debug)ypbind 100007 NIS binderwalld 100008 rwall, shutdownyppasswdd 100009 yppasswd

Communication Semantics• TCP or UDP ?• Sun RPC semantics defined as function of

underlying transport protocol.• RPC on UDP: calls can be lost or duplicated.

• at-least-once semantics if caller receives reply.• zero-or-more semantics if caller does not receive

reply. Programming with zero-or-more semantics: idempotent procedure calls.

• Sun RPC retransmission mechanism:– non-adaptive timeouts– fixed number of retransmissions

Remote Programs & protocols Ports