Networking Implementations (part 1)

CPS210Spring 2006

Papers

The Click Modular Router Robert Morris

Lightweight Remote Procedure Call Brian Bershad

Procedure Calls

main (int, char**) { char *p = malloc (64); foo (p);}

foo (char *p) { p[0] = ‘\0’;}

Code + Data

Heap

main: argc, argv

Stack

0x54320

0xf324

0xfa3964

foo: 0xf33c, 0xfa3964

RPC basics

Want network code to look local Leverage language support

3 components on each side User program (client or server) Stub procedures RPC runtime support

Building an RPC server

Define interface to server IDL (Interface Definition Language)

Use stub compiler to create stubs Input: IDL, Output: client/server stub

code Server code linked with server stub Client code linked with client stub

RPC Binding Binding connects clients to servers Two phases: server export, client import In Java RMI

rmic compiles IDL into ServerObj_{Skel,Stub} Export looks like this

Naming.bind (“Service”, new ServerObj()); ServerObj_Skel dispatches requests to input ServerObj

Import looks like this Naming.lookup("rmi://host/Service"); Returns a ServerObj_Stub (subtype of ServerObj)

Remote Procedure Calls (RPC)

main (int, char**) { char *p = malloc (64); foo (p);}

// client stubfoo (char *p) { // bind to server socket s (“remote”); // invoke remote server s.send(FOO); s.send(marsh(p)); // copy reply memcpy(p,unmarsh(s.rcv())); // terminate s.close();}

Code + Data

Heap

main: argc, argv

Stack

foo: 0xf33c, 0xfa3964

Code + Data

Heap

RPC_dispatch: socket

Stack

stub: 0xd23c, &s

// serverfoo (char *p) { p[0] = ‘\0’;}

foo_stub (s) { // alloc, unmarshall char *p2 = malloc(64); s.recv(p2, 64); // call server foo(p2); // return reply s.send(p2, 64);}

RPC_dispatch (s) { int call; s.recv (&call); // do dispatch switch (call) { … case FOO: // call stub foo_stub(s); …} s.close ();}

foo: 0xd23c, 0xfb3268

1) Bind2) Invoke and reply3) Terminate

RPC Questions

Does this abstraction make sense? You always know when a call is remote

What is the advantage over raw sockets?

When are sockets more appropriate? What about strongly typed languages?

Can type info be marshaled efficiently?

LRPC Context

In 1990, micro-kernels were all the rage Split OS functionality between “servers” Each server runs in a separate addr

space Use RPC to communicate

Between apps and micro-kernel Between micro-kernel and servers

Micro-kernels argument

Easy to protect OS from applications Run in separate protection modes Use HW to enforce

Easy to protect apps from each other Run in separate address spaces Use naming to enforce

How do we protect OS from itself? Why is this important?

Mach architecture

Kernel

Userprocess

Fileserver

PagerMemoryserver

Processsched.

Comm.

Network

LRPC Motivation Overwhelmingly, RPCs are intra-machine RPC on a single machine is very expensive

Many context switches Much data copying between domains

Result: monolithic kernels make a comeback Run servers in kernel to minimize overhead Sacrifices safety of isolation

How can we make intra-machine RPC fast? (without chucking microkernels altogether)

Baseline RPC cost

Null RPC call void null () { return; }

1. Procedure call2. Client to server: trap + context

switch3. Server to client: trap + context

switch4. Return to client

Sources of extra overhead Stub code Marshaling and unmarshaling arguments

User1 Kernel, KernelUser2, back again Access control (binding validation) Message enquing and dequeuing Thread scheduling

Client and server have separate thread pools Context switches

Change virtual memory mappings Server dispatch

LRPC Approach

Optimize for the common case: Intra-machine communication

Idea: decouple threads from address spaces For LRPC call, client provides server

Argument stack (A-frame) Concrete thread (one of its own)

Kernel regulates transitions between domains

1) Binding

Code + Data

Heap

Stack

Code + Data

Heap

Stack

LRPC runtimeLRPC runtime

Kernel

C S

Clerk

PDL(S) set_name{addr:0x54320, conc:1, A_stack_sz:12}

import “S”

main: argc, argv

// server codechar name[8];set_name(char *newname) { int i, valid=0; for (i=0;i<8;i++) { if(newname[i]==‘\0’){ valid=1; break; } } if (valid) return strcpy(name, newname); return –EINVAL;}

0x54320

C import req.A-stack (12 bytes)

A-stack (12 bytes)

0x74a28LR{}

0x74a28 0x761c2

BindObj

BindObj

2) Calling

Code + Data

Heap

Stack

Code + Data

Heap

Stack

LRPC runtimeLRPC runtime

Kernel

C S

Clerk

PDL(S) set_name{addr:0x54320, conc:1, A_stack_sz:12}

main: argc, argv

// server codechar name[8];set_name(char *newname) { int i, valid=0; for (i=0;i<8;i++) { if(newname[i]==‘\0’){ valid=1; break; } } if (valid) return strcpy(name, newname); return –EINVAL;}

0x54320

A-stack (12 bytes)

A-stack (12 bytes)

”foo”

“foo”

0x74a28LR{}

0x74a28 0x761c2

BindObj

set_name: “foo”

&BindObj,0x7428,set_name

0x74a28LR{Csp,Cra}

server_stub:

set_name: 0x761c2

”foo”, 0

“foo”, 0

Data copying

Questions

Is fast IPC still important? Are the ideas here useful for

VMs? Just how safe are servers from

clients?