Reducing Interprocess Communication Overhead in Concurrent Programs

Reducing Interprocess Communication Overhead

in Concurrent Programs

Erik StenmanKostis Sagonas

http://www.erlang.org/index.html

Info

rmati

onst

ekn

olo

gi

ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 2

Motivation Concurrency as an abstraction is

important. Systems that need to interact with the outside

world are hard to model without concurrency. Unfortunately concurrency costs.

There are two types of runtime overheads: ”Direct overhead” of concurrency primitives. ”Indirect overhead” from hiding the data-flow from

the optimizing compiler.

Info

rmati

onst

ekn

olo

gi


Goal of this work Reduce the overhead of concurrency

in concurrent programs.

Idea Optimize the code that implements

process communication.We call this interprocess optimization, and we will present three techniques:1. Rescheduling send.2. Direct dispatch send3. Interprocess inlining.

Info

rmati

onst

ekn

olo

gi


Rescheduling Send Typically, when a process sends a

message, it is because it wants the receiver to act on that message.

It is therefore in the interest of the sender to yield to the receiver and allow it to promptly act on the sent message.

We call this type of send, a rescheduling send.

Info

rmati

onst

ekn

olo

gi


Implementation: The send operation suspends the

currently active (sending) process.

Benefits: lower message passing latency. better cache behavior (the receiver has

access to the message while it is still hot in the cache).

Rescheduling Send

Info

rmati

onst

ekn

olo

gi


Direct Dispatch Send The sender contributes the remaining part of its

time-slice to the receiver (hoping this would lead to a faster response).

The receiver then is woken up directly (ignoring the ready queue). Overhead of the scheduler is eliminated.

If the receiver also performs a direct dispatch send back to the sender, interprocess communication becomes as fast as a function call.

Info

rmati

onst

ekn

olo

gi


Interprocess Inlining Merge code from the sender with

code from the receiver.

Info

rmati

onst

ekn

olo

gi


Process The receiver

Process The sender

A message

Info

rmati

onst

ekn

olo

gi


Info

rmati

onst

ekn

olo

gi


Info

rmati

onst

ekn

olo

gi


Known communication protocol.Can be optimized.Process only needs to be ready to receive the communication.

Info

rmati

onst

ekn

olo

gi


The state of process has changed.

Without really participating in the actual communication.

Info

rmati

onst

ekn

olo

gi


Interprocess Inlining Merge code from the sender with

code from the receiver. In the merged code, the overhead of

the communication protocol can be optimized away.

We suggest using profiling to find situations where this is feasible.

This requires that the optimized code is guarded by a runtime test.

Info

rmati

onst

ekn

olo

gi


An example in Erlang…ref_server(V) -> receive {‘++’,From}-> From !

{self(),V+1}, ref_server(V+1); {‘:=’,From,NewV}-> From !

{self(),NewV}, ref_server(NewV); {‘=’,From}-> From !

{self(),V}, ref_server(V) end

inc(Beta) ->

Beta ! {‘++’,self()},

receive

{Beta,Answer} ->

Answer;

end.

Info

rmati

onst

ekn

olo

gi


…could be rewritten asref_server(V) -> receive {‘++’,From}-> From !

{self(),V+1}, ref_server(V+1); {‘:=’,From,NewV}-> From !

{self(),NewV}, ref_server(NewV); {‘=’,From}-> From !

{self(),V}, ref_server(V) end

inc´(Beta) ->

if ready(Beta)->

B_V=get(‘V’,Beta)+1,

save(‘V’,B_V,Beta),

B_V;

true ->

inc(Beta)

end.

Info

rmati

onst

ekn

olo

gi


Code merging: the general case

g - Head

Createmessage

Send

-Tail

g - Head

Receive

g – tail

’ - Head

Createmessage

Send

-Tail

Extractedg – tail

Ready

Copy

Restoreβ-State

Saveβ-State

Copy of-Tail

Yes

Info

rmati

onst

ekn

olo

gi


Code merging

Can only be done if: Code explosion does not occur. The code does not suspend. The control flow does not escape the

included code. The extracted code terminates

(o/w, the sending process might hang after code merging.)

Info

rmati

onst

ekn

olo

gi


Code merging These instructions are not extracted:

A call or a return. Instructions that lead to the suspension of the

process. Throwing an exception. Some BIFs are large and uncommon and not

worth the effort to adapt for external execution.

Non-terminating code is unacceptable. Hence, we do not accept any loops in the

control flow graph.

Info

rmati

onst

ekn

olo

gi


Code merging

g() ->

receive

{hi,From}->

From ! {self(),fine};

_ -> ignore

end,

g().

Createmessage

tail

send

f(Beta) ->

Beta ! {hi,self()},

receive

{Beta,fine} ->

right;

{Beta,_Other} ->

wrong

end.

Info

rmati

onst

ekn

olo

gi


Code merging

g() ->

receive

{hi,From}->


_ -> ignore

end,

g().

Test

f-tail

f(Beta) -> Mess = {hi,self()}, if ready(Beta)-> NEWCODE; true -> Beta ! Mess, receive {Beta,fine} -> right; {Beta,_Answer} -> wrong endend.

Info

rmati

onst

ekn

olo

gi


Code merging

g() ->

receive

{hi,From}->


_ -> ignore

end,

g().

NEWCODE: -Mbox = [{hi,self()}],

case -Mbox of [{hi, -From}]-> -From ! -{-self(),fine};

_ -> -ignore-end,save_state(Beta),COPY-OF-f-TAIL

Info

rmati

onst

ekn

olo

gi


Code merging

g() ->

receive

{hi,From}->


_ -> ignore

end,

g().


case [{hi,self()}]of [{hi, -From}]-> -From ! -{-self(),fine};

_ -> -ignore-end,save_state(Beta),COPY-OF-f-TAIL

Info

rmati

onst

ekn

olo

gi


Code merging

g() ->

receive

{hi,From}->


_ -> ignore

end,

g().


-From = self(), -From ! -{-self(),fine},

save_state(Beta),COPY-OF-f-TAIL

Info

rmati

onst

ekn

olo

gi


Code merging

g() ->

receive

{hi,From}->


_ -> ignore

end,

g().


-From = self(), self()! -{-self(),fine},


Info

rmati

onst

ekn

olo

gi


Return messages An important special case Handled by applying the same type

of rewriting. The ready test is replaced by a test

that checks that both mailboxes are empty.

Info

rmati

onst

ekn

olo

gi


More Code mergingg() ->

receive

{hi,From}->


_ -> ignore

end,

g().


-From = self(), self()! -{-self(),fine},


Info

rmati

onst

ekn

olo

gi



receive

{hi,From}->


_ -> ignore

end,

g().


-From = self(), -Mbox = {-self(),fine},


Info

rmati

onst

ekn

olo

gi



receive

{hi,From}->


_ -> ignore

end,

g().

NEWCODE:-Mbox = {-self(),fine},


Info

rmati

onst

ekn

olo

gi



receive

{hi,From}->


_ -> ignore

end,

g().

NEWCODE: -Mbox = {-self(),fine},

save_state(Beta),

case -Mbox of {Beta,fine} ->

right;

{Beta,_Other} ->

wrong

end

Info

rmati

onst

ekn

olo

gi



receive

{hi,From}->


_ -> ignore

end,

g().


save_state(Beta),case {-self(),fine} of

{Beta,fine} -> right; {Beta,_Other} -> wrong end

Info

rmati

onst

ekn

olo

gi



receive

{hi,From}->


_ -> ignore

end,

g().


save_state(Beta),

right

Info

rmati

onst

ekn

olo

gi



receive

{hi,From}->


_ -> ignore

end,

g().

NEWCODE: save_state(Beta),

right

Info

rmati

onst

ekn

olo

gi



receive

{hi,From}->


_ -> ignore

end,

g().

f´(Beta) -> if ready(Beta)->

save_state(Beta), right; true -> f(Beta)end.

Info

rmati

onst

ekn

olo

gi


This optimization gives Access to variables in another

process with almost no overhead. (Two reads, one test, and two writes.)

The overhead of the communication protocol can be reduced. (Creating a

tuple and switching on it.)

Info

rmati

onst

ekn

olo

gi


Profiling We have profiled parts of Eddie and

the AXD code and found: That each send goes to one particular

receive. The receiving process is typically

suspended with an empty mailbox. The same profiling tool could be used in a

dynamic compiler to find pairs of senders/receivers whose inter-process communication can be optimized.

Info

rmati

onst

ekn

olo

gi


Conclusions Presented several different methods

for cross-process optimization that reduce the overhead of interprocess communication.

No modifications to existing code are required.

Open issue is their integration into the Erlang development environment

Info

rmati

onst

ekn

olo

gi


Praeterea censeo "0xCA" scribere Erlang posse.Happi

That’s all folks!

Info

rmati

onst

ekn

olo

gi


Problem How does one find senders and

receivers that communicate? Static analysis

Problematic in a language with dynamic typing, dynamic process creation, communication through mailboxes, and asynchronous communication.

Profiling & dynamic recompilation

Info

rmati

onst

ekn

olo

gi


Interprocess inlining Do:

Find two communicating processes. Merge code from the sender with code

from the receiver. Optimize the merged code.

Get: Reduced message passing. Short-circuited code for switching on

messages.

Info

rmati

onst

ekn

olo

gi


An example in Erlang…

g() ->

receive

{hi,From}->


_ -> ignore

end,

g().

f(Beta) ->

Beta ! {hi,self()},

receive

{Beta,fine} ->

right;

{Beta,_Other} ->

wrong

end.

Info

rmati

onst

ekn

olo

gi


…could be rewritten as

g() ->

receive

{hi,From}->


_ -> ignore

end,

g().

f´(Beta) -> if ready(Beta)->

save_state(Beta), right; true -> Beta ! {hi,self()}, receive {Beta,fine} -> right; {Beta,_Other} -> wrong end end.

Documents

Reducing Interprocess Communication Overhead in Concurrent Programs