1

Distribuerede systemer og sikkerhed – 14. marts 2002

From Coulouris, Dollimore and Kindberg

Distributed Systems: Concepts and Design

Edition 3, © Addison-Wesley 2001

Presentation based on slides for the book:

Slides modified by Jens B Jorgensen, University of Aarhus

2

Chapter 11: Coordination and Agreement

From Coulouris, Dollimore and Kindberg

Distributed Systems: Concepts and Design

Edition 3, © Addison-Wesley 2001

3

Failure model

Assumptions: Independent processes, reliable channels, no Byzantine errors.

Failure detector: May be unreliable (yield “Unsupected” or “Suspected”). May be reliable (yield “Unsuspected” or “Failed”).

Crashedrouter

4

Distributed mutual exclusion

Situation: A number of processes want to access some shared resource.

Problem: Prevent interference, maintain consistency; critical section.

Examples: Shared files. Car park monitoring.

5

Distributed mutual exclusion – basics

Application-level protocol: enter() resourceAccess() exit()

Correctness criteria: Safety Liveness ->-fairness

Performance measures: Bandwith. Client delay. Throughput. Synchronization delay.

6

Distributed mutual exclusion – central server algorithm

Server

1. Requesttoken

Queue ofrequests

2. Releasetoken

3. Granttoken

4

2

p4

p3p

2

p1

7

Distributed mutual exclusion – ring-based algorithm

pn

p2

p3

p4

Token

p1

8

Distributed mutual exclusion – time-stamp based algorithm

On initializationstate := RELEASED;

To enter the sectionstate := WANTED;Multicast request to all processes; request processing deferred hereT := request’s timestamp;Wait until (number of replies received = (N – 1));state := HELD;

On receipt of a request <Ti, pi> at pj (i ≠ j)if (state = HELD or (state = WANTED and (T, pj) < (Ti, pi)))then

queue request from pi without replying; else

reply immediately to pi;end if

To exit the critical sectionstate := RELEASED;reply to any queued requests;

9

Distributed mutual exclusion – time-stamp based algorithm example

p3

34

Reply

34

41

4141

34

p1

p2

ReplyReply

10

Elections

Situation: A unique process to play a particular role among a set of processes must be chosen.

Problem: All processes must agree on the choice.Examples:

Central server algorithm for mutual exclusion. Coordinator process in Berkeley algorithm for internal

clock synchronization.

11

Elections – basics

Protocol: A process may call an election. Any process is either participant or non-participant in an election. The elected process should be chosen as the one with largest

identifier.

Correctness criteria: Safety. Liveness.

Performance measures: Bandwidth. Turn-around time.

12

Elections – ring-based algorithm

N processes arranged in a ring; a coordinator must be elected; no failures occur.

Initially, each process is non-participant. Some process pi sends an election message, elctn(i). When a process pr receives a message elctn(i):

If r<i then forward elctn(i); participant(pr) := true; endif; If (r>i and not(participant(pr))) then

forward (elctn(r)); participant(pr) := true; endif;

If (r>i and participant(pr)) then skip (* do not forward *); endif; If r=i then participant(pr) := false; send(elctd(r)); endif;

When a process pr receives an elected message, elctd(c): participant(pr) := false; elected(r) := c; if r=c then skip (* do not forward *) else forward (elctd(c)); endif;

13

Elections – ring-based algorithm example

24

15

9

4

3

28

17

24

1

Note: The election was started by process 17.The highest process identifier encountered so far is 24. Participant processes are shown darkened

14

Multicast communication

The aim is for each of a group of processes to receive copies of the messages sent to the group.

A process issues only one multicast operation to send a message to a group of processes instead of issuing multiple send operations.

Multicast operations may provide: Delivery guarantees. Efficiency. Convenience to programmers.

15

Multicast communication – basics

Failure model: Reliable channels, processes may crash.

Processes are member of groups (open or closed).Operations:

multicast(g,m). deliver(m).

Closed group Open group

16

Multicast communication – basic multicast

B-multicast based on a reliable one-to-one send operation: To B-multicast(g,m), for each p in g, send(p,m). On receive(m) at p, B-deliver(m) at p.

May be implemented using threads to perform the send operations concurrently.

May suffer from the ack-implosion problem.

17

Multicast communication – reliable multicast

Integrity: A correct process p delivers a message m at most once. Furthermore p in group(m) and m was supplied to a multicast operation by sender(m).

Validity: If a correct process multicasts message m, then it will eventually deliver m (assumption: closed groups).

Agreement: If a correct process delivers message m, then all other correct processes in group(m) will eventually deliver(m).

18

Multicast communication – reliable multicast algorithm

19

Multicast communication – orderings; bulletin board example

Bulletin board: os.interesting

Item From Subject

23 A.Hanlon Mach

24 G.Joseph Microkernels

25 A.Hanlon Re: Microkernels

26 T.L’Heureux RPC performance

27 M.Walker Re: Mach

end

20

Multicast communication – ordering relations

FIFO ordering: If a correct process issues multicast(g,m) and then multicast(g,m’), then every correct process that delivers m’ will deliver m before m’.

Causal ordering: If multicast(g,m) -> multicast(g,m’), then any correct process that delivers m’ will deliver m before m’.

Total ordering: If a correct process delivers m before it delivers m’, then any other correct process that delivers m’ will deliver m before m’.

Hybrid ordering relations exist.

21

Multicast communication – ordering examples

F3

F1

F2

T2

T1

P1 P2 P3

Time

C3

C1

C2

Notice the consistent ordering of totally ordered messages T1 and T2,

the FIFO-related messages F1 and F2 and the causally

related messages C1 and C3 – and the otherwise arbitrary delivery ordering of messages.

22

Multicast communication – bulletin board example revisited

Bulletin board: os.interesting

Item From Subject

23 A.Hanlon Mach

24 G.Joseph Microkernels

25 A.Hanlon Re: Microkernels

26 T.L’Heureux RPC performance

27 M.Walker Re: Mach

end

23

Summary

Failures and failure detection.Distributed mutual exclusion:

Central-server algorithm. Ring-based algorithm. Time-stamp based algorithm.

Elections: Ring-based algorithm.

Multicast communication Basic multicast. Reliable multicast. Ordering semantics.