98
Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Embed Size (px)

Citation preview

Page 1: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Link Reversal Algorithms

Jennifer L. WelchDISC 2014 Tutorial

Page 2: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

2

What is Link Reversal?

Distributed algorithm design technique

Used in solutions for a variety of problems routing, leader election, mutual

exclusion, scheduling, resource allocation,…

Model problem as a directed graph and reverse the direction of links appropriately

Use local knowledge to decide which links to reverse

Page 3: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Outline

1. Routing in a Graph: Correctness2. Routing in a Graph: Complexity3. Routing and Leader Election in a

Distributed System4. Mutual Exclusion in a Distributed

System5. Scheduling in a Graph6. Resource Allocation in a Distributed

System3

Page 4: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

ROUTING IN A GRAPH: CORRECTNESS

Section 1

4

Page 5: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Routing [Gafni & Bertsekas 1981]

Undirected connected graph represents communication topology of a system

Unique destination node

5

D

1 2 3

4 5 6

Page 6: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Routing [Gafni & Bertsekas 1981]

Assign virtual directions to the graph edges (links) s.t. if nodes forward

messages over the links, they reach the destination

Directed version of the graph (orientation) must be acyclic have destination as only

sink

Thus every node has path to destination.

6

D

1 2 3

4 5 6

Page 7: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Mending Routes

What happens if some edges go away?Might need to change the virtual

directions on some remaining edges (reverse some links)

More generally, starting with an arbitrary directed graph, each vertex should decide independently which of its incident links to reverse

7

Page 8: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Mending Routes Example

8

D

1 2 3

4 5 6

Page 9: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Sinks

A vertex with no outgoing links is a sink.

The property of being a sink can be detected locally.

A sink can then reverse some incident links

Basis of several algorithms…

9

Page 10: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Full Reversal Routing Algorithm

10

Input: directed graph G with destination vertex D

Let S(G) be set of sinks in G other than D

while S(G) is nonempty do reverse every link incident on a vertex

in S(G)G now refers to resulting directed

graph

Page 11: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

11

Full Reversal (FR) Routing Example

D

1 2 3

4 5 6

D

1 2 3

4 5 6

D

1 2 3

4 5 6

D

1 2 3

4 5 6

D

1 2 3

4 5 6

D

1 2 3

4 5 6

Page 12: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Why Does FR Terminate? Suppose it does not. Let W be vertices that take infinitely

many steps nonempty by assumption

Let X be vertices that take finitely many steps nonempty since it includes D

Consider neighboring vertices w in W, x in X exist since graph is connected

Consider first step by w after last step by x: link is w g x and stays that way forever.

Then w cannot take any more steps, contradiction.

12

X(stop)D

W(don’tstop)

w

x

Page 13: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Why is FR Correct?

Assume input graph is acyclic. Acyclicity is preserved at each iteration:

Any new cycle introduced must include a vertex that just took a step, but such a vertex is now a source (has no incoming links)

When FR terminates, no vertex, except possibly D, is a sink.

A DAG must have at least one sink: if no sink, then a cycle can be constructed

Thus output graph is acyclic and D is the unique sink. 13

Page 14: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Pair Algorithm

Can implement FR by having each vertex v keep an ordered pair (c,v), the height (or vertex label) of vertex v c is an integer counter that can be

incremented v is the id of vertex v

View link between v and u as being directed from vertex with larger height to vertex with smaller height (compare pairs lexicographically)

If v is a sink then v sets c to be 1 larger than maximum counter of all v’s neighbors

14

easy to argue

Page 15: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Pair Algorithm Example

15

0

1 2 3

(0,2)(0,1)

(1,0)

(2,3)(2,1)

Page 16: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Pair Algorithm Example

16

0

1 2 3

(0,2)(0,1)

(1,0)

(2,3)(2,1) (3,2)

Page 17: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Pair Algorithm Example

17

0

1 2 3

(0,2)(0,1)

(1,0)

(2,3)(2,1) (3,2)

Page 18: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Partial Reversal Routing Algorithm

Try to avoid repeated reversals of the same link.

Vertices keep track of which incident links have been reversed recently.

Link (u,v) is reversed by v iff the link has not been reversed by u since the last iteration in which v took a step.

18

Page 19: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

19

Partial Reversal (PR) Routing Example

D

1 2 3

4 5 6

D

1 2 3

4 5 6

D

1 2 3

4 5 6

D

1 2 3

4 5 6

D

1 2 3

4 5 6

D

1 2 3

4 5 6

Page 20: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Why is PR Correct?

Termination can be proved similarly as for FR: difference is that it might take two steps by w after last step by x until link is w g x .

20

X(stop)D

W(don’tstop)

w

x

Page 21: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Why is PR Correct?

Preservation of acyclicity is more involved, deferred to later.Alternate geometric proof due to

Radeva and Lynch 2011

21

Page 22: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Triple Algorithm

Can implement PR by having each vertex v keep an ordered triple (a,b,v), the height (or vertex label) of vertex v a and b are integer counters v is the id of vertex v

View link between v and u as being directed from vertex with larger height to vertex with smaller height (compare triples lexicographically)

If v is a sink then v sets a to be 1 greater than smallest a of all its

neighbors sets b to be 1 less than smallest b of all its neighbors

with new value of a (if none, then leave b alone)22

not so easy to argue

Page 23: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Triple Algorithm Example

23

0

1 2 3

(0,0,2)(0,0,1)

(0,1,0)

(0,2,3)(1,0,1)

Page 24: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Triple Algorithm Example

24

0

1 2 3

(0,0,2)(0,0,1)

(0,1,0)

(0,2,3)(1,0,1) (1,-1,2)

Page 25: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Triple Algorithm Example

25

0

1 2 3

(0,0,2)(0,0,1)

(0,1,0)

(0,2,3)(1,0,1) (1,-1,2)

Page 26: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

General Vertex Label Algorithm

Generalization of Pair and Triple algorithms

Assign a label to each vertex s.t. labels are from a totally ordered, countably

infinite set new label for a sink depends only on old

labels for the sink and its neighbors sequence of labels taken on by a vertex

increases without bound

26

Page 27: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Correctness of General Vertex Label Algorithm

Termination: Similar to the arguments for LR and PR.

Difference is that it might take several steps by w after last step by x until link is w g x.

However it will happen because w’s label increases without bound and eventually is larger than x’s final label.

27

X(stop)D

W(don’tstop)

w

x

Page 28: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Correctness of General Vertex Label Algorithm

Acyclicity is preserved because labels are from a totally ordered set

28

Page 29: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Relationship Between Algorithms

29

FR PR

Pair Algorithm Triple Algorithm

General Vertex Label Algorithm

?

Page 30: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Binary Link Labels Routing [Charron-Bost et al. 2013]

Alternate way to implement and generalize FR and PR

Instead of unbounded vertex labels, apply binary link labels to input DAG link directions are independent of labels (in

contrast to algorithms using vertex labels) Algorithm for a sink:

if at least one incident link is labeled 0, then reverse all incident links labeled 0 and flip labels on all incident links

if no incident link is labeled 0, then reverse all incident links but change no labels

30

Page 31: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Binary Link Labels Example

31

0

1 2 3

0

0

1

1

Page 32: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Binary Link Labels Example

32

0

1 2 30

0

1

1

Page 33: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Binary Link Labels Example

33

0

1 2 3

0

10

1

Page 34: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Why is BLL Correct?

Termination can be proved very similarly to termination for PR.Takes 1 or 2 steps until

link is w g x.

34

X(stop)D

W(don’tstop)

w

x

Page 35: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Why is BLL Correct?

What about acyclicity preservation? Depends on initial labeling:

35

0

00 1

0

1

3

2

0

10 1

0

1

3

2

Page 36: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Conditions on Initial Labeling

All labels are the sameall 1’s => Full Reversalall 0’s => Partial Reversal

Every vertex has all incoming links labeled the same (“uniform” labeling)

Both of the above are special cases of a more general condition that is necessary and sufficient for preserving acyclicity 36

Page 37: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Relationship Between Algorithms Revisited

37

FR PR

Pair Algorithm Triple Algorithm

General Vertex Label Algorithm

?

Binary Link Label Algorithm

Page 38: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

ROUTING IN A GRAPH: COMPLEXITY

Section 2

38

Page 39: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

What About Complexity?

Busch et al. (2003,2005) initiated study of the performance of link reversal routing

39

Work complexity of a vertex: number of steps taken by the vertex

Global work complexity: sum of work complexity of all vertices

Page 40: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Worst-Case Work Complexity Bounds [Busch et al.]

bad vertex: has no (directed) path to dest.

Pair algorithm (Full Reversal): for every input, global work

complexity is O(n2), where n is number of initial bad vertices

for every n, there exists an input with n bad vertices with global work complexity Ω(n2)

Triple algorithm (~Partial Reversal): same as Pair algorithm (if appropriately initialized)

40

Page 41: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Exact Work Complexity Bounds

A more fine-grained question: Given any input graph and any vertex in that graph, exactly how many steps does that vertex take?

Busch et al. answered this question for FR.

Charron-Bost et al. answered this question for BLL (as long as labeling satisfies Acyclicity Condition): includes FR and PR. 41

Page 42: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Definitions

Let C = <v1, v2, …, vk> be a chain in the labeled input DAG (series of vertices s.t. either (vi,vi+1) or (vi+1,vi) is a link).

r: number of links that are labeled 1 and rightway ((vi,vi+1) is a link, i.e., directed away from D)

s: number of occurrences of vertices s.t. the two adjacent links are incoming and labeled 0

Res: 1 if last link in X is labeled 0 and rightway, else 0

42

Page 43: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Example of Definitions

For chain <D,7,6,5>: r = 1, s = 0, Res = 1For chain <D,1,2,3,4,5>: r = 2, s = 1, Res

= 043

D0

0

1

1 0 1

7

1 2 3 8

6 5

4

1 01

Page 44: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

44

Grouping the Vertices D is set of all vertices with an incoming link

labeled 0 and either an incoming link labeled 1 or an outgoing link

S is all other vertices (all incoming links, if any, labeled 1 or a sink with all links labeled 0)

0

1

1

1

1

0

1

0

00

0group S

group D

Page 45: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

BLL Work Complexity Theorem: The work complexity of vertex v

ismin{r(C) + s(C) + Res(C) : C is a chain from D to v}

if v is in Smin{2r(C) + 2s(C) + Res(C) : C is a chain from D to v}

if v is in D Proof is by showing how different

combinations of r, s, and Res are either invariant or increase by certain amounts throughout execution until a chain becomes a path from v to D.

45

Page 46: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Work Complexity for FR Recall: The work complexity of vertex v is

min{r(C) + s(C) + Res(C) : C is a chain from D to v} if v is in Smin{2r(C) + 2s(C) + Res(C) : C is a chain from D to v} if v is in D

For FR (all labels are 1), all vertices are in S, s(C) = 0, and Res(C) = 0. Thus formula becomesminimum, over all chains from D to v, of number of links in the chain that are directed away from D

Worst-case graph for global work complexity:D 1 2 … n

vertex i has work complexity i global work complexity then is Θ(n2) 46

Page 47: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Work Complexity for PR

For PR (all labels are 0 initially), work complexity of vertex v is min, over all D-to-v chains, of s + Res if v is

a sink or a source initially min, over all D-to-v chains, of 2s + Res if v

is neither a sink nor a source initially Worst-case graph for global work

complexity:D 1 2 3 … n

work complexity of vertex i is Θ(i) global work complexity is Θ(n2)

47

Page 48: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Comparing FR and PR

Looking at worst-case global work complexity shows no difference – both are quadratic in number of bad vertices

Can use game theory to show some meaningful differences (Charron-Bost et al. 2013)

Consider class of uniform labelings: for each vertex, all incoming links have the same binary label

A vertex’s choice of initial label for its incoming links can be viewed as its strategy in a game

48

0 0

0

1 1

1

orShould v play ?v v

Page 49: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Comparing FR and PR with Game Theory

Compare FR and PR among all uniform labelings (every vertex initially has all its incoming links with same label)

Global work complexity of FR can be larger than optimal (w.r.t. all uniform labelings) by a factor of Θ(n)

Global work complexity of PR is never more than twice the optimal

Thus PR is a safer bet than than FR w.r.t. global work complexity

49

Page 50: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Time Complexity

Time complexity is number of iterations in greedy execution (all sinks take step in each iteration)

Busch et al. (2003, 2005) observed that time complexity cannot exceed global work complexity Thus O(n2) iterations for both pair (FR) and

triple (~PR) algorithms Busch et al. also showed graphs on which

pair (FR) and triple (~PR) algorithms require Ω(n2) iterations 50

Page 51: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Time Complexity

Charron-Bost et al. (2011) derived an exact formula for the last iteration in which any vertex takes a step in any graph for BLL…

51

Page 52: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

FR Time Complexity: Recurrence for Work

Let Wi(t) be number of steps vertex i has taken by iteration t Identify a recurrence relation for Wi(t) based on

understanding how vertex i and its neighbors take turns being sinks:

52

0 1 32

o o o ot = 0

0 1 32

o o 1 ot = 1

0 1 32

o 1 1 1t = 2

0 1 32

o 1 2 1t = 3

0 1 32

o 1 2 2t = 4

Wi(t+1) = min{Wj(t)+1,Wk(t) : j is initially incoming to i,k is initially outgoing from i}

Page 53: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Linear Recurrence Wi(t+1) = min{Wj(t)+1,Wk(t) :

j is initially incoming to i, k is initially outgoing from i}

This recurrence is linear in the min-plus algebra

Represent system of equations as a matrix A

W(t+1) = A W(t) = At O (O is all-zero vector)

Classical duality arguments give us an expression for the time vector using max-plus algebra

Formula for time complexity of general BLL is rather technical

53

Page 54: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

FR Time Complexity Formula

Theorem: For every bad vertex v, termination time of v is1 + max{len(C): C is chain ending at v with r = σv – 1}

where σv is the work complexity of v Worst-case graph for global time

complexity:

54

D

n-1n

n/2+1

n/221

n/2+2

n/2+3

vertex n/2 has work complexity n/2;consider chain that goes around the loopcounter-clockwise n/2-1 times starting andending at n/2: has r = n/2-1 and length Θ(n2)

Page 55: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

BLL Time Complexity

What about other link labelings? Transform to FR! In more detail: for every labeled input

graph G (satisfying the Acyclicity Condition), construct another graph T(G) s.t. for every execution of BLL on G, there is a

“corresponding” execution of FR on T(G) time complexities of relevant vertices in the

corresponding executions are the same

55

Page 56: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Idea of Transformation

If a vertex v is initially in the category D (a sink with some links labeled 0 and some labeled 1, or not a sink with an incoming link labeled 0), then its incident links are partitioned into two sets: all links in one set reverse at odd-numbered steps by v all links in the other set reverse at even-numbered

steps by v Transformation replaces each vertex in D with

two vertices, one corresponding to odd steps by v and the other to even steps, and inserts appropriate links

Vertices initially in category S reverse all their links at every step they take, which is already FR-like behavior

56

Page 57: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Example of LR->FR Transformation

57

LR input FR input

0

0

0

1 1

Yellow vertex in graph on left is in D, so it is replacedwith two vertices in graph on right

Page 58: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

PR Time Complexity

Theorem: For every vertex v with no path to D originally, termination time of v is 1 + max{len(C): C is a chain ending at v with s +

Res = σv – 1} if v is a sink or a source initially 1 + max{len(C): C is a chain ending at v with 2s

+ Res = σv – 1} otherwise

Worst-case graph for global time complexity:D 1 2 3 … n/2 n/2+1 … n

Vertex n/2 has work complexity n/2. Consider chain that starts at n/2, ends at n/2, and goes back and forth between n/2 and n making (n-2)/4 round trips. 2s+Res for this chain is n/2-1, and length is Θ(n2). 58

Page 59: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

FR vs. PR Again

On chain on previous slide, PR has quadratic time complexity.

But FR on that chain has linear time complexity in fact, FR has linear time complexity

on any tree On chain on previous slide, PR has

(slightly) better global work complexity than FR.

59

Page 60: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

ROUTING AND LEADER ELECTION IN A DISTRIBUTED SYSTEM

Section 3

60

Page 61: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

From Graph to Distributed System

To adapt previous ideas to a distributed system: Node (processor) is a vertex Communication channel is an edge (link)

Issues to be overcome: Neighboring nodes need to communicate to

agree on which way the link between them should be directed: delays and losses

Topology can change due to movement and failures; might not always be connected

61

Page 62: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Routing in a Dynamic System

In any execution that experiences a finite number of topology changes, after the last topology change:every node in same connected

component as D (destination) should have a path to D

every node not in the same component as D should stop trying to find a route to D or forward a message to D

62

Page 63: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

What’s Wrong with FR?

63

D

21

3

D

21

3

D

21

3

D

21

3

Page 64: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

TORA [Park & Corson 1997]

Modify the generalized algorithm of Gafni & Bertsekas using increasing vertex labels

Vertex labels, or heights, are 5-tuples one entry is current time: Temporally Ordered

Routing Algorithm Every node in same connected component as D

eventually has a path to D in the directed version of the communication graph induced by the heights

Clever use of additional entries in the heights allows node to tell when they are partitioned from D and should stop participating

64

Page 65: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

65

Heights in TORA

[ t , oid , r , d , i ]

reference level delta

time this ref. levelwas started

id of nodeoriginatingthis ref.level

reflectionbit

ordersnodes withsame ref.level

id of node,breaks ties

Page 66: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

TORA Overview

Route Creation: use standard spanning tree construction ideas to set ref levels to (0,0,0) and deltas to distances from D

Route Maintenance and Partition Detection: see next slide

Route Erasure: When partition is detected, flood “clear” messages throughout component

66

Page 67: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

67

TORA Route Maintenance

If node i loses last outgoing link: due to a link failure (Case Generate):

set ref level to (current time, i, 0), a full reversal due to a height change and

nbrs don’t have same ref level (Case Propagate): adopt max ref level and set d to effect a partial reversal

nbrs have same ref level with r = 0 (Case Reflect): adopt new ref level, set r to 1, set d to 0 (full reversal)

nbrs have same ref level with r = 1 and oid = i (Case Detect):

Partition! Start process of erasing routes.

Page 68: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

68

TORA Example – Partition

D

1 2 3 4

5

Generate

D

1 2 3 4

5

Propagate Propagate

D

1 2 3 4

5Reflect

Reflect

D

1 2 3 4

5

PropagatePropagate

D

1 2 3 4

5

Detect

Page 69: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

TORA Discussion

Works best with perfectly synchronized clocks

How to prove correctness? Gafni & Bertsekas result does not directly

apply because of asynchronous delay in updating neighbors about new heights

Other issues remain: partition detection, route creation, route erasure

Can be adapted to solve leader election: when partition is detected, elect a new leader! (Cf. Ingram et al. 2013)

69

Page 70: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

MUTUAL EXCLUSION IN A DISTRIBUTED SYSTEM

Section 4

70

Page 71: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Link Reversal for Mutual Exclusion [Snepscheut 1987]

Goal: no-lockout mutual exclusion in a message-passing system with a tree communication topology

Solution: Pass around a unique “token” message.

71

Page 72: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Link Reversal for Mutual Exclusion [Snepscheut 1987]

impose logical directions on communication channels s.t. token holder is unique sink

when a node has the token, it can enter critical section

When node needs the token, it sends “request” message on its unique outgoing link – toward token holder

When a node receives a request, it remembers it in a FIFO queue and forwards it toward the token-holder (if not already waiting)

when token holder responds to a request, it forwards the token to the neighbor at the head of the queue, and reverses direction of that link 72

Page 73: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

From a Tree to a DAG

For a general communication topology, the previous algorithm can be run on a spanning tree overlay of the graph.

However, this does not take advantage of the redundancy offered by additional links.

Instead, direct all links in the graph: request message can be forwarded on any

outgoing link when a node receives the token, all its

outgoing links are reversed, to make it a sink73

Page 74: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

From a Tree to a DAG

74

Page 75: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Related Work

Naimi and Trehel 1987: analyzed expected message complexity

Raymond 1989: gave fault-tolerant extension

Dhamdhere and Kulkarni 1994: considered dynamic topogies

Walter et al. 2001: handled dynamic topologies by adding vertex labels

75

Page 76: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

SCHEDULING IN A GRAPHSection 5

76

Page 77: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Scheduling in a Graph[Barbosa & Gafni 1989]

What happens when the Full Reversal routing algorithm is executed without a destination? I.e., every vertex in the graph does a

reversal when it is a sink Call this algorithm FRND (FR with No

Destination). When a vertex is a sink, it is said to be

scheduled: can take some action with the guarantee that none of its neighbors are scheduled at the same time. 77

Page 78: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Behavior of FRND

Claim: FRND maintains acyclicity. Same proof as for FR.

Claim: Every vertex is a sink infinitely often. By Claim 1, at each iteration there is at least

one sink, so FRND never terminates. If some vertices take finitely many steps and

some take infinitely many, then use same argument as for showing FR terminates to get a contradiction.

Thus every vertex is scheduled infinitely often.

78

Page 79: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

FRND Example

79

0 21 3 4 5

0 21 3 4 5

0 21 3 4 5

0 21 3 4 5

0 21 3 4 5

0 21 3 4 5

Page 80: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Behavior of FRND

Claim: In the greedy execution of FRND from an initial DAG, eventually the pattern of sinks becomes periodic.Let G1, G2,… be the sequence of graph

states in the execution. Since finite number of vertices, Gi = Gj for some i and j. Thus Gi+1 = Gj+1, etc.

Note: every vertex appears at least once in every period.

80

Page 81: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Q&A about FRND

How long until the execution becomes periodic? At most polynomial number of iterations

[Malka & Rajsbaum 1991] How long is the period?

At least 2 iterations, since neighbors cannot be sinks simultaneously

Can be as bad as exponential [Malka et al. 1993]

How “fair” is the period? every vertex takes same number of steps…

81

Page 82: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Period is Fair

Claim 1: Difference in total number of steps taken by u and v at any iteration is at most the distance between them. Proof is by induction on the distance.

Claim 2: Every vertex takes same number of steps in the period. Suppose u appears a times and v appears b

times with b > a. After k-th execution of the period, u has taken ka steps and v has taken kb steps. Eventually kb – ka exceeds the distance between u and v, contradicting Claim 1. 82

Page 83: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Multiplicity and Concurrency

Multiplicity of the period is the number of times that each vertex takes a step in the period.

Concurrency is ratio of multiplicity m to the period length p, i.e., m/p.

Concurrency is fraction of iterations during which any given vertex takes steps.at most 1/2at least 1/n

83

Page 84: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Multiplicity and Concurrency

Multiplicity of the period is the number of times that each vertex takes a step in the period.

Concurrency is ratio of multiplicity m to the period length p, i.e., m/p.

Concurrency is fraction of iterations during which any given vertex takes steps.at most 1/2at least 1/n

84

Page 85: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Concurrency Example

85

Period length is 5multiplicity is 2concurrency is 2/5

Page 86: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Exact Expression for Concurrency

Claim 1: For any initial orientation of a tree, the greedy execution of FRND reaches a periodic orientation with length 2 and multiplicity 1, so concurrency = 1/2.

Claim 2: For any periodic orientation G of a non-tree graph, the concurrency is equal to the minimum, over all (simple) circuits k in G, of the fraction of links in k that are right-way.

86

Page 87: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Choosing a Good Initial Orientation For trees, the initial orientation is

unimportant: they all lead to a period with concurrency 1/2

For non-trees, the initial orientation can make a big difference to the concurrency: consider a ring of n vertices, where n is even if initially there is just 1 sink, there will never

be more than 1 sink in any orientation: concurrency is 1/n

if initially every other vertex is a sink, vertices keep alternating: concurrency is 1/2

Unfortunately, determining the best orientation is NP-complete!

87

Page 88: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

RESOURCE ALLOCATION IN A DISTRIBUTED SYSTEM

Section 6

88

Page 89: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Resource Allocation in a Distributed System [Chandy & Misra 1984]

Dining philosophers (or resource allocation) problem is a generalization of the mutual exclusion problem: conflict graph: vertices correspond to the

nodes, edge between I and j means I and j compete for exclusive access to a resource

Ensure exclusion: no two neighbors in the conflict graph are in their critical sections simultaneously

Ensure fairness: every node requesting access to its critical section eventually is granted access 89

Page 90: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

First Solution

Use FRND on the conflict graph: when a vertex is a sink, it can enter its critical section Every vertex is a sink infinitely often no two neighbors are sinks simultaneously

Issues: How to adapt FRND to asynchronous

message passing? Why bother a node that is not interested in

entering its critical section?

90

Page 91: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Chandy & Misra’s Solution

Key data structure is precedence graph, directed version of conflict graph

Precedence graph is represented in a distributed fashion by having each node keep a variable for each neighbor indicating who yields to whom variables are initialized so that precedence graph is

acyclic Each pair of neighbors i and j share a token to

ensure exclusion if i doesn’t have token when it wants to enter C.S. it

sends request to j j sends back the token immediately if j is in its

remainder section or if it is in its trying section and i has precedence over j, otherwise j defers the request from i

91

Page 92: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Chandy & Misra’s Solution

Thus precedence graph is used to arbitrate between contending neighbors, but otherwise is ignored.

Once i has all its tokens, it enters the C.S. When i leaves the C.S., it satisfies all deferred

requests and does a full reversal in the precedence graph

92

Page 93: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Correctness Ideas

Management of tokens ensures exclusion.

By starting with an acyclic conflict graph and only modifying it with full reversal, the precedence graph is always acyclic:no deadlock can be caused by a cycle

of nodes waiting on each other

93

Page 94: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Conclusion Other applications of link reversal

include:distributed queueingk-mutual exclusionpublish/subscribesimulated annealinggraph coloringneural networks

Appeal of the approach is using local knowledge to solve global problems

94

Page 95: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

Open Questions

Explore consequences of work and time bounds for BLL on other applications of link reversal

Discover effects on bounds of distributed system realities:message delays failuresongoing topology changes

95

X

X

Page 96: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

96

Acknowledgements

Welch and Walter, “Link Reversal Algorithms,” Synthesis Lectures on Distributed Computing Theory #8, Morgan & Claypool Publishers, 2011. Editor Nancy Lynch

My collaborators: Bernadette Charron-Bost, Matthias Fuegger, Antoine Gaillard, Rebecca Ingram, Navneet Malpani, Mira Radeva, Patrick Shields, Nitin Vaidya, Saira Viqar, Jennifer Walter, Josef Widder

Funding: National Science Foundation, Texas Higher Education Coordinating Board

Page 97: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

References Barbosa and Gafni, “Concurrency in Heavily Loaded Systems,” ACM

TOPLAS 1989. Busch, Surapaneni and Tirthapura, “Analysis of Link Reversal

Routing Algorithms for Mobile Ad Hoc Networks,” SPAA 2003. Busch and Tirthapura, “Analysis of Link Reversal Routing

Algorithms,” SIAM JOC 2005. Chandy and Misra, “The Drinking Philosophers Problem,” ACM

TOPLAS 1984. Charron-Bost, Fuegger, Welch and Widder, “Full Reversal Routing as

a Linear Dynamical System,” SIROCCO 2011. Charron-Bost, Fuegger, Welch and Widder, “Partial is Full,”

SIROCCO 2011. Charron-Bost, Gaillard, Welch and Widder, “Link Reversal Routing

with Binary Link Labels: Work Complexity,” SIAM JOC 2013. Dhamdere and Kulkarni, “A Token-Based k-Resilient Mutual

Exclusion Algorithm for Distributed Systems,” IPL 1994. Gafni and Bertsekas, “Distributed Algorithms for Generating Loop-

Free Routes in Networks with Frequently Changing Topology,” IEEE Trans. Comm. 1981.

97

Page 98: Link Reversal Algorithms Jennifer L. Welch DISC 2014 Tutorial

References Ingram, Radeva, Shields, Viqar, Walter and Welch, “A Leader

Election Algorithm for Dynamic Networks with Causal Clocks,” Distributed Computing 2013.

Malka, Moran and Zaks, “A Lower Bound on the Period Length of a Distributed Scheduled,” Algorithmica 1993.

Malka and Rajsbaum, “Analysis of Distributed Algorithms Based on Recurrence Relations”, WDAG 1991.

Naimi and Trehel, “An Improvement of the log n Distributed Algorithm for Mutual Exclusion,” ICDCS 1987.

Park and Corson, “A Highly Adaptive Distributed Routing Algorithm for Mobile Wireless Networks,” INFOCOM 1997.

Radeva and Lynch, “Partial Reversal Acyclicity,” PODC 2011. Raymond, “A Tree-Based Algorithm for Distributed Mutual

Exclusion,” ACM TOCS 1989. van de Snepscheut, “Fair Mutual Exclusion on a Graph of

Processes,” Distributed Computing 1987. Walter, Welch and Vaidya, “A Mutual Exclusion Algorithm for Ad Hoc

Mobile Networks,” Wireless Networks, 2001.

98