Embed Size (px)
DESCRIPTION
UBI529. 3. Distributed Graph Algorithms. Distributed Algorithms Models. Interprocess Communication method : accessing shared memory, point-to-point or broadcast messages, or remote procedure calls. • Timing model : synchronous or asynchronous models. - PowerPoint PPT Presentation
Citation preview
UBI529
3. Distributed Graph Algorithms
2
Distributed Algorithms Models
Interprocess Communication method: accessing shared memory, point-to-point or broadcast messages, or remote procedure calls.
• Timing model: synchronous or asynchronous models.
• Failure models : reliable or faulty behavior; Byzantine failures (failed processor can behave arbitrarily).
3
We assume
A distributed network—Modeled as a graph. Nodes are processors and edges are communication links.
• Nodes can communicate directly (only) with their neighbors through the edges.
• Nodes have unique processor identities.
• Synchronous model: Time is measured in rounds (time steps).
• One message (typically of size O(log n)) can be sent through an edge in a time step. A node can send messages simultaneously through all its edges at once in a round.
• No failure of nodes or edges. No malicious nodes.
2.1 Vertex and Tree Coloring
• Vertex Coloring
• Sequential Vertex Coloring Algorithms
• Distributed Synchronous Vertex Coloring Algorithm
• Distributed Tree Coloring Algorithms
5
Preliminaries
Vertex Coloring Problem: Given undirected Graph G = (V,E). Assign a color cu to each vertex u Є V such that if e = (v,w) Є E, then cu ≠ cw Aim is to use the minimum number of colors.
Definition 2.1.1 : Given an undirected Graph, chromatic number Χ(G) is the minimum number of colors to color it. A vertex k-coloring uses exactly k colors. If X(G) = k, G is k-colorable but not (k-1) colorable.Calculating X(G) is NP-hard. 3-coloring decision is NP-complete.
Applications :Assignment of radio frequencies : Colors represent frequencies, transmitters are the vertices. If two stations are neighbors when they interfere.University course scheduling : Vertices are courses, students edgesFast register allocation for computer programming : Vertices are variables, they are neigbors if they can be active at the same time.
6
Sequential Algorithm for Vertex Coloring
Algorithm 2.1.1 : Sequential Vertex ColoringInput : G with v1,v2, ..., vn
Output : Vertex Coloring f : VG -> {1,2,3,..}
1. For i =1 to n do 2. f(vi) := smallest color number that does not conflict by
any of the other colored neighbors of vi
3. Return Vertex Coloring f
7
Vertex Coloring Algorithms
Definition 2.1.2 : The number of neighbors of a vertex v is called the degree of v δ(v). The maximum degree vertex in a Graph G is called the the Graph degree Δ(G) = Δ.
Theorem 2.1.1 : The algorithm is correct and terminates in O(n) steps. The algorithm uses Δ +1 colors.Proof: Correctness and termination are straight-forward. Since each node has at most Δ neighbors, there is always at least one color free in the range {1, …, Δ+1}.
Remarks:• For many graphs coloring can be done with much less than Δ +1 colors.• This algorithm is not distributed; only one processor is active at a time. But: Use idea of Algorithm 1.4 to define “local” coloring subroutine 1.7
8
Heuristic Vertex Coloring Algorithm : Largest Degree First
Idea : (Two observations) A vertex of a large degree is more difficult to color than a smaller degree vertex. Also, a vertex with more colored neighbors will be more difficult to color later
Algorithm 2.1.1 : Largest Degree First Algorithm
Input : G with v1,v2, ..., vn
Output : Vertex Coloring f : VG -> {1,2,3,..}
1. While there are uncolored vertices of G2. Among the uncolored max. degree vertices
Choose vertex v with the max. Colored degree 3. Assign smallest possible k to v : f(v) := k4. Return Vertex Coloring fThe coloring in the diagram is v3,v1,v2,v4,v8,v6,v7,v5
Colored degree : # of different colors used to color neighbors of v
9
Coloring Trees : A Distributed Algorithm
Lemma 2.1.1: X(Tree) <= 2.
Proof: If the distance of a node to the root is odd (even), color it 1 (0). An odd node has only even neighbors and vice versa.If we assume that each node knows its parent (root has no parent) and children in a tree, this constructive proof gives a very simple algorithm.
Algorithm 2.1.3 [Slow tree coloring]:
1. Root sends color 0 to children. (Root is colored 0)2. When receiving a message x from parent, a node u picks color cu = 1-x, and sends cu to its children
10
Distributed Tree Coloring
Remarks:
• With the proof of Lemma 2.1.1, the algorithm 2.13 is correct.• The time complexity of the algorithm is the height of the tree.• When the root is chosen randomly, this can be up to the diameter of the tree.
2.2 Distributed Tree based Communication Algorithms
• Broadcast
• Convergecast
• BFS Tree Construction
12
Broadcast
Broadcasting means sending a message from a source node to all othernodes of the network.
Two basic broadcasting approaches are flooding andspanning tree-based broadcast.
Flooding:
A source node s wants to send a message to allnodes in the network. s simply forwards the message over all its edges.
Any vertex v != s, upon receiving the message forthe first time (over an edge e) forwards it on everyother edge.
Upon receiving the message again it does nothing.
13
Broadcast
Definition 2.2.1 [Broadcast]: A broadcast operation is initiated by a single processor, the source. The source wants to send a message to all other nodes in the system.
Definition 2.2.2 [Distance, Radius, Diameter]:
• The distance between two nodes u, v in an undirected graph is the number of hops of a minimum path between u and v.
• The radius of a node u in a graph is the maximum distance between u and any other node. The radius of a graph is the minimum radius of any node in the graph.
• The diameter of a graph is the maximum distance between two arbitrary nodes.
14
Broadcast
Theorem 2.2.1 [Lower Bound]: The message complexity of a broadcast is at least n-1. The radius of the graph is a lower bound for the time complexity.Proof: Every node must receive the message.
Remarks:• You can use a pre-computed spanning tree to do the broadcast with tight message complexity.• If the spanning tree is a breadth-first spanning tree (for a given source), then also the time complexity is tight.Definition 2.2.3 : A graph (system/network) is clean if the nodes do not know the topology of the graph.Theorem 2.2.2 [Clean Lower Bound]: For a clean network, the number of edges is a lower bound for the broadcast message complexity.Proof: If you do not try every edge, you might miss a whole part of the graph behind it.
15
Flooding
Algorithm 2.2.1 [Flooding]: The source sends the message to all neighbors. Each node receiving the message the first time forwards to all (other) neighbors.
Remarks:
• If node v receives the message first from node u, then node v calls node u “parent”. This parent relation defines a spanning tree T. If the flooding algorithm is executed in a synchronous system, then T is a breadth-first spanning tree (with respect to the root).
• More interestingly, also in asynchronous systems the flooding algorithm terminates after r time units, where r is the radius of the source. (But note that the constructed spanning tree needs not be breadth-first.)
16
Flooding Analysis
Theorem : The message complexity of flooding is (|E|) and the time complexity is (D), where D is the diameter of G.
Proof. The message complexity follows from the fact that each edge delivers the message at least once and at most twice (one in each direction). To show the time complexity, we use induction on t to show that after t time units, the message has already reached everyvertex at a distance of t or less from the source
17
Broadcast Over a Rooted Spanning Tree
Suppose processors already have information about a rooted spanning tree of the communication topology
tree: connected graph with no cycles spanning tree: contains all processors rooted: there is a unique root node
Implemented via parent and children local variables at each processor
indicate which incident channels lead to parent and children in the rooted spanning tree
18
Broadcast Over a Rooted Spanning Tree: A Simple Algorithm
1. root initially sends msg to its children2. when a node receives msg from its parent
sends msg to its children terminates (sets a local boolean to true)
Synchronous model:
time is depth of the spanning tree, which is at most n - 1 number of messages is n - 1, since one message is sent over
each spanning tree edge
Asynchronous model:
same time and messages
19
Tree Broadcast
Assume that a spanning tree has been constructed.
Theorem . For every n-vertex graph G with a spanning tree T rooted at r0, the message complexity of broadcast is n−1 and time complexity is depth(T).
A broadcast algorithm can be used to construct a spanning tree in G.
The message complexity of broadcast is asymptotically equivalent to the message complexity of spanning tree construction.
Using a breadth-first spanning tree, we get theoptimal message and time complexities for broadcast.
20
Convergecast
Again, suppose a rooted spanning tree has already been computed by the processors
parent and children variables at each processor
Do the opposite of broadcast:
leaves send messages to their parents
non-leaves wait to get message from each child, then send combined info to parent
21
Convergecast
g h
a
b c
d e f
g h
d e,g f,h
c,f,hb,d
solid arrows: parent-child relationships
dotted lines:non-tree edges
22
Finding a Spanning Tree Given a Root
a distinguished processor is known, to serve as the rootroot sends M to all its neighborswhen non-root first gets M
set the sender as its parent send "parent" msg to sender send M to all other neighbors
when get M otherwise
send "reject" msg to sender
use "parent" and "reject" msgs to set children variables and know when to terminate
23
Execution of Spanning Tree Alg.
g h
a
b c
d e f
Synchronous: always givesbreadth-first search (BFS) tree
g h
a
b c
d e f
Asynchronous: not necessarily BFS tree
Both models:O(m) messagesO(diam) time
2.3 Distributed Minimum Spanning Tree Algorithms
25
Minimum Spanning Tree
Minimum spanning tree. Given a connected graph G = (V, E) with real-valued edge weights ce, an MST is a subset of the edges T E
such that T is a spanning tree whose sum of edge weights is minimized.
Cayley's Theorem. There are nn-2 spanning trees of Kn.
5
23
10
21
14
24
16
6
4
189
7
11 8
5
6
4
9
7
11 8
G = (V, E) T, eT ce = 50
can't solve by brute force
26
Applications
MST is fundamental problem with diverse applications.
Network design– telephone, electrical, hydraulic, TV cable, computer, road
Approximation algorithms for NP-hard problems– traveling salesperson problem, Steiner tree
Indirect applications
– max bottleneck paths– LDPC codes for error correction– image registration with Renyi entropy– learning salient features for real-time face verification– reducing data storage in sequencing amino acids in a protein– model locality of particle interactions in turbulent fluid flows– autoconfig protocol for Ethernet bridging to avoid cycles in a
network Cluster analysis.
27
Greedy Algorithms
Kruskal's algorithm. Start with T = . Consider edges in ascending order of cost. Insert edge e in T unless doing so would create a cycle.
Reverse-Delete algorithm. Start with T = E. Consider edges in descending order of cost. Delete edge e from T unless doing so would disconnect T.
Prim's algorithm. Start with some root node s and greedily grow a tree T from s outward. At each step, add the cheapest edge e to T that has exactly one endpoint in T.
Remark. All three algorithms produce an MST.
28
Greedy Algorithms
Simplifying assumption. All edge costs ce are distinct.
Cut property. Let S be any subset of nodes, and let e be the min cost edge with exactly one endpoint in S. Then the MST contains e.
Cycle property. Let C be any cycle, and let f be the max cost edge belonging to C. Then the MST does not contain f.
f C
S
e is in the MST
e
f is not in the MST
29
Cycles and Cuts
Cycle. Set of edges the form a-b, b-c, c-d, …, y-z, z-a.
Cutset. A cut is a subset of nodes S. The corresponding cutset D is the subset of edges with exactly one endpoint in S.
Cycle C = 1-2, 2-3, 3-4, 4-5, 5-6, 6-1
13
8
2
6
7
4
5
Cut S = { 4, 5, 8 }Cutset D = 5-6, 5-7, 3-4, 3-5, 7-8
13
8
2
6
7
4
5
30
Cycle-Cut Intersection
Claim. A cycle and a cutset intersect in an even number of edges.
Pf. (by picture)
13
8
2
6
7
4
5
S
V - S
C
Cycle C = 1-2, 2-3, 3-4, 4-5, 5-6, 6-1Cutset D = 3-4, 3-5, 5-6, 5-7, 7-8 Intersection = 3-4, 5-6
31
Greedy Algorithms
Simplifying assumption. All edge costs ce are distinct.
Cut property. Let S be any subset of nodes, and let e be the min cost edge with exactly one endpoint in S. Then the MST T* contains e.
Pf. (exchange argument) Suppose e does not belong to T*, and let's see what happens. Adding e to T* creates a cycle C in T*. Edge e is both in the cycle C and in the cutset D corresponding
to S there exists another edge, say f, that is in both C and D.
T' = T* { e } - { f } is also a spanning tree. Since ce < cf, cost(T') < cost(T*). This is a contradiction. ▪
f
T*
e
S
32
Greedy Algorithms
Simplifying assumption. All edge costs ce are distinct.
Cycle property. Let C be any cycle in G, and let f be the max cost edge belonging to C. Then the MST T* does not contain f.
Pf. (exchange argument) Suppose f belongs to T*, and let's see what happens. Deleting f from T* creates a cut S in T*. Edge f is both in the cycle C and in the cutset D corresponding
to S there exists another edge, say e, that is in both C and D.
T' = T* { e } - { f } is also a spanning tree. Since ce < cf, cost(T') < cost(T*). This is a contradiction. ▪
f
T*
e
S
33
Prim's Algorithm: Proof of Correctness
Prim's algorithm. [Jarník 1930, Dijkstra 1957, Prim 1959] Initialize S = any node. Apply cut property to S. Add min cost edge in cutset corresponding to S to T, and add
one new explored node u to S.
S
34
Implementation: Prim's Algorithm
Prim(G, c) { foreach (v V) a[v] Initialize an empty priority queue Q foreach (v V) insert v onto Q Initialize set of explored nodes S
while (Q is not empty) { u delete min element from Q S S { u }
foreach (edge e = (u, v) incident to u) if ((v S) and (ce < a[v]))
decrease priority a[v] to ce
}
Implementation. Use a priority queue ala Dijkstra. Maintain set of explored nodes S. For each unexplored node v, maintain attachment cost a[v] =
cost of cheapest edge v to a node in S. O(n2) with an array; O(m log n) with a binary heap.
35
Kruskal's Algorithm: Proof of Correctness
Kruskal's algorithm. [Kruskal, 1956] Consider edges in ascending order of weight. Case 1: If adding e to T creates a cycle, discard e according
to cycle property. Case 2: Otherwise, insert e = (u, v) into T according to cut
property where S = set of nodes in u's connected component.
Case 1
v
u
Case 2
e
eS
36
Implementation: Kruskal's Algorithm
Kruskal(G, c) { Sort edges weights so that c1 c2 ... cm. T
foreach (u V) make a set containing singleton u
for i = 1 to m (u,v) = ei
if (u and v are in different sets) { T T {ei} merge the sets containing u and v } return T}
Implementation. Use the union-find data structure. Build set T of edges in the MST. Maintain set for each connected component. O(m log n) for sorting and O(m (m, n)) for union-find.
are u and v in different connected components?
merge two components
m n2 log m is O(log n) essentially a constant
37
38
Distributed Spanning tree construction
Chang-Robert’s algorithm
{The root is known}
Uses signals and acks, similar
to the termination detection
algorithm. Uses the same rule
for sending acknowledgment.
0
1 2
3 4
5
root
For a graph G=(V,E), a spanning tree is a maximally connected subgraph T=(V,E’), E’ E,such that if one more edge is added, then the subgraph is no more a tree. Used for broadcasting in a network.
Question: What if the root is not
designated?
39
Chang Roberts Spanning Tree Algprogram probe-echodefine N : integer (no. of neighbors)
C, D : integer;initially parent :=i; C=0; D=0;
{for the initiator}
send probes to each neighbor;D:=no. of neighbors;do D!=0 echo -> D:=D-1 od {D=0 signals end}
{ for a non-initator process i>0}
do parent parent=i C=0 -> C:=1; parent := sender; if i is not a leaf -> send probes to non –
parent neighbors; D:= no. of non-parent neighbors
fi; echo -> D:=D-1; probe sender != parent -> send echo to sender; C=1 D=0 -> send echo to parent; C:=0;od
40
Graph traversal
Many applications of exploring an unknown graph by a visitor
(a token or mobile agent or a robot). The goal of traversal
is to visit every node at least once, and return to the starting point.
- How efficiently can this be done?
- What is the guarantee that all nodes will be visited?
- What is the guarantee that the algorithm will terminate?
Consider web-crawlers, exploration of social networks,graph layouts for visualization or drawing etc.
41
Graph traversal and Spanning Tree Formation
Rule 1. Send the token towards each neighbor exactly once.
Rule 2. If rule 1 is not applicable, then send the token to the parent.
Tarry’s algorithm is one of the oldest (1895)
0 2
3
4 5
root
6
1
5
A possible route is: 0 1 2 5 3 1 4 6 2 6 4 1 3 5 2 1 0
Nodes and their parent pointers generate a spanning tree that may not be DFS
42
Distributed MST
Def MST Fragment : In a weighted graph G = (V,E,w), a tree T in G is called an MST fragment of G, i there exists an MST of G such that T is a subgraph of that MST.
Def MWOE : An edge e is an outgoing edge of a MST fragment T, iff exactly one of its endpoints belongs to T. The minimum weight outgoingedge is denoted MWOE(T).Lemma : Consider a MST fragment T of a graph G = (V, E, w). Lete = MWOE(T). Then T U e is a MST fragment as well.
Proof : Let TM be an MST containing T. If TM contains T we are done.Otherwise, let e’ be an edge that connects T to the rest of TM.Clearly, e’ is an outgoing edge of T and w(e’)>=w(e). Adding e to TM, creates a graph C with a cycle through e and e’. Discarding e’ from C yields a new T’ M with w(T’ M) >= w(TM).
43
Minimum Spanning Tree
Given a weighted graph G = (V, E), generate a spanning tree T = (V, E’)
such that the sum of the weights of all the edges is minimum.
Applications
On Euclidean plane, approximate solutions to the traveling salesman
problem,
Lease phone lines to connect the different offices with a minimum cost,
Visualizing multidimensional data (how entities are related to each other)
We are interested in distributed algorithms only
The traveling salesman problemasks for the shortest route to visit a collection of cities and return to
the starting point.
44
Example
45
Sequential algorithms for MST
Review (1) Prim’s algorithm and (2) Kruskal’s algorithm.
Theorem. If the weight of every edge is distinct, then the MST is unique.
1
2
3
4
5
6
7
8
9
e0 2
3
51
4
6
T1T2
46
Gallagher-Humblet-Spira (GHS) Algorithm
GHS is a distributed version of Prim’s
algorithm.
Bottom-up approach. MST is recursively
constructed by fragments joined by an edge
of least cost.
3
7
5
Fragment Fragment
47
Challenges
1
2
3
4
5
6
7
8
9
e0 2
3
51
4
6
T1T2
Challenge 1. How will the nodes in a given fragment identify the edge to be used to connect with a different fragment?
A root node in each fragment is the coordinator
48
Challenges
1
2
3
4
5
6
7
8
9
e0 2
3
51
4
6
T1T2
Challenge 2. How will a node in T1 determine if a given edge connects to a node of a different tree T2 or the same tree T1? Why will node 0 choose the edge e with weight 8, and not the edge with weight 4?
Nodes in a fragment acquire the same name before augmentation.
49
Two main steps
Each fragment has a level. Initially each node is a fragment at level 0.
(MERGE) Two fragments at the same level L combine to form a fragment
of level L+1
(ABSORB) A fragment at level L is absorbed by another fragment at level
L’ (L < L’)
50
Least weight outgoing edge
To test if an edge is outgoing, each node sends a test message through a candidate edge. The receiving node may send accept or reject.
Root broadcasts initiate in its own fragment, collects the report from other nodes about eligible edges using a convergecast, and determines the least weight outgoing edge.
1
2
3
4
5
6
7
8
9
e0 2
3
51
4
6
T1T2
test
reject
accept
51
Accept of reject?
Case 1. If name (i) = name (j) then send rejectCase 2. If name (i)≠name (j)level (i) level (j) then send acceptCase 3. If name (i) ≠ name (j) level (i) > level (j) then wait until level (j) = level (i).
Levels can only increase.
Question: Can fragments wait for ever and lead to a deadlock?
test
Let i send test to j
reject
test
52
Delayed response
join
initiate
test
Level 5 Level 3
A B
B is about to change its level to 5. So B does notsend an accept reponse to A in response to test
53
The major steps
Repeat
Test edges as outgoing or notDetermine lwoe - it becomes a tree edgeSend join (or respond to join)Update level & name & identify new coordinator
until done
54
Classification of edges
Basic (initially all branches are basic)Branch (all tree edges)Rejected (not a tree edge)
Branch and rejected are stable attributes
55
Wrapping it up
Merge
The edge through which the join
message is sent, changes its status to
branch, and becomes a tree edge.
Each root broadcasts an
(initiate, L+1, name) message
to the nodes in its own fragment.
T T’
(join, L, T)
(join, L’, T’)
(a)
level=L level = L’
L= L’
T T’
level=Llevel = L’
(b) L > L’
(join, L’, T;)
Example of merge
56
Wrapping it up
Absorb
T’ receives an initiate
message.
This indicates that the fragment
at level L has been
absorbed by the
other fragment at level L’.
They collectively search for the
lwoe.
The edge through which the
join message was sent,
changes
its status to branch.
T T’
(join, L, T)
(join, L’, T’)
(a)
level=L level = L’
L= L’
T T’
level=Llevel = L’
(b) L > L’
(join, L’, T;)
initiate
Example of absorb
57
Example
4
1
0 2
5
6 3
1
3
2
4 7
8
9
5
6
58
Example
4
1
0 2
5
6 3
1
3
2
4 7
8
9
5
6
merge merge
merge
59
Example
4
1
0 2
5
6 3
1
2
4 7
8
9
5
6
merge
absorb
3
60
Example
4
1
0 2
5
6 3
1
2
4 7
8
9
5
6
3
absorb
61
Message complexity
At least two messages (test + reject) must pass through eachrejected edge. The upper bound is 2|E| messages.
At each of the log N levels, a node can receive at most (1) oneinitiate message and (2) one accept message (3) one joinmessage (4) one test message not leading to a rejection, and(5) one changeroot message.
So, the total number of messages has an upper bound of2|E| + 5N logN
62
MST Algorithms: Theory
Deterministic comparison based algorithms. O(m log n) [Jarník, Prim, Dijkstra, Kruskal, Boruvka] O(m log log n). [Cheriton-Tarjan 1976, Yao 1975] O(m (m, n)). [Fredman-Tarjan 1987] O(m log (m, n)). [Gabow-Galil-Spencer-Tarjan 1986] O(m (m, n)). [Chazelle 2000]
Holy grail. O(m).
Notable. O(m) randomized. [Karger-Klein-Tarjan 1995] O(m) verification. [Dixon-Rauch-Tarjan 1992]
Euclidean. 2-d: O(n log n).compute MST of edges in Delaunay k-d: O(k n2). dense Prim
63
Distributed MST Algorithms
Gallager, Humblet, & Spira ’83: O(n log n) running time message: O(|E| + n log n) (optimal)
Chin & Ting ’85: O(n log log n) timeGafni ’85: O(n log*n) Awerbuch ’87: O(n), existentially optimalGaray, Kutten, & Peleg ’98: O(D + n0.61), Diameter DKutten & Peleg ’98:
Elkin ’04: , μ is called MST radius
– Cannot detect termination unless μ is given as input.
Peleg & Rabinovich (’99) showed a lower bound of for running time.
nnDO log*
nO ~
n~
64
Distributed Graph Algs : Other areas of interest
Distributed Cycle/Knot Detection
Distributed Center Finding
Distributed Connected Dominating Set Construction in MANETs, WSNs
Distributed Clustering based on Graph Partitioning
65
References
Introduction to Graph Theory, Douglas West, Prentice Hall, 2000 (basics)
Graph Theory and Its Applications, Gross and Yellen, CRC Press, 1998(basics)Distributed Algorithm Course notes, J.Welch, TAMU (flooding and tree algorithms)CS590A Fall 2007 G. Pandurangan 1, Purdue University
Distributed Computing Principles Course Notes, Roger Wattenhofer, ETH (Coloring algorithms)Introduction to Algorithm Design, Kleinman, Tardos, Prentice-Hall, 2005 (MST dependent)
22C:166 Distributed Systems and Algorithms Course, Sukumar Ghosh, University of Iowa (routing part heavily dependent)
