View
215
Download
0
Embed Size (px)
Citation preview
CPSC 411, Fall 2008: Set 9 1
CPSC 411Design and Analysis
of Algorithms
Set 9: More Graph AlgorithmsProf. Jennifer Welch
Fall 2008
CPSC 411, Fall 2008: Set 9 2
Generic MST Algorithm input: weighted undirected graph
G = (V,E,w) T := empty set while T is not yet a spanning tree of G
find an edge e in E s.t. T U {e} is a subgraph of some MST of G
add e to T return T (as MST of G)
CPSC 411, Fall 2008: Set 9 3
Kruskal's Algorithm as a Special Case of Generic Alg. Consider edges in increasing order of
weight Add the next edge iff it doesn't cause
a cycle At any point, T is a forest (set of
trees); eventually T is a single tree
CPSC 411, Fall 2008: Set 9 4
Idea of Prim's Algorithm Instead of growing the MST as possibly
multiple trees that eventually all merge, grow the MST from a single node, so that there is only one tree at any point.
Also a special case of the generic algorithm: at each step, add the minimum weight edge that goes out from the tree constructed so far.
CPSC 411, Fall 2008: Set 9 5
Prim's Algorithm input: weighted undirected graph G = (V,e,w) T := empty set S := {any node in V} while |T| < |V| - 1 do
let (u,v) be a min wt. outgoing edge (u in S, v not in S) add (u,v) to T add v to S
return (S,T) (as MST of G)
CPSC 411, Fall 2008: Set 9 6
Prim's Algorithm Example
a
b
h
i
c
g
d
f
e
4
8 7
8
11
2
7 6
1 2
4 14
9
10
CPSC 411, Fall 2008: Set 9 7
Correctness of Prim's Algorithm Let Ti be the tree represented by (S,T)
at the end of iteration i. Show by induction on i that Ti is a
subtree of some MST of G. Basis: i = 0 (before first iteration). T0
contains just a single node, and thus is a subtree of every MST of G.
CPSC 411, Fall 2008: Set 9 8
Correctness of Prim's Algorithm Induction: Assume Ti is a subtree of
some MST M. We must show Ti+1 is a subtree of some MST.
Let (u,v) be the edge added in iteration i+1.
u vTi
Ti+1
Case 1: (u,v) is in M.Then Ti+1 is also asubtree of M.
CPSC 411, Fall 2008: Set 9 9
Correctness of Prim's AlgorithmCase 2: (u,v) is not in M. There is a path P in M from u to v, since M
spans G. Let (x,y) be the first edge in P with one
endpoint in Ti and the other not in Ti.
u v
Ti
x
yP
CPSC 411, Fall 2008: Set 9 10
Correctness of Prim's Algorithm Let M' = M - {(x,y)} U {(u,v)} M' is also a spanning tree of G. w(M') = w(M) - w(x,y) + w(u,v) ≤ w(M)
since (u,v) is min wt outgoing edge So M' is also an MST and
Ti+1 is subtree
M' u v
Ti
x
y
Ti+1
CPSC 411, Fall 2008: Set 9 11
Implementing Prim's Algorithm How do we find minimum weight
outgoing edge? First cut: scan all adjacency lists at
each iteration. Results in O(VE) time. Try to do better.
CPSC 411, Fall 2008: Set 9 12
Implementing Prim's Algorithm Idea: have each node not yet in the
tree keep track of its best (cheapest) edge to the tree constructed so far.
To find min wt. outgoing edge, find minimum among these values use a priority queue to store the best edge
info (insert and extract-min operations)
CPSC 411, Fall 2008: Set 9 13
Implementing Prim's Algorithm When a node v is added to T, some
other nodes might have their best edges affected, but only neighbors of v add decrease-key operation to the priority
queueu
vTi
Ti+1 w
w's best edge to Ti
check if this edge is cheaper for w
x
x's best edge to Ti
v's best edge to Ti
CPSC 411, Fall 2008: Set 9 14
Details on Prim's AlgorithmAssociate with each node v two fields: best-wt[v] : if v is not yet in the tree, then
it holds the min. wt. of all edges from v to a node in the tree. Initially infinity.
best-node[v] : if v is not yet in the tree, then it holds the name of the node u in the tree s.t. w(v,u) is v's best-wt. Initially nil.
CPSC 411, Fall 2008: Set 9 15
Details on Prim's Algorithm input: G = (V,E,w)// initialization initialize priority queue Q to contain all
nodes, using best-wt values as keys let v0 be any node in V decrease-key(Q,v0,0)
// last line means change best-wt[v0] to 0 and adjust Q accordingly
CPSC 411, Fall 2008: Set 9 16
Details on Prim's Algorithm while Q is not empty do
u := extract-min(Q) // node w/ smallest best-wt
if u is not v0 then add (u,best-node[u]) to T for each neighbor v of u do
if v is in Q and w(u,v) < best-wt[v] then best-node[v] := u decrease-key(Q,v,w(u,v))
return (V,T) // as MST of G
CPSC 411, Fall 2008: Set 9 17
Running Time of Prim's AlgorithmDepends on priority queue implementation. Let Tins be time for insert Tdec be time for decrease-key Tex be time for extract-minThen we have |V| inserts and one decrease-key in the
initialization: O(VTins+Tdec) |V| iterations of while
one extract-min per iteration: O(VTex) total
CPSC 411, Fall 2008: Set 9 18
Running Time of Prim's Algorithm Each iteration of while includes a for
loop. Number of iterations of for loop varies,
depending on how many neighbors the current node has
Total number of iterations of for loop is O(E).
Each iteration of for loop: one decrease key, so O(ETdec) total
CPSC 411, Fall 2008: Set 9 19
Running Time of Prim's Algorithm O(V(Tins + Tex) + ETdec) If priority queue is implemented with
a binary heap, then Tins = Tex = Tdec = O(log V) total time is O(E log V)
(Think about how to implement decrease-key in O(log V) time.)
CPSC 411, Fall 2008: Set 9 20
Shortest Paths in a Graph We already saw the Floyd-Warshall
algorithm to compute all pairs of shortest paths
Let's review two important single-source shortest path algorithms: Dijkstra's algorithm Bellman-Ford algorithm
CPSC 411, Fall 2008: Set 9 21
Single Source Shortest Path Problem Given: directed or undirected graph
G = (V,E,w) and source node s in V Find: For each t in V, a path in G from
s to t with minimum weight Warning! Negative weights are a
problem:
s t4
5
CPSC 411, Fall 2008: Set 9 22
Shortest Path Tree Result of a SSSP algorithm can be
viewed as a tree rooted at the source Why not use breadth-first search? Works fine if all weights are the same:
weight of each path is (a multiple of) the number of edges in the path
Doesn't work when weights are different
CPSC 411, Fall 2008: Set 9 23
Dijkstra's SSSP Algorithm Assumes all edge weights are nonnegative Similar to Prim's MST algorithm Start with source node s and iteratively
construct a tree rooted at s Each node keeps track of tree node that
provides cheapest path from s (not just cheapest path from any tree node)
At each iteration, include the node whose cheapest path from s is the overall cheapest
CPSC 411, Fall 2008: Set 9 24
Prim's vs. Dijkstra's
s
5
4
1
6
Prim's MST
s
5
4
1
6
Dijkstra's SSSP
CPSC 411, Fall 2008: Set 9 25
Implementing Dijkstra's Alg. How can each node u keep track of its best
path from s? Keep an estimate, d[u], of shortest path
distance from s to u Use d as a key in a priority queue When u is added to the tree, check each of
u's neighbors v to see if u provides v with a cheaper path from s: compare d[v] to d[u] + w(u,v)
CPSC 411, Fall 2008: Set 9 26
Dijkstra's Algorithm input: G = (V,E,w) and source node s// initialization d[s] := 0 d[v] := infinity for all other nodes v initialize priority queue Q to contain
all nodes using d values as keys
CPSC 411, Fall 2008: Set 9 27
Dijkstra's Algorithm while Q is not empty do
u := extract-min(Q) for each neighbor v of u do
if d[u] + w(u,v) < d[v] thend[v] := d[u] + w(u,v)decrease-key(Q,v,d[v])parent(v) := u
CPSC 411, Fall 2008: Set 9 28
Dijkstra's Algorithm Example
a b
d e
c
2
84 9
2
4
12
10 6 3
source is node a
0 1 2 3 4 5Q abcd
ebcde cde de d Ø
d[a]
0 0 0 0 0 0
d[b]
∞ 2 2 2 2 2
d[c] ∞ 12 10 10 10 10
d[d]
∞ ∞ ∞ 16 13 13
d[e]
∞ ∞ 11 11 11 11
iteration
CPSC 411, Fall 2008: Set 9 29
Correctness of Dijkstra's Alg. Let Ti be the tree constructed after i-th
iteration of while loop: nodes not in Q edges indicated by parent variables
Show by induction on i that the path in Ti
from s to u is a shortest path and has distance d[u], for all u in Ti.
Basis: i = 1. s is the only node in T1 and d[s] = 0.
CPSC 411, Fall 2008: Set 9 30
Correctness of Dijkstra's Alg. Induction: Assume Ti is a correct shortest path tree
and show for Ti. Let u be the node added in iteration i. Let x = parent(u).
s x
Ti
u
Ti+1
Need to show path in Ti+1 from s to u is a shortest path, and has distance d[u]
CPSC 411, Fall 2008: Set 9 31
Correctness of Dijkstra's Alg
s
x
Ti
u
Ti+1
P, path in Ti+1
from s to u
(a,b) is first edge in P' thatleaves Ti
a
b
P', anotherpath from s to u
CPSC 411, Fall 2008: Set 9 32
Correctness of Dijkstra's Alg
s
x
Ti
u
Ti+1
a
bP'
PLet P1 be part of P' before (a,b.)
Let P2 be part of P' after (a,b).
w(P') = w(P1) + w(a,b) + w(P2)
≥ w(P1) + w(a,b) (nonneg wts)
≥ wt of path in Ti from s to a + w(a,b) (inductive hypothesis)
≥ w(s->x path in Ti) + w(x,u) (alg chose u in iteration i and
d-values are accurate, by inductive hypothesis
= w(P).
So P is a shortest path, and d[u] is accurate after iteration i+1.
CPSC 411, Fall 2008: Set 9 33
Running Time of Dijstra's Alg. initialization: insert each node once
O(V Tins)
O(V) iterations of while loop one extract-min per iteration => O(V Tex) for loop inside while loop has variable number of
iterations…
For loop has O(E) iterations total one decrease-key per iteration => O(E Tdec)
Total is O(V (Tins + Tex) + E Tdec)
CPSC 411, Fall 2008: Set 9 34
Using Fancier Heap Implementations O(V(Tins + Tex) + ETdec) If priority queue is implemented with a
binary heap, then Tins = Tex = Tdec = O(log V) total time is O(E log V)
There are fancier implementations of the priority queue, such as Fibonacci heap: Tins = O(1), Tex = O(log V), Tdec = O(1) (amortized) total time is O(V log V + E)
CPSC 411, Fall 2008: Set 9 35
Using Simpler Heap Implementations O(V(Tins + Tex) + ETdec) If graph is dense, so that |E| = (V2), then it
doesn't help to make Tins and Tex to be at most O(V).
Instead, focus on making Tdec be small, say constant.
Implement priority queue with an unsorted array: Tins = O(1), Tex = O(V), Tdec = O(1) total is O(V2)
CPSC 411, Fall 2008: Set 9 36
What About Negative Edge Weights? Dijkstra's SSSP algorithm requires all
edge weights to be nonnegative even more restrictive than outlawing
negative weight cycles Bellman-Ford SSSP algorithm can
handle negative edge weights even "handles" negative weight cycles by
reporting they exist
CPSC 411, Fall 2008: Set 9 37
Bellman-Ford Idea Consder each edge (u,v) and see if u
offers v a cheaper path from s compare d[v] to d[u] + w(u,v)
Repeat this process |V| - 1 times to ensure that accurate information propgates from s, no matter what order the edges are considered in
CPSC 411, Fall 2008: Set 9 38
Bellman-Ford SSSP Algorithm input: directed or undirected graph G = (V,E,w)//initialization initialize d[v] to infinity and parent[v] to nil for all v in V
other than the source initialize d[s] to 0 and parent[s] to s// main body for i := 1 to |V| - 1 do
for each (u,v) in E do // consider in arbitrary order if d[u] + w(u,v) < d[v] then
d[v] := d[u] + w(u,v) parent[v] := u
CPSC 411, Fall 2008: Set 9 39
Bellman-Ford SSSP Algorithm// check for negative weight cycles for each (u,v) in E do
if d[u] + w(u,v) < d[v] then output "negative weight cycle exists"
CPSC 411, Fall 2008: Set 9 40
Running Time of Bellman-Ford O(V) iterations of outer for loop O(E) iterations of inner for loop O(VE) time total
CPSC 411, Fall 2008: Set 9 41
Bellman-Ford Example
s
c
a
b3
—44
2
1
process edges in order(c,b)(a,b)(c,a)(s,a)(s,c)
<board work>
CPSC 411, Fall 2008: Set 9 42
Correctness of Bellman-FordAssume no negative-weight cycles.Lemma: d[v] is never an underestimate of the
actual shortest path distance from s to v.Lemma: If there is a shortest s-to-v path
containing at most i edges, then after iteration i of the outer for loop, d[v] is at most the actual shortest path distance from s to v.
Theorem: Bellman-Ford is correct.Proof: Follows from these 2 lemmas and fact
that every shortest path has at most |V| - 1 edges.
CPSC 411, Fall 2008: Set 9 43
Correctness of Bellman-Ford Suppose there is a negative weight
cycle. Then the distance will decrease even
after iteration |V| - 1 shortest path distance is negative infinity
This is what the last part of the code checks for.
CPSC 411, Fall 2008: Set 9 44
Speeding Up Bellman-Ford The previous example would have
converged faster if we had considered the edges in a different order in the for loop move outward from s
If the graph is a DAG (no cycles), we can fully exploit this idea to speed up the running time
CPSC 411, Fall 2008: Set 9 45
DAG Shortest Path Algorithm input: directed graph G = (V,E,w) and
source node s in V topologically sort G d[v] := infinity for all v in V d[s] := 0 for each u in V in topological sort order do
for each neighbor v of u do d[v] := min{d[v],d[u] + w(u,v)}
CPSC 411, Fall 2008: Set 9 46
DAG Shortest Path Algorithm Running Time is O(V + E). Example: <board work>