8/3/2019 22348266 Graphs Topo Intro
1/58
Graph Traversalsand
Minimum Spanning Trees
15-211: Fundamental Data Structures
and Algorithms
Rose Hoberman
April 8, 2003
8/3/2019 22348266 Graphs Topo Intro
2/58
Announcements
8/3/2019 22348266 Graphs Topo Intro
3/58
315-211: Fundamental DataStructures and Algorithms Rose HobermanApril 8, 2003
Announcements
Readings: Chapter 14
HW5: Due in less than one week! Monday, April 14, 2003, 11:59pm
8/3/2019 22348266 Graphs Topo Intro
4/58
415-211: Fundamental DataStructures and Algorithms Rose HobermanApril 8, 2003
Today
More Graph Terminology (some review)
Topological sort
Graph Traversals (BFS and DFS)
Minimal Spanning Trees
After Class... Before Recitation
8/3/2019 22348266 Graphs Topo Intro
5/58
GraphTerminology
8/3/2019 22348266 Graphs Topo Intro
6/58
615-211: Fundamental DataStructures and Algorithms Rose HobermanApril 8, 2003
Paths and cycles
Apath is a sequence of nodesv
1, v
2, , v
Nsuch that (v
i,v
i+1)E for 0
8/3/2019 22348266 Graphs Topo Intro
7/58
715-211: Fundamental DataStructures and Algorithms Rose HobermanApril 8, 2003
Cycles
PIT
BOS
JFK
DTW
LAX
SFO
8/3/2019 22348266 Graphs Topo Intro
8/58
815-211: Fundamental DataStructures and Algorithms Rose HobermanApril 8, 2003
More useful definitions
In a directed graph:
The indegree of a node v is the number ofdistinct edges (w,v)E.
The outdegree of a node v is the number of
distinct edges (v,w)
E.
A node with indegree 0 is a root.
8/3/2019 22348266 Graphs Topo Intro
9/58
915-211: Fundamental DataStructures and Algorithms Rose HobermanApril 8, 2003
Trees are graphs
A dagis a directed acyclic graph.
A tree is a connected acyclic undirected
graph.
A forestis an acyclic undirected graph (notnecessarily connected), i.e., each connectedcomponent is a tree.
8/3/2019 22348266 Graphs Topo Intro
10/58
1015-211: Fundamental DataStructures and Algorithms Rose HobermanApril 8, 2003
Example DAG
Watch
Socks
Shoes
Undershorts
Pants
Belt Tie
Shirt
Jacket
a DAG implies an
ordering on events
8/3/2019 22348266 Graphs Topo Intro
11/58
1115-211: Fundamental DataStructures and Algorithms Rose HobermanApril 8, 2003
Example DAG
Watch
Socks
Shoes
Undershorts
Pants
Belt Tie
Shirt
Jacket
In a complex DAG, it
can be hard to find a
schedule that obeys
all the constraints.
8/3/2019 22348266 Graphs Topo Intro
12/58
Topological Sort
8/3/2019 22348266 Graphs Topo Intro
13/58
1315-211: Fundamental DataStructures and Algorithms Rose HobermanApril 8, 2003
Topological Sort
For a directed acyclic graph G = (V,E)
A topological sort is an ordering of all of Gsvertices v
1, v
2, , v
nsuch that...
Formally: for every edge (vi,v
k) in E, i
8/3/2019 22348266 Graphs Topo Intro
14/58
1415-211: Fundamental DataStructures and Algorithms Rose HobermanApril 8, 2003
Topological sort
There are often many possible topologicalsorts of a given DAG
Topological orders for this DAG:
1,2,5,4,3,6,7
2,1,5,4,7,3,6
2,5,1,4,7,3,6
Etc.
Each topological order is a feasible schedule.
1
4
76
3 5
2
8/3/2019 22348266 Graphs Topo Intro
15/58
1515-211: Fundamental DataStructures and Algorithms Rose HobermanApril 8, 2003
Topological Sorts for Cyclic
Graphs?
Impossible!1 2
3
If v and w are two vertices on a cycle, thereexist paths from v to w andfrom w to v.
Any ordering will contradict one of these paths
8/3/2019 22348266 Graphs Topo Intro
16/58
1615-211: Fundamental DataStructures and Algorithms
Rose HobermanApril 8, 2003
Topological sort algorithm
Algorithm
Assume indegree is stored with each node. Repeat until no nodes remain:
Choose a root and output it. Remove the root and all its edges.
Performance
O(V
2 +
E), if linear search is used to find a root.
8/3/2019 22348266 Graphs Topo Intro
17/58
1715-211: Fundamental DataStructures and Algorithms
Rose HobermanApril 8, 2003
Better topological sort
Algorithm: Scan all nodes, pushing roots onto a stack.
Repeat until stack is empty:
Pop a root r from the stack and output it. For all nodes n such that (r,n) is an edge, decrement nsindegree. If 0 then push onto the stack.
O( V + E ), so still O(V2) in worst case, but better
for sparse graphs. Q: Why is this algorithm correct?
8/3/2019 22348266 Graphs Topo Intro
18/58
1815-211: Fundamental DataStructures and Algorithms
Rose HobermanApril 8, 2003
Correctness
Clearly any ordering produced by this algorithm isa topological order
But...
Does every DAG have a topological order, and ifso, is this algorithm guaranteed to find one?
8/3/2019 22348266 Graphs Topo Intro
19/58
Quiz Break
8/3/2019 22348266 Graphs Topo Intro
20/58
2015-211: Fundamental DataStructures and Algorithms
Rose HobermanApril 8, 2003
Quiz
Prove: This algorithm never gets stuck, i.e. if there
are unvisited nodes then at least one ofthem has an indegree of zero.
Hint: Prove that if at any point there are unseen
vertices but none of them have an indegreeof 0, a cycle must exist, contradicting our
assumption of a DAG.
8/3/2019 22348266 Graphs Topo Intro
21/58
2115-211: Fundamental DataStructures and Algorithms
Rose HobermanApril 8, 2003
Proof
See Weiss page 476.
8/3/2019 22348266 Graphs Topo Intro
22/58
Graph Traversals
8/3/2019 22348266 Graphs Topo Intro
23/58
2315-211: Fundamental DataStructures and Algorithms
Rose HobermanApril 8, 2003
Graph Traversals
Both take time: O(V+E)
8/3/2019 22348266 Graphs Topo Intro
24/58
2415-211: Fundamental DataStructures and Algorithms
Rose HobermanApril 8, 2003
Use of a stack
It is very common to use a stack to keep trackof: nodes to be visited next, or
nodes that we have already visited. Typically, use of a stack leads to a depth-first
visit order.
Depth-first visit order is aggressive in thesense that it examines complete paths.
8/3/2019 22348266 Graphs Topo Intro
25/58
2515-211: Fundamental DataStructures and Algorithms
Rose HobermanApril 8, 2003
Topological Sort as DFS
do a DFS of graph G
as each vertex v is finished (all of itschildren processed), insert it onto the front of
a linked list return the linked list of vertices
why is this correct?
8/3/2019 22348266 Graphs Topo Intro
26/58
2615-211: Fundamental DataStructures and Algorithms
Rose HobermanApril 8, 2003
Use of a queue
It is very common to use a queue to keeptrack of: nodes to be visited next, or
nodes that we have already visited. Typically, use of a queue leads to a breadth-
firstvisit order.
Breadth-first visit order is cautious in the
sense that it examines every path of length ibefore going on to paths of length i+1.
8/3/2019 22348266 Graphs Topo Intro
27/58
2715-211: Fundamental DataStructures and Algorithms
Rose HobermanApril 8, 2003
Graph Searching ???
Graph as state space (node = state, edge = action)
For example, game trees, mazes, ...
BFS and DFS each search the state space for a bestmove. If the search is exhaustive they will find the samesolution, but if there is a time limit and the search space islarge...
DFS explores a few possible moves, looking at the effectsfar in the future
BFS explores many solutions but only sees effects in thenear future (often finds shorter solutions)
8/3/2019 22348266 Graphs Topo Intro
28/58
2815-211: Fundamental DataStructures and Algorithms
Rose HobermanApril 8, 2003
Minimum Spanning Trees
8/3/2019 22348266 Graphs Topo Intro
29/58
2915-211: Fundamental DataStructures and Algorithms
Rose HobermanApril 8, 2003
Problem: Laying Telephone Wire
Central office
8/3/2019 22348266 Graphs Topo Intro
30/58
3015-211: Fundamental DataStructures and Algorithms
Rose HobermanApril 8, 2003
Wiring: Nave Approach
Central office
Expensive!
8/3/2019 22348266 Graphs Topo Intro
31/58
3115-211: Fundamental DataStructures and Algorithms
Rose HobermanApril 8, 2003
Wiring: Better Approach
Central office
Minimize the total length of wire connecting the customers
8/3/2019 22348266 Graphs Topo Intro
32/58
3215-211: Fundamental DataStructures and Algorithms
Rose Hoberman
April 8, 2003
Minimum Spanning Tree (MST)(see Weiss, Section 24.2.2)
it is a tree (i.e., it is acyclic)
it covers all the vertices V contains |V| - 1 edges
the total cost associated with tree edges is theminimum among all possible spanning trees
not necessarily unique
A minimum spanning tree is a subgraph of anundirected weighted graph G, such that
8/3/2019 22348266 Graphs Topo Intro
33/58
3315-211: Fundamental DataStructures and Algorithms
Rose Hoberman
April 8, 2003
Applications of MST
Any time you want to visit all vertices in a graph at
minimum cost (e.g., wire routing on printed circuit boards,sewer pipe layout, road planning)
Internet content distribution
$$$, also a hot research topic Idea: publisher produces web pages, content distribution network
replicates web pages to many locations so consumers can accessat higher speed
MST may not be good enough! content distribution on minimum cost tree may take a long time!
Provides a heuristic for traveling salesman problems. Theoptimum traveling salesman tour is at most twice thelength of the minimum spanning tree (why??)
8/3/2019 22348266 Graphs Topo Intro
34/58
3415-211: Fundamental DataStructures and Algorithms
Rose Hoberman
April 8, 2003
How Can We Generate a MST?
a
ce
d
b2
45
9
6
4
5
5
a
ce
d
b2
45
9
6
4
5
5
8/3/2019 22348266 Graphs Topo Intro
35/58
3515-211: Fundamental DataStructures and Algorithms
Rose Hoberman
April 8, 2003
Prims Algorithm
Initializationa. Pick a vertex rto be the rootb. SetD(r) = 0, parent(r) = nullc. For all vertices v V, v r, setD(v) =
d. Insert all vertices into priority queueP,using distances as the keys
a
ce
d
b2
45
9
6
4
5
5
e a b c d
0
Vertex Parente -
8/3/2019 22348266 Graphs Topo Intro
36/58
3615-211: Fundamental DataStructures and Algorithms
Rose Hoberman
April 8, 2003
Prims Algorithm
WhilePis not empty:
1. Select the next vertex u to add to the treeu = P.deleteMin()
2. Update the weight of each vertex wadjacent tou which is not in the tree (i.e., wP)
Ifweight(u,w)< D(w),a.parent(w) = u
b.D(w) = weight(u,w)c. Update the priority queue to reflectnew distance forw
8/3/2019 22348266 Graphs Topo Intro
37/58
3715-211: Fundamental DataStructures and Algorithms
Rose Hoberman
April 8, 2003
Prims algorithm
a
ce
d
b2
45
9
6
4
5
5
d b c a
4 5 5
Vertex Parente -b ec ed e
The MST initially consists of the vertex e, and we updatethe distances and parent for its adjacent vertices
Vertex Parent
e -b -c -d -
d b c a
e
0
8/3/2019 22348266 Graphs Topo Intro
38/58
3815-211: Fundamental DataStructures and Algorithms
Rose Hoberman
April 8, 2003
Prims algorithm
a
ce
d
b2
45
9
6
4
5
5
a c b
2 4 5
Vertex Parente -b ec d
d ea d
d b c a
4 5 5
Vertex Parent
e -b ec ed e
8/3/2019 22348266 Graphs Topo Intro
39/58
3915-211: Fundamental DataStructures and Algorithms
Rose Hoberman
April 8, 2003
Prims algorithm
a
ce
d
b2
45
9
6
4
5
5
c b
4 5
Vertex Parente -b ec dd ea d
a c b
2 4 5
Vertex Parente -b ec dd ea d
8/3/2019 22348266 Graphs Topo Intro
40/58
4015-211: Fundamental DataStructures and Algorithms
Rose Hoberman
April 8, 2003
Prims algorithm
a
ce
d
b2
45
9
6
4
5
5
b
5
Vertex Parente -b e
c dd ea d
c b
4 5
Vertex Parente -b ec dd ea d
8/3/2019 22348266 Graphs Topo Intro
41/58
4115-211: Fundamental DataStructures and Algorithms
Rose Hoberman
April 8, 2003
Prims algorithm
Vertex Parente -b ec dd ea d
a
ce
d
b2
45
9
6
4
5
5
The final minimum spanning tree
b
5
Vertex Parent
e -b ec dd ea d
8/3/2019 22348266 Graphs Topo Intro
42/58
4215-211: Fundamental DataStructures and Algorithms
Rose Hoberman
April 8, 2003
Running time of Prims algorithm
(without heaps)
Initialization of priority queue (array): O(|V|)
Update loop: |V| calls Choosing vertex with minimum cost edge: O(|V|)
Updating distance values of unconnectedvertices: each edge is considered only onceduring entire execution, for a total of O(|E|)updates
Overall cost without heaps:
When heaps are used, apply same analysis as forDijkstras algorithm (p.469) (good exercise)
O(|E| + |V| 2)
8/3/2019 22348266 Graphs Topo Intro
43/58
4315-211: Fundamental DataStructures and Algorithms
Rose Hoberman
April 8, 2003
Prims Algorithm Invariant
At each step, we add the edge (u,v) s.t. theweight of(u,v) is minimum among all edgeswhere u is in the tree and v is not in the tree
Each step maintains a minimum spanning tree ofthe vertices that have been included thus far
When all vertices have been included, we have aMST for the graph!
8/3/2019 22348266 Graphs Topo Intro
44/58
4415-211: Fundamental DataStructures and Algorithms
Rose Hoberman
April 8, 2003
Correctness of Prims
This algorithm adds n-1 edges without creating a cycle,
so clearly it creates a spanning tree of any connectedgraph (you should be able to prove this).
But is this a minimum spanning tree?
Suppose it wasn't.
There must be point at which it fails, and in particularthere must a single edge whose insertion first
prevented the spanning tree from being a minimumspanning tree.
8/3/2019 22348266 Graphs Topo Intro
45/58
4515-211: Fundamental DataStructures and Algorithms
Rose Hoberman
April 8, 2003
Correctness of Prims
Let V' be the vertices incident with edges in S
Let T be a MST ofGcontaining all edges in S, but not (x,y).
Let Gbe a connected,undirected graph
Let S be the set ofedges chosen by Primsalgorithm beforechoosing an errorful
edge (x,y)
xy
8/3/2019 22348266 Graphs Topo Intro
46/58
4615-211: Fundamental DataStructures and Algorithms
Rose Hoberman
April 8, 2003
Correctness of Prims
xy
v
w
There is exactly one edge on this cycle with exactly onevertex in V, call this edge (v,w)
Edge (x,y) is not in T, sothere must be a path inT fromxto ysince T isconnected.
Inserting edge (x,y) into
T will create a cycle
8/3/2019 22348266 Graphs Topo Intro
47/58
4715-211: Fundamental Data
Structures and Algorithms
Rose Hoberman
April 8, 2003
Correctness of Prims
Since Prims chose (x,y)over(v,w), w(v,w) >= w(x,y).
We could form a new spanning tree T by swapping (x,y)for(v,w) in T (prove this is a spanning tree).
w(T) is clearly no greater than w(T)
But that means T is a MST
And yet it contains all the edges in S, and also (x,y)
...Contradiction
8/3/2019 22348266 Graphs Topo Intro
48/58
4815-211: Fundamental Data
Structures and Algorithms
Rose Hoberman
April 8, 2003
Another Approach
a
ce
d
b2
45
9
6
4
5
5
Create a forest of trees from the vertices Repeatedly merge trees by adding safe edges
until only one tree remains A safe edge is an edge of minimum weight which
does not create a cycle
forest: {a}, {b}, {c}, {d}, {e}
8/3/2019 22348266 Graphs Topo Intro
49/58
4915-211: Fundamental Data
Structures and Algorithms
Rose Hoberman
April 8, 2003
Kruskals algorithm
Initializationa. Create a set for each vertex v Vb. Initialize the set of safe edgesA
comprising the MST to the empty set
c. Sort edges by increasing weight
a
ce
d
b2
45
9
6
4
5
5
F = {a}, {b}, {c}, {d}, {e}A =E= {(a,d), (c,d), (d,e), (a,c),
(b,e), (c,e), (b,d), (a,b)}
8/3/2019 22348266 Graphs Topo Intro
50/58
5015-211: Fundamental Data
Structures and Algorithms
Rose Hoberman
April 8, 2003
Kruskals algorithm
For each edge (u,v) Ein increasing orderwhile more than one set remains:
Ifu and v, belong to different sets Uand Va. add edge (u,v) to the safe edge set
A = A{(u,v)}
b. merge the sets Uand VF = F- U- V + (U V)
ReturnA
Running time bounded by sorting (or findMin)
O(|E|log|E|), or equivalently, O(|E|log|V|) (why???)
8/3/2019 22348266 Graphs Topo Intro
51/58
5115-211: Fundamental Data
Structures and Algorithms
Rose Hoberman
April 8, 2003
Kruskals algorithm
E= {(a,d), (c,d), (d,e), (a,c),(b,e), (c,e), (b,d), (a,b)}
Forest{a}, {b}, {c}, {d}, {e}{a,d}, {b}, {c}, {e}
{a,d,c}, {b}, {e}{a,d,c,e}, {b}{a,d,c,e,b}
A
{(a,d)}
{(a,d), (c,d)}{(a,d), (c,d), (d,e)}{(a,d), (c,d), (d,e), (b,e)}
a
c
e
db
2
45
9
6
4
5
5
8/3/2019 22348266 Graphs Topo Intro
52/58
5215-211: Fundamental Data
Structures and Algorithms
Rose Hoberman
April 8, 2003
After each iteration, every tree in the forest is a MSTof the vertices it connects
Algorithm terminates when all vertices are connectedinto one tree
Kruskals Algorithm Invariant
8/3/2019 22348266 Graphs Topo Intro
53/58
5315-211: Fundamental Data
Structures and Algorithms
Rose Hoberman
April 8, 2003
Correctness of Kruskals
This algorithm adds n-1 edges without creating a
cycle, so clearly it creates a spanning tree of anyconnected graph (you should be able to prove this).
But is this a minimum spanning tree?
Suppose it wasn't.
There must be point at which it fails, and in particularthere must a single edge whose insertion firstprevented the spanning tree from being a minimumspanning tree.
8/3/2019 22348266 Graphs Topo Intro
54/58
5415-211: Fundamental Data
Structures and Algorithms
Rose Hoberman
April 8, 2003
Correctness of Kruskals
Let e be this first errorful edge.
Let K be the Kruskal spanning tree
Let S be the set of edges chosen by Kruskals algorithm before choosing e
Let T be a MST containing all edges in S, but not e.
K TS
e
8/3/2019 22348266 Graphs Topo Intro
55/58
5515-211: Fundamental Data
Structures and Algorithms
Rose Hoberman
April 8, 2003
Correctness of Kruskals
Proof(by contradiction):
Assume there exists someedge e in T - S, w(e) = w(e) forall edges e in T - S
However, since e is not in K (why??), it must have
been discarded because it caused a cycle with some ofthe other edges in S.
But e + S is a subgraph ofT, which means it cannotform a cycle ...Contradiction
8/3/2019 22348266 Graphs Topo Intro
56/58
5615-211: Fundamental Data
Structures and Algorithms
Rose Hoberman
April 8, 2003
Correctness of Kruskals
Inserting edge e into T will create a cycle
There must be an edge on this cycle which is not in K (why??).Call this edge e
e must be in T - S, so(by our lemma) w(e) >= w(e)
We could form a new spanning tree T by swapping e fore in T
(prove this is a spanning tree). w(T) is clearly no greater than w(T)
But that means T is a MST
And yet it contains all the edges in S, and also e
...Contradiction
8/3/2019 22348266 Graphs Topo Intro
57/58
5715-211: Fundamental Data
Structures and Algorithms
Rose Hoberman
April 8, 2003
Greedy Approach
Like Dijkstras algorithm, both Prims and Kruskalsalgorithms are greedy algorithms
The greedy approach works for the MST problem;however, it does not work for many otherproblems!
8/3/2019 22348266 Graphs Topo Intro
58/58
Thats All!