Minimum Spanning Tree in Graph

- Week 13

2

Problem: Laying Telephone Wire

Central office

3

Wiring: Naive Approach

Central office

Expensive!

4

Wiring: Better Approach

Central office

Minimize the total length of wire connecting ALL customers

5

Spanning trees Suppose you have a connected undirected graph:

Connected: every node is reachable from every other node Undirected: edges do not have an associated direction

...then a spanning tree of the graph is a connected subgraph which contains all the vertices and has no cycles.

A connected,undirected graph

Four of the spanning trees of the graph

All 16 of its Spanning TreesComplete Graph

7

Minimum-cost spanning trees

Suppose you have a connected undirected graph with a weight (or cost) associated with each edge.

The cost of a spanning tree would be the sum of the costs of its edges.

A minimum-cost spanning tree is a spanning tree that has the lowest cost.

A B

E D

F C

16

19

21 11

33 14

1810

6

5

A connected, undirected graph

A B

E D

F C

1611

18

6

5

A minimum-cost spanning tree

8

Minimum Spanning Tree (MST)

it is a tree (i.e., it is acyclic)

it covers all the vertices V contains |V| - 1 edges

the total cost associated with tree edges is the minimum among all possible spanning trees

not necessarily unique.

A minimum spanning tree is a subgraph of an undirected weighted graph G, such that

Tree = connected graph without cycles

10

How Can We Generate a MST?

a

ce

d

b2

45

9

6

4

5

5

a

ce

d

b2

45

9

6

4

5

5

11

Finding minimum spanning trees

There are two basic algorithms for finding minimum-cost spanning trees, and both are greedy algorithms.

Kruskal’s algorithm: Start with no nodes or edges in the spanning tree, and repeatedly add the cheapest edge that does not create a cycle

Here, we consider the spanning tree to consist of edges only

Prim’s algorithm: Start with any one node in the spanning tree, and repeatedly add the cheapest edge, and the node it leads to, for which the node is not already in the spanning tree.

Here, we consider the spanning tree to consist of both nodes and edges

12

Kruskal’s algorithm

The steps are:

1. The forest is constructed - with each node in a separate tree. 2. The edges are placed in a priority queue. 3. Until we've added n-1 edges, (1). Extract the cheapest edge from the priority queue, (2). If it forms a cycle, reject it. Else add it to the forest. Adding it to the forest will join two trees together.

Every step will have joined two trees in the forest together, so that at the end, there will only be one tree in T.

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Complete Graph

1

4

2

5

2

5

4

3

4

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

A AB D

B B

B

C D

J C

C

E

F

D

D H

J E G

F FG I

G GI J

H J JI

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Sort Edges (in reality they are placed in a priority queue - not sorted - but sorting them

makes the algorithm easier to visualize)

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Add Edge

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Add Edge

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Add Edge

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Add Edge

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Add Edge

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Cycle

Don’t Add Edge

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Add Edge

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Add Edge

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Add Edge

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Cycle

Don’t Add Edge

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Add Edge

4

1

2

2 1

3

32

4

A

B C

D

E F

G

HI

J

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Minimum Spanning Tree Complete Graph

Visualization of Kruskal's algorithm

“repeatedly add the cheapest edge that does not create a cycle”

Time complexity of Kruskal's Algorithm

Running Time = O(E log E) (E = # edges)

Testing if an edge creates a cycle can be slow unless a complicated data structure called a “union-find” structure is used.

This algorithm works best, of course, if the number of edges is kept to a minimum.

30

Prim’s algorithm

The steps are:

1. Initialize a tree with a single node arbitrarily chose from graph. 2. Repeat until all nodes are in the tree: (1). Find the node from the graph with the smallest connecting edge to the tree, (2). Add it to the tree

Every step will have joined one node, so that at the end we will have one new graph with all the nodes and it will be a minimum spanning tree of the original graph.

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Complete Graph

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Old Graph New Graph

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Old Graph New Graph

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Old Graph New Graph

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Old Graph New Graph

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Old Graph New Graph

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Old Graph New Graph

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Old Graph New Graph

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Old Graph New Graph

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Old Graph New Graph

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Old Graph New Graph

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

4

1

2

2 1

3

32

4

A

B C

D

E F

G

HI

J

Complete Graph Minimum Spanning Tree

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Unlike Kruskal’s, Prim’s algorithm doesn’t need to see all of the graph at once. It can deal with it one piece at a time.

It also doesn’t need to worry if adding an edge will create a cycle since this algorithm deals primarily with the nodes, and not the edges.

Summary of Prim's Algorithm

Time complexity of Prim's Algorithm

Running Time = O(E + V log V) (E = #edges, V = #nodes)

45

Time Complexity Review

Kruskal’s algorithm: O(e log e) Prim’s algorithm: O( e log v)

Kruskal’s algorithm is preferable on sparse graphs, i.e., where e is very small compared to the total number of possible edges.

Prim’s algorithm is easy to implemented, but the number of vertices needs to be kept to a minimum in addition to the number of edges.

46

1. How does Google quickly find web pages that contain a search term? Answer

2. How can a subsequence of DNA be quickly found within the genome? Answer

3. How does your operating system track which memory (disk or RAM) is free? Answer

4. How to generate a Maze game? Answer, Example implementation

5. Etc…

Slide 11 on the Week 1 – Introduction of the class “Example applications of data structures and algorithms”

47

Mazes Typically,

Every location in a maze is reachable from the starting location

There is only one path from start to finish

If the cells are “vertices” and the open doors between cells are “edges,” this describes a spanning tree

Since there is exactly one path between any pair of cells, any cells can be used as “start” and “finish”

This describes a spanning tree

48

Mazes as spanning trees

While not every maze is a spanning tree, most can be represented as such.

The nodes are “places” within the maze.

There is exactly one cycle-free path from any node to any other node.

49

Building a maze I

This algorithm requires two sets of cells

the set of cells already in the spanning tree, IN

the set of cells adjacent to the cells in the spanning tree (but not in it themselves), called the FRONTIER

Start with all walls present

Pick any cell and put it into IN (red)

• Put all adjacent cells, that aren’t in IN, into FRONTIER (blue)

50

Building a maze II

Repeatedly do the following:

Remove any one cell C from FRONTIER and put it in IN

Erase the wall between C and some one adjacent cell in IN

Add to FRONTIER all the cells adjacent to C that aren’t in IN (or in FRONTIER already)

• Continue until there are no more cells in FRONTIER

• When the maze is complete (or at any time), choose the start and finish cells

Generating a maze by Prim’s algorithm

Prim's algorithm has many applications, such as in the generation of this maze, which applies Prim's algorithm to a randomly weighted grid graph.

Click on to see the Demo.

51