Introduction to Algorithms - MIT OpenCourseWare · Introduction to Algorithms 6.046J/18.401J...

Introduction to Algorithms6.046J/18.401J

LECTURE 16Greedy Algorithms (and

Graphs)• Graph representation• Minimum spanning trees• Optimal substructure• Greedy choice• Prim’s greedy MST

algorithmProf. Charles E. Leiserson

Graphs (review)Definition. A directed graph (digraph)G = (V, E) is an ordered pair consisting of• a set V of vertices (singular: vertex),• a set E ⊆ V × V of edges.In an undirected graph G = (V, E), the edge set E consists of unordered pairs of vertices.In either case, we have |E | = O(V 2). Moreover, if G is connected, then |E | ≥ |V | – 1, which implies that lg |E | = Θ(lgV). (Review CLRS, Appendix B.)

Adjacency-matrix representation

The adjacency matrix of a graph G = (V, E), where V = {1, 2, …, n}, is the matrix A[1 . . n, 1 . . n]given by

A[i, j] = 1 if (i, j) ∈ E,0 if (i, j) ∉ E.

Adjacency-matrix representation

The adjacency matrix of a graph G = (V, E), where V = {1, 2, …, n}, is the matrix A[1 . . n, 1 . . n]given by

A[i, j] = 1 if (i, j) ∈ E,0 if (i, j) ∉ E.

A 1 2 3 41234

0 1 1 00 0 1 00 0 0 00 0 1 0

Θ(V 2) storage ⇒ denserepresentation.

Adjacency-list representationAn adjacency list of a vertex v ∈ V is the list Adj[v]of vertices adjacent to v.

Adj[1] = {2, 3}Adj[2] = {3}Adj[3] = {}Adj[4] = {3}

For undirected graphs, |Adj[v] | = degree(v).For digraphs, |Adj[v] | = out-degree(v).

Adj[1] = {2, 3}Adj[2] = {3}Adj[3] = {}Adj[4] = {3}

Minimum spanning trees

Input: A connected, undirected graph G = (V, E)with weight function w : E → R.• For simplicity, assume that all edge weights are

distinct. (CLRS covers the general case.)

Minimum spanning trees

Input: A connected, undirected graph G = (V, E)with weight function w : E → R.• For simplicity, assume that all edge weights are

distinct. (CLRS covers the general case.)

∑∈

vuwTw),(

),()( .

Output: A spanning tree T — a tree that connects all vertices — of minimum weight:

Example of MST

Optimal substructureMST T:

(Other edges of Gare not shown.)

Optimal substructure

vRemove any edge (u, v) ∈ T.

MST T:

Optimal substructure

vRemove any edge (u, v) ∈ T.

MST T: (Other edges of Gare not shown.)

vRemove any edge (u, v) ∈ T. Remove any edge (u, v) ∈ T. Then, T is partitioned into two subtrees T1 and T2.

Theorem. The subtree T1 is an MST of G1 = (V1, E1), the subgraph of G induced by the vertices of T1:

V1 = vertices of T1,E1 = { (x, y) ∈ E : x, y ∈ V1 }.

Similarly for T2.

Proof of optimal substructure

w(T) = w(u, v) + w(T1) + w(T2).Proof. Cut and paste:

If T1′were a lower-weight spanning tree than T1 for G1, then T ′ = {(u, v)} ∪ T1′ ∪ T2 would be a lower-weight spanning tree than T for G.

Do we also have overlapping subproblems?•Yes.

Great, then dynamic programming may work!•Yes, but MST exhibits another powerful property which leads to an even more efficient algorithm.

Do we also have overlapping subproblems?•Yes.

Hallmark for “greedy”algorithms

Greedy-choice propertyA locally optimal choice

is globally optimal.

Hallmark for “greedy”algorithms

Greedy-choice propertyA locally optimal choice

is globally optimal.

Theorem. Let T be the MST of G = (V, E), and let A ⊆ V. Suppose that (u, v) ∈ E is the least-weight edge connecting A to V – A. Then, (u, v) ∈ T.

Proof of theoremProof. Suppose (u, v) ∉ T. Cut and paste.

∈ A∈ V – A

(u, v) = least-weight edge connecting A to V – A

∈ A∈ V – A

(u, v) = least-weight edge connecting A to V – A

Consider the unique simple path from u to v in T.

∈ A∈ V – A

u(u, v) = least-weight edge connecting A to V – A

Consider the unique simple path from u to v in T. Swap (u, v) with the first edge on this path that connects a vertex in A to a vertex in V – A.

∈ A∈ V – A

u(u, v) = least-weight edge connecting A to V – A

vT ′:

Consider the unique simple path from u to v in T. Swap (u, v) with the first edge on this path that connects a vertex in A to a vertex in V – A.A lighter-weight spanning tree than T results.

Prim’s algorithmIDEA: Maintain V – A as a priority queue Q. Key each vertex in Q with the weight of the least-weight edge connecting it to a vertex in A.Q ← Vkey[v] ←∞ for all v ∈ Vkey[s] ← 0 for some arbitrary s ∈ Vwhile Q ≠ ∅

do u ← EXTRACT-MIN(Q)for each v ∈ Adj[u]

do if v ∈ Q and w(u, v) < key[v]then key[v] ← w(u, v) ⊳ DECREASE-KEY

π[v] ← u

At the end, {(v, π[v])} forms the MST.

Example of Prim’s algorithm

∈ A∈ V – A

∞∞

∞∞ ∞∞

∞∞ 00

∞∞

∈ A∈ V – A

∞∞

∞∞ ∞∞

∞∞ 00

∞∞

∈ A∈ V – A

∞∞

∞∞ 77

∞∞ 00

∞∞

∈ A∈ V – A

∞∞

∞∞ 77

∞∞ 00

∞∞

∈ A∈ V – A

∞∞ 00

∈ A∈ V – A

∞∞ 00

∈ A∈ V – A

1414 00

∈ A∈ V – A

1414 00

∈ A∈ V – A

1414 00

∈ A∈ V – A

Analysis of PrimQ ← Vkey[v] ←∞ for all v ∈ Vkey[s] ← 0 for some arbitrary s ∈ Vwhile Q ≠ ∅

do if v ∈ Q and w(u, v) < key[v]then key[v] ← w(u, v)

π[v] ← u

Θ(V)total

π[v] ← u

Θ(V)total

|V |times

π[v] ← u

degree(u)times

|V |times

Θ(V)total

Analysis of Prim

Handshaking Lemma ⇒Θ(E) implicit DECREASE-KEY’s.

Q ← Vkey[v] ←∞ for all v ∈ Vkey[s] ← 0 for some arbitrary s ∈ Vwhile Q ≠ ∅

π[v] ← u

degree(u)times

|V |times

Θ(V)total

Analysis of Prim

Handshaking Lemma ⇒Θ(E) implicit DECREASE-KEY’s.

Q ← Vkey[v] ←∞ for all v ∈ Vkey[s] ← 0 for some arbitrary s ∈ Vwhile Q ≠ ∅

π[v] ← u

degree(u)times

|V |times

Θ(V)total

Time = Θ(V)·TEXTRACT-MIN + Θ(E)·TDECREASE-KEY

Analysis of Prim (continued)

TotalQ T TDECREASE-KEYEXTRACT-MIN

array O(V) O(1) O(V2)

array O(V) O(1) O(V2)binary heap O(lg V) O(lg V) O(E lg V)

Fibonacci heap

O(lg V)amortized

O(1)amortized

O(E + V lg V)worst case

MST algorithms

Kruskal’s algorithm (see CLRS):• Uses the disjoint-set data structure (Lecture 10).• Running time = O(E lg V).

MST algorithms

Kruskal’s algorithm (see CLRS):• Uses the disjoint-set data structure (Lecture 10).• Running time = O(E lg V).

Best to date:• Karger, Klein, and Tarjan [1993].• Randomized algorithm.• O(V + E) expected time.

Introduction to Algorithms - MIT OpenCourseWare · Introduction to Algorithms 6.046J/18.401J...

Documents

Chương I: sắp xếp kiểu trộn, sắp xếp kiểu chèn · Web viewMIT OpenCourseWare 6.046J Introduction to Algorithms, Fall 2005 Please use the following citation format:

Design and Analysis of Algorithms - MIT OpenCourseWare · Design and Analysis of Algorithms 6.046J/18.401J L ECTURE 7 . Skip Lists • Data structure ... Design of two linked lists

Introduction to Algorithms 6.046J/18.401J/SMA5503

Algorithms: Greedy Algorithms

Introduction to Algorithms 6.046J Lecture 2 Prof. Shafi Goldwasser

Introduction to Algorithms...Introduction to Algorithms 6.046J/18.401J LECTURE 20 Network Flow I • Flow networks • Maximum-flow problem • Flow notation • Properties of flow

Distributed Algorithms 6.046J, Spring, 2015 Part 2 · • Round 2: – If you joined the MIS, ... 505 leaf leaf Asynchronous Distributed ... Distributed Algorithms, Chapter 8

Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 1 Prof. Charles E. Leiserson

Introduction to Algorithms - Duke Universityreif/courses/alg... · Introduction to Algorithms 6.046J/18.401J LECTURE 1 Analysis of Algorithms • sort • analysis • sort • Recurrences

Class on Design and Analysis of Algorithms, … and Analysis of Algorithms May 23, 2015 Massachusetts Institute of Technology 6.046J/18.410J Profs. Erik Demaine, Srini Devadas, and

Lecture Notes 10: Red-black trees - MIT OpenCourseWare · PDF fileIntroduction to Algorithms 6.046J/18.401J LECTURE 10 Balanced Search Trees • Red-black trees • Height of a red-black

Introduction to Algorithms 6.046J/18.401J L ECTURE 16 Greedy Algorithms (and Graphs) Graph representation Minimum spanning trees Optimal substructure Greedy

Introduction to Algorithms - UAB Barcelonamat.uab.cat/~alseda/MasterOpt/DemaineLecture.pdfIntroduction to Algorithms 6.046J/18.401J LECTURE 17 Shortest Paths I • Properties of shortest

Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 3 Prof. Erik Demaine

Introduction to Algorithms - courses.csail.mit.edu · Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 3 Prof. Piotr Indyk. February 11, 2003 (c) ... • Divide-and-conquer

Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson L2.1 Introduction to Algorithms 6.046J/18.401J LECTURE9 Randomly built binary

Introduction to Algorithms - Duke Universityreif/courses/alglectures/...Introduction to Algorithms 6.046J/18.401J LECTURE 21 Network Flow II • Max-flow, min-cut theorem • Ford-Fulkerson

Introduction to Algorithms - IIT-Computer Sciencewan/cs330/Shortest-Paths.pdf · Introduction to Algorithms 6.046J/18.401J Prof. Charles E. Leiserson LECTURE 14 Shortest Paths I •

Introduction to Algorithms 6.046J/18 - MIT OpenCourseWare · PDF fileProblem sets 12. Describing algorithms ... September 7, 2005 Introduction to Algorithms L1.4 ... Input: 8 2 4 9