Greedy Algorithms
• Like dynamic programming algorithms, greedy algorithms are usually designed to solve optimization problems
• Unlike dynamic programming algorithms, – greedy algorithms are iterative in nature.– An optimal solution is reached from local
optimal solutions.– This approach does not work all the time.– A proof that the algorithm does what it claims
is needed, and usually not easy to get.
Fractional Knapsack Problem
• Given n items of sizes s1, s2, …, sn and values v1, v2, …, vn and size C, the problem is to find x1, x2, …, xn that maximize
subject to
n
iiivx
1
n
iii Csx
1
Solution to Fractional Knapsack Problem
• Consider yi = vi / si
• What is yi?
• What is the solution?
Shortest Paths Problems
Input: A graph with non-negative weights or costs associated with each edge.
Output: The list of edges forming the shortest path.
Sample problems:– Find shortest path between two named vertices– Find shortest path from S to all other vertices– Find shortest path between all pairs of vertices
Will actually calculate only distances, not paths.
Shortest Paths Definitions
(A, B) is the shortest distance from vertex A to B.
length(A, B) is the weight of the edge connecting A to B.– If there is no such edge, then length(A, B) = .
A
D
C
B10
8
1
7
5
Single-Source Shortest Paths
• Problem: Given G=(V,E), start vertex s, find the shortest path from s to all other vertices.– Assume V={1, 2, …, n} and s=1
• Solution: A greedy algorithm called Dijkstra’s Algorithm
Dijkstra’s Algorithm Outline
• Partition V into two sets: X={1} and Y= {2, 3, …, n}
• Initialize [i] for 1 i n as follows: …• Select yY such that [y] is minimum
[y] is the length of the shortest path from 1 to y that uses only vertices in set X.
• Remove y from Y, add it to X, and update [w] for each wY where (y,w) E if the path through y is shorter.
Dijkstra’s AlgorithmInput: A weighted directed graph G = (V,E), where V = {1,2,…,n}Output: The distance from vertex 1 to every other vertex in G.1. X = {1}; Y = {2,3,…,n}; [1]=0;2. for y = 2 to n do3. if y is adjacent to 1 then [y]=length[1,y];4. else [y]= end if5. end for6. for j = 1 to n – 1 do7. Let y Y be such that [y] is minimum;8. X = X {y} // add vertex y to X9. Y = Y - {y} // delete vertex y from Y10. for each edge (y,w) do11. if w Y and [y] + length[y,w] < [w] then12. [w] = [y] + length[y,w] 13. end for14. end for
Correctness of Dijkstra’s Algorithm
• Lemma: In Dijkstra’s algorithm, when a vertex y is chosen in Step 7, if its label [y] is finite then [y] = [y].
Proof:
Time Complexity
• Mainly depends on how we implement step 7, i.e., finding y s.t. [y] is minimum.
• Approach 1: Scan through the vector representing current distances of vertices in Y:
• Approach 2: Use a min-heap to maintain vertices in the set Y:
Minimum Cost Spanning Trees
• Minimum Cost Spanning Tree (MST) Problem:
Input: An undirected weighted connected graph G.Output: The subgraph of G that 1) has minimum total cost as measured by summing
the weights of all the edges in the subset, and 2) keeps the vertices connected.
– What does such subgraph look like?
Kruskal’s Algorithm
Initially, each vertex is in its own MST.
Merge two MST’s that have the shortest edge between them.– Use a priority queue to order the unprocessed
edges. Grab next one at each step.
How to tell if an edge connects two vertices that are already in the same MST?– Use the UNION/FIND algorithm with parent-
pointer representation.
Kruskal’s MST Algorithm
Sort the edges of E(G) by weight in non-decreasing order;
For each vertex v V(G) do New_Tree(v); // creating tree with one root node v
T=; // MST initialized to empty
While |T| < n - 1 do Let (u,v) be the next edge in E(G) if FIND(u) FIND(v) then T = T (u,v); UNION(u,v);
Return T;
Correctness of Kruskal’s Algorithm
• Lemma: Algorithm Kruskal correctly finds a minimum cost spanning tree in a weighted undirected graph.
Proof:
• Theorem: Algorithm Kruskal’s finds a minimum cost spanning tree in a weighted undirected graph in O(m log m) time, where m is the number of edges in G.
Prim’s Algorithm
• Very similar to Dijkstra’s algorithm
• Grows a minimum spanning tree from an arbitrary vertex u V
Prim’s Algorithm
1. T = ; X={1}; Y = V(G) – {1};2. while Y do3. Let (x,y) be of minimum weight
such that x X and y Y.
4. X = X {y};5. Y = Y – {y};
6. T = T {(x,y)};7. end while;
Prim’s MST Implementation
• As with Dijkstra’s algorithm, the key issue is
determining which vertex is next closest.• As with Dijkstra’s algorithm, the alternative is to
use a priority queue (min-heap).• Running times for the two implementations are
identical to the corresponding Dijkstra’s algorithm implementations.
• For implementation details of the corresponding algorithms, check the book pages 245 and 247.