Upload
mrzaggy
View
239
Download
0
Embed Size (px)
Citation preview
8/14/2019 Lecture8 - Greedy Algorithms
1/61
COMP2230 Introduction to
Algorithmics
Lecture 8
Slides based on:A. Levitin. The Design and Analysis of Algorithms
R. Johnsonbaugh & M. Schefer. Algorithms
Slides by P. Moscato and Y. Lin
8/14/2019 Lecture8 - Greedy Algorithms
2/61
Lecture Overview
Greedy Algorithms Text, Chapter 7
8/14/2019 Lecture8 - Greedy Algorithms
3/61
Change-Making Problem
We have an infinite supply of coins of differentdenominations d1> d2> . . . > dm.
We want to pay any given amount, using theminimumnumber of coins.
(This problem is faced every day by millions andmillions of cahiers . . .)
8/14/2019 Lecture8 - Greedy Algorithms
4/61
Change-Making Problem
For example, we have an infinite supply of each of thefollowing
1c, 2c, 5c, 10c, 20c, 50c,1,2
How would you pay for
1.05 2.40 30c 85c
http://en.wikipedia.org/wiki/2_euro_coinshttp://en.wikipedia.org/wiki/2_euro_coinshttp://en.wikipedia.org/wiki/2_euro_coinshttp://en.wikipedia.org/wiki/2_euro_coinshttp://en.wikipedia.org/wiki/2_euro_coinshttp://en.wikipedia.org/wiki/2_euro_coinshttp://en.wikipedia.org/wiki/2_euro_coinshttp://en.wikipedia.org/wiki/2_euro_coins8/14/2019 Lecture8 - Greedy Algorithms
5/61
Making change (cont)
This algorithm makes change for an amount Ausing coins of denominationsdenom[1] > denom[2] > > denom[n] = 1.
Input Parameters: denom, AOutput Parameters: None
greedy_coin_change(denom,A) {i= 1
while (A> 0) {
c= A/denom[i]
println(use+ c + coins of denomination+ denom[i])
A= A- c* denom[i]
i= i+ 1}
}
8/14/2019 Lecture8 - Greedy Algorithms
6/61
Making change (cont)
greedy_coin_change is a Greedy Algorithm keeps track of sum of coins paid so far checks whether there is still more to pay
If there is more to pay, algorithm adds coins of largest amountpossible (so as not to exceed amount needed)
It never changes its mind
Be greedy seems like a good idea! Could you see when greed is bad?
8/14/2019 Lecture8 - Greedy Algorithms
7/61
You should expect thisBad cases
Greed might end up with a solution which is not optimal (checkcoin denominations 1, 6 and 10 and use greedy for 22).
Greed might end up with a solution which is not feasible (if
the smallest coin denomination is larger than 1, solution mightnot always be feasible, e.g. coin denominations 3, 4 and 5 andamount 6).
8/14/2019 Lecture8 - Greedy Algorithms
8/61
General characteristicsof greedy algorithms
On each step, Greedy algorithms make a choice that is:
Feasible (satisfies the constraints)
Locally Optimal (the best choice among all feasible choices)
Irrevocable (cannot be changed latter)
8/14/2019 Lecture8 - Greedy Algorithms
9/61
General characteristicsof greedy algorithms
Greedy algorithms generally involve:
A computable selection functionused to prioritize (whichcandidate is most desirable?)
set of candidates to be considered for inclusion in solution a set of candidates which have already been chosen a set of candidates which have already been rejected
solution function (to check if our set isa solution) feasible function, to check if our set is feasible, i.e. can be
extended to make a solution For optimization problems an objective function
(what is our score ?)
8/14/2019 Lecture8 - Greedy Algorithms
10/61
Greed: the form
Make-change version: candidates
(large) set of coins
solution function check if coins chosen =
exact amount paid feasible set
total value of coins < totalamount to be paid
selection function choose highest valued coin
of remaining set objective function count # of coins
(C,S)
{C is the set of candidates}
S { will hold the solution}
C and not (S)
x (C)
C C \ {x}
(S {x})
S (S {x})
(S)
S
no solution
8/14/2019 Lecture8 - Greedy Algorithms
11/61
Greed: the general form
(C,S)
{C is the set of candidates}
S { will hold the solution}
C and not (S)
x (C)
C C \ {x}
(S {x})
S (S {x})
(S)
S
no solution
( )
const = {c0, c1, ....}
{ will hold the solution}
0 {sum of coins in S}
x largest item in Csuch that s + x val
no such item
No solution
S S {a coin of value x}
s s + x
8/14/2019 Lecture8 - Greedy Algorithms
12/61
Minimum Spanning Trees
MST-Problem
Input:G=(V,E,W) a connectedundirected graph with non-negative weights on edges
Output:a connected sub-graphG=(V,T) of G such that the sum
of the edge lengths of G isminimized.
10
5
8
7
16
2
1
8
8/14/2019 Lecture8 - Greedy Algorithms
13/61
MST (cont)
MST must be a tree. Why ?
Applications: minimum cost networks
cabling
transport 5
8
2
1
8/14/2019 Lecture8 - Greedy Algorithms
14/61
Greedy strategies for MST
Identify what are thecandidates
What would the startingpoint be?
How do we select the nextcandidate? what about rejection?
When are we finished?
edges? vertices?something else?
8/14/2019 Lecture8 - Greedy Algorithms
15/61
Greedy by edges
start with an empty edgeset T
select next shortest edge(not already seen)
reject if creates a cycle
stop when we haveselected n-1 edges
what does the selectedset represent?
Greedy strategies for MST
What are our candidates?
What would the startingpoint be?
How do we select the nextcandidate? what about rejection?
When are we finished?
8/14/2019 Lecture8 - Greedy Algorithms
16/61
Greedy by vertices
Start with a single node is there a special one we
should start at?
Select shortest edge Reject if it doesn't connect
a node in current set to anode we havent seen yet
Stop when we haveconnected all |V| nodes
What are our candidates?
What would the startingpoint be?
How do we select the nextcandidate? what about rejection?
When are we finished?
Greedy strategies for MST
K k l G d b d
8/14/2019 Lecture8 - Greedy Algorithms
17/61
Kruskal: Greedy by edgesExample
Connected Components:
start: {1} {2} {3} {4} {5} {6} {7} {8}
1 2
5643
4
8
64
37
1 2 3
4 5 6
7
8/14/2019 Lecture8 - Greedy Algorithms
18/61
Prims algorithm: Greedy byvertices - Example
starting set: VT= {1}
1 2
5643
4
8
64
37
1 2 3
4 5 6
7
8/14/2019 Lecture8 - Greedy Algorithms
19/61
The greedy template on MST
Candidates are edgesin G Set of edges is a solution if it is a tree which spans V Set of edges is feasible if doesnt contain a cycle Objective function:
minimize total length of edges in the solution
Terminology:
Promising set of edges: one which can be extended to form optimal
solution
Leaving edge: edge with exactly one end in particular set of vertices
8/14/2019 Lecture8 - Greedy Algorithms
20/61
Promising Edges
Promising Lemma BTW, lemmas are auxiliary statements used to form a basis for more
relevant statements (theorems) to be used later on; both lemmas andtheorems need to be proved otherwise they are just conjectures
Given: G = (V,E,W) as before Given VTV Given T E a promising set of edges, such that no edge
leaves VT Let e be the shortest edge leaving VT
Then T {e} is promising
8/14/2019 Lecture8 - Greedy Algorithms
21/61
Proof of Promising Lemma
The proof will be by contradiction we assume that what were
going to prove is false, and thenshow that thatassumption isfalse
assume T {e} is notpromising
We have a promising set (T)
and the edge e leaving VT
T
eVT
8/14/2019 Lecture8 - Greedy Algorithms
22/61
Proof (cont) T is promising, so there is a MST U containing
edges of T
T is the section of the MST U that weve alreadyfound consider U T {e} not promising, so U {e} must have a cycle
- why? because T {e} not promising and thus U must
have another edge uleaving VTon the path
between the end vertices of e AND umust be atleast as large as e, that is, w(u) w(e) (because eis the shortest edge leaving VT)
So, lets create a new tree T, by deleting u fromU and adding in e
T= U\{u}{e}
Since U was MST, so is T
Now T{e} T,
So T
{e} is promising - so we have our proof !
T
e
u
8/14/2019 Lecture8 - Greedy Algorithms
23/61
Kruskals Algorithm
Initially, set of edges T is empty
Each node of G = (V,T) is in itself a connected component
Each step: smallest not-yet-chosen candidate edge e is considered
If T{e}contains no cycle, that is, if e bridges two differentconnected components, then e is added to T
Otherwise, e is rejected (and never considered again!)
Each addition to T reduces the number of connected components by 1(why?)
Terminate when only one connected component remains.
8/14/2019 Lecture8 - Greedy Algorithms
24/61
Kruskals Algorithm - Pseudocode
Algorithm Kruskal(G)
//Input: A weighted connected graph G//Output: A minimum spanning tree of G, given by its set of edges ETSort E in non-decresing order of the edge weights
w(ei1) . . . w(ei|E|)
ET; //initializing the set of tree edgescounter 0; //initializing the number of tree edgesk 0; //initializing the number of processed edgeswhilecounter < |V|-1 do
k k + 1;
ifET
{eik
} is acyclic
ETET {eik};
counter counter + 1;
returnET
8/14/2019 Lecture8 - Greedy Algorithms
25/61
Kruskal - an example
Connected Components: start: {1} {2} {3} {4} {5} {6} {7} Shortest edge: (1,2); no cycle accept; connected components: {1,2} {3} {4} {5} {6} {7}
Shortest edge: (2,3); no cycle accept; connected components: {1,2,3} {4} {5} {6} {7} Shortest edge: (4,5); no cycle accept; connected components: {1,2,3} {4,5} {6} {7} Shortest edge: (6,7); no cycle accept; connected components: {1,2,3} {4,5} {6,7} Shortest edge: (1,4); no cycle accept; connected components: {1,2,3,4,5} {6,7} Shortest edge: (2,5); creates cycle reject Shortest edge: (4,7); no cycle accept; connected components:{1,2,3,4,5,6,7}
1 2
5643
4
8
64
37
1 2 3
4 5 6
7
8/14/2019 Lecture8 - Greedy Algorithms
26/61
Correctness of Kruskals Algorithm
Apply Promising Lemma and use induction
Empty set is promising
Assume by inductive hypothesis that set Tof kedgesselected by the Kruskals algorithm is promising. If ashortest edge edoes not create a cycle then byPromising Lemma T{e} is also promising.
8/14/2019 Lecture8 - Greedy Algorithms
27/61
Implementing Kruskals Algorithm
Kruskals algorithm can be efficiently implementedusing disjoint sets.
Remember operations on disjoint sets: makeset(x) find(x) tells us which component x is in
union(a,b) merges two components
8/14/2019 Lecture8 - Greedy Algorithms
28/61
Kruskal algorithm
Kruskals algorithm finds a minimum spanning tree in aconnected, weighted graph with vertex set {1, ... , n} .The input to the algorithm is edgelist, an array ofedge, and n. The members of edge are
vand w, the vertices on which the edge is incident. weight, the weight of the edge.
The output lists the edges in a minimum spanning
tree. The function sortsorts the array edgelistinnondecreasing order of weight.
8/14/2019 Lecture8 - Greedy Algorithms
29/61
Algorithm 7.2.4 Kruskals Algorithm
Input Parameters: edgelist, nOutput Parameters: Nonekruskal(edgelist,n) {
sort(edgelist)
fori= 1 to n
makeset(i)
count= 0
i= 1
while(count< n- 1) {
if(findset(edgelist[i].v) != findset(edgelist[i].w)) {println(edgelist[i].v+ + edgelist[i].w)
count= count+ 1
union(edgelist[i].v,edgelist[i].w)
}
i= i+ 1
}
}
8/14/2019 Lecture8 - Greedy Algorithms
30/61
Analysis of Kruskals AlgorithmAssume we have n nodes and |E| edges.
Worst case: we have to look at all edges and nodes.
Q(|E| log |E|) to sort edges
Q(n) to initialize disjoint node sets
O(|E|log n) for findset and union operations At most 2 |E| findset
n-1 unions
|E| n-1, so n-1 = O(|E|)
Each findset or union takes O(log n) log |E| is Q( log n)
the total is:
Q(|E| log n)
8/14/2019 Lecture8 - Greedy Algorithms
31/61
Prims Algorithm
Greedy by verteces Initially set VTconsists of a single vertex of V and T is empty
Each step: the smallest not-yet-chosen candidate edge e isconsidered
If e leaves VTthen e is added to T and the endpoint not in VTisadded to VT
otherwise e is rejected.
Terminate when VT=V (all nodes used up)
8/14/2019 Lecture8 - Greedy Algorithms
32/61
Prims algorithm Pseudocode
algorithm Prim(G)
//Input: A weighted connected graph G
//Output: A minimum spanning tree of G, given by its set of edges ETVT{v0}
ETfori = 1 to |V|-1
find a minimum weight edge e*=(v*,u*), among all
edges leaving VT (that is, v*VT, u* VT)
VTVT{u*}
ETET {e*}
returnET
8/14/2019 Lecture8 - Greedy Algorithms
33/61
Prims algorithm - example
starting set: VT= {1} {1,2}
{1,2,3} {1,2,3,4} {1,2,3,4,5} {1,2,3,4,5,7} {1,2,3,4,5,7,6}
1 2
5643
4
8
64
37
1 2 3
4 5 6
7
8/14/2019 Lecture8 - Greedy Algorithms
34/61
Is Prims Algorithm Correct?
Apply Promising Lemma anduse induction
Tree containing a singlevertex is promising
Assume by inductivehypothesis that T= (VT, ET)of kedges selected by thePrims algorithm is promising.
Let e*=(v*,u*), be a shortestedge leaving VT. Then byPromising Lemma T = (VT{u*}, ET{e*} ) is alsopromising.
e
v
u
8/14/2019 Lecture8 - Greedy Algorithms
35/61
Analysis of Prims algorithm
To improve complexity, for each vertex u not in the current treewe can maintain the nearest tree vertex and the weight of thecorresponding edge; we call this the label of vertex u
When we add a new vertex u* to the tree, we need to: add u* to VT
for each vertex u in V-VT which is connected to u* by a shorteredgethen its label, we need to update the label
Time Complexity of Prims algorithm will depend on the datastructures used
If we use adjacency matrix for the graph and an unorderedarray for the list of labels then time complexity is Q(n2) If we use adjacency lists for graph G and min-heap for list of
labels, then the time complexity is O(|E| log n)
8/14/2019 Lecture8 - Greedy Algorithms
36/61
Greedy Shortest Path
Whats difference in this toPrim/Kruskal Algorithms?
Is this just another MST?
What are our candidates?
What do we start with?When are we finished?
What do we return?
What does a general step in thealgorithm look like?
Set of C = candidate nodes (yetto be considered)
Set of S = nodes already chosen
S initially contains source
S maintained so that currentshortest paths are stored
At each step, select closestnode to S in C
Stop when all nodes in S
8/14/2019 Lecture8 - Greedy Algorithms
37/61
Dijkstras Algorithmalgorithm Dijakstra(G,s)
//Diajkstras algorithm for a single source (s) shortest paths
//Input: A weighted connected graph G = (V,E,W)with non-negative weights and a vertex s
//Output: The length dvof a shortest path from sto vfor eachvertex v
forevery vertex v in V do
dv; pvnull
Penqueue(Q,v,d) //initialise vertex priority in the priority queueds0; decrease(Q,v,dv)//updatepriority of s with dsVT
fori = 1 to |V|-1 do
u* Pdequeue(Q) //dequeuethe minimum priority element
VTVT{u*}
forevery vertex in V-VTthat is adjacent to u* do
ifdu*+ w(u*,u) < du
dudu*+ w(u*,u); puu*;decrease(Q,v,dv)
http://localhost/var/www/apps/conversion/tmp/scratch_3//updatehttp://localhost/var/www/apps/conversion/tmp/scratch_3//dequeuehttp://localhost/var/www/apps/conversion/tmp/scratch_3//dequeuehttp://localhost/var/www/apps/conversion/tmp/scratch_3//update8/14/2019 Lecture8 - Greedy Algorithms
38/61
Dijkstra Example
6
5
1 2
4 3
25
10
50
200
20
30
20
5
12
1
0
4
8/14/2019 Lecture8 - Greedy Algorithms
39/61
6
5
1 2
4 3
25 50
200
20
30
20
5
1
0
1 node 1 PQ = [50,200,10,,25]
4
node 4 PQ= [40,30,-x-,,22]
6
node 6 PQ= [40,30,-x-,22,-x-]
5
node 5 PQ= [40,30,-x-,-x-,-x-]
3
node 3 PQ= [35,-x-,-x-,-x-,-x-]
stop
10
12
8/14/2019 Lecture8 - Greedy Algorithms
40/61
Is it correct?
Terminology: A path from source to v is
specialif all intermediate nodesbelong to S
We use an array D to storelength of shortest specialpaths
We need to prove 2 things: If node i 1 is in S, then D[i]
gives length of shortest pathfrom 1 to i
If node i is not in S, then D[i]gives shortest specialpath from1 to i
How could we prove this? consider the proof techniques
weve seen so far, which style ismore appropriate?
Why is this a good thing to prove? Why might this proof be more
straight forward than provingcorrectness after its all finished?
8/14/2019 Lecture8 - Greedy Algorithms
41/61
Proof
(1) If node i 1 is in S, then D[i]gives length of shortest path from1 to i
(2) If node i is not in S, then D[i]
gives shortest specialpath from 1to i
Basis?Inductive assumption?
assume both conditions are truejustbeforewe add node v
(1) Prove path to v passes onlythroughnodes in S.
How? What other options are there?
(2) Consider node w v which is not in S How do we know the special path to
w is still shortest if v is included in it? if v is not included?
8/14/2019 Lecture8 - Greedy Algorithms
42/61
8/14/2019 Lecture8 - Greedy Algorithms
43/61
Prims algorithm Pseudocode
algorithm Prim(G)//Input: A weighted connected graph G//Output: A minimum spanning tree of G, given by its
set of edges ETVT{v0}
ET
fori = 1 to |V|-1
find a minimum weight edge e*=(v*,u*), among all edges leaving VT(that is, v*V
T, u* V
T)
VTVT{u*}
ETET {e*}
returnET
8/14/2019 Lecture8 - Greedy Algorithms
44/61
To improve complexity, for each vertex u not in thecurrent tree we can maintain the nearest tree vertex
and the weight of the corresponding edge; we callthis the label of vertex u.
When we add a new vertex u* to the tree, we need to: add u* to VT
for each vertex u in V-VT which is connected to u*by a shorter edgethen its label, we need to updatethe label
8/14/2019 Lecture8 - Greedy Algorithms
45/61
Example
Tree vertices:a(-,-)
Fringe vertices:b(a,3), e(a,6), f(a,5)
Unseen vertices:
c(-,), d(-, )
Nearest vertex:b(a,3)
a f d
b
e
c1
3 4 4 6
55
62
8
8/14/2019 Lecture8 - Greedy Algorithms
46/61
Example
Tree vertices:a(-,-), b(a,3)
Fringe vertices:c(b,1),e(a,6), f(b,4)
Unseen vertices:
d(-, )
Nearest vertex:c(b,1)
a f d
b
e
c1
3 4 4 6
55
62
8
8/14/2019 Lecture8 - Greedy Algorithms
47/61
Example
Tree vertices:a(-,-), b(a,3), c(b,1)
Fringe vertices:e(a,6), f(b,4), d(c, 6)
Unseen vertices:
Nearest vertex:f(b,4)
a f d
b
e
c1
3 4 4 6
55
62
8
8/14/2019 Lecture8 - Greedy Algorithms
48/61
Example
Tree vertices:a(-,-), b(a,3), c(b,1), f(b,4)
Fringe vertices:e(f,2), d(f, 5)
Unseen vertices:
Nearest vertex:e(f,2)
a f d
b
e
c1
3 4 4 6
55
62
8
8/14/2019 Lecture8 - Greedy Algorithms
49/61
Example
Tree vertices:a(-,-), b(a,3), c(b,1), f(b,4),
e(f,2)
Fringe vertices:d(f, 5)
Unseen vertices:
Nearest vertex:d(f, 5)
a f d
b
e
c1
3 4 4 6
55
62
8
8/14/2019 Lecture8 - Greedy Algorithms
50/61
Example
Tree vertices:a(-,-), b(a,3), c(b,1), f(b,4),
e(f,2), d(f, 5)
Fringe vertices:
Unseen vertices:
Nearest vertex:
a f d
b
e
c1
3 4 4 6
55
62
8
8/14/2019 Lecture8 - Greedy Algorithms
51/61
Analysis of Prims algorithm
Time Complexity of Prims algorithm will depend onthe data structures used; note that the list of labelson fringe vertices is a priority queue
If we use adjacency matrix for the graph and anunordered array for the list of labels then timecomplexity is Q(n2): To create a list of labels Q(n)
At each step finding a minimum in the list and updating thelist takes Q(n)
There are n-1 steps, thus Q(n2) in total
8/14/2019 Lecture8 - Greedy Algorithms
52/61
8/14/2019 Lecture8 - Greedy Algorithms
53/61
Dijkstras algorithm is very similar to Prims. Toimprove complexity, for each vertex u not in thecurrent tree we can maintain the parent tree vertexand the distance from the source; we call this thelabel of vertex u.
When we add a new vertex u* to the tree, we need to: add u* to VT for each vertex u in V-VT which is connected to u*
by an edgeof the weight w(u*,u) such that du*+w(u*,u) < duthen we need to update the label of uby du*+ w(u*,u) and set parent to u*
Dijkstras Algorithm for computing a single-source shortest paths
8/14/2019 Lecture8 - Greedy Algorithms
54/61
8/14/2019 Lecture8 - Greedy Algorithms
55/61
8/14/2019 Lecture8 - Greedy Algorithms
56/61
Example
Tree vertices:a(-,0)
Fringe vertices:b(a,3), e(a,6), f(a,5)
Unseen vertices:
c(-,), d(-, )
Nearest vertex:b(a,3)
a f d
b
e
c1
3 4 4 6
55
62
8
8/14/2019 Lecture8 - Greedy Algorithms
57/61
Example
Tree vertices:a(-,0), b(a,3)
Fringe vertices:e(a,6), f(a,5), c(b,1+3)
Unseen vertices:
d(-, )
Nearest vertex:c(b,1+3)
a f d
b
e
c1
3 4 4 6
55
62
8
8/14/2019 Lecture8 - Greedy Algorithms
58/61
Example
Tree vertices:a(-,0), b(a,3), c(b,4)
Fringe vertices:e(a,6), f(a,5), d(c,6+4)
Unseen vertices:
Nearest vertex:f(a,5)
a f d
b
e
c1
3 4 4 6
55
62
8
8/14/2019 Lecture8 - Greedy Algorithms
59/61
Example
Tree vertices:a(-,0), b(a,3), c(b,4), f(a,5)
Fringe vertices:e(a,6), d(c,6+4)
Unseen vertices:
Nearest vertex:e(a,6)
a f d
b
e
c1
3 4 4 6
55
62
8
8/14/2019 Lecture8 - Greedy Algorithms
60/61
Example
Tree vertices:a(-,0), b(a,3), c(b,4), f(a,5), e(a,6)
Fringe vertices:
d(c,6+4)
Unseen vertices:
Nearest vertex:d(c,10)
a f d
b
e
c1
3 4 4 6
55
62
8
8/14/2019 Lecture8 - Greedy Algorithms
61/61
Example
Tree vertices:a(-,0), b(a,3), c(b,4),
f(a,5), e(a,6), d(c,10)
Fringe vertices:
Unseen vertices:
Nearest vertex:
a f d
b
e
c1
3 4 4 6
55
62
8