CS 4407 Algorithms Greedy Algorithms

1

CS 4407

Algorithms

Greedy Algorithms

Prof. Gregory Provan

Department of Computer Science

University College Cork

CS 4407, Algorithms University College Cork,

Gregory M. Provan

Overview

Motivation

Abstract greedy algorithm

Examples

– Minimum spanning tree

– Scheduling

– Activity selection


Gregory M. Provan

3

Fundamental Techniques

There are some algorithmic tools that are

quite specialised. They are good for

problems they are intended to solve, but

they are not very versatile.

There are also more fundamental (general)

algorithmic tools that can be applied to a

wide variety of different data structure and

algorithm design problems.


Gregory M. Provan

Motivation

Greedy algorithms are efficient ways to solve

hard problems

– May not be optimal though

Objectives

– When is a greedy algorithm optimal?

– Can we prove bounded optimality?

• Approximation algorithms

– Can we show sub-classes for optimality?


Gregory M. Provan

5

The Greedy Method

An optimisation problem (OP) is a problem that involves searching through a set of configurations to find one that minimises or maximizes an objective function defined on these configurations

The greedy method solves a given OP going through a sequence of (feasible) choices

The sequence starts from well-understood starting configuration, and then iteratively makes the decision that seems best from all those that are currently possible.


Gregory M. Provan

6

The Greedy Method

The greedy approach does not always lead to an optimal solution.

The problems that have a greedy solution are said to posses the greedy-choice property.

The greedy approach is also used in the context of hard (difficult to solve) problems in order to generate an approximate solution.


Gregory M. Provan

Greedy Algorithms

A greedy algorithm always makes the choice that looks best at the moment.

It makes a local optimal choice in the hope that this choice will lead to a globally optimal solution.

Greedy algorithms yield optimal solutions for many (but not all) problems.

2020/2/6


Gregory M. Provan

The Greedy Method

The greedy method is a general algorithm design paradigm, built

on the following elements:

– configurations: different choices, collections, or values to find

– objective function: a score assigned to configurations, which we want to

either maximize or minimize

It works best when applied to problems with the greedy-choice

property:

– a globally-optimal solution can always be found by a series of local

improvements from a starting configuration.

– Never change a previous choice

8


Gregory M. Provan

Greedy Algorithms

When are greedy algorithms optimal?

Must satisfy the optimal substructure

property

– Optimal sub-problem must participate in optimal

problem


Gregory M. Provan

What makes a greedy algorithm?

Feasible – Has to satisfy the problem’s constraints

Locally Optimal – The greedy part

– Has to make the best local choice among all feasible choices available on that step

• If this local choice results in a global optimum then the problem has optimal substructure

Irrevocable – Once a choice is made it can’t be un-done on subsequent steps of the

algorithm

Simple examples: – Playing chess by making best move without lookahead

– Giving fewest number of coins as change

Simple and appealing, but don’t always give the best solution


Gregory M. Provan

Abstract Greedy Algorithm

Inputs: heuristic H, Loop Invariant LI, Cost

Function CF

Greedy (list)

– L Sort list by H, CF, R //R is result//

– Do while L

• R R head(L) if addition does not violate LI

• L L - head(L)

– End do

– Return R


Gregory M. Provan

Variants of Greedy Method

Greedy method sorts the objects in our task

Two main methods

– Global sort: sort at the beginning

– Repeated sort: sort for every iteration


Gregory M. Provan

Objectives

Define greedy algorithm

– Heuristic function

Determine optimality

Prove optimality (or sub-optimality)

Page 14


Gregory M. Provan

Proving Optimality

Sub-optimality is easy to show (if present)

– Provide counter-example

Optimality is more challenging

Techniques

– Greedy property always holds

– “greedy stays ahead”

– Exchange methods


Gregory M. Provan

Examples

Counting money

Scheduling

Minimum Spanning tree

Huffman encoding

Dijkstra’s algorithm (single-source shortest

paths)


Gregory M. Provan

Example: Counting money

Suppose you want to count out a certain amount of money, using the fewest possible bills and coins

A greedy algorithm would do this would be: At each step, take the largest possible bill or coin that does not overshoot – Example: To make $6.39, you can choose:

• a $5 bill

• a $1 bill, to make $6

• a 25¢ coin, to make $6.25

• A 10¢ coin, to make $6.35

• four 1¢ coins, to make $6.39

For US money, the greedy algorithm always gives the optimum solution


Gregory M. Provan

A failure of the greedy algorithm

In some (fictional) monetary system, “krons” come in 1 kron, 7 kron, and 10 kron coins

Using a greedy algorithm to count out 15 krons, you would get – A 10 kron piece

– Five 1 kron pieces, for a total of 15 krons

– This requires six coins

A better solution would be to use two 7 kron pieces and one 1 kron piece – This only requires three coins

The greedy algorithm results in a solution, but not in an optimal solution


Gregory M. Provan

Sample MST

6

5

4

2

9

15

14

10

38


Gregory M. Provan

Spanning Tree

Definition

– A spanning tree of a graph G is a tree (acyclic) that

connects all the vertices of G once

• i.e. the tree “spans” every vertex in G

– A Minimum Spanning Tree (MST) is a spanning tree on a

weighted graph that has the minimum total weight

w T w u vu v T

( ) ( , ),

such that w(T) is minimum


Gregory M. Provan

Sample MST

Which links to make this a MST?

6

5

4

2

9

15

14

10

38

Optimal substructure: A subtree of the MST must in turn be a

MST of the nodes that it spans.


Gregory M. Provan

MST Claim

Claim: Say that M is a MST

– If we remove any edge (u,v) from M then this

results in two trees, T1 and T2.

– T1 is a MST of its subgraph while T2 is a MST of

its subgraph.

– Then the MST of the entire graph is T1 + T2 + the

smallest edge between T1 and T2

– If some other edge was used, we wouldn’t have

the minimum spanning tree overall


Gregory M. Provan

Greedy Algorithm

We can use a greedy algorithm to find the

MST.

– Two common algorithms

• Kruskal

• Prim


Gregory M. Provan

Example: Kruskal’s MST Algorithm

Idea: Greedily construct the MST

– Go through the list of edges and make a forest

that is a MST

– At each vertex, sort the edges

– Edges with smallest weights examined and

possibly added to MST before edges with higher

weights

– Edges added must be “safe edges” that do not

ruin the tree property.


Gregory M. Provan

Scheduling Problem Statement

INPUT: A set S = { I1, I2, ···, In} of n event time-intervals Ik = sk , fk, k =1..n, where sk = start time of Ik , fk = finishing time of Ik , ( sk < fk ).

OUTPUT: A maximum cardinality subset C S of mutually compatible intervals

(i.e., no overlapping pairs).

Example: S = the intervals shown below, C = the blue intervals, C is not the unique optimum. |C| =4. Can you spot another optimum solution?

time


Gregory M. Provan

Task Scheduling

Given: a set T of n tasks, each having: – A start time, si

– A finish time, fi (where si < fi)

Goal: Perform all the tasks using a minimum number of “machines.”

27

1 9 8 7 6 5 4 3 2

Machine 1

Machine 3

Machine 2


Gregory M. Provan

Task Scheduling Algorithm

Greedy choice: consider tasks by their start time and use as few machines as possible with this order.

– Run time: O(n log n).

Correctness: Suppose there is a better schedule.

– We can use k-1 machines

– The algorithm uses k

– Let i be first task scheduled on machine k

– Task i must conflict with k-1 other tasks

– K mutually conflict tasks

– But that means there is no non-conflicting schedule using k-1 machines

28

Algorithm taskSchedule(T)

Input: set T of tasks w/ start time si and finish time fi

Output: non-conflicting schedule with minimum number of machines

m 0 {no. of machines}

while T is not empty

remove task i w/ smallest si

if there’s a machine j for i then

schedule i on machine j

else

m m + 1

schedule i on machine m


Gregory M. Provan

Example

Given: a set T of n tasks, each having: – A start time, si

– A finish time, fi (where si < fi)

– [1,4], [1,3], [2,5], [3,7], [4,7], [6,9], [7,8] (ordered by start)

Goal: Perform all tasks on min. number of machines

29

1 9 8 7 6 5 4 3 2

Machine 1

Machine 3

Machine 2

A banquet hall manager has received a list of reservation requests

for the exclusive use of her hall for specified time intervals

She wishes to grant the maximum number of reservation requests

that have no time overlap conflicts

Help her select the maximum number of conflict free time intervals

Event Scheduling Example


Gregory M. Provan

Activity-Selection Problem

Problem: get your money’s worth out of a

carnival

– Buy a wristband that lets you onto any ride

– Lots of rides, each starting and ending at

different times

– Your goal: ride as many rides as possible

• Another, alternative goal that we don’t solve here:

maximize time spent on rides

Welcome to the activity selection problem


Gregory M. Provan

Activity-Selection

Formally:

– Given a set S of n activities

si = start time of activity i

fi = finish time of activity i

– Find max-size subset A of compatible activities

Assume (wlog) that f1 f2 … fn

1 2

3 4

5

6


Gregory M. Provan

Activity Selection Problem

Problem: – Schedule an exclusive resource in competition with other

entities.

Formal Definition: – Set S={1,2, … n} of activities.

– Each activity has a start time si and a finish time fj, where si <fj.

– Activities i and j are compatible if they do not overlap.

The activity selection problem – select a maximum-size set of mutually compatible

activities.


Gregory M. Provan

Greedy Choice Property

Activity selection problem exhibits the

greedy choice property:

– Locally optimal choice globally optimal sol’n

Theorem: if S is an activity selection problem

sorted by finish time, then optimal solution

A S such that {1} A

– Sketch of proof: if optimal solution B that does

not contain {1}, can always replace first activity

in B with {1} (Why?). Same number of activities,

thus optimal.


Gregory M. Provan

Activity Selection: A Greedy Algorithm

So actual algorithm is simple:

– Sort the activities by finish time

– Schedule the first activity

– Then schedule the next activity in sorted list

which starts after previous activity finishes

– Repeat until no more activities

Intuition is even more simple:

– Always pick the shortest ride available at the

time


Gregory M. Provan

Greedy Activity Selection

Just march through each activity by finish

time and schedule it if possible:


Gregory M. Provan

Activity Selection Example

Schedule job 1, then try rest: (end up with 1, 4, 8):

T=1 T=2 T=3 T=4 T=5 T=6 T=7 T=8 T=9 T=10 T=11 T=12 T=13

1 1 1 1

Runtime?


Gregory M. Provan

Activity Selection: Optimal Substructure

Let k be the minimum activity in A (i.e., the

one with the earliest finish time). Then A - {k}

is an optimal solution to S’ = {i S: si fk}

– In words: once activity #1 is selected, the

problem reduces to finding an optimal solution

for activity-selection over activities in S

compatible with #1

– Proof: if we could find optimal solution B’ to S’

with |B| > |A - {k}|,

• Then B U {k} is compatible

• And |B U {k}| > |A|


Gregory M. Provan

Some Greedy Heuristics

Greedy iteration step: From among undecided intervals, select the interval I that looks BEST.

Commit to I if it’s conflict-free (i.e., doesn’t overlap with the committed ones so far).

Reject I otherwise.

Greedy 1: BEST = earliest start time (min sk).

Greedy 2: BEST = latest finishing time (max fk).

Greedy 3: BEST = shortest interval (min fk – sk).

Greedy 4: BEST = overlaps with fewest # of undecided intervals.

Documents

CS 4407 Algorithms Greedy Algorithms