Greedy Algorithms

Definition

• An approach to solving optimisation problems

• A sequence of steps involving choices that are– Feasible– Locally optimal– Irrevocable

• Hope that a sequence of locally optimal choices will give a global optimal solution

Greedy is good

• Fast to code

• Simple

• Usually easy to debug

Greedy is not always good

• Change-making problem

• Greedy works for some coin denominations, not all

• Take coin denominations of A = 1, B = 5, C = 7

• Suppose we wish to want to make up 17 with the least number of coins:– Greedy: C: 2, A: 3 – Optimal: C: 1, B: 2

Sample Problem: Barn Repair [1999 USACO Spring Open]

• There is a long list of stalls, some of which need to be covered with boards. You can use up to N (1 <= N <= 50) boards, each of which may cover any number of consecutive stalls. Cover all the necessary stalls, while covering as few total stalls as possible.

The solution

• Cover all the occupied stalls with a single board

• Now remove the largest unoccupied section possible, leaving two boards

• Do this for each board until the limit is reached

• Suppose n = 3

Stalls Example

Empty cell

Occupied cell

Stalls: why it works

• Suppose the optimal solution does not contain the gap removed, but does contain a smaller one

• We combine the two boards at each end of the smaller gap and split the board covering the larger gap in two

• Thus we create a solution using the same number of boards and covering fewer stalls

• We can always remove the largest gap

Time Scheduling

• You have a number of activities you want to perform in a certain period of time

• Specified by their starting and ending times

• But you can only do one activity at a time and some overlap

• Want to find a schedule that allows you to do the most number of activities

Time Scheduling 2

• An exhaustive search through all possibilities is infeasible

• There are 2^n possibilities (some, are of course invalid)

• With n = 50, no computer in the world would finish

Time Schedule 3

• There is a simple greedy solution

• Choose the activity that finishes first

• Then choose the next activity after that which finishes first

• And so on

Time Schedule 4

Why it works:If we choose the activity that finishes first, we

maximise the amount of time remaining after the activity is done

If we choose any other activity, the remaining time is shorter

There is no disadvantage in choosing the activity that finishes first since we can choose to do all the activities afterward as in the other case and possibly more

Time Schedule Pseudocode

N // number of activitiesActivity [] I // array of activitiesSort I in increasing order of finishing timeA = {0} //set of chosen activitiesL = 0For( i = 1; i < N; i++)

if(I[i].start > L.finish)A = A + {i}L = i

Time Schedule Example

Solution:• Debug • Chess • Starcraft • TopCoder• Shower• Eating and drinking

contest

Classic Examples of Greedy Algorithms

• Prim’s Algorithm

• Kruskal’s algorithm

• Dijkstra’s algorithm

Divide and Conquer

Outline

• Break a problem into smaller problems of the same type (non-overlapping, usually)

• Solve the smaller problems (easier)

• Use this to solve the bigger problem

Brainless Example

• Suppose we want to add the numbers:

• a1, …, an.

• This can be done as follows– add the first floor(n/2) numbers– add the last ceil(n/2) numbers– add the above two sums– If n =1, the number itself is the result

• The procedure is applied recursively

Pseudocode for example

Function add(int [] nums, start, end)if(start - end = 0)

return nums[start];else

return add(nums, start, (start+end)/2) +add(nums, (start+end)/2 +1, end);

Binary Search 1

• An efficient algorithm for searching for an element in a sorted list

• Suppose we have a list of integers and want to know if K is in the list– First see if the K is in the middle position, m– If so, stop– If list[m] < K we search the upper half of the

list with the same procedure– If list[m] > K we search the lower half

Binary Search 2

Recursive pseudocode:Function search(int [] list, start, end, K)

m = (start + end)/2if (list [m] = K) return true;if(list[m] > K)

return search(list, start, m-1, K);if(list[m] < K)

return search(list, m+1, end, K);

Binary Search 3

Non-recursive pseudocode:Function search(int [] list, K) start = 0, end = list.length-1 while(start <= end)

m = (start+end)/2 If (list[m] = K) return true; Else if(list[m] < K) start = m+1 Else if (list[m] > K) end = m-1

return false

Binary Search

3 14 27 31 39 42 55 70 74 81 85 93 98

3 14 27 31 39 42 55 70 74 81 85 93 98

3 14 27 31 39 42 55 70 74 81 85 93 98

Search for K = 70

0 1 2 3 4 5 6 7 8 9 10 11 12

Binary search

• Comparison to sequential search:– O(log

2 n) vs O(n)

– Expect N/2 comparisons for sequential search– With 1 million elements binary search will take

at most 20 comparisons

Living Dead Parrot (Keegan Carruthers-Smith)

• Pet Shop Owner with a lot of cages with parrots in them• The owner needs to find out which cages have parrots that are alive

but the door to the back of his store is locked. • The cages are in a single row. • Rope-and-pulley system is used to rattle the cages• A parrot will squawk if its cage is rattled• Pulleys allow ranges of cages to be rattled

• You must work out which cages have parrots that are alive in them. The owner can work out how many live parrots are in the range a to b by listening to the number of squawks. The owner’s time is limited, so he can only pull on the pulley up to M times. The cages are numbered from 1 to N. There are K live parrots.

Parrot Solution

• The idea is similar to a binary search

• First query the range [1,N] to find how many parrots are alive

• Now query [1,floor(N/2)] and you also have how many are alive in [floor(N/2)+1,N]

• Do this recursively until the range length is the same as the number of parrots alive

Parrots pseudocodealive_parrots = boolean array of size N

set each value in alive_parrots to falsefunction search(a, b, num_alive):

if (num_alive == 0)return

if (b - a + 1 == num_alive)set alive_parrots to true in range [a,b]c = (a+b)/2q = query(a,c)search(a, c, q)search(c+1, b, num_alive - q)

search(1, N, query(1,N))

Bisection Method for root finding

• We want to find a solution to the equation f(x) = 0 where f(x) is a continuous function

• Suppose we have two points and, a and b, such that f(a) < 0 and f(b) > 0 or vice versa

• Then f(x) = 0 for some x in (a,b)

Bisection Method 2

• We proceed as follows:

• Find the midpoint of the interval: c = (a+b)/2

• Now – either f(c) = 0 (unlikely)– f(a) and f(c) have opposite sign– f(c) and f(b) have opposite sign

• Suppose f(a) and f(c) are opposite, then the root is in the interval (a,c) and we can repeat the procedure until a sufficiently small interval has been found

Bisection Method 3

Other classic D&C Algorithms

• Merge sort

• Quicksort

References

• Introduction to the Design and Analysis of Algorithms. Levitin

• Wikipedia

• SACO server: http://olympiad.cs.uct.ac.za

• TopCoder: http://www.topcoder.com/tc?module=Static&d1=tutorials&d2=greedyAlg