Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Chapter 2

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1

Chapter 2 Chapter 2

Fundamentals of the Analysis Fundamentals of the Analysis of Algorithm Efficiencyof Algorithm Efficiency

Copyright © 2007 Pearson Addison-Wesley. All rights reserved.


Towers of Hanoi (from Wikipedia)Towers of Hanoi (from Wikipedia)

If n>1, then somewhere along the sequence of moves, the If n>1, then somewhere along the sequence of moves, the largest disk must be moved from peg f to another peg, largest disk must be moved from peg f to another peg, preferably to peg t. preferably to peg t.

The only situation that allows this move is when all smaller The only situation that allows this move is when all smaller n-1 disks are on peg r. n-1 disks are on peg r. • First all h-1 smaller disks must go from f to r. First all h-1 smaller disks must go from f to r.

• Move the largest disk Move the largest disk

• Finally move the h-1 smaller disks from peg r to peg t. Finally move the h-1 smaller disks from peg r to peg t.

Now the problem is reduced to moving h-1 disks from one Now the problem is reduced to moving h-1 disks from one peg to another one, first from f to r and subsequently from peg to another one, first from f to r and subsequently from r to t r to t • The same method can be used both times by renaming the pegs. The same method can be used both times by renaming the pegs.

• The same strategy can be used to reduce the h-1 problem to h-2, h-The same strategy can be used to reduce the h-1 problem to h-2, h-3, and so on until only one disk is left. This is called 3, and so on until only one disk is left. This is called recursion..

1-2


Towers of HanoiTowers of Hanoi

The following is a procedure for moving a tower of h disks The following is a procedure for moving a tower of h disks from a peg f onto a peg t, with r being the remaining third from a peg f onto a peg t, with r being the remaining third peg:peg:• Step 1: If h>1 then first move the h-1 smaller disks from peg f to Step 1: If h>1 then first move the h-1 smaller disks from peg f to

peg r.peg r.

• Step 2: Now the largest disk, i.e. disk h-1 can be moved from peg f Step 2: Now the largest disk, i.e. disk h-1 can be moved from peg f to peg t.to peg t.

• Step 3: If h>1 then again use this procedure to move the h-1 smaller Step 3: If h>1 then again use this procedure to move the h-1 smaller disks from peg r to peg t.disks from peg r to peg t.

1-3


Example 2: The Tower of Hanoi PuzzleExample 2: The Tower of Hanoi Puzzle

http://en.wikipedia.org/wiki/Tower_of_Hanoi http://www.cut-the-knot.org/recurrence/hanoi.shtml Recursive Solution (Move n>1 disks from P1 to P3

• Move recursively n-1 disks from P1 to P2 (P3 is Aux)

• Then move the largest disk from P1 to P3

• Move recursively n-1 disks from P2 to P3 (P1 is Aux)

Recursive Algorithm Analysis• M(n) = M(n-1) + 1 + M(n-1) for n>1 , M(1) = 1

• M(n) = 2M(n-1)+1, M(1) = 1

1-4


Example 2: The Tower of Hanoi PuzzleExample 2: The Tower of Hanoi Puzzle

1

2

3

Recurrence for number of moves:Recurrence for number of moves:


Solving recurrence for number of movesSolving recurrence for number of moves

M(M(nn) = 2M() = 2M(nn-1) + 1, M(1) = 1-1) + 1, M(1) = 1

Use Backward SubstitutionUse Backward Substitution

M(n) = 2M(n-1)+1M(n) = 2M(n-1)+1

=2[2M(n-2)+1]+1 = 2=2[2M(n-2)+1]+1 = 222M(n-2)+2+1M(n-2)+2+1

=2=222[2M(n-3)+1]+2+1 [2M(n-3)+1]+2+1 = 2= 233M(n-3)+2M(n-3)+222+2+1+2+1

PatternPattern

M(n) = 2M(n) = 2iiM(n-i)+2M(n-i)+2i-1i-1+2+2i-2i-2+…+2+1+…+2+1

= 2= 2iiM(n-i)+2M(n-i)+2ii-1-1

Initial condition: n=1 or i = n-1 Initial condition: n=1 or i = n-1

= 2= 2nn-1 -1 exponential algorithm : still optimal exponential algorithm : still optimal


Tree of calls for the Tower of Hanoi PuzzleTree of calls for the Tower of Hanoi Puzzle

n

n-1 n-1

n-2 n-2 n-2 n-2

1 1

... ... ...2

1 1

2

1 1

2

1 1

2


Example 3: Counting #bitsExample 3: Counting #bits

Basic operation: additions Recurrence relation:

A(n) = A(floor(n/2)) + 1 for n>1, A(1) = 0

Problem: floor(n/2) makes backward substitution not work when n is not a power of 2Estimate: for only cases where n is a power of 2


Slightly Updated RecurrenceSlightly Updated Recurrence

A(2A(2kk) = A(2) = A(2k-1k-1) + 1 for k>0, A(2) + 1 for k>0, A(200) = 0) = 0

Use backward substitutionUse backward substitutionA(2A(2kk) = A(2) = A(2k-1k-1) + 1) + 1

= [A(2= [A(2k-2k-2) + 1]+1 = A(2) + 1]+1 = A(2k-2k-2) + 2) + 2

= [A(2= [A(2k-3k-3) + 1]+2 = A(2) + 1]+2 = A(2k-3k-3) + 3) + 3

……

= A(2= A(2k-ik-i) + i) + i

……

= A(2= A(2k-kk-k) + k) + k

Thus,Thus,

A(2A(2kk) = A(1) + k = k) = A(1) + k = k

n=2n=2kk and hence k = log and hence k = log22nn A(n) = log2n = A(n) = log2n = ΘΘ(log n)(log n) 1-9

Chapter 3Chapter 3

Brute ForceBrute Force

Copyright © 2007 Pearson Addison-Wesley. All rights reserved.


Brute ForceBrute Force

A straightforward approach, usually based directly on the A straightforward approach, usually based directly on the problem’s statement and definitions of the concepts involvedproblem’s statement and definitions of the concepts involved

Just do it, and don’t be fancyJust do it, and don’t be fancy

Examples:Examples: Computing Computing aan n ((a a > 0, > 0, nn a nonnegative integer) a nonnegative integer)

Computing Computing nn!!

Multiplying two matricesMultiplying two matrices

Searching for a key of a given value in a listSearching for a key of a given value in a list


So, When and Why Brute ForceSo, When and Why Brute Force

Why not brute force?Why not brute force?• You’ll never get an elegant algorithmYou’ll never get an elegant algorithm

• You’ll never give yourself a chance to see the hidden patterns You’ll never give yourself a chance to see the hidden patterns necessary for getting an optimal solutionnecessary for getting an optimal solution

• For problems where solutions that are better than all pairs exist, For problems where solutions that are better than all pairs exist, brute force will not be optimalbrute force will not be optimal

Why brute force?Why brute force?• Exceedingly easy to conceptualizeExceedingly easy to conceptualize

• The method can be applied to a large range of problems (not true The method can be applied to a large range of problems (not true for a lot of the other strategies we’ll see)for a lot of the other strategies we’ll see)

• Works well for small values (input sizes) and its easy to implementWorks well for small values (input sizes) and its easy to implement

• Expense of a more advanced algorithm may not be justifiedExpense of a more advanced algorithm may not be justified

– Only a few difficult instances to solveOnly a few difficult instances to solve• Makes a good yard stickMakes a good yard stick

– i.e. We can do at least this well very easilyi.e. We can do at least this well very easily


Brute-Force Sorting AlgorithmBrute-Force Sorting Algorithm

Selection SortSelection Sort Scan the array to find its smallest element and Scan the array to find its smallest element and swap it with the first element. Then, starting with the second swap it with the first element. Then, starting with the second element, scan the elements to the right of it to find the element, scan the elements to the right of it to find the smallest among them and swap it with the second elements. smallest among them and swap it with the second elements. Generally, on pass Generally, on pass i i (0 (0 i i n-n-2), find the smallest element in 2), find the smallest element in AA[[i..n-i..n-1] and swap it with 1] and swap it with AA[[ii]:]:

AA[0] [0] . . . . . . AA[[ii-1] | -1] | AA[[ii], . . . , ], . . . , AA[[minmin], . . ., ], . . ., AA[[nn--1] 1]

in their final positionsin their final positions

Example: 7 3 2 5Example: 7 3 2 5

Example: 89 45 68 90 29 34 17Example: 89 45 68 90 29 34 17


Analysis of Selection SortAnalysis of Selection Sort


AnalysisAnalysis

Time efficiency:Time efficiency:• CC (n) = (n) = i=0 to n-2i=0 to n-2j=i+1 to n-1j=i+1 to n-111

= = i=0 to n-2i=0 to n-2[(n-1)-(i+1)+1][(n-1)-(i+1)+1]

= = i=0 to n-2i=0 to n-2(n-1-i)(n-1-i)

= (n-1) + (n-2) + (n-3) + … + 1= (n-1) + (n-2) + (n-3) + … + 1

by (S2)by (S2)

= ((n-1)n)/2 = (n= ((n-1)n)/2 = (n22 – n) / 2 – n) / 2

approx. = napprox. = n22/2 /2 O(n O(n22))

Space efficiency:Space efficiency:

Stability:Stability:

Note: # swaps is only Θ(n), or exactly n-1 -Is this good or bad?


Brute-Force Sorting Algorithm #2Brute-Force Sorting Algorithm #2

Bubble SortBubble Sort Scan the array looking at each consecutive Scan the array looking at each consecutive pair of elements. Swap the two elements if they are out of pair of elements. Swap the two elements if they are out of order. Repeatedly scan the array in an all-pairs manner.order. Repeatedly scan the array in an all-pairs manner.

See pg 100 for the pseudocodeSee pg 100 for the pseudocode

Example:Example:• 7 3 2 57 3 2 5

• 89 45 68 90 29 34 1789 45 68 90 29 34 17

Note: we can do better by stopping if no swaps are neededNote: we can do better by stopping if no swaps are needed• i.e. – the list is sortedi.e. – the list is sorted


AnalysisAnalysis

Time efficiency:Time efficiency:• CC (n) = (n) = i=0 to n-2i=0 to n-2j=0 to n-2-ij=0 to n-2-i11

= = i=0 to n-2i=0 to n-2[(n-2-i)-0+1][(n-2-i)-0+1]

= = i=0 to n-2i=0 to n-2(n-1-i)(n-1-i)

= (n-1) + (n-2) + (n-3) + … + 1= (n-1) + (n-2) + (n-3) + … + 1

by (S2)by (S2)

= ((n-1)n)/2 = (n= ((n-1)n)/2 = (n22 – n) / 2 – n) / 2

approx. = napprox. = n22/2 /2 O(n O(n22))

Space efficiency:Space efficiency:

Stability:Stability:

Note: How many swaps must take place in the worst case?

That is, look at the code for Selection Sort and Bubble Sort and note where the swap commands are.


Improving Brute ForceImproving Brute Force

With brute force, its often possible to do small things that With brute force, its often possible to do small things that decrease the run-timedecrease the run-time• These don’t generally change the asymptotic boundsThese don’t generally change the asymptotic bounds

Example: Sequential SearchExample: Sequential Search• Make sure that you have the value you are looking for appended to Make sure that you have the value you are looking for appended to

the end of the listthe end of the list

– That way, the item will have to be found and we That way, the item will have to be found and we won’t have to constantly check for the end of the listwon’t have to constantly check for the end of the list

• Obviously, we could first sort our listObviously, we could first sort our list


Brute-Force String MatchingBrute-Force String Matching

patternpattern: a string of : a string of mm characters to search for characters to search for texttext: a (longer) string of : a (longer) string of nn characters to search in characters to search in problem: find a substring in the text that matches the patternproblem: find a substring in the text that matches the pattern

Brute-force algorithmBrute-force algorithm

Step 1 Align pattern at beginning of textStep 1 Align pattern at beginning of text

Step 2 Moving from left to right, compare each character ofStep 2 Moving from left to right, compare each character of pattern to the corresponding character in text until pattern to the corresponding character in text until

– all characters are found to match (successful search); orall characters are found to match (successful search); or

– a mismatch is detecteda mismatch is detected

Step 3 While pattern is not found and the text is not yetStep 3 While pattern is not found and the text is not yet exhausted, realign pattern one position to the right and exhausted, realign pattern one position to the right and repeat Step 2 repeat Step 2


Examples of Brute-Force String MatchingExamples of Brute-Force String Matching

• Pattern: Pattern: 001011 001011 Text: Text: 10010101101001100101111010 10010101101001100101111010

• Pattern: Pattern: happyhappy

Text: Text: It is never too late to have a It is never too late to have a happy childhood.happy childhood.

b Pattern: Pattern: happyhappy Text: happhapphapphappy Text: happhapphapphappy

b Pattern: aaabPattern: aaab

Text: aaaaaaaaaaaaaaaaaaaaaaaaaaaabText: aaaaaaaaaaaaaaaaaaaaaaaaaaaab


Pseudocode and Efficiency Pseudocode and Efficiency

Efficiency: Worst Case?Efficiency: Worst Case?

Expected Average?Expected Average?


Brute-Force Polynomial EvaluationBrute-Force Polynomial Evaluation

Problem: Find the value of polynomialProblem: Find the value of polynomial

pp((xx) = ) = aannxxnn + + aann-1-1xxnn-1 -1 +… ++… + a a11xx1 1 + + aa0 0

at a point at a point xx = = xx00

Brute-force algorithmBrute-force algorithm

Efficiency:Efficiency:

pp 0.00.0forfor ii nn downtodownto 0 0 dodo powerpower 1 1

forfor jj 1 1 toto ii dodo //compute //compute xxii powerpower powerpower xx pp pp + + aa[[ii] ] powerpower

returnreturn pp


Polynomial Evaluation: ImprovementPolynomial Evaluation: Improvement

We can do better by evaluating from right to left:We can do better by evaluating from right to left:

pp((xx) = ) = aannxxnn + + aann-1-1xxnn-1 -1 +… ++… + a a11xx1 1 + + aa0 0

at a point at a point xx = = xx00

Better brute-force algorithmBetter brute-force algorithm

Efficiency:Efficiency:

pp aa[0][0]powerpower 1 1forfor ii 1 1 toto nn dodo

powerpower powerpower x x pp p p + + aa[[ii] ] powerpowerreturnreturn pp

Documents

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Chapter 2