RAIK 283: Data Structures & Algorithms

1

Recursive Algorithm Analysis

Dr. Ying [email protected]

RAIK 283: Data Structures & RAIK 283: Data Structures & AlgorithmsAlgorithms

September 13, 2012

2

Giving credit where credit is due:Giving credit where credit is due:• Most of the lecture notes are based on the slides Most of the lecture notes are based on the slides

from the Textbook’s companion websitefrom the Textbook’s companion websitehttp://www.aw-bc.com/info/levitinhttp://www.aw-bc.com/info/levitin

• Several slides are from Hsu Wen Jing of the Several slides are from Hsu Wen Jing of the National University of SingaporeNational University of Singapore

• I have modified them and added new slidesI have modified them and added new slides

RAIK 283: Data Structures & RAIK 283: Data Structures & AlgorithmsAlgorithms

3

Example: a recursive algorithmExample: a recursive algorithm

Algorithm:Algorithm:if if nn=0 then =0 then FF((nn) := 1) := 1

else else FF((nn) := ) := FF((nn-1) * -1) * nnreturn return FF((nn) )

4

Algorithm F(n)

// Compute the nth Fibonacci number recursively

//Input: A nonnegative integer n

//Output: the nth Fibonacci number

if n 1 return n

else return F(n-1) + F(n-2)

Example: another recursive Example: another recursive algorithmalgorithm

5

Recurrence RelationRecurrence Relation Recurrence RelationRecurrence Relation

6

Recurrence RelationRecurrence Relation Recurrence Relation:Recurrence Relation: an equation or inequality that an equation or inequality that

describes a function in terms of its value on smaller describes a function in terms of its value on smaller inputsinputs

7

Example: a recursive algorithmExample: a recursive algorithm Algorithm:Algorithm:

if if nn=0 then =0 then FF((nn) := 1) := 1else else FF((nn) := ) := FF((nn-1) * -1) * nn

return return FF((nn) )

What does this algorithm compute?What does this algorithm compute?

8

Example: a recursive algorithmExample: a recursive algorithm Algorithm:Algorithm:



What does this algorithm compute?What does this algorithm compute?

What’s the basic operation of this algorithm?What’s the basic operation of this algorithm?

9

Example: recursive evaluation Example: recursive evaluation of of nn ! ! Recursive definition of Recursive definition of nn!:!: Algorithm:Algorithm:



M(n): number of multiplications to compute M(n): number of multiplications to compute nn! ! with this recursive algorithmwith this recursive algorithm

10




M(n): number of multiplications to compute M(n): number of multiplications to compute nn! ! with this recursive algorithmwith this recursive algorithm

Could we establish a Could we establish a recurrence relationrecurrence relation for for deriving M(n)? deriving M(n)?

11

Example: recursive evaluation Example: recursive evaluation of of nn ! ! Definition:Definition: n n ! = 1*2! = 1*2*…*(n-*…*(n-1)*1)*nn Recursive definition of Recursive definition of nn!:!: Algorithm:Algorithm:



M(n) = M(n-1) + 1M(n) = M(n-1) + 1 Initial Condition: M(0) = ?Initial Condition: M(0) = ?

12




M(n) = M(n-1) + 1M(n) = M(n-1) + 1 Initial condition: M(0) = 0Initial condition: M(0) = 0 Explicit formula for M(n) in terms of n only? Explicit formula for M(n) in terms of n only?

13

Time efficiency of recursive Time efficiency of recursive algorithmsalgorithmsSteps in analysis of recursive algorithms: Steps in analysis of recursive algorithms: Decide on parameter Decide on parameter nn indicating indicating input sizeinput size Identify algorithm’s Identify algorithm’s basic operationbasic operation Determine Determine worstworst, , averageaverage, and , and bestbest case for inputs of size case for inputs of size nn

Set up a recurrence relation and initial condition(s) for Set up a recurrence relation and initial condition(s) for CC((nn)-the number of times the basic operation will be )-the number of times the basic operation will be executed for an input of size executed for an input of size nn

Solve the recurrence to obtain a closed form or Solve the recurrence to obtain a closed form or determine the order of growth of the solution (see determine the order of growth of the solution (see Appendix B)Appendix B)

14

EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi Problem:Problem:

• Given three pegs (A, B, C) and n disks of different sizes Given three pegs (A, B, C) and n disks of different sizes • Initially, all the disks are on peg A in order of size, the largest on Initially, all the disks are on peg A in order of size, the largest on

the bottom and the smallest on top the bottom and the smallest on top • The goal is to move all the disks to peg C using peg B as an The goal is to move all the disks to peg C using peg B as an

auxiliaryauxiliary• Only 1 disk can be moved at a time, and a larger disk cannot be Only 1 disk can be moved at a time, and a larger disk cannot be

placed on top of a smaller oneplaced on top of a smaller one

A C

B

n disks

15

EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi Design a recursive algorithm to solve this problem:Design a recursive algorithm to solve this problem:

• Given three pegs (A, B, C) and n disks of different sizes Given three pegs (A, B, C) and n disks of different sizes • Initially, all the disks are on peg A in order of size, the largest on Initially, all the disks are on peg A in order of size, the largest on

the bottom and the smallest on top the bottom and the smallest on top • The goal is to move all the disks to peg C using peg B as an The goal is to move all the disks to peg C using peg B as an

auxiliaryauxiliary• Only 1 disk can be moved at a time, and a larger disk cannot be Only 1 disk can be moved at a time, and a larger disk cannot be

placed on top of a smaller oneplaced on top of a smaller one

A C

B

n disks

16

EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi

Step 1: Solve simple case when n<=1?Step 1: Solve simple case when n<=1?Just trivialJust trivial

A C

B

A C

B

Move(A, C)

17


Step 2: Assume that a smaller instance can be Step 2: Assume that a smaller instance can be solved, i.e. can move n-1 disks. Then?solved, i.e. can move n-1 disks. Then?

A C

B

A C

B

A C

B

18


A C

BA C

B

A C

B

A C

B

19


A C

BA C

B

A C

B

A C

B

TOWER(n, A, B, C)

20


A C

BA C

B

A C

B

A C

B

TOWER(n, A, B, C)

TOWER(n-1, A, C, B)

Move(A, C)

TOWER(n-1, B, A, C)

21


TOWER(n, A, B, C) {

TOWER(n-1, A, C, B); Move(A, C);TOWER(n-1, B, A, C)}

if n<1 return;

22


TOWER(n, A, B, C) {


if n<1 return;

Algorithm analysis: Algorithm analysis: Input size? Basic operation?Input size? Basic operation?

23


TOWER(n, A, B, C) {


if n<1 return;

Algorithm analysis: Algorithm analysis: Do we need to differentiate best case, worst Do we need to differentiate best case, worst

case & average case for inputs of size n?case & average case for inputs of size n?

24


TOWER(n, A, B, C) {


if n<1 return;

Algorithm analysis: Algorithm analysis: Set up a recurrence relation and initial Set up a recurrence relation and initial

condition(s) for C(n)condition(s) for C(n)

25


TOWER(n, A, B, C) {


if n<1 return;

Algorithm analysis: Algorithm analysis: C(n) = 2C(n-1)+1C(n) = 2C(n-1)+1

26

In-Class ExerciseIn-Class Exercise P. 76 Problem 2.4.1 (c): solve this recurrence relation: P. 76 Problem 2.4.1 (c): solve this recurrence relation:

x(n) = x(n-1) + n for n>0, x(0)=0x(n) = x(n-1) + n for n>0, x(0)=0 P. 77 Problem 2.4.4: consider the following recursive algorithm: P. 77 Problem 2.4.4: consider the following recursive algorithm:

• Algorithm Q(n)Algorithm Q(n)// Input: A positive integer n// Input: A positive integer nIf n = 1 return 1If n = 1 return 1else return Q(n-1) + 2 * n – 1else return Q(n-1) + 2 * n – 1

• A. Set up a recurrence relation for this function’s values and solve it A. Set up a recurrence relation for this function’s values and solve it to determine what this algorithm computesto determine what this algorithm computes

• B. Set up a recurrence relation for the number of multiplications B. Set up a recurrence relation for the number of multiplications made by this algorithm and solve it. made by this algorithm and solve it.

• C. Set up a recurrence relation for the number of C. Set up a recurrence relation for the number of additions/subtractions made by this algorithm and solve it. additions/subtractions made by this algorithm and solve it.

27

Example: BinRec(n) Example: BinRec(n)

Algorithm BinRec(n)

//Input: A positive decimal integer n

//Output: The number of binary digits in n’s binary representation

if n = 1 return 1

else return BinRec( n/2 ) + 1

28

Smoothness ruleSmoothness ruleIf T(n) If T(n) (f(n)) (f(n)) for values of n that are powers of b, where b for values of n that are powers of b, where b 2, 2,

then then

T(n) T(n) (f(n))(f(n))

29

Example: BinRec(n) Example: BinRec(n)

Algorithm BinRec(n)

//Input: A positive decimal integer n

//Output: The number of binary digits in n’s binary representation

if n = 1 return 1

else return BinRec( n/2 ) + 1

If C(n) If C(n) (f(n)) (f(n)) for values of n that are powers of b, where b for values of n that are powers of b, where b 2, 2, then then C(n) C(n) (f(n))(f(n))

30

Fibonacci numbersFibonacci numbers The Fibonacci sequence:The Fibonacci sequence:

0, 1, 1, 2, 3, 5, 8, 13, 21, … 0, 1, 1, 2, 3, 5, 8, 13, 21, …

Fibonacci recurrence:Fibonacci recurrence:F(F(nn) = F() = F(nn-1) + F(-1) + F(nn-2) -2) F(0) = 0 F(0) = 0 F(1) = 1F(1) = 1

2nd 2nd order linear homogeneous order linear homogeneous recurrence relation recurrence relation

with constant coefficientswith constant coefficients

31

Solving linear homogeneous Solving linear homogeneous recurrence relations with constant recurrence relations with constant coefficientscoefficients

Easy first: 1Easy first: 1stst order LHRRCCs: order LHRRCCs:CC((nn) = ) = a Ca C((nn -1) -1) CC(0) = (0) = tt … Solution: … Solution: CC((nn) =) = t a t ann

Extrapolate to 2Extrapolate to 2ndnd order orderLL((nn) = ) = aa LL((nn-1) + -1) + bb LL((nn-2) … A solution?: -2) … A solution?: LL((nn) = ) = ??

Characteristic equation (quadratic)Characteristic equation (quadratic) Solve to obtain roots Solve to obtain roots rr11 and and rr2 2 ((quadratic formula)) General solution to RR: linear combination of General solution to RR: linear combination of rr11

nn and and rr22

nn

Particular solution: use initial conditionsParticular solution: use initial conditions

http://en.wikipedia.org/wiki/Quadratic_equation

32

Solving linear homogeneous Solving linear homogeneous recurrence relations with constant recurrence relations with constant coefficientscoefficients

Easy first: 1Easy first: 1stst order LHRRCCs: order LHRRCCs:CC((nn) = ) = a Ca C((nn -1) -1) CC(0) = (0) = tt … Solution: … Solution: CC((nn) =) = t a t ann

Extrapolate to 2Extrapolate to 2ndnd order orderLL((nn) = ) = aa LL((nn-1) + -1) + bb LL((nn-2) … A solution?: -2) … A solution?: LL((nn) = ) = ??

Characteristic equation (quadratic)Characteristic equation (quadratic) Solve to obtain roots Solve to obtain roots rr11 and and rr2 2 ((quadratic formula)) General solution to RR: linear combination of General solution to RR: linear combination of rr11

nn and and rr22

nn

Particular solution: use initial conditionsParticular solution: use initial conditions

Explicit Formula for Explicit Formula for Fibonacci Number: F(n) = F(n-1) +F(n-2)Fibonacci Number: F(n) = F(n-1) +F(n-2)

http://en.wikipedia.org/wiki/Quadratic_equation

33

1. Definition based recursive algorithm1. Definition based recursive algorithm

Computing Fibonacci numbersComputing Fibonacci numbers

Algorithm F(n)

// Compute the nth Fibonacci number recursively



if n 1 return n

else return F(n-1) + F(n-2)

34

2. Nonrecursive brute-force algorithm2. Nonrecursive brute-force algorithm


Algorithm Fib(n)

// Compute the nth Fibonacci number iteratively



F[0] 0; F[1] 1

for i 2 to n do

F[i] F[i-1] + F[i-2]

return F[n]

35


3. Explicit formula algorithm3. Explicit formula algorithm

integernearest the torounded )2

51(5

1)( nnF

Special care in its implementation: Special care in its implementation:

Intermediate results are irrational numbersIntermediate results are irrational numbersTheir approximations in the computer are accurate enoughTheir approximations in the computer are accurate enough

Final round-off yields a correct result Final round-off yields a correct result

36

In-Class Exercises In-Class Exercises What is the explicit formula for A(n)?What is the explicit formula for A(n)?

A(A(nn) = 3A() = 3A(nn-1) – 2A(-1) – 2A(nn-2) A(0) = 1 A(1) = 3-2) A(0) = 1 A(1) = 3

P.83 2.5.3. Climbing stairs: Find the number of P.83 2.5.3. Climbing stairs: Find the number of different ways to climb an n-stair stair-case if each different ways to climb an n-stair stair-case if each step is either one or two stairs. (For example, a 3-step is either one or two stairs. (For example, a 3-stair staircase can be climbed three ways: 1-1-1, 1-stair staircase can be climbed three ways: 1-1-1, 1-2, and 2-1.)2, and 2-1.)

Documents

RAIK 283: Data Structures & Algorithms