Recurrence Equations

Algorithm : Design & Analysis

[4]

In the last class…

Recursive Procedures Analyzing the Recursive Computation. Induction over Recursive Procedures Proving Correctness of Procedures

Recurrence Equations

Recursive algorithm and recurrence equation

Solution of the Recurrence equations Guess and proving Recursion tree Master theorem

Divide-and-conquer

Recurrence Equation: Concept

A recurrence equation: defines a function over the natural number n in term of its own value at one or more integers

smaller than n Example: Fibonacci numbers

Fn=Fn-1+Fn-2 for n2

F0=0, F1=1

Recurrence equation is used to express the cost of recursive procedures.

Linear Homogeneous Relation

knmnnn ararara 2211

is called linear homogeneous relation of degree k.

1)2( nn cc 31 nn aa

21 nnn fff 22

1 nnn ggg

Yes No

Characteristic Equation

For a linear homogeneous recurrence relation of degree k

the polynomial of degree k

is called its characteristic equation.

The characteristic equation of linear homogeneous recurrence relation of degree 2 is:

knmnnn ararara 2211

kkkk rxrxrx 2

21

1

0212 rxrx

Solution of Recurrence Relation

If the characteristic equation of the

recurrence relation has two distinct

roots s1 and s2, then

where u and v depend on the initial conditions, is the

explicit formula for the sequence.

If the equation has a single root s, then, both s1 and s2 in

the formula above are replaced by s

0212 rxrx

2211 nnn arara

nnn vsusa 21

Fibonacci Sequence

f1=1

f2=1

fn= fn-1+ fn-2

f1=1

f2=1

fn= fn-1+ fn-2

1, 1, 2, 3, 5, 8, 13, 21, 34, ......

Explicit formula for Fibonacci Sequence The characteristic equation is x2-x-1=0, which has roots:

2

51

2

5121

sands

Note: (by initial conditions) 11 22

212211 vsusfandvsusf

which results:nn

nf

2

51

5

1

2

51

5

1

Determining the Upper Bound

Example: T(n)=2T(n/2) +n Guess

T(n)O(n)? T(n)cn, to be proved for c large enough

T(n)O(n2)? T(n)cn2, to be proved for c large enough

Or maybe, T(n)O(nlogn)? T(n)cnlogn, to be proved for c large enough

Try and fail to prove T(n)cn: T(n)=2T(n/2)+n 2c(n/2)+n 2c(n/2)+n = (c+1)n

T(n) = 2T(n/2)+n 2(cn/2 lg (n/2))+n cn lg (n/2)+n = cn lg n – cn log 2 +n = cn lg n – cn + n cn log n for c1Note

: the pro

of is i

nvalid

for T(1

)=1

Recursion Tree

T(size) nonrecursive cost

The recursion tree for T(n)=T(n/2)+T(n/2)+n

T(n) n

T(n/4) n/4T(n/4) n/4T(n/4) n/4T(n/4) n/4

T(n/2) n/2T(n/2) n/2

Recursion Tree Rules

Construction of a recursion tree work copy: use auxiliary variable root node expansion of a node:

recursive parts: children nonrecursive parts: nonrecursive cost

the node with base-case size

Recursion tree equation

For any subtree of the recursion tree, size field of root =

Σnonrecursive costs of expanded nodes +

Σsize fields of incomplete nodes Example: divide-and-conquer:

T(n) = bT(n/c) + f(n) After kth expansion:

1

0

)(k

ii

ik

k

c

nfb

c

nTbnT

Evaluation of a Recursion Tree

Computing the sum of the nonrecursive costs of all nodes.

Level by level through the tree down. Knowledge of the maximum depth of the

recursion tree, that is the depth at which the size parameter reduce to a base case.

Recursion Tree

T(n) n

T(n/4) n/4T(n/4) n/4T(n/4) n/4T(n/4) n/4

T(n/2) n/2T(n/2) n/2

Work copy: T(k)=T(k/2)+T(k/2)+kWork copy: T(k)=T(k/2)+T(k/2)+k

At this level: T(n)=n+2(n/2)+4T(n/4)=2n+4T(n/4)At this level: T(n)=n+2(n/2)+4T(n/4)=2n+4T(n/4)

n/2d(size 1)

T(n)=nlgn

Recursion Tree for T(n)=3T(n/4)+(n2)

cn2

T(1) T(1) T(1) T(1) T(1)T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1)

c(n/16)2c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2c(n/16)2 c(n/16)2 c(n/16)2c(n/16)2

…… ……

c(n/4)2 c(n/4)2 c(n/4)2

log4n

cn2

2

16

3cn

22

16

3cn

…

3log4n

Total: O(n2)Note: 3loglog 443 nn

Verifying “Guess” by Recursive Tree

)()(13

16

)(

163

1

1

)(16

3

)(16

3)(

23log2

3log2

0

3log2

1log

0

3log2

4

4

4

4

4

nOncn

ncn

ncn

ncnnT

i

i

n

i

i

cdwhendn

cndn

cnnd

cnnd

cnnTnT

13

1616

3

)4/(3

4/3

)4/(3)(

2

22

22

22

2

Inductive hypothesisInductive hypothesis

Common Recurrence Equation

Divide and Conquer T(n) = bT(n/c) + f(n)

Chip and Conquer T(n) = T(n - c) + f(n)

Chip and Be Conquered T(n) = bT(n - c) + f(n)

Recursion Tree for T(n)=bT(n/c)+f(n)

f(n)

T(1) T(1) T(1) T(1) T(1)T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1)

f(n/c2)f(n/c2) f(n/c2) f(n/c2) f(n/c2)f(n/c2) f(n/c2) f(n/c2)f(n/c2)

…… ……

f(n/c) f(n/c) f(n/c)

logcn

f(n)

)/( cnbf

)/( 22 cnfb

…

bcnlog

Note: bn cc nb loglog

b

b

Total ?

Solving the Divide-and-Conquer

The recursion equation for divide-and-conquer, the general case:T(n)=bT(n/c)+f(n)

Observations: Let base-cases occur at depth D(leaf), then

n/cD=1, that is D=lg(n)/lg(c) Let the number of leaves of the tree be L, then

L=bD, that is L=b(lg(n)/lg(c)). By a little algebra: L=nE, where E=lg(b)/lg(c),

called critical exponent.

c

bnb

c

n

bc

nc

n

bL lg

lglglg

lg

lg

lglg

lg

222lg

lg

c

bnb

c

n

bc

nc

n

bL lg

lglglg

lg

lg

lglg

lg

222lg

lg

Divide-and-Conquer: the Solution

The recursion tree has depth D=lg(n)/ lg(c), so there are about that many row-sums.

The 0th row-sum is f(n), the nonrecursive cost of the root.

The Dth row-sum is nE, assuming base cases cost 1, or (nE) in any event.

The solution of divide-and-conquer equation is the nonrecursive costs of all nodes in the tree, which is the sum of the row-sums.

Little Master Theorem

Complexity of the divide-and-conquer case 1: row-sums forming a geometric series:

T(n)(nE), where E is critical exponent

case 2: row-sums remaining about constant: T(n)(f(n)log(n))

case 3: row-sums forming a decreasing geometric series:

T(n)(f(n))

Master Theorem

Loosening the restrictions on f(n) Case 1: f(n)O(nE-), (>0), then:

T(n)(nE) Case 2: f(n)(nE), as all node depth

contribute about equally:T(n)(f(n)log(n))

case 3: f(n)(nE+), (>0), and f(n)O(nE+), (), then:

T(n)(f(n))

The positive is critical, resulting gaps between cases as well

The positive is critical, resulting gaps between cases as well

Using Master Theorem

)lg()(,3

)()(lg)(,793.03log,4,3

lg4

3)(3

)(lg)(,2),(1)(,0,2

3,1

13

2)(2

)()(,1),()(,2,3,9

39)(1

21.121.04

0

21

nnnTappliescase

nOnnnnfEcb

nnn

TnTExample

nnTappliescasennfEcb

nTnTExample

nnTappliescasenOnnfEcb

nn

TnTExample

EE

E

Looking at the Gap

T(n)=2T(n/2)+nlgn a=2, b=2, E=1, f(n)=nlgn We have f(n)=(nE), but no >0 satisfies

f(n)=(nE+), since lgn grows slower that n for any small positive .

So, case 3 doesn’t apply. However, neither case 2 applies.

0any for 0ln

lim

n

nn

Proof of the Master Theorem

)lg(/)lg(

0

)(cn

dd

d

c

nfbnT

dEdEd

E

d c

nf

c

n

c

nf

)lg(/)lg(

0

)lg(/)lg(

0

)()(

cn

dddEd

cn

d

d

c

nf

cc

nfbnT

(Note: in asymptotic analysis, f(n)(nE+) leads to f(n) is about (nE+), ignoring the coefficients.

(Note: in asymptotic analysis, f(n)(nE+) leads to f(n) is about (nE+), ignoring the coefficients.

Case 3 as an example

Decreasing geo. series

Home Assignment

pp.143- 3.7 3.8 3.9 3.10