CSE 3101

EECS 3101

Prof. Andy Mirzaian

STUDY MATERIAL:

• [CLRS] Appendix A, chapter 4• Lecture Note 3

2

Recurrence Relations:

Summations:

3

TOPICS

Summations & Recurrence Relations arise in the running time analysis of algorithms.

SUMMATIONS: Classic Methods: Arithmetic, Geometric, Harmonic, Bounded Tail Approximating Summation by Integration Summation by Parts ASSUME: ASymptotic SUmmation Made Easy

RECURRENCE RELATIONS: Iteration method Recursion tree method Divide-&-Conquer Recurrence: The Master Method Guess-&-Verify method Full History Recurrences Variable Substitution method

4

SUMMATION

)()3()2()1()()(

Ν:

1

nffffifnS

fn

i

R

5

Classic Methods

Arithmetic (or polynomial): S(n) = Q(n f(n)) f(n) = n: 1 + 2 + ··· + n = n(n+1)/2 = Q( n2 ) f(n) = n2: 12 + 22 + ··· + n2 = n(n+1)(2n+1)/6 = Q( n3 ) f(n) = nd: 1d + 2d + ··· + nd = Q( nd+1 ), for any constant real d > -1

Geometric (or exponential): S(n) = Q(f(n)) f(n) = xn: 1 + x + x2 + ··· + xn = (xn+1 –1)/(x –1) for any constant real 0|x|1 = Q(xn) if x > 1 (Geometric increasing) f(n) = 2n: 1+ 2 + 22 + ··· + 2n = Q(2n) (with x = 2)

Harmonic: S(n) = Q(log n) f(n) = 1/n: H(n) = 1 + 1/2 + 1/3 + ··· + 1/n ln n = Q(log n)

Bounded Tail: S(n) = Q(1) 1 + x + x2 + ··· + xn = (1 – xn+1)/(1-x) = Q(1) if 0<x<1. (Geometric decreasing) f(n) = 2-n: 1 + 1/2 + 1/4 + ··· + 1/2n = Q(1) (with x = ½) f(n) = n-2: 1 + 1/22 + 1/ 32 + ··· + 1/n2 = Q(1) (Arithmetic decreasing)

6

Approximating by

x

f(x)

f : f monotonically non-decreasing

m-1 m m+1 m+2 . . . n n+1

n

m

n

mi

n

mdx)x(f)i(fdx)x(f

1

1

7

Approximating by

NOTE: direction of inequalities changed

x

f(x)

f : f monotonically non-increasing

m-1 m m+1 m+2 . . . n n+1

n

m

n

mi

n

mdx)x(f)i(fdx)x(f

1

1

8

Approximating by

S(n) = f(1) + f(2) + … + f(n) (m = 1) Example 1: S(n) = 13 + 23 + … + n3 .

f(x) = x3 : f and f(x) dx = x3 dx = x4 / 4. n4 / 4 S(n) ( (n+1)4 – 1)/ 4 S(n) = Q(n4)

Example 2: H(n) = 1 + 1/2 + 1/3 + … + 1/n .

f(x) = 1/x : f and f(x) dx = dx /x = ln x. 1 + ln n H(n) ln (n+1) H(n) = Q(log n)

Caution against dividing by zero!Separate f(1) from the summation. 9

Summation by Parts: Arithmetic f(n) = n : S(n) = f(1) + f(2) + f(3) + ··· + f(n).

S(n) = 1 + 2 + 3 + ······································································· + n

0 ³ n/2 · n/2 W(n2)

S(n) = Q(n2)

W(n2)

n·n O(n2)

+ (n/2 –1) + n/2 + (n/2 +1) + (n/2 + 2) +

n/2 n/2

10

Summation by Parts: Harmonic

H(n) = Q(log n)

nnH 1161

151

141

131

121

111

101

91

81

71

61

51

41

31

211)(

> 8(1/16) = ½ > 4(1/8) = ½ >2(1/4) = ½ ½ 1

³ 1 + ½ log n W(log n)

< 2(1/2)=1 < 4(1/4) =1<1 < 8(1/8) = 11

1+ log n O(log n)

11

ASSUME : ASymptotic SUmmation Made Easy

f: N + : S(n) = f(1) + f(2) + f(3) + ··· + f(n)

maxn(f) = max { f(1) , f(2) , … , f(n) }

minn(f) = min { f(1) , f(2) , … , f(n) }

aven(f) = average {f(1) , f(2), … , f(n) }

S(n) = n · aven(f)

0 minn(f) aven(f) maxn(f)

f minn(f) = f(1) , maxn(f) = f(n)

f minn(f) = f(n) , maxn(f) = f(1)

max{ n · minn(f), maxn(f) } S(n) n · maxn(f)

ASSUME: A method for finding g(n) (and hence, S(n)).

S(n) = Q( g(n) · maxn(f)) for some 1 g(n) n

12

ASSUME on monotone f(n)

f: N + S(n) = f(1) + f(2) + f(3) + ··· + f(n)

FACT: f is monotone exactly one of the following holds:

(1) f(n) = 0 n (2) f(n) = Q(1)

(3) f(n) = w(1) and f(4) f(n) = o(1) and f and f(1) > 0

Cases (1) & (2): g(n) = Q(n), S(n) = Q(g(n) f(n)) = Q(n f(n))

Case (3): Considered next.

Case (4): Similar methods apply. Exercise.13

ASSUME for f & f(n) = w(1)

Step 1) Find m such that log f(m) = log f(n) – Q(1)

Step 2) g(n) n – m [ Note: 1 g(n) n ]

THEOREM: f & g S(n) = Q(f(n)g(n)).

nm

g(n)

f(n)

f(m) cf(n) f(m+1)f(n)g(n)

f

0 < c < 1

S(n) = f(1) + ··· + f(n)

14

ASSUME: examples

Polynomial: f(n) = nd (const. d 0) :

log f(m) = log f(n) – Q(1) d log m = d log n – d = d log n/2 m = n/2 g(n) = n–m = n/2.

f & g S(n) = Q(f(n)g(n)) = Q(nd+1).

Exponential: f(n) = 2n :

log f(m) = log f(n) – Q(1) m = n – 1 g(n) = n–m = 1

f & g S(n) = Q(f(n)g(n)) = Q(2n).

15

log(1 x) = Q( ln(1 – x) ) = – Q(x) for x = o(1)

Taylor Series Expansion :

3)3(

2)2()1(

!3)0(

!2)0(

!1)0(

)0()( xf

xf

xf

fxf

f (x) = ln(1– x) : 432432

)1ln( xxxxx

Example:

n

n

nn

n

n

nn

n

nn

xn

nnn

1

log

loglog

log

1)(loglog

] take[1loglog

1log))(loglog(

16

ASSUME: examples

Super-polynomial sub-exponential 1: f(n) = : log f(m) = log f(n) – Q(1) = – 1 [Now square both sides] m = ( – 1)2 = n – 2 + 1

g(n) = n–m = 2 –1 = Q( ).

f & g S(n) = Q(f(n)g(n)) = Q().

Super-polynomial sub-exponential 2: f(n) = 2n/log n : log f(m) = log f(n) – Q(1)

(*) m/log m = n/log n – Q(1) [Multiply across by log m log n ] m = n – Q(log n) [Verify this satisfies (*) with “=” ] m/log m = (n – Q(log n)) / log (n – Q(log n))

= ( n – Q(log n) ) / (log n)(1 – Q(1/n)) [See previous page] = ( n/log n – Q(1) ) / (1 – Q(1/n)) = n/log n – Q(1) [Why? Multiply sides by 1 – Q(1/n)]

g(n) = n – m = Q(log n)

f & g S(n) = Q(f(n)g(n)) = Q(2n/log n log n). 17

ASSUME: Lower Bound Proof

THEOREM: f & g S(n) = f(1) + ··· + f(n) = Q(f(n)g(n)).

Lower bound: S(n) S(n) – S(m) = f(m+1) + f(m+2) + ··· + f(n) shaded rectangular area since f = f(m+1) g(n) c f(n) g(n) W( f(n) g(n) ).

nm+1

g(n)

f(n)

f(m) cf(n) f(m+1)f(m+1)g(n)

f

m

18

ASSUME: Upper Bound Proof

THEOREM: f & g S(n) = f(1) + ··· + f(n) = Q(f(n)g(n)).

Upper bound: S(n) sum of shaded rectangular areas since f = g(n0)f(n0) + g(n1)f(n1) + g(n2)f(n2) + ···

g(n) [ f(n0) + f(n1) + f(n2) + ··· ] since g

g(n)f(n) [ 1 + c + c2 + ··· ] f(ni+1) c f(ni) g(n)f(n) / (1 – c) classic geometric decreasing sum O( f(n) g(n) ).

n0 = ng(n0)

f(n0)

cf(n0) f(n1)g(n0)f(n0)

f

n1

g(n1)

g(n1)f(n1)

n2

g(n2)

g(n2)f(n2)

n3

c2f(n0) cf(n1) f(n2)

19

Caution!

nloglog

)n(f 22

f(n) is a step function sandwiched between n and .

Why is “g” not true?

To guarantee conclusion of the theorem, its premise must hold. The premise insists that not only f, but also g holds.

f without g is possible. The following is an example:

However, with Simple Analytical Functions (SAF), i.e., functions composed of only constants, arithmetic, exponential, logarithm, and functional composition, such a “bizarre” situation will not arise, and hence, the statement requiring “g” can be omitted from the premise of the theorem.

20

Simple Analytical Function (SAF)SAF : a function composed of a finite number of applications of constants, arithmetic, exponential, logarithm, & functional composition.

Example: n

nn

nnn

72

loglog2

/)(log614

FACTS: A SAF f(n) has the following properties: a) log f(n) and f(x)/x are also SAFs. b) has only a finite # of roots. (does not oscillate) c) is asymptotically sign invariant. (past its last real root) d) is asymptotically monotonic. (part (c) on its derivative) e) In ASSUME: f(n) g(n) f) when comparing SAFs asymptotically, “n0nn0” can be replaced by the phrase “for infinitely many n” (“for i.m. n” for short). E.g., “for all n that are positive integer powers of 2”. (“past last real root”)

Our short article says more on this topic.The next table shows a summary of ASSUME on SAF.

21

http://www.cse.yorku.ca/~andy/courses/3101/Slides/SAF.pdf

f(n) : a SAFS(n) = f(1) + f(2) + ··· + f(n)

Sum TypeExample

f(n)

Geometric(or exponential)

Arithmetic(or polynomial)

quasi

Harmonic

BoundedTail

,

22

A Simple form

The SAFs we encounter, usually have the following simple form: nlognt)n(f ednfor some real constants t>0, d, e. We classify S(n) for such f(n) as:

t > 1 Geometric (increasing)

t = 1

d > -1 Arithmetic

d = -1 quasi Harmonic

d < -1 Bounded tail (polynomially decreasing)

0 < t < 1 Bounded tail (geometrically decreasing)

23

RECURRENCERELATIONS

)log()(

O(1)n for )1(

O(1) n for )(2T)( 2

n

nnnT

O

nnT

24

The Iteration Method

The iteration method (or unwinding the recurrence):T(n) = f(n) + T(n –1)

= f(n) + f(n-1) + T(n –2)

= f(n) + f(n-1) + f(n-2) + T(n –3)

= …

= f(n) + f(n-1) + f(n-2) + ··· + f(3) + f(2) + T(1)

= f(n) + f(n-1) + f(n-2) + ··· + f(3) + f(2) + f(1) + T(0)

= f(n) + f(n-1) + f(n-2) + ··· + f(3) + f(2) + f(1).

1for 1

0for 0

n)n(f)n(T

n)n(T

Example: f(n) = Q(n) T(n) = Q(n2).

f(n) = Q(2n) T(n) = Q(2n).

f(n) is the driving function of the recurrence

25


T(n) = n + 2T(n/2)

= n + 2 [n/2 + 2T(n/22)]

= 2n + 22 T(n/22)

= 2n + 22 [n/22 + 2 T(n/23)]

= 3n + 23 T(n/23)

= 3n + 23 [n/23 + 2T(n/24)]

= 4n + 24 T(n/24)

= …

= kn + 2k T(n/2k) take k = 1+ log n, T(n/2k) = 0

= n(1 + log n)

= Q(n log n )

1for 2

1for 0

2 nnT

n)n(T n

26


T(n) = n2 + 2T(n/2) = n2 + 2 [(n/2)2 + 2T(n/22)]

= n2(1+ ½) + 22 T(n/22)

= n2(1+ ½) + 22 [(n/22)2 + 2 T(n/23)]

= n2(1+ ½ + ½2) + 23 T(n/23)

= n2(1+ ½ + ½2) + 23 [(n/23)2 + 2T(n/23)]

= n2(1+ ½ + ½2 + ½3) + 24 T(n/24)

= …

= n2 (1+ ½ + ½2+ ½3 + ··· + ½k ) + 2k+1 T(n/2k+1)

take k = log n, T(n/2k+1) = 0

= n2 (1+ ½ + ½3 + ½5 + ··· + ½k )

= n2 · Q(1) geometric decreasing = Q(n2 )

1for 2

1for 02

2 nnT

n)n(T n

27

T(n)

The Recursion Tree Method

1for 2

1for 0

2 n)n(fT

n)n(T n

f(n)f(n)

T(n/2) T(n/2)

2f(n/2)f(n/2)

T(n/4) T(n/4) T(n/4) T(n/4)

f(n/2)

T(n/4) T(n/4)

4f(n/4)f(n/4)

T(n/8) T(n/8)

f(n/4)

T(n/8) T(n/8)

f(n/4)

T(n/8) T(n/8)

f(n/4)

T(n/8) T(n/8)8f(n/8)f(n/

8)f(n/8)

f(n/8)

f(n/8)

f(n/8)

f(n/8)

f(n/8)

f(n/8)

T(n/16) T(n/16)

n

i

i

nnn

inf

fffnfnTlog

0

842

22

)(8)(4)(2)()(

28

T(n)


4for

4for 0

24 nnTT

n)n(T nn

nn

T(n/4) T(n/2)

3n/4n/4

T(n/16) T(n/8) T(n/8) T(n/4)

n/2

T(n/4)

(3/4)2nn/16

T(n/64) T(n/32)

n/8

T(n/32) T(n/16)

n/8

T(n/32) T(n/16)

n/4

T(n/16) T(n/8) (3/4)3nn/

64n/32

n/32

n/16

n/32

n/16

n/16

n/8

T(n/256) T(n/16)

).(41

)(3

432

43

43

34

324

34

3

nOnn

nnnnnT

CLAIM: T(n) = Q(n).

Lower bound: T(n) W(n) obvious

log 2

n

log 4

n

Upper bound

29


4for

4for 0

24 nnTT

n)n(T nn

FACT:(1) + < 1 a b T(n) = Q(n) [linear]

(2) + = 1 a b T(n) = Q(n log n) (3) + > 1 a b T(n) = Q(nd) [super-

linear poly] where d > 1 is the unique constant that satisfies the equation ad + bd = 1. [See the guess-&-verify method later & Exercise 9.]

This is a special case of the following recurrence:

T(n) = T(an) + T(bn) + Q(n)

where 0 a < 1 and 0 b < 1 are real constant parameters.

30

Divide-&-Conquer

MergeSort is the prototypical divide-&-conquer algorithm.Here is a high level description:

Algorithm MergeSort( S ) § John von Neumann [1945]

Pre-Condition: input S is a sequence of numbers

Post-Condition: output is a sorted permutation of the input sequence

Base: if |S| 1 then return S

Divide: Divide S into its left and right halves S=L, R, |L||R| ½|S|

Conquer: L’ MergeSort( L ); R’ MergeSort( R )

Combine: S’ Merge( L’, R’ )

Output: return S’

end

31

MergeSort Example

2315 41 82 37 92 6631 34 73 2519 3315 16 58

2315 41 82 37 926631 34 732519 3315 16 58

2315 41 8237 926631 34 732519 3315 16 58

23 15 41 82 37 92 66 31 34 73 25 19 33 15 16 58

INPUT

2315 41 8237 926631 34 732519 3315 16 58

OUTPUT32

procedure MS( s, f ) § sort A[s..f] Base: if s f then returnDivide: m (s+f)/2Conquer: MS( s, m –1 ) ; MS( m, f )Combine: Merge( s, m, f ) end

Algorithm MergeSort( A[1..n] ) B[0..n+1] auxiliary array for mergingMS(1,n)end

procedure Merge( s, m, f )B[s-1 .. m-2] A[s,m-1] B[m .. f] A[m,f] B[m-1] B[f+1] § barrier sentinel

i s-1 ; j mfor k s .. f do if B[i] B[j]

then A[k] B[i]; i++ else A[k] B[j]; j++end

B

s f

A

n

s-1 mm-1 f+1i j

left half right half

merged so far

k

Merge time = Q(n), where n = f – s +1.

MERGE Loop Invariant:

33

procedure MS( s, f ) § sort A[s..f] Base: if s f then returnDivide: m (s+f)/2Conquer: MS( s, m –1 ) ; MS( m, f )Combine: Merge( s, m, f ) end

Algorithm MergeSort( A[1..n] ) B[0..n+1] auxiliary array for mergingMS(1,n)end

MergeSort Recurrence:

T(n) = Q(1) n 1

T(n) = T( n/2 ) + T( n/2 ) + Q(n) n>1

Simplified Recurrence:

T(n) = 2 T(n/2) + Q(n) for n > 1 T(n) = Q(1) for n 1

procedure Merge( s, m, f )B[s-1 .. m-2] A[s,m-1] B[m .. f] A[m,f] B[m-1] B[f+1] § barrier sentinel

i s-1 ; j mfor k s .. f do if B[i] B[j]

then A[k] B[i]; i++ else A[k] B[j]; j++end

Solution:

T(n) = Q( n log n). 34

Divide-&-Conquer Recurrence

a = # of recursive calls (a > 0)n/b = size of each sub-instance called recursively (b > 1)f(n) = cost of divide + combine steps (every thing except the recursive calls).

In MergeSort we have:a = 2 recursive calls of size n/b = n/2 each.Cost of divide = Q(1)Cost of Combine, i.e., merge, is Q(n).So, f(n) = Q(n).

𝑇 (𝑛 )={𝑎𝑇 (𝑛𝑏 )+ 𝑓 (𝑛) ∀𝑛>1

Θ (1 ) ∀𝑛≤ 1

35

Recursion Tree:

a logb n = n logb a (compare their logarithms)

h(n) = ak ·Q(1) = Q(ak) = Q(alogbn ) = (Q alogbn) = (Q nlogba)k = logbn = Q( log n).

f(n)

h(n) = Q(n logba )

ak-1f(n/bk-1)

a2f(n/b2)

af(n/b)

f(n)

f(n/b) f(n/b) f(n/b)

a

f(n/b2) f(n/b2) f(n/b2) f(n/b2)

f(n/bk-1) f(n/bk-1)

Q(1) Q(1)Q(1)Q(1)

k = logbn

a k

11

1

n)(

n)n(f)(aT)n(T b

n

36

a logb n = n logb a (compare their logarithms)

h(n) = ak ·Q(1) = Q(ak) = Q(alogbn ) = (Q alogbn) = (Q nlogba)k = logbn = Q( log n).

f(n)

h(n) = Q(n logba )

ak-1f(n/bk-1)

a2f(n/b2)

af(n/b)

S(n) = S

T(n) = S(n) + h(n) = Q( max{S(n), h(n)})

h(n) = ( Q n logba )

12

12

1

0

)()()(

)(

k

i

bnk

bn

bn

k

ibni

fafaafnf

fanS

1)1(

1)()()(

n

nnfaTnT b

n

37

Homogeneous Solution: contribution of the leaves of the recursion tree.

h(n) = (Q n logba )

1)1(

1)()(

n

nahnh b

n

10

1)()()(

n

nnfaSnS b

n

Special Solution: contribution of the internal nodes of the recursion tree.

1)1(

1)()()(

n

nnfaTnT b

n

T(n) = S(n) + h(n)

)(

)()(,

)(

)()( :Define

nh

nfnr

nh

nSnQ

10

1)()()(

n

nnrQnQ b

n

T(n) = h(n)( Q(n) + 1 )

32)()(bn

bn

bn rrrnrnQ

f(n) is a SAF r(n) is a SAF

38

)(

)()(,

)(

)()(

nh

nfnr

nh

nSnQ T(n) = h(n)( Q(n) + 1 )

k

i

ibrnQ1

)(f(n) is a SAF r(n) is a SAF

Let n = bk for integer k = (log n).

r(n) r(bi) Q(n)=Q( ? ) T(n)=Q( ? )

W(n+e) W(b+ie) r(n) f(n)

O(n-e) O(b-ie) 1 h(n)

Q(loge n) e > -1

Q(ie) r(n) k f(n)log n

Q(log-1 n) Q(1/i) log k h(n)log log n

O(loge n) e < -1

O(i-e) 1 h(n)39

Compare with CLRS. See Exercise 4.

f(n)»

h(n)

(e > -1) f(n)

h(n) (e = -1)

(e < -1) f(n)

«h(n)

THE MASTER THEOREMSuppose f(n) is a SAF, and > 0 &e e are constant reals.

11

1

n)(

n)n(f)(aT)n(T b

n

h(n) = Q( n logba )

40

f(n) = Q( tn nd log e n) constants t > 0, d, e.

t > 1 T(n) = Q(f(n))

t = 1

bd > a T(n) = Q(f(n))

bd = a

d = logb a

e > -1 T(n) = Q(f(n) log n)

e = -1 T(n) = Q(h(n) log log n)

e < -1 T(n) = Q(h(n))

bd < a T(n) = Q(h(n))

0 < t < 1 T(n) = Q(h(n))

11

1

n)(

n)n(f)(aT)n(T b

n

h(n) = Q( n logba )

41

ASSUME [ ASymptotic SUmmation Made Easy ] g(n) where

Sum Type :

Geometric

Arithmetic

Harmonic , e = a real constant

Bounded Tail

MASTER METHOD: , ,

=

CSE 3101 Sums & Recurrences Fact Sheet

42

Example 1: Assume a = 2, b = 4, f(n) = Q(nd log2 n). Express T(n) based on the constant parameter d.

Solution:

log b a = log 4 2 = ½ h(n) = ()

d > ½ T (n) = (f(n)) = ()

d = ½ T (n) = (f(n) log n) = ( )

d < ½ T (n) = (h(n)) = ()

Solution:logb a = log4 2 = ½ h(n) = Q()

d > ½ T(n) = Q(f(n)) = Q() d = ½ T(n) = Q(f(n) log n) = Q( ) d < ½ T(n) = Q(h(n)) = Q()

11

1

n)(

n)n(f)(aT)n(T b

n

h(n) = Q( n logba )

43

Example 2: Assume b = 3, f(n) = Q(n4 / log n). Express T(n) based on the constant parameter a.

Solution:

h(n) = (n lo g3 a) , bd = 3 4 = 81

a < 81 T (n) = (f(n)) = (n 4 / log n)

a = 81 T (n) = (h(n) log log n) = (n 4 log log n)

a > 81 T (n) = (h(n)) = (n lo g3a)

Solution: h(n) = Q(nlog3a) , bd = 34 = 81

a < 81 T(n) = Q(f(n)) = Q(n4 / log n)a = 81 T(n) = Q(h(n) log log n) = Q(n4 log log n)a > 81 T(n) = Q(h(n)) = Q(nlog3a)

11

1

n)(

n)n(f)(aT)n(T b

n

h(n) = Q( n logba )

44

Guess-&-VerifyBASIC INGRIDIANTS:

Guess a solution to the recurrence.

Verify it by mathematical induction. The induction variable must be a natural number. E.g., height of the recursion tree (max recursion depth), or size of the recursion tree (# times you apply the recurrence). n may also be a candidate if it “stays” integer.

If you spot a place where the verification does not go through, examine it to help revise your guess, and try again.

One key guiding principle, taken with a grain of salt, is to first make sure the leading term (with its constant coefficient) matches on the LHS and RHS of the recurrence. That will show you to guess higher or lower the next time! After figuring out the leading term, you can apply the same principle to find the lower order terms.

45

Guess-&-Verify: Example 1

46

• Clearly . Is ?

• Guess # 1:

lower order terms (to be determined)

• Plug this guess in the recurrence and verify:

• LHS leading term < RHS leading term guess is too low.


47

• Guess # 2:



• LHS leading term > RHS leading term guess is too high.


48

• Guess # 3:



=

• LHS leading term = RHS leading term correct guess of high order term.

• Solve the residual recurrence for (with revised boundary condition):

(must have , why?)

• Solution: and

Guess-&-Verify: Example 2Recurrence: T(n) = 2T( n/2 ) + n2 , T(0) = 0

if n is even = 2 [ a(n/2)2 + b(n/2)+c ] + n2 not “” for odd n, if b<0 = (1+ a/2)n2 + bn + 2c an2 + bn + c our guessed upper-bound

T(n) n2 W(n2) is obvious. Let’s see if O(n2) is an upper-bound:

Guess: T(n) an2 + bn + c (for some constants a>0, b, c to be determined)

We need: for all even n 1: an2 + bn + c (1+ a/2)n2 + bn + 2c, i.e., (a/2 – 1)n2 – c 0.

Basis (n = 0): T(0) = 0 a02 + b0 + c c 0we need

Ind. Step (n 1): T(n) = 2T( n/2 ) +n2

2 [ a n/22 + b n/2 + c ] + n2 by induction hypothesis

49


Ind. Step (n 1): T(n) = 2T( n/2 ) +n2

2 [ a n/22 + b n/2 + c ] + n2 by induction hypothesis

T(n) n2 W(n2) is obvious. Let’s see if O(n2) is an upper-bound:

Guess: T(n) an2 + bn + c (for some constants a>0, b, c to be determined)

We need: for all even n 1: an2 + bn + c (1+ a/2)n2 + bn + 2c, i.e., (a/2 – 1)n2 – c 0.

We need: for all odd n 1:an2 + bn + c (1+ a/2)n2 + (b-a)n + (2c-b+a/2), i.e., (a/2 – 1)n2 + an + (b – c – a/2) 0.

if n is odd = 2 [ a((n-1)/2)2 + b((n-1)/2)+c ] + n2 = (1+ a/2)n2 + (b-a)n + (2c-b+a/2) an2 + bn + c our guessed upper-bound

a = 2b = -1c = 0

works for all n 0.

n2 T(n) 2n2 - n

Basis (n = 0): T(0) = 0 a02 + b0 + c c 0we need

50


Exercise 8: Using this same method, verify the following

tighter LOWER BOUND:

n1: 2n2 – n – 2n log n T(n) 2n2 – n.

T(n) = 2n2 – n – O(n log n). a = 2b = -1c = 0

works for all n 0.

n2 T(n) 2n2 - n

For more examples ofguess-&-verify see

Lecture Note 3&

Sample Solutions …. 51

Full History Recurrence

Such recurrences often arise in average-case analysis.

0),()()( :1 Example1

0

nnfiTnT

n

i

Values of T(n) for some small n:T(0) = f(0)T(1) = T(0) + f(1) = f(0) + f(1)T(2) = T(0) + T(1) + f(2) = 2f(0) + f(1) + f(2)T(3) = T(0) + T(1) + T(2) + f(3) = 4f(0) + 2f(1) + f(2)+ f(3) T(4) = T(0) + T(1) + T(2) + T(3) + f(4) = 8f(0) + 4f(1) + 2f(2) + f(3) + f(4)

0,)(2

)( :2 Example1

0

nniT

nnT

n

i

This is the QuickSort expected time recurrence (details shown later in LS5)

52

Full History Recurrence

1n),1n(f)n(f)1n(T)1n(T)n(T :Subtract

) 1n (i.e., 01n),1n(f)i(T)1n(T :1-nn2n

0i

0nfor )0(f

1n)1n(f)n(f)1n(T2)n(T :Rearrange

Now there is only one appearance of T(.) on the RHS. Continue with conventional methods.

0n),n(f)i(T)n(T :1 Example1n

0i

A Solution Method:

53

Variable Substitution: Example 1Sometimes we can considerably simplify the recurrence by a change of variable.

T(n) = T(n/2) + log n [assume the usual boundary condition T(O(1))=O(1).]

Change of variable: n = 2m, i.e., m = log n.

Now T(n) = T(2m). It is some function of m. Rename it S(m). That is, T(n) = T(2m) = S(m).

The recurrence becomes:

S(m) = S(m-1) + m.

By the iteration method: S(m) = Q(m2).So, T(n) = T(2m) = S(m) = Q(m2) = Q(log2 n).

T(n) = Q(log2 n). 54

Variable Substitution: How?

T( n ) = T( n/2 ) + f(n)

Recurrence

T(n) = 0 for n < 1

Boundary Condition

Now back substitute:T(n) = T(g(m)) = S(m) = S(g-1(n)), where g-1 is functional inverse of g.

See another example on the next page

g(m) g(m-1)

Rename: n = g(m), n/2 = g(m-1), S(m) = T(g(m)), h(m) = f(g(m))

g(m)

Now solve 2 recurrences:1) g(m) = 2g(m-1), g(m) = 2mg(0) = 2m .

g(0) = 1 our choice

2) S(m) = S(m-1) + h(m) S(m) = h(0)+h(1)+ ··· +h(m). S(0) = T(g(0)) = T(1) = f(1) = h(0)

55

Variable Substitution: Example 2

T() = T() + 1

g(m) g(m-1)

Rename: n = g(m), = g(m-1), S(m) = T(g(m)).

m

gmgmgmgmg 2222 )0()3()2()1()(32

g(0) = 2 n = g(m) = 22m m = log log n.

S(m) = S(m-1)+1 S(m) = Q(m) = Q(log log n).S(0) = T(2) = Q(1)

T(n) = Q(log log n).

56

Exercises

57

1. For each pseudo-code below derive the simplified asymptotic running time in Q(?) notation.

(a) for i 1 .. n do for j 1 .. 2*i do print( i+j )

(c) for i 1 .. n do j 1 while j i do j j+3

(e) for i 1 .. n do j 1 while j i do j j+j

(g) for i 1 .. n do j 2 while j n do j j*j

(i) for i 1 .. n do j 1 while i*j n do j j+1

(k) for i 1 .. n do j n while i < j*j do j j – 2

(b) for i 1 .. n do for j 2*i .. n do print( i+j )

(d) for i 1 .. n do j 1 while i+j n do j j+3

(f) for i 1 .. n do j 1 while j*j i do j j+1

(h) for i 1 .. n do j 2 while j i do j j*j

(j) for i 1 .. n do j 1 while i*i j do j j+1

(l) for i 1 .. n do j n while i j*j do j j div 2 58

•

59

4. The Master Theorem: CLRS versus these Slides:

The case “f(n)/h(n) = W(n+e)” of the Master Theorem [page 94 of CLRS]

has the following extra condition:

(*) if af(n/b) < cf(n) for some constant c < 1 and all sufficiently large n.

The same theorem on page 40 of these slides instead uses the condition:

(**) f(n) is a SAF.

Show that (**) implies (*) in the case “f(n)/h(n) = W(n+e)” .[Hint: Consider the SAF r(n) = f(n)/h(n) = W(n+e). So, r(bi) = W(b+ie) is increasing at least

exponentially as a function of i. Then, with some extra derivation, show that r(n/b) / r(n) < c < 1

for some constant c and all sufficiently large n. c = b– /2e works. ]

5. Exercise 6 of Lecture Note 3 (page 9) solves the recurrence T(n) = 3T(n/3 +5) + n/2

by the guess-&-verify method. Such recurrences can also be solved by the

variable substitution method. Consider the somewhat generalized recurrence

T(n) = a T(n/b + c) + f(n), where a>0, b>1, and c are constants, and f(n) is the

driving function which is a SAF.

a) Show the substitution S(n) = T(n + cb/(b-1)) results in the new recurrence S(n) = aS(n/b) + f(n + cb/(b-1)). [Now this can be solved by the Master Theorem.]

b) Use this method to solve the recurrence T(n) = 4T(n/2 +7) + n2.60

•

niTinTn

i

1

0)()( (m)

61

•

62

(n)

8. Show that the recurrence

has the solution:

9. Show that the recurrence

for any real constants 0 a < 1, 0 b < 1, c > 0, > 0, has the following solutions:

a) + < 1 a b T(n) = Q(n) b) + = 1 a b T(n) = Q(n log n) c) + > 1 a b T(n) = Q(nd), where d > 1 is the unique constant

that satisfies the equation: ad + bd = 1.

63

•

64

12. Derive the most simplified answers to the following summations.Mention which methods you used to derive your solution.a) b) c) d)

e) .

65

13. Recursion Time Analysis:A certain recursive algorithm takes an input list of elements. Divides the list into sub-lists, each with . Recursively solves each of these smaller sub-instances. Then spends an additional time to combine the solutions of these sub-instances to obtain the solution of the main instance. As a base case, if the size of the input list is at most a specified positive constant, then the algorithm solves such a small instance directly in time.a) Express the recurrence relation that governs the time complexity of this algorithm.b) Derive the solution to this recurrence relation: . Mention which methods you used to derive your solution.

66

END

67

Documents

CSE 3101