1 Recursion, Recurrences and Induction Supplementary Notes Prepared by Raymond Wong Presented by...

Recursion, Recurrences and Induction

Supplementary Notes

Prepared by Raymond WongPresented by Raymond Wong

e.g.1 (Page 7) Tower of Hanoi

When n = 1 (i.e., no. of disks = 1) I want to move all n disks from i to j

i j{i, j}

e.g.1 Tower of Hanoi

i j{i, j}

We move ONE disk from i to j.

No. of disk moves = 1

Disk move:Moving ONE disk from i to j

i j{i, j}

Top disk

Bottom disk

i j{i, j}

We move ONE top disk from i to {i, j}.

Disk move:Moving ONE top disk from i to {i, j}

i j{i, j}

We move ONE bottom disk from i to j.

Moving ONE bottom disk from i to j

i j{i, j}

We move ONE top disk from {i, j} to j.

Moving ONE top disk from {i, j} to j

n can be any positive integer I want to move all n disks from i to j

i j{i, j}

Top disk group

containing n-1 disks

Bottom disk

Virtual top box

i j{i, j}

We move ONE top disk group from i to {i, j}.

No. of disk moves = no. of disk moves for one top disk group

Disk move:Moving ONE top disk group from i to {i, j}

i j{i, j}

We move ONE top disk group from {i, j} to j.

Moving ONE top disk group from {i, j} to j

+ no. of disk moves for one top disk group

i j{i, j}

No. of disk moves = 2.no. of disk moves for one top disk group

No. of disk moves for n

No. of disk moves for n-1

The top disk group corresponds to top n-1 disks

top n-1 disks

The bottom disk corresponds to the largest disk

the largest disk

Let M(n) be the total number of disk moves needed for n disks.

M(n) M(n-1)

Note that M(1) = 1

top n-1 disks

the largest disk

M(n) M(n-1)

Why is it correct?

This is because we are implicitly using induction.We assume that we can move top n-1 disks correctly.

Let p(n) is a statement that the algorithm is correct for n.

Step 1: Prove that p(1) (i.e., the base case) is true.

Step 2: Prove that “p(n-1) p(n)” is true for all n > 1.Step 2(a): Assume that p(n-1) is true for n > 1.

Verify that p(1) is true

Step 2(b): According to p(n-1), we deduce that p(n) is true.

Inductive Hypothesis

Inductive Step

Note that M(1) = 1

top n-1 disks

the largest disk

M(n) M(n-1)

Why is it correct?

This is because we are implicitly using induction.We assume that we can move top n-1 disks correctly.

Let p(n) is a statement that the algorithm is correct for n.

Note that M(1) = 1

i j{i, j}

This case is obviously true.

i j{i, j}

We move ONE top disk group from i to {i, j}.

We assume that moving n-1 disks is

correct.

i j{i, j}

We assume that moving n-1 disks is

correct.

e.g.2 (Page 11) Given M(1) = 1 and

M(n) = 2M(n-1) + 1 for n > 1 What are the values of M(1), M(2), M(3), M(4) and M(5)?M(1) = 1

M(2) = 2M(1) + 1 = 2.1 + 1 = 3

M(3) = 2M(2) + 1 = 2.3 + 1 = 7

M(4) = 2M(3) + 1 = 2.7 + 1 = 15

M(5) = 2M(4) + 1 = 2.15 + 1 = 31

21 - 1

22 - 1

23 - 1

24 - 1

25 - 1

It “seems” that M(n) = 2n-1

e.g.3 (Page 12)

Illustration of “Proof by mathematical induction” (Weak Induction)

We are going to prove the following claim C: statement P(n) is true for each positive integer n, namely 1, 2, …

P(1) true

Step 1: Prove that P(1) (i.e., the base case) is true.

Step 2: Prove that “P(n-1) P(n)” is true for all n > 1.

Step 2(a): Assume that P(n-1) is true for n > 1.

Verify that P(1) is true

Step 2(b): According to P(n-1), we deduce that P(n) is true.

Inductive Hypothesis

Inductive Step

e.g.4 (Page 17) Let S(n) be the number of subsets of a set X of size n. What are the values of S(0), S(1), S(2) and S(3)?

When n = 0, X = {} There is a subset, namely an empty set (i.e., {})

When n = 1, X = {1} There are 2 subsets, namely {} and {1}.

Thus, S(0) = 1

Thus, S(1) = 2

When n = 2, X = {1, 2}There are 4 subsets, namely {}, {1}, {2}, {1, 2}.Thus, S(2) = 4

When n = 3, X = {1, 2, 3}There are 8 subsets, namely {},{1},{2},{3},{1, 2},{1, 3},{2, 3},{1, 2, 3}

Thus, S(3) = 8

It “seems” that S(n) = 2n

e.g.4Consider the 8 subsets of {1, 2, 3}

{} {1} {2} {1, 2}

{3} {1, 3} {2, 3} {1, 2, 3}

All 4 possible subsets of {1, 2}

Each set is the same as the above set by adding a number 3

All possible subsets of a set of size 2.

The total number of such subsets isequal to S(2).

All possible subsets of a set of size 2 by adding a number 3

The total number of such subsets isequal to S(2).

The total number of subsets of a set of size 3 (i.e., S(3) ) is equal to2.S(2)

In general, we derive that S(n) = 2.S(n-1)

We want to find a recursion formula for S(n).

e.g.5 (Page 21)

0 1 2 3 n-1 n…

Raymond borrows a loan with initial amount A (e.g., $100,000) from the bank.

Raymond needs to return a monthly payment M (e.g., $1000) every month.

Raymond returns a monthly payment M (e.g., $1000) to the bank.

Raymond returns M

Suppose T(n) is the amount of money in the bank at the n-th month.

T(0) T(1) T(2) T(3) T(n-1) T(n)

0 1 2 3 n-1 n…

Raymond borrows a loan with initial amount A (e.g., $100,000) from the bank.

Raymond needs to return a monthly payment M (e.g., $1000) every month.

Raymond returns a monthly payment M (e.g., $1000) to the bank.

Raymond returns M

Suppose T(n) is the amount of money in the bank at the n-th month.

T(0) T(1) T(2) T(3) T(n-1) T(n)=A =T(0)*(1+0.01p/12) – M

Suppose the interest rate is p% per year

Suppose the interest rate is (p/12)% per month.

Suppose the interest rate is (0.01p/12) per month.

=T(1)*(1+0.01p/12) – M

=T(n-1)*(1+0.01p/12) – M

e.g.6 (Page 24)

T(0) = bT(n) = rT(n-1) + a if n > 0

Derive a closed form for T(n) by a “top-down” approach (or called iterating recurrence)

T(0) = bT(n) = rT(n-1) + a if n > 0

T(n) = rT(n-1) + a = r(rT(n-2) + a) + a

= r2T(n-2) + ra + a = r2(rT(n-3) + a) + ra + a = r3T(n-3) + r2a + ra + a = r3(rT(n-4) + a) + r2a + ra + a = r4T(n-4) + r3a + r2a + ra + a

= …= rnT(n-n) + rn-1a + rn-2a + … + ra + a

= rnT(0) + a(rn-1 + rn-2 + … + r + 1)

= rnT(0) + a(rn-1 + rn-2 + … + r + r0)

= rnb + a ri

Note that

=1 – rn 1 – r

if r 1= rnb + a 1 – rn

1 – r

e.g.7 (Page 25)

T(0) = bT(n) = rT(n-1) + a if n > 0

Derive a closed form for T(n) by a “bottom-up” approach

T(0) = bT(n) = rT(n-1) + a if n > 0

T(0) = b

T(1) = rT(0) + a = rb + a

T(2) = rT(1) + a = r(rb+a) + a = r2b + ra + a

T(3) = rT(2) + a = r(r2b + ra + a) + a = r3b + r2a+ ra + a

T(n) = rnb + rn-1a + rn-2a + … + ra + a

= rnb + a(rn-1 + rn-2 + … + r + 1)= rnb + a(rn-1 + rn-2 + … + r + r0)

= rnb + a ri

Note that

=1 – rn 1 – r

if r 1= rnb + a 1 – rn

1 – r

e.g.7 (Page 31)

We want to prove the following corollary.Corollary 4.2: The formula for the sum of a geometric series with r 1 is

=1 – rn 1 – r

Corollary 4.2: The formula for the sum of a geometric series with r 1 is

=1 – rn 1 – r

Theorem 4.1 is shown as follows.

Theorem 4.1:If T(0) = b T(n) = rT(n-1) + a if n > 0then = rnb + a 1 – rn

1 – rT(n)

Let T(0) = 0

Let T(n) = ri

for n > 0

We want to show that T(n) = 1 – rn 1 – r

by using Theorem 4.1 (which is relatedto recursion).

We want to write T(n) in a recursion form. Consider T(n) = rn-1 + rn-2 + … + r + r0

= (rn-1 + rn-2 + … + r) + r0

= r(rn-2 + rn-3 + … + r0) + 1= r T(n-1) + 1 This is in a recursion

form.We know that b = 0We know that a = 1

According to Theorem 4.1, we have

= rnb + a 1 – rn 1 – r

= rn0 + 11 – rn 1 – r

1 – rn 1 – r

e.g.8 (Page 32) We want to prove the following

theorem.Lemma 4.3: Let r 1 be a positive value independent of n.Let t(n) be the largest term in the geometric series

Then, the value of the geometric series is O(t(n)).

e.g.,If r = 0.5, then we have the value V of geometric series is equal to 0.50 + 0.51 + 0.52 + … + 0.5n-1

1 + 0.5 + 0.25 + … + 0.5n-1

The largest term t(n) in this series is equal to 1 (= 0.50)

e.g.,If r = 2, then we have the value V of geometric series is equal to 20 + 21 + 22 + … + 2n-1

1 + 2+ 2 + … + 2n-1

The largest term t(n) in this series is equal to 2n-1

Lemma 4.3 states that V = O(t(n)) = O(1)

Lemma 4.3 states that V = O(t(n)) = O(2n-1)

Why is it correct?

Lemma 4.3: Let r 1 be a positive value independent of n.Let t(n) be the largest term in the geometric series

Consider two case.

Case 1: r < 1

Case 2: r > 1

From our previous slide, we know that t(n) = r0 = 1

From our previous slide, we know that t(n) = rn-1

Consider Case 1

We know that =1 – rn 1 – r

1 – r

Note that r is a constant. 1

1 – rThus, is a constant.

= O(1)

= O(t(n))

Note that we can show that = (t(n)) ri

Lemma 4.3: Let r 1 be a positive value independent of n.Let t(n) be the largest term in the geometric series

Consider two case.

Case 1: r < 1

Case 2: r > 1

From our previous slide, we know that t(n) = r0 = 1

From our previous slide, we know that t(n) = rn-1

Consider Case 2

We know that =1 – rn 1 – r

Note that r is a constant. r

1 – rThus, is a constant.

= O(rn-1)= O(t(n))

=rn – 1 r – 1

r – 1

r – 1rn-1

Note that we can show that = (t(n)) ri

e.g.9 (Page 35) First-order Linear Recurrence

Dependent on one-step backward (i.e., T(n-1))

E.g., T(n) = f(n)T(n-1) + g(n) Second-order Linear Recurrence

Dependent on two-steps backward (i.e., T(n-2))E.g., T(n) = T(n-1) +2T(n-2)

First-order Non-Linear Recurrence Linear means the power of T(n-1) is 1. Non-linear means the power of T(n-1) is not 1. E.g., T(n) = (T(n-1))2 + 3

We are interested in this recurrence where f(n) is equal to a constant r.

e.g.10 (Page 37)

Prove the following Theorem 4.5 by induction.

Theorem 4.5: For any positive constants a and r, and any function g defined on the non-negative integers, the solution to the first-order linear recurrence

isT(n) =

rT(n-1) + g(n) if n > 0,a if n = 0,

T(n) = rna + g(i) (*)

e.g.10

isT(n) =

rT(n-1) + g(n) if n > 0,a if n = 0,

T(n) = rna + g(i) (*)

Step 1: Prove that P(0) (i.e., the base case) is true.

Let P(n) be “ ” rn-i

T(n) = rna + g(i)

T(0) = a (by definition)

= r0aWe want to show that

T(0) = r0a + g(i)

That is, T(0) = r0a

Thus, P(0) is true.

e.g.10

isT(n) =

rT(n-1) + g(n) if n > 0,a if n = 0,

T(n) = rna + g(i) (*)

T(n) = rna + g(i)

Step 2: Prove that “P(n-1) P(n)” is true for all n > 0.Step 2(a): Assume that P(n-1) is true for n > 0.

r(n-1)-i

T(n-1) = rn-1a + g(i)That is,

Consider T(n) = rT(n-1) + g(n) (by definition) r(n-1)-i

rn-1a + g(i)= r( ) + g(n)

We want to show that

T(n) = rna + g(i)

r(n-1)-i

rna + r g(i)= + g(n)

r1+(n-1)-i

rna + g(i)= + g(n)

rna + g(i)= + rn-ng(n)

rna + g(i)=

e.g.10

isT(n) =

rT(n-1) + g(n) if n > 0,a if n = 0,

T(n) = rna + g(i) (*)

T(n) = rna + g(i)

Step 2: Prove that “P(n-1) P(n)” is true for all n > 0.Step 2(a): Assume that P(n-1) is true for n > 0.

r(n-1)--i

T(n-1) = rn-1a + g(i)That is,

We want to show that

T(n) = rna + g(i)

Thus, P(n) is true.We prove that “P(n-1) P(n)” is true for all n > 0

By Mathematical Induction,

T(n) = rna + g(i)

e.g.11 (Page 39)

We want to find the closed form of T(n)where

T(0) = 6 T(n) = 4T(n-1) + 2n if n > 0

T(0) = 6T(n) = 4T(n-1) + 2n if n > 0

e.g.11

T(0) = 6T(n) = 4T(n-1) + 2n if n > 0

isT(n) =

rT(n-1) + g(n) if n > 0,a if n = 0,

T(n) = rna + g(i) (*)

We can write the closed form directly by using Theorem 4.5.

We have r = 4 g(n) = 2n a= 6

Thus, by using Theorem 4.5, we have

T(n) = rna + g(i)

i 1= 4n.6 + 4n-i . 2i

i 1= 4n.6 +4n 4-i . 2i

e.g.11

T(0) = 6T(n) = 4T(n-1) + 2n if n > 0

We have r = 4 g(n) = 2n a= 6

T(n) = rna + g(i)

i 1= 4n.6 + 4n-i . 2i

i 1= 4n.6 +4n 4-i . 2i

i 1= 4n.6 +4n (22)-i . 2i

i 1= 4n.6 +4n 2-2i . 2i

i 1= 4n.6 +4n 2-i

i 1= 4n.6 +4n

2( ) i

i 1= 4n.6 +4n

2( ) i-1

i= 4n.6 +4n

2( ) i

= 4n.6 +4n . 1

1 – (½)n

1 – ½

= 4n.6 +4n . 1

1 – (½)n

= 4n.6 +4n . (1 – (½)n)

= 4n.6 +4n – 4n(½)n

= 4n.7 – 22n . 2-n = 4n.7 – 2n

e.g.12 (Page 40)

We want to find the closed form of T(n)where

T(0) = 10 T(n) = 3T(n-1) + n if n > 0

T(0) = 10T(n) = 3T(n-1) + n if n > 0

e.g.12

T(0) = 10T(n) = 3T(n-1) + n if n > 0

isT(n) =

rT(n-1) + g(n) if n > 0,a if n = 0,

T(n) = rna + g(i) (*)

We have r = 3 g(n) = n a= 10

T(n) = rna + g(i)

i 1= 3n.10 + 3n-i . i

i 1= 3n.10 + 3n 3-i . i

i 1= 3n.10 + 3n i . 3-i

e.g.12

T(0) = 10T(n) = 3T(n-1) + n if n > 0

We have r = 3 g(n) = n a= 10

T(n) = rna + g(i)

i 1= 3n.10 + 3n-i . i

i 1= 3n.10 + 3n 3-i . i

i 1= 3n.10 + 3n i . 3-i

Theorem 4.6: For any real number x 1,

i 1i . xi =

nxn+2 – (n+1)xn+1 + x

(1-x)2

Suppose that we know the following theorem.

i 1i . 3-i =

i 1i . (1/3)i

n(1/3)n+2 – (n+1)(1/3)n+1 + (1/3)

(1-1/3)2=

n.3-(n+2) – (n+1)3-(n+1) + 3-1

(2/3)2=

. (n.3-(n+2) – (n+1)3-(n+1) + 3-1)=32

e.g.12

T(0) = 10T(n) = 3T(n-1) + n if n > 0

We have r = 3 g(n) = n a= 10

T(n) = rna + g(i)

i 1= 3n.10 + 3n-i . i

i 1= 3n.10 + 3n 3-i . i

i 1= 3n.10 + 3n i . 3-i

i 1i . xi =

nxn+2 – (n+1)xn+1 + x

(1-x)2

i 1i . 3-i . (n.3-(n+2) – (n+1)3-(n+1) + 3-1)=

. (n.3-(n+2)+2 – (n+1)3-(n+1)+2 + 3-1+2)=1

. (n.3-n – (n+1)3-n+1 + 31)=1

4. (n.3-n – (n+1)3.3-n + 3)=

e.g.12

T(0) = 10T(n) = 3T(n-1) + n if n > 0

We have r = 3 g(n) = n a= 10

T(n) = rna + g(i)

i 1= 3n.10 + 3n-i . i

i 1= 3n.10 + 3n 3-i . i

i 1= 3n.10 + 3n i . 3-i

i 1i . xi =

nxn+2 – (n+1)xn+1 + x

(1-x)2

i 1i . 3-i . (n.3-n – (n+1)3.3-n + 3)=

. (n.3-n – (3n+3).3-n + 3)=1

. (n.3-n – 3n.3-n – 3.3-n + 3)=1

. (– 2n.3-n – 3.3-n + 3)=1

e.g.12

T(0) = 10T(n) = 3T(n-1) + n if n > 0

We have r = 3 g(n) = n a= 10

T(n) = rna + g(i)

i 1= 3n.10 + 3n-i . i

i 1= 3n.10 + 3n 3-i . i

i 1= 3n.10 + 3n i . 3-i

i 1i . xi =

nxn+2 – (n+1)xn+1 + x

(1-x)2

i 1i . 3-i . (– 2n.3-n – 3.3-n + 3)=

. [(– 2n – 3).3-n + 3]=1

=–2n – 3

43-n +

i 1i . 3-i =

–2n – 3

43-n +

e.g.12

T(0) = 10T(n) = 3T(n-1) + n if n > 0

We have r = 3 g(n) = n a= 10

T(n) = rna + g(i)

i 1= 3n.10 + 3n-i . i

i 1= 3n.10 + 3n 3-i . i

i 1= 3n.10 + 3n i . 3-i

i 1i . 3-i =

–2n – 3

43-n +

= 3n.10 + 3n (–2n – 3

43-n +

= 3n.10 +–2n – 3

4+ 3n . 3

= 3n.(10 + ) – 2n + 3

= 3n. – 2n + 3

1 Recursion, Recurrences and Induction Supplementary Notes Prepared by Raymond Wong Presented by...

Documents

Asymptotic Notation, Recurrences: Substitution, …sourav/Lecture-02.pdfAsymptotic Notation, Recurrences: Substitution, Iteration, Master Method Lecture 2 L2.2 Solving recurrences

1 Intro to Crypto and Mod Supplementary Notes Prepared by Raymond Wong Presented by Raymond Wong

Introduction to stem cells Stem cell Community Forum Raymond Wong, PhD Neuroregeneration Research Unit, CERA

Recurrences – II

Recurrences: Methods and Examples

Wong raymond symposium2016_present

1 Basic Counting Supplementary Notes Prepared by Raymond Wong Presented by Raymond Wong

Merge Sort and Recurrences - UTKweb.eecs.utk.edu/.../2-Jan-14-Mergesort-Recurrences-no-answers.pdf · Merge Sort and Recurrences . COSC 581, ... using merge sort. • Combine: Merge

Recurrences

1 Advanced Induction Supplementary Notes Prepared by Raymond Wong Presented by Raymond Wong

COMP53311 Data Warehouse Prepared by Raymond Wong Presented by Raymond Wong raywong@cse

1 Inverses and GCDs Supplementary Notes Prepared by Raymond Wong Presented by Raymond Wong

1 Binomial Coefficient Supplementary Notes Prepared by Raymond Wong Presented by Raymond Wong

1 Quantifiers Supplementary Notes Prepared by Raymond Wong Presented by Raymond Wong

1 Independence Supplementary Notes Prepared by Raymond Wong Presented by Raymond Wong

1 Variance of RVs Supplementary Notes Prepared by Raymond Wong Presented by Raymond Wong

1 Random Variables Supplementary Notes Prepared by Raymond Wong Presented by Raymond Wong

Recurrences Slides

Polar Bears Cecilia Mak Jacqueline Wong Raymond Yang Helen Li Period 3, Life Science MS. KWON

1 Inclusion-Exclusion Supplementary Notes Prepared by Raymond Wong Presented by Raymond Wong