Compare the growth rate of functionshuding/331material/notes2.pdfOutline 1 Compare the growth rate...

Outline

1 Compare the growth rate of functions

2 Limit Test

3 L’Hospital Rule

4 Stirling Formula

c©Hu Ding (Michigan State University) CSE 331 Algorithm and Data Structures 1 / 18

Compare the growth rate of functions

We have two algorithms A1 and A2 for solving the same problem,with runtime functions T1(n) and T2(n), respectively. Whichalgorithm is more efficient?We compare the growth rate of T1(n) and T2(n).If T1(n) = Θ(T2(n)), then the efficiency of the two algorithms areabout the same (when n is large).If T1(n) = o(T2(n)), then the efficiency of the algorithm A1 will bebetter than that of algorithm A2 (when n is large).By using the definitions, we can directly show whetherT1(n) = O(T2(n)), or T1(n) = Ω(T2(n)). However, it is not easy toprove the relationship of two functions in this way.

We have two algorithms A1 and A2 for solving the same problem,with runtime functions T1(n) and T2(n), respectively. Whichalgorithm is more efficient?

We compare the growth rate of T1(n) and T2(n).If T1(n) = Θ(T2(n)), then the efficiency of the two algorithms areabout the same (when n is large).If T1(n) = o(T2(n)), then the efficiency of the algorithm A1 will bebetter than that of algorithm A2 (when n is large).By using the definitions, we can directly show whetherT1(n) = O(T2(n)), or T1(n) = Ω(T2(n)). However, it is not easy toprove the relationship of two functions in this way.

We have two algorithms A1 and A2 for solving the same problem,with runtime functions T1(n) and T2(n), respectively. Whichalgorithm is more efficient?We compare the growth rate of T1(n) and T2(n).

If T1(n) = Θ(T2(n)), then the efficiency of the two algorithms areabout the same (when n is large).If T1(n) = o(T2(n)), then the efficiency of the algorithm A1 will bebetter than that of algorithm A2 (when n is large).By using the definitions, we can directly show whetherT1(n) = O(T2(n)), or T1(n) = Ω(T2(n)). However, it is not easy toprove the relationship of two functions in this way.

We have two algorithms A1 and A2 for solving the same problem,with runtime functions T1(n) and T2(n), respectively. Whichalgorithm is more efficient?We compare the growth rate of T1(n) and T2(n).If T1(n) = Θ(T2(n)), then the efficiency of the two algorithms areabout the same (when n is large).

If T1(n) = o(T2(n)), then the efficiency of the algorithm A1 will bebetter than that of algorithm A2 (when n is large).By using the definitions, we can directly show whetherT1(n) = O(T2(n)), or T1(n) = Ω(T2(n)). However, it is not easy toprove the relationship of two functions in this way.

We have two algorithms A1 and A2 for solving the same problem,with runtime functions T1(n) and T2(n), respectively. Whichalgorithm is more efficient?We compare the growth rate of T1(n) and T2(n).If T1(n) = Θ(T2(n)), then the efficiency of the two algorithms areabout the same (when n is large).If T1(n) = o(T2(n)), then the efficiency of the algorithm A1 will bebetter than that of algorithm A2 (when n is large).

By using the definitions, we can directly show whetherT1(n) = O(T2(n)), or T1(n) = Ω(T2(n)). However, it is not easy toprove the relationship of two functions in this way.

We have two algorithms A1 and A2 for solving the same problem,with runtime functions T1(n) and T2(n), respectively. Whichalgorithm is more efficient?We compare the growth rate of T1(n) and T2(n).If T1(n) = Θ(T2(n)), then the efficiency of the two algorithms areabout the same (when n is large).If T1(n) = o(T2(n)), then the efficiency of the algorithm A1 will bebetter than that of algorithm A2 (when n is large).By using the definitions, we can directly show whetherT1(n) = O(T2(n)), or T1(n) = Ω(T2(n)). However, it is not easy toprove the relationship of two functions in this way.

Outline

2 Limit Test

3 L’Hospital Rule

4 Stirling Formula

Limit TestLimit Test is a powerful method for comparing functions.

Limit TestLet T1(n) and T2(n) be two functions. Let c = limn→∞

T1(n)T2(n) .

1 If c is a constant > 0, then T1(n) = Θ(T2(n)).

2 If c = 0, then T1(n) = o(T2(n)).

3 If c =∞, then T1(n) = ω(T2(n)).

4 If c does not exists (or if we do not know how to compute c), the limit testfails.

Proof of (1): c = limn→∞T1(n)T2(n) means: ∀ε > 0, there exists n0 ≥ 0 such that for

any n ≥ n0:∣∣∣ T1(n)

T2(n) − c∣∣∣ ≤ ε; or equivalently: c− ε ≤ T1(n)

T2(n) ≤ c + ε. Let ε = c/2and let c1 = c− ε = c/2 and c2 = c + ε = 3c/2, we have

c1T2(n) ≤ T1(n) ≤ c2T2(n)

for all n ≥ n0. Thus T1(n) = Θ(T2(n)) by definition.

Limit TestLimit Test is a powerful method for comparing functions.

Limit TestLet T1(n) and T2(n) be two functions. Let c = limn→∞

T1(n)T2(n) .

1 If c is a constant > 0, then T1(n) = Θ(T2(n)).

2 If c = 0, then T1(n) = o(T2(n)).

3 If c =∞, then T1(n) = ω(T2(n)).

4 If c does not exists (or if we do not know how to compute c), the limit testfails.

Proof of (1): c = limn→∞T1(n)T2(n) means: ∀ε > 0, there exists n0 ≥ 0 such that for

any n ≥ n0:∣∣∣ T1(n)

T2(n) − c∣∣∣ ≤ ε; or equivalently: c− ε ≤ T1(n)

T2(n) ≤ c + ε. Let ε = c/2and let c1 = c− ε = c/2 and c2 = c + ε = 3c/2, we have

c1T2(n) ≤ T1(n) ≤ c2T2(n)

for all n ≥ n0. Thus T1(n) = Θ(T2(n)) by definition.c©Hu Ding (Michigan State University) CSE 331 Algorithm and Data Structures 4 / 18

Example

Example 1

T1(n) = 10n2 + 15n− 60, T2(n) = n2

limn→∞

T2(n)= lim

n→∞

10n2 + 15n− 60n2 = lim

n→∞(10+

15n−60

n2 ) = 10+0−0 = 10

Since 10 is a constant > 0, we have T1(n) = Θ(T2(n)) = Θ(n2) by thestatement 1 of Limit Test (as expected).

Log function

The log functions are very useful in algorithm analysis.

lg = log2 n

log n = log10 n

ln n = loge n

(ln n is the log function with the natural base e = 2.71828 . . .).

Log function

The log functions are very useful in algorithm analysis.

lg = log2 n

log n = log10 n

ln n = loge n

(ln n is the log function with the natural base e = 2.71828 . . .).

Log base change formula

For any 1 < a, b, logb n =loga nloga b = logb a · loga n.

Proof: Let k = logb n. By definition: n = bk.

Take loga on both sides: loga n = loga(bk) = k · loga bThis implies: logb n = k =

loga nloga b .

Let n = a in this formula and note 1 = loga a:

logb a =loga aloga b

loga b

This proves the second part of the formula.

Proof: Let k = logb n. By definition: n = bk.Take loga on both sides: loga n = loga(bk) = k · loga b

This implies: logb n = k =loga nloga b .

loga b

Proof: Let k = logb n. By definition: n = bk.Take loga on both sides: loga n = loga(bk) = k · loga bThis implies: logb n = k =

loga nloga b .

loga b

loga nloga b .

loga b

loga nloga b .

loga b

Outline

2 Limit Test

3 L’Hospital Rule

4 Stirling Formula

L’Hospital Rule

L’Hospital RuleIf limn→∞ f (n) = 0 and limn→∞ g(n) = 0, then

limn→∞

g(n)= lim

n→∞

f ′(n)

g′(n)

If limn→∞ f (n) =∞ and limn→∞ g(n) =∞, then

limn→∞

g(n)= lim

n→∞

f ′(n)

g′(n)

L’Hospital Rule

L’Hospital RuleIf limn→∞ f (n) = 0 and limn→∞ g(n) = 0, then

limn→∞

g(n)= lim

n→∞

f ′(n)

g′(n)

If limn→∞ f (n) =∞ and limn→∞ g(n) =∞, then

limn→∞

g(n)= lim

n→∞

f ′(n)

g′(n)

Example

Example 2T1(n) = n2 + 6, T2(n) = n lg n. (Recall: lg n = log2 n.)

limn→∞T1(n)T2(n)

= limn→∞n2+6n lg n = limn→∞

= limn→∞1− 6

ln 2·n(by L’Hospital Rule)

= ln 2 limn→∞(n− 6n) = ln 2(∞− 0) =∞

By Limit Test, we have n2 + 6 = ω(n lg n).

Example

Example 3T1(n) = (ln n)k, T2(n) = nε, where k > 0 is any (large) constant andε > 0 is any (small) constant. (Recall: ln n = loge n.)

= limn→∞(ln n)k

nε (use L’Hospital Rule)

= limn→∞k(ln n)k−1×(1/n)

εn(ε−1)

= kε limn→∞

(ln n)k−1

nε (use L’Hospital Rule again and simplify)= k(k−1)

ε2 limn→∞(ln n)k−2

nε (use L’Hospital Rule k times).....

= k(k−1)···2·1εk limn→∞

1nε = 0

So by Limit Test, (ln n)k = o(nε) for any k and ε. For example, takek = 100 and ε = 0.01, we have (ln n)100 = o(n0.01).

Example

Example 3T1(n) = (ln n)k, T2(n) = nε, where k > 0 is any (large) constant andε > 0 is any (small) constant. (Recall: ln n = loge n.)

= limn→∞(ln n)k

nε (use L’Hospital Rule)

= limn→∞k(ln n)k−1×(1/n)

εn(ε−1)

= kε limn→∞

(ln n)k−1

nε (use L’Hospital Rule again and simplify)= k(k−1)

ε2 limn→∞(ln n)k−2

nε (use L’Hospital Rule k times).....

= k(k−1)···2·1εk limn→∞

1nε = 0

So by Limit Test, (ln n)k = o(nε) for any k and ε. For example, takek = 100 and ε = 0.01, we have (ln n)100 = o(n0.01).

Example

Example 4T1(n) = nk, T2(n) = an, where k > 0 is any (large) constant and a > 1 isany constant bigger than 1.

= limn→∞nk

an (using L’Hospital Rule)

= limn→∞k·nk−1

ln a·an = kln a limn→∞

nk−1

an ( using L’Hospital Rule k times)= k(k−1)···2·1

(ln a)k limn→∞n0

= k(k−1)···2·1(ln a)k limn→∞

1an = 0

So by Limit Test, nk = o(an) for any k > 0 and a > 1. For example, takek = 1000 and a = 1.001, we have n1000 = o((1.001)n).

Example

Example 4T1(n) = nk, T2(n) = an, where k > 0 is any (large) constant and a > 1 isany constant bigger than 1.

= limn→∞nk

an (using L’Hospital Rule)

= limn→∞k·nk−1

ln a·an = kln a limn→∞

nk−1

an ( using L’Hospital Rule k times)= k(k−1)···2·1

(ln a)k limn→∞n0

= k(k−1)···2·1(ln a)k limn→∞

1an = 0

So by Limit Test, nk = o(an) for any k > 0 and a > 1. For example, takek = 1000 and a = 1.001, we have n1000 = o((1.001)n).

Example

Example 5T1(n) = loga n, T2(n) = logb n, where a > 1 and b > 1 are any twoconstants bigger than 1.

By the Log Base Change Formula: logb n = logb a · loga n

Thus: limn→∞T1(n)T2(n)

= limn→∞loga nlogb n = limn→∞

loga nlogb a·loga n = 1

logb a

Since 1logb a > 0 is a constant, we have loga n = Θ(logb n) by Limit Test.

So: the growth rates of the log functions are the same for anybase > 1.

Example

By the Log Base Change Formula: logb n = logb a · loga nThus: limn→∞

T1(n)T2(n)

logb a

Example

T1(n)T2(n)

logb a

Example

T1(n)T2(n)

logb a

Example

Example 6T1(n) = an, T2(n) = bn, where 1 < a < b are any two constants.

We have: limn→∞T1(n)T2(n)

= limn→∞an

bn = limn→∞(ab)n = 0.

Thus: an = o(bn) (for any 1 < a < b) by Limit Test.

The list of common functions:The following list shows the functions commonly used in algorithmanalysis, in the order of increasing growth rate (a, b, c, d, k, ε arepositive constants, ε < 1, k > 1, d > 1 and a < b):

c, logd n, (logd n)k, nε, n, nk, an, bn, n!, nn

in the sense that if f (n) and g(n) are any two consecutive functions inthe list, we have f (n) = o(g(n))

Example

= limn→∞an

Example

= limn→∞an

Example

= limn→∞an

Examples

Example 7T1(n) = n! and T2(n) = an (a > 1)

limn→∞an

L’Hospital Rule doesn’t help: We don’t know how to take derivativeof n!

a1· a

2· · · a

2dae︸︷︷︸2dae terms

· a2dae+ 1

· · · an︸︷︷︸

(n−2dae) termsThe first part is a constant c > 0. In the second part, each term < 1/2.So:

0 ≤ limn→∞

n!≤ c · lim

n→∞(12

)(n−2dae) = 0

By Limit Test: an = o(n!).

Examples

limn→∞an

a1· a

2· · · a

· a2dae+ 1

· · · an︸︷︷︸

0 ≤ limn→∞

n!≤ c · lim

n→∞(12

)(n−2dae) = 0

Examples

limn→∞an

a1· a

2· · · a

· a2dae+ 1

· · · an︸︷︷︸

0 ≤ limn→∞

n!≤ c · lim

n→∞(12

)(n−2dae) = 0

Examples

limn→∞an

a1· a

2· · · a

· a2dae+ 1

· · · an︸︷︷︸

(n−2dae) terms

The first part is a constant c > 0. In the second part, each term < 1/2.So:

0 ≤ limn→∞

n!≤ c · lim

n→∞(12

)(n−2dae) = 0

Examples

limn→∞an

a1· a

2· · · a

· a2dae+ 1

· · · an︸︷︷︸

0 ≤ limn→∞

n!≤ c · lim

n→∞(12

)(n−2dae) = 0

Examples

limn→∞an

a1· a

2· · · a

· a2dae+ 1

· · · an︸︷︷︸

0 ≤ limn→∞

n!≤ c · lim

n→∞(12

)(n−2dae) = 0

Examples

limn→∞an

a1· a

2· · · a

· a2dae+ 1

· · · an︸︷︷︸

0 ≤ limn→∞

n!≤ c · lim

n→∞(12

)(n−2dae) = 0

By Limit Test: an = o(n!).c©Hu Ding (Michigan State University) CSE 331 Algorithm and Data Structures 15 / 18

Outline

2 Limit Test

3 L’Hospital Rule

4 Stirling Formula

Stirling Formula

Stirling Formula√

2πn(n

112n+1 ≤ n! ≤

√2πn

or: n! =√

2πn(n

1 + Θ

))When n = 10;

n! = 3628800.√

2πn( n

)n= 3598696, 99% accurate.

Examples

Example 7 (another solution)T1(n) = n! and T2(n) = an (a > 1)

limn→∞

an ≥ limn→∞

√2πn

)n= lim

n→∞

√2πn · lim

n→∞

The first limit is∞. The second limit goes to∞∞. So it’s also∞. Thuslimn→∞

n!an =∞ and n! = ω(an) by limit test.

Example 8T1(n) = nn and T2(n) = n!

By using similar method as in Example 7, we can show: n! = o(nn)

Examples

limn→∞

an ≥ limn→∞

√2πn

)n= lim

n→∞

√2πn · lim

n→∞

Examples

limn→∞

an ≥ limn→∞

√2πn

)n= lim

n→∞

√2πn · lim

n→∞

Compare the growth rate of functionshuding/331material/notes2.pdfOutline 1 Compare the growth rate...

Documents

1 L’Hospital: Self-healing Secure Routing for Mobile …netlab.cs.ucla.edu/wiki/files/CSD-TR040055.pdf1 L’Hospital: Self-healing Secure Routing for Mobile Ad-hoc Networks Jiejun

COLB Compare

1. Data Compare for Oracle 1 documentation...Data Compare for Oracle 1 documentation About Data Compare for Oracle Data Compare for Oracle lets you compare and deploy Oracle table

2 Divide & Conquer Strategy and Merge Sort - cse.msu.eduhuding/331material/notes3.pdf · 2 Divide & Conquer Strategy and Merge Sort ... extra space. c Hu Ding (Michigan State University)

Compare Data Center Tutorial Using TFCompanion to compare spectra

Compare and Contrast...doctor vet Compare and Contrast Name:!_ ©ADP . firefighter police officer Compare and Contrast Name:!_ ©ADP . firefighter police officer Compare and

Public.xml Compare

Compare Fractions

La visió de l’hospital

Db Compare

“L’Hospital Clinic de Barcelona

JUNY-JULIOL DE 2013 martorellmartorell.cat › ca › files › doc830 › af-but-187-.pdf · 2018-10-24 · subtipus de malaltia minoritÀria en una pacient de l’hospital de martorell

กรณีศึกษาการเปรียบเทียบไฟร์วอลล์ Compare Firewalls Case Study Compare Firewalls Case Study.pdf · หัวข้อโครงงาน

Yasenkanna Compare

More Than - The Blade · Online Ordering TOLEDO ... COMPARE SET AT COMPARE SET AT COMPARE SET AT COMPARE SET AT $799 COMPARE SET AT $1399 ... your chairs, sofas, cushions, etc

Access Compare

Uji Compare

Literatur - Springer978-3-540-77314-6/1.pdf · lis mathematischen Vorlesungen über die Methode der Integrale und anderes, aufgeschrieben zum Gebrauch des Herrn Marquis de l’Hospital

Cover Compare

Compare & Contrast