Daaa 2 Asymptotic_ Notation

8/10/2019 Daaa 2 Asymptotic_ Notation

1/28

Dr. Md. Rafiqul Islam INTRODUCTION TO ALGORITHMS Slide #

CHAPTER 2

ASYMPTOTIC NOTATION


2/28

INTRODUCTION TO ALGORITHMSD RAFIQUL ISLAM

2

ASYMPTOTIC NOTATION

To estimate complexity (time and spacecomplexities) of algorithms several notations

are used in literatures.

They are O (Big oh), (theta), (Omega)

notations.

These notations are known as asymptotic

notations.


3/28


3

O-notation

The big-Oh O-Notationasymptotic upper bound

f(n) = O(g(n)),if there exists constants c>0and n0>0, s.t.

f(n)

c g(n)for n n0

f(n)andg(n)are functions over non-

Used for worst-caseanalysis

)(nf( )c g n

0n Input Size

Run

ningTime


4/28


4

O-notation

Example 2.1:The function )(23 nOn as nn 423 for all .2n Here g(n) = n, c= 4.

)(2410 22 nOnn as 22 112410 nnn for all .5n Here g(n) = n2, c=11.

)2(2*6 2 nn On as nn n 2*72*6 2 for .4n

)(33 2nOn as 2333 nn for all .2n )(2410 42 nOnn as

42 102410 nnn for .2n You can go to up. Because, O is for upper bound.

)1(23 On as 23 n is not less than or equal to cfor any constant cand all

.0nn ).(24102

nOnn You cannot go down or lower.


5/28


5

O-notation

Simple Rule:Drop lower order terms and constant factors.

50 n log n is O(n log n)

7n- 3 is O(n)8n2log n + 5n2+ n is O(n2log n)

Note: Although for 50 n log nwe can write

O(n3), it is expected that an approximation is ofthe smallest possible order.


6/28


6

The big-Omega

W-Notation

asymptotic lower bound

f(n) = (g(n))if there exists constants c>0and n0>0, such that

f(n) >= c g(n) for n n0

Used to describe best-case

running times or lower bounds

of algorithmic problems.

E.g., lower-bound of searching

in an unsorted array is (n). Input Size

RunningTime

)(nf

( )c g n

0n

-notation


7/28


7

-notation

Example 2.2:The function )(23 nn W as nn 323 for all 1n (the inequality holds for

,0n but the definition of requires an 00 n ).

)(33 nn W as nn 333 for .1n )(2410 22 nnn W as

22 2410 nnn for .1n )2(2*6 2 nn n W as nn n 22*6 2 for .1n

Observe also that ),1(33 Wn ),(2410 2 nnn W

),1(2410 2 W nn ),(2*6 22 nnn W ),(2*6 2 nnn W and

).1(2*6 2 Wnn You can go down, but you cannot go up as you wish.


8/28


8

The big-Theta Q-Notation

asymptoticly tight bound

f(n) = (g(n))if there exists constants c1>0, c2>0,and n0>0, s.t. for

n

n0c1g(n)

f(n)

c2g(n)

f(n)=Q(g(n)) if and only if

f(n)=O(g(n)), f(n)=W(g(n))

O(f(n)) is often abused

instead of Q(f(n))

-notation

Input Size

Run

ningTime

)(nf

0n

)(ngc 2

)(ngc 1


9/28


9

-notation

Example 2.3:

The function )(23 nn Q as nn 323 for all 2n and nn 423 for all .2n So

,31 c ,42 c and .20 n ),(33 nn Q ).(241022 nnn Q ),2(2*6

2 nn n Q

and ).(log4log*10 nn Q

Observe that ),1(23 Qn ),(332nn Q ),(2410

2 nnn Q

),1(2410 2 Q nn ),(2*622 nnn Q ),(2*6

1002 nnn Q and ).1(2*62 Q nn

You cannot go up or down as you wish. It is tight bound.


10/28


10

Theorem

Theorem 2.1

For any two function )(nf and ),(ng ))(()( ngnf Q if and only if ))(()( ngOnf and

(g(n)).)( Wnf


11/28


11

Typical growth rate

Table 2.1:Typical growth rate

Function Name

c

nlog

n

n2

n3

2n

Constantlogarithmic

linearquadratic

cubicexponential

n

nnnnnn 2loglog32

for all .4n


12/28


12

Recurrences

A recurrence is an equation or inequalitythat describes a function in terms of itsvalue on smaller inputs.

The running time of merge sortalgorithm could be describe by the recurrence

.1if)2/(2

1if)(nnnT

ncnT

The solution of the above recurrence claim to be )log()( nnOnT or )log()( nnnT .

There are three methods for solving recurrences that is, for obtaining asymptotic or

O bounds on the solution.


13/28


13

Recursion Tree

A recursion tree is a convenient way to visualizewhat happens when a recurrence is iterated.


14/28


1414

Recursion tree ofT(n)=2T(n/2)+cn


15/28


15

Iteration method

The iteration method converts the recurrence into a summation and then

relives on techniques for bounding summations to solve the recurrence.

122

constanta,1)(

nn)/T(n

isananT


16/28


16

Analysis

knk

nTk

......

nn

T

nn

T

nn

T

nnn

T

nn

TnT

2

2

3

23

23

38

8

2

4

4

2422

22


17/28


18/28


18

The master method

The master method provides a cookbook method for solving recurrences of the form

(2.3)

Where and are constant and is an asymptotically positive function. The

master method requires memorization of three cases, but then the solution of many recurrences

can be determined quite easily, often without pencil and paper.The recurrence (2.3) describes the running time of an algorithm that divides a problem of size

n into a subproblems, each of size , where a and b are positive constants. The a

subproblems are solved recursively, each in time . The cost of dividing the problem and

combining the result of the subproblems is described by the function .


19/28


19

The master method

The master method depends on the following theorem.

Theorem 2.2 (master theorem)

Let and be constants, let be a function, and let be defined

on the nonnegative integers by the recurrence

,

Where we interpret to mean either or . The can be

bounded asymptotically as follows.

1. If for some constant , then

2.

If then

3.

If for some constant , and if for

some constant and all sufficiently large n, then .


20/28


20

Master method

T(n) = a T(n/b) + f(n)At first we will find nlogbawith respect to the above

recurrence and see the followings:

1) If nlog

ba

>f(n) polynomially (in power of n)T(n) = (nlogba)

2) If f(n) = nlogba,T(n) = (nlogbalgn)

3) If f(n) > nlogba, polynomially, T(n) = (f(n))

[lg means log base 2].


21/28


21

Use of Master method

T(n) = 9T(n/3) +nHere a = 9, b = 3 and f(n) = n.

nlogba = nlog39 = n2> n. We got case 1,

that is nlogba > f(n), So we can write

T(n) = (nlog39) = (n2)


22/28


22


T(n) = T (2n/3) + 1T(n) = divide deno. and num. by 2

Here a =1, b = 3/2 and f(n) =1.

nlogba= nlog3/21= n0= 1. That is, nlogba= f(n)According to case 2 we can write

T(n) = (nlogbalgn) = (1.lgn) = (lgn)

1)

23

( n

T


23/28


23


T(n) = 3 T(n/4) + nlgn)Here a =3, b = 4 and f(n) = nlgn.

nlogba = nlog43= n0.793. That is, nlgn > n0.793

orf(n) >nlogba. So, according to case 3 we

can write

T(n) =(f(n)= (nlgn).


24/28


24


T(n) = 3 T(n/4) + nlgn)Here a =3, b = 4 and f(n) = nlgn.

nlogba = nlog43= n0.793. That is, nlgn > n0.793

orf(n) >nlogba. So, according to case 3 we

can write

T(n) =(f(n)= (nlgn).


25/28


25

Failure of Master method

T(n) = 2 T(n/2) + nlgnHere a =2, b = 2 and f(n) = nlgn.

nlogba = nlog22= n. That is, nlgn > n, but

not greater in polynomial (in power of n)Consequebtly this recurrence can not be

solved using master theorem.


26/28


26


T(n) = 7 T(n/2) + n2

Here a =7, b = 2 and f(n) = n2.

nlogba = nlog27= n2.8. That is, n2.8> n2or

nlogba> f(n). So, according to case 1 wecan write

T(n) =(f(n))= (n2.8)= (nlg7)


27/28


27

References

1. Introduction to algorithms, Third EditionThomas H. Corman, Charles E. Leiserson, Ronald L.

Rivest.

2. Fundamentals of computer algorithmEllis Horowitz, Sartaj Sahni and Rajasekaran.


28/28

INTRODUCTION TO ALGORITHMS 28

This is the end of chapter 2

THANK YOU

Documents

Daaa 2 Asymptotic_ Notation