Today’s Agenda Asymptotic Analysis of algorithm A good formal mathematical understanding of...

Today’s Agenda

Asymptotic Analysis of algorithm

A good formal mathematical understanding of algorithms running times and Big O notation

We measure algorithms and data structures against another using Asymptotic Analysis

CS2336: Computer Science II

Example

Suppose you write a program that manages inventory, for a nation wide chain of stores

every night you have to process all of the changes that happened in inventory in every store across the country that day!

10,000 ms to read inventory from the disk

10ms to process each transactions

for n, when n is a large number, the total time takes:

(10,000+10n) ms

for example there are 1 million transactions a day!

CS2336: Computer Science II

Asymptotic Analysis

Study of how algorithms behave as size of the data you processing grows very large! As it goes to infinity!

Big O notation , represent upper bound in a function. Has a very formal mathematical meaning!

CS2336: Computer Science II

Big O notation

Let n be size of the program’s input size can be number of bits that our program has to

read from disk for the inventory example! it can be numbers of items, how many items I want

to insert to my list!for Asymptotic analysis we don’t care how exactly n is

measured!

Let T(n) be a real world function, that describe some actual process in real. for example an actual running time of your inventory

program in ms on an input of n items. T(n) = (10,000 + 10n)

Let F(n) be another function-preferably simple function! (we want to use it as a upper bound)

CS2336: Computer Science II

Informal Big O

Complexity of T(n) is O( F(n) ) if and only if T(n) <= c F(n)

whenever when n is big and for some large constant c.

How big is big? choose n big enough to make T(n) function to fit under F(n)

How large the c have to be? choose c big enough to make T(n) function to fit under F(n)

CS2336: Computer Science II

Example

T(n) = 10,000+10n lets try f(n) = n, c=20

CS2336: Computer Science II

10000

20000

30000

1000

T(n)cf(n) = 20n

how they behave when n go to infinity!

T(n)<=cf(n) for any n>=1000SO T(n) complexity is O(n)

Formal Big O

O(f(n)) is an infinite set of all functions T(n) that satisfy :

There exist positive constants c and N such that, for all n >= N ,

T(n) <= c f(n)

N is 1000 in our example

CS2336: Computer Science II

Example

1,000,000n O(n)

Big O doesn’t care about the constant factors!

n3+n2+1 O(n3)

Big O is usually used to pick the fastest growing term.

CS2336: Computer Science II

Table of important big O setsFunction

common name

O(1) constant

O(log n) logarithmic

O(log 2 n ) log- squared

O(n1/2 ) root n

O(n) linear

O(n log n)

O(n2) quadratic

O(n3) cubic

O(n4) quartic

O(2n) exponential

O(en) exponential (but very more so!)

O(n!) < O(nn)

CS2336: Computer Science II

Asymptotically en is much more faster than 2n

Logarithms

If you don’t know:

logab is equal to logc

b / logca

you need to review logarithms again!

CS2336: Computer Science II

binary search : recursion tree

T(n) = T(n/2) + 1

= T(n/22) + 2

= T(n/23) + 3

= T(n/24) + 4

= …….. = T(n/2k) + k = T(1) + k = 1 + log2n

O(log2n)

if n=1 => T(1) = 1

n/2k = 1 => k = log2 n

CS2336: Computer Science II

https://math.dartmouth.edu/archive/m19w03/public_html/Section5-1.pdf

A finished recursion tree

CS2336: Computer Science II

The Master Theorem

n is the size of the problem.

a is the number of sub-problems in the recursion.

n/b is the size of each sub-problem.

f (n) is the cost of the work done outside the recursive calls, which includes the cost of dividing the problem and the cost of merging the solutions to the sub-problems.

CS2336: Computer Science II

The Master Theorem

The master theorem concerns recurrence relations of the form:

f(n) = nc

1. if logb a < c, T(n) = O(nc),

2. if logb a = c, T(n) = O(nc log n),

3. if logb a > c, T(n) = O(nlogb

a).

if n = 1

if n > 1

https://math.dartmouth.edu/archive/m19w03/public_html/Section5-2.pdf

Example T(n) = 8T(n/2) + 1000n2

T(n) = T(n/2) + 1

T(n) = 9T(n/3) +

T(n) = 2T(n/2) + n

T(n) = 3T(n/3) + n2

Example T(n) = 8T(n/2) + 1000n2 a = 8 , b = 2, c = 2

logba = log2

8= 3 > c case 3 T(n) = O(n3)

T(n) = T(n/2) + 1 a = 1 , b = 2, c = 0

logba = log2

1= 0 = c case 2 T(n) = O(n0logn) = O(logn)

T(n) = 9T(n/3) + n a = 9, b = 3, c = 1

logba = log3

9= 2 > c case 3 T(n) = O(n2)

T(n) = 2T(n/2) + n a=2,b=2,c=1

logba = log2

2= 1 = c case 2 T(n) = O(n1logn) = O(n logn)

T(n) = 3T(n/3) + n2 a=3,b=3,c=2

logba = log3

3= 1 < c case 1 T(n) = O(n2)