Today’s Agenda Asymptotic Analysis of algorithm A good formal mathematical understanding of algorithms running times and Big O notation We measure

Today’s Agenda

Asymptotic Analysis of algorithm

A good formal mathematical understanding of algorithms running times and Big O notation

We measure algorithms and data structures against another using Asymptotic Analysis

CS2336: Computer Science II

Example

Suppose you write a program that manages inventory, for a nation wide chain of stores

every night you have to process all of the changes that happened in inventory in every store across the country that day!

10,000 ms to read inventory from the disk

10ms to process each transactions

for n, when n is a large number, the total time takes:

(10,000+10n) ms

for example there are 1 million transactions a day!


Asymptotic Analysis

Study of how algorithms behave as size of the data you processing grows very large! As it goes to infinity!

Big O notation , represent upper bound in a function. Has a very formal mathematical meaning!


Big O notation

Let n be size of the program’s input size can be number of bits that our program has to

read from disk for the inventory example! it can be numbers of items, how many items I want

to insert to my list!for Asymptotic analysis we don’t care how exactly n is

measured!

Let T(n) be a real world function, that describe some actual process in real. for example an actual running time of your inventory

program in ms on an input of n items. T(n) = (10,000 + 10n)

Let F(n) be another function-preferably simple function! (we want to use it as a upper bound)


Informal Big O

Complexity of T(n) is O( F(n) ) if and only if T(n) <= c F(n)

whenever when n is big and for some large constant c.

How big is big? choose n big enough to make T(n) function to fit under F(n)

How large the c have to be? choose c big enough to make T(n) function to fit under F(n)


Example

T(n) = 10,000+10n lets try f(n) = n, c=20


10000

20000

30000

1000

T(n)cf(n) = 20n

how they behave when n go to infinity!

T(n)<=cf(n) for any n>=1000SO T(n) complexity is O(n)

Formal Big O

O(f(n)) is an infinite set of all functions T(n) that satisfy :

There exist positive constants c and N such that, for all n >= N ,

T(n) <= c f(n)

N is 1000 in our example


Example

1,000,000n O(n)

Big O doesn’t care about the constant factors!

n3+n2+1 O(n3)

Big O is usually used to pick the fastest growing term.


Table of important big O setsFunction

common name

O(1) constant

O(log n) logarithmic

O(log 2 n ) log- squared

O(n1/2 ) root n

O(n) linear

O(n log n)

O(n2) quadratic

O(n3) cubic

O(n4) quartic

O(2n) exponential

O(en) exponential (but very more so!)

O(n!) < O(nn)


Asymptotically en is much more faster than 2n

Logarithms

If you don’t know:

logab is equal to logc

b / logca

you need to review logarithms again!


binary search : recursion tree

T(n) = T(n/2) + 1

= T(n/22) + 2

= T(n/23) + 3

= T(n/24) + 4

= …….. = T(n/2k) + k = T(1) + k = 1 + log2n

O(log2n)

if n=1 => T(1) = 1

n/2k = 1 => k = log2 n


https://math.dartmouth.edu/archive/m19w03/public_html/Section5-1.pdf

A finished recursion tree


The Master Theorem

n is the size of the problem.

a is the number of sub-problems in the recursion.

n/b is the size of each sub-problem.

f (n) is the cost of the work done outside the recursive calls, which includes the cost of dividing the problem and the cost of merging the solutions to the sub-problems.


The Master Theorem

The master theorem concerns recurrence relations of the form:

f(n) = nc

1. if logb a < c, T(n) = O(nc),

2. if logb a = c, T(n) = O(nc log n),

3. if logb a > c, T(n) = O(nlogb

a).

if n = 1

if n > 1

https://math.dartmouth.edu/archive/m19w03/public_html/Section5-2.pdf

Example T(n) = 8T(n/2) + 1000n2

T(n) = T(n/2) + 1

T(n) = 9T(n/3) +

T(n) = 2T(n/2) + n

T(n) = 3T(n/3) + n2

Example T(n) = 8T(n/2) + 1000n2 a = 8 , b = 2, c = 2

logba = log2

8= 3 > c case 3 T(n) = O(n3)

T(n) = T(n/2) + 1 a = 1 , b = 2, c = 0

logba = log2

1= 0 = c case 2 T(n) = O(n0logn) = O(logn)

T(n) = 9T(n/3) + n a = 9, b = 3, c = 1

logba = log3

9= 2 > c case 3 T(n) = O(n2)

T(n) = 2T(n/2) + n a=2,b=2,c=1

logba = log2

2= 1 = c case 2 T(n) = O(n1logn) = O(n logn)

T(n) = 3T(n/3) + n2 a=3,b=3,c=2

logba = log3

3= 1 < c case 1 T(n) = O(n2)

Documents

Today’s Agenda Asymptotic Analysis of algorithm A good formal mathematical understanding of algorithms running times and Big O notation We measure