2. Analysis of Algorithms

8/9/2019 2. Analysis of Algorithms

http://slidepdf.com/reader/full/2-analysis-of-algorithms 1/26

ANALYSIS OFALGORITHMS

Introducing Some Limited Concepts about Algoritms



Outline

!""icienc# o" Algoritms

!$amples

r er o ro%t

&orst' A(erage and )est Case Anal#sis



!""icienc# o" Algoritms T%o "undamental *uestions+

Ho% "ast %ill m# program run, -Time Comple$it#.

Ho% muc memor# it %ill ta/e,

o" operations per"ormed b# te algoritm

&e %ill assume tat all operations ta/e te same

amount o" time0



!$amplesFahrenheit to Celsius Converter (code)



!$amplesFahrenheit to Celsius Converter (analysis)

Tree aritmetic operationsOne subtraction' once multiplication' and one di(ision

'

generali1e and sa# %e a(e tree operations



!$amplesAdding Array Elements (code)



!$amplesAdding Array Elements (analysis)

For an arra# o" si1e N' o% man# operations are rougl# e$ecuted,

Statement Frequency of Execution

int sum 2 3 4

int i2 3 4 4

Te total number o" operations clearl# depends on input si1e

&e %ill not obsess about te small details-constant "actors.

i 5 list0lengt N

i66 N

sum2sum6list-i. N

Total 27N68



!$amplesAdding Square Matrices(code)

For an N N matri$' o% man# operations arerougl# e$ecuted,



!$amplesAdding Square Matrices(analysis)

Statement Frequency of Execution

dbouble9:9: sum000 4

int i2 3 4 4

i 5 list0lengt N

i66 N

int ; 2 3 N

;5 list0lengt N8

;66 N8

sum9i:9;: 2 a9i:9;: 6 b9i:9;: N8

Total 27N86 7N 6 8



Order o" Gro%tEffect of Problem Sie

Onl# large problem si1es are %ort stud#ing<

Modern computers are (er# "ast0

usuall# complete in under a minute

Our anal#sis o" algoritm "ocuses on o% an

algoritm scales %it problem si1es0



Order o" Gro%t!ro"th #ate of Functions

Consider+ " -n. 2 n8 6 433n 6 log -n. 6 4333

Let us consider o% tis "unction bea(es "or small



Order o" Gro%t!ro"th #ate of Functions(cont$d)

For te pre(ious "unction' " -n. 2 n8 6 433n 6 log -n. 64333' %e can see For large n' " -n. n8

Tis enables us to use te appro$imation+ " -n. n8' "or asu c en # arge n

Tis simpli"ication %or/ed despite te large coe""icient in "ront o"n

Generall#' "or most "unctions' lo%er order terms become lessrele(ant as te (alue o" n gets larger

Tis =as#mptotic= simpli"ication is (er# use"ul "or anal#1ingalgoritms Remember tat onl# large problems are %ort stud#ing



Order o" Gro%t%ig&' otation

Instead o" sa#ing an algoritm ta/es 7n8 6 >n 6 8steps' its simpler to lea(e out lo%er order terms suc

as >n and 8 and e(en te coe""icient 7 and sa te

gro%t rate is O-n8. -pronounced as big?o o" n8.

)ig?O is te most %idel# used as#mptotic notation in

algoritm anal#sis

&ill be used trougout te course



Order o" Gro%t)ig?O Notation ? Common Sense Approac

)ig?O enables us to speci"# o% an algoritm@s per"ormancescales %it input si1e

&e %ill discard lo%er order terms since te# are dominated na dominates nb i" a b+ "or e$ample n7 dominates n8

an# e$ponential dominates an# pol#nomial+ 7n dominates Bn

an# pol#nomial dominates an# logaritm+ n dominates -log -n..7

Multiplicati(e constants can be omitted+ 4>n8 becomes n8

Constant terms can be important-especiall# i" te# are large.0

)ut+ &e cannot easil# stud# algoritms %itout te simplicit#a""orded b# big?O notation0

Constant terms depend ea(il# on te ard%are arcitecture ? soits ard to come up %it an accurate number



Order o" Gro%t%ig&' otation & Formal efinition

enition Let T-n. and " -n. be "unctions "rom positi(e

20

-%ic means T-N. gro%s no "aster tan " -n.. i"

tere are positi(e constants c and n3 suc tat T-N.

2 c"-N. %en N n30



Order o" Gro%t!ro"th #ate Com*arisons

I" %e assume a gi(en operation ta/es onenanosecond-43?D. on some computer' %e can

construct te "ollo%in table



Order o" Gro%t+hat do "e mean by a fast algorithm,

&e cannot simpl# sa# a "ast algoritm is one tat runs*uic/l# %en gi(en a real input instance

Tis de"inition depends on te speed o" te computer ? a gor ms run as on some mac nes an s o% # on

oter macines

i""erent algoritms bea(e di""erentl# on di""erentinput si1es

A some%at standard denition %e %ill "ollo%

efinition

An algorithm if fast if it is growth rate is O(n) or less.



&orst' A(erage and )est CaseSequential Search (code)



&orst' A(erage and )est CaseSequential Search (analysis)

Our anal#sis depends on te input

)est Case+ O-4. I" te item being searced is "ound at te rst position

Tis appens onl# once in a %ile

ors ase+ I" te item being searced is "ound at te end o" te list'or NOT "ound at

all

Te is te upper bound

A(erage Case

Hard to determine anal#ticall# unless %e /no% te t#pe o" input %e areprocessing

Re"lects te actual situation



&orst' A(erage and )est Case

%inary Search ("hat is it,)

A searc algoritm tat %or/s onl# on sorted lists0 Similar to o% #ou %ould searc "or a %ord in te

A (er# %idel# used searc algoritm




%inary Search (the big idea)

9Image ta/en "rom+ Sed%ic/ and &a#ne' Algoritms' Fourt ed0:




)inar# Searc -code.




%inary Search (analysis)

Our anal#sis depends on te input

)est Case+ O-4. I" te item being searced is "ound e$actl# at te middle

Tis appens onl# once in a %ile

ors ase+ og n ? %e % scuss s s or # I" te item being searced is "ound at te end o" te list' or NOT "ound

at all

Te is te upper bound

A(erage Case

Hard to determine anal#ticall# unless %e /no% te t#pe o" input %e areprocessing

Re"lects te actual situation




%inary Search ("orst case analysis)

Te meaning o" log8-n. 2 m+ i" #ou di(ide n b# 8o(er and o(er' m times' #ou get 40




%inary Search ("orst case analysis) & cont$d

In binar# searc' %e /eep on reducing te searc domain b# a "actor o"

t%o

I" te initial input as N elements+

&e a(e NE8 items to be searced in second iteration' NE88 iterations in te

tird iteration' NE87 in te "ourt iteration000

In te %orst case' te searc goes on until one element is le"t in te list on' sa#'

te Mt iteration

)ased on our pre(ious obser(ation o" te logaritm' "or an input si1e o" N

and a total o" M iterations-a"ter %ic onl# one item is le"t.' %e a(e te

relation sip




+orst- Average- and %est Case

)est Case Considers cases tat are unli/el#0 Not used muc0

Indicates te lo%er bound on te gro%t0

A(erage Case+ ard to e(aluate' but (er# use"ul &orst Case+

er# use"ul since it tells us te upper bound on te gro%trate

Muc easier to calculate tan a(erage case ? %ill be used(er# o"ten0

Documents

2. Analysis of Algorithms