Embed Size (px)
Citation preview
Mod
ule
#6
– O
rder
s of
Gro
wth
M
odul
e #
6 –
Ord
ers
of G
row
th
UCDavis,ECS20,Spring2019DiscreteMathematicsforComputerScienceProf.RaissaD’Souza(slidesadoptedfromMichaelFrankandHalukBingöl)
Lecture12
Complexity of Algorithms 3.2-3.3
Mod
ule
#6
– O
rder
s of
Gro
wth
RecallLec10onalgorithmsHowlongdoeseachalgorithmtaketorun?• Howdothenumberofopera1onsscalewithinputlengthn?
• Worst-casecomplexity• e.g.Searchlistoflengthnandtheitemtofindisinposi1onan
• Best-casecomplexity• e.g.Searchlistoflengthnandtheitemtofindisinposi1ona1
• Average-casecomplexity/Typical-casecomplexity
Mod
ule
#6
– O
rder
s of
Gro
wth
Averagecasecomplexity • e.g.,foranunorderedlistoflengthn
• Withprobability1/ntheelementisinposi1onajforjrangingfrom1ton.
• LetM(j)denotethenumberofopera1onsneededifthedesireditemisinposi1onaj
• Averagecase:
(1/n)Σnj=1M(j)
Mod
ule
#6
– O
rder
s of
Gro
wth
Specifyinganalgorithm:OurPseudocodeLanguage:
• procedurename(argument:type)• variable:=expression• beginstatementsend• {comment}• ifcondi8onthenstatement[elsestatement]
• forvariable:=ini8alvaluetofinalvaluestatement• whilecondi8onstatement• returnexpression
Mod
ule
#6
– O
rder
s of
Gro
wth
procedureprocname(arg:type)• OQenjustsay“procname(arg)”withoutthetype.
• Declaresthatthefollowingtextdefinesaprocedurenamedprocnamethattakesinputs(arguments)namedargwhicharedataobjectsofthetypetype.• Example:proceduremaximum(L:listofintegers)[statementsdefiningmaximum…]
Mod
ule
#6
– O
rder
s of
Gro
wth
variable:=expression• Anassignmentstatementevaluatestheexpressionexpression,thenreassignsthevariablevariabletothevaluethatresults.• Exampleassignmentstatement:v:=3x+7(Ifxis2,changesvto13.)
• Alsouse• v<-3x+7
Mod
ule
#6
– O
rder
s of
Gro
wth
Informalstatement• Some1meswemaywriteastatementasaninformalEnglishimpera1ve,ifthemeaningiss1llclearandprecise:e.g.,“swapxandy”
• Keepinmindthatrealprogramminglanguagesneverallowthis.
• Whenweaskforanalgorithmtodoso-and-so,wri1ng“Doso-and-so”isn’tenough!• Breakdownalgorithmintodetailedsteps.
Mod
ule
#6
– O
rder
s of
Gro
wth
Swaptwovalues • V1:=10• V2:=20
badswap(V1,V2)V1=V2V2=V1(WhatendsupbeingthevalueofV2?)
swap(V1,V2)
temp:=V1V1=V2V2=temp(Thisoneworks)
Mod
ule
#6
– O
rder
s of
Gro
wth
beginstatementsend
• Groupsasequenceofstatementstogether:beginstatement1statement2…statementnend
• Allowsthesequencetobeusedjustlikeasinglestatement.• Mightbeused:
• AQeraproceduredeclara1on.• InanifstatementaQerthenorelse.• Inthebodyofafororwhileloop.Curly braces {} are used instead
in many languages.
Mod
ule
#6
– O
rder
s of
Gro
wth
{comment}• Notexecuted(doesnothing).• Natural-languagetextexplainingsomeaspectoftheproceduretohumanreaders.
• Alsocalledaremarkinsomerealprogramminglanguages,e.g.BASIC.
• Example,mightappearinamaxprogram:• {Notethatvisthelargestintegerseensofar.}
Mod
ule
#6
– O
rder
s of
Gro
wth
ifconditionthenstatement
• Evaluatetheproposi1onalexpressioncondi8on.• Iftheresul1ngtruthvalueisTrue,thenexecutethestatementstatement;
• otherwise,justskiponaheadtothenextstatementaQertheifstatement.
• Variant:ifcondthenstmt1elsestmt2• Likebefore,butifftruthvalueisFalse,executesstmt2.
Mod
ule
#6
– O
rder
s of
Gro
wth
whileconditionstatement
• Evaluatetheproposi1onal(Boolean)expressioncondi8on.
• Iftheresul1ngvalueisTrue,thenexecutestatement.
• Con1nuerepea1ngtheabovetwoac1onsoverandoverun1lfinallythecondi8onevaluatestoFalse;thenproceedtothenextstatement.
Mod
ule
#6
– O
rder
s of
Gro
wth
whileconditionstatement
• Alsoequivalenttoinfinitenestedifs,likeso:ifcondi8onbeginstatementifcondi8onbeginstatement…(con8nueinfinitenestedif’s)endend
Mod
ule
#6
– O
rder
s of
Gro
wth
forvar:=initialto6inalstmt
• Ini8alisanintegerexpression.
• Finalisanotherintegerexpression.
• Seman:cs:Repeatedlyexecutestmt,firstwithvariablevar:=ini8al,thenwithvar:=ini8al+1,thenwithvar:=ini8al+2,etc.,thenfinallywithvar:=final.
• Ques:on:Whathappensifstmtchangesthevalueofvar,orthevaluethatini8alorfinalevaluatesto?
Mod
ule
#6
– O
rder
s of
Gro
wth
forvar:=initialto6inalstmt
• Forcanbeexactlydefinedintermsofwhile,likeso:
begin var := initial while var ≤ final begin stmt var := var + 1 end end
Mod
ule
#6
– O
rder
s of
Gro
wth
NowwritingAlg1:Vindmax
Mod
ule
#6
– O
rder
s of
Gro
wth
Alg.#3:BinarySearch• Basicidea:Oneachstep,lookatthemiddleelementoftheremaininglisttoeliminatehalfofit,andquicklyzeroinonthedesiredelement.
<x >x <x <x
Mod
ule
#6
– O
rder
s of
Gro
wth
Alg.#3:BinarySearch
• procedure binary search (x:integer, a1, a2, …, an: distinct integers) i := 1 {left endpoint of search interval} j := n {right endpoint of search interval} while i<j begin {while interval has >1 item}
m := ⎣(i+j)/2⎦ {midpoint} if x>am then i := m+1 else j := m
end if x = ai then location := i else location := 0 return location
Mod
ule
#6
– O
rder
s of
Gro
wth
SortingAlgorithms• Sor1ngisacommonopera1oninmanyapplica1ons.
• E.g.spreadsheetsanddatabases• Itisalsowidelyusedasasubrou1neinotherdata-processingalgorithms.
• Twosor1ngalgorithmsshownintextbook:• Bubblesort• Inser1onsort However, these are not
very efficient, and you should not use them on large data sets!
Mod
ule
#6
– O
rder
s of
Gro
wth
BubbleSort(Alg#4)• Smallestelements“float”uptothetopofthelist,likebubblesinacontainerofliquid.
30 31 1
32 33 2
34 3
30 1
31 32 2
33 3
34
1 30 31 2
32 3
33 34
1 30 2
31 3
32 33 34
1 2
30 3
31 32 33 34
1 2 3
30 31 32 33 34
Mod
ule
#6
– O
rder
s of
Gro
wth
Review§3.1:Algorithms
• Characteris1csofalgorithms.
• Pseudocode.• Examples:Maxalgorithm,linearsearch&binarysearchalgorithms,sor1ng.
• Intui1velyweseethatbinarysearchismuchfasterthanlinearsearch,buthowdoweanalyzetheefficiencyofalgorithmsformally?
• Usemethodsofalgorithmiccomplexity,whichu1lizetheorder-of-growthconceptsfrom§1.8.
Mod
ule
#6
– O
rder
s of
Gro
wth
Seehandout7• Linearsearch(unsortedlist)• Binarysearch(needssortedlist)• Bubblesort• Inser1onsort
• (note“forkin0to-1”doesnotexecute)• Greedyalgorithms
• Makethebestchoicegivenwhatyouknow.• Internetrou1ngalgorithms• Makingchangewithminimalnumberofcoins
Mod
ule
#6
– O
rder
s of
Gro
wth
Howlongdoeseachalgorithmtaketorun?• Howdothenumberofopera1onsscalewithinputlengthn?
• Worst-casecomplexity• e.g.Searchlistoflengthnandtheitemtofindisinposi1onan
• Best-casecomplexity• e.g.Searchlistoflengthnandtheitemtofindisinposi1ona1
• Average-casecomplexity/Typical-casecomplexity
Mod
ule
#6
– O
rder
s of
Gro
wth
Howtowegeneralizethis? • Ingeneralwedonotcareabouttheexactnumberofopera1onsrequired.Wewanttoes1mateorboundthisbysomefunc1onofn.(Wherenisthelengthoftheinput)
Mod
ule
#6
– O
rder
s of
Gro
wth
Growthoffunctions(§3.2)• Forfunc1onsovernumbers,weoQenneedtoknowaroughmeasureofhowfastafunc8ongrows.
• Iff(x)isfastergrowingthang(x),thenf(x)alwayseventuallybecomeslargerthang(x)inthelimit(forlargeenoughvaluesofx).
• Usefulinengineeringforshowingthatonedesignscalesbeserorworsethananother.
Mod
ule
#6
– O
rder
s of
Gro
wth
VisualizingOrdersofGrowth• Onagraph,asyougototheright,thefaster-growingfunc-1onalwayseventuallybecomesthelargerone...
fA(n)=30n+8
Increasing n →
fB(n)=n2+1
Valu
e of
func
tion →
fB(n) > fA(n), for n>k
Mod
ule
#6
– O
rder
s of
Gro
wth
Anotherexample,graphically• Note30n+8isn’tlessthannanywhere(n>0).
• Itisn’tevenlessthan31neverywhere.
• Butitislessthan31neverywheretotherightofn=8.
n>k=8 →
Increasing n →
Valu
e of
func
tion →
n
30n+8 cn = 31n
30n+8 ∈O(n)
Mod
ule
#6
– O
rder
s of
Gro
wth
Commonreferencefunctions
Mod
ule
#6
– O
rder
s of
Gro
wth
Handout8
Mod
ule
#6
– O
rder
s of
Gro
wth
RelationsBetweentheRelations• Subsetrela1onsbetweenorder-of-growthsets.
R→R
Ω( f ) O( f ) Θ( f ) • f
Mod
ule
#6
– O
rder
s of
Gro
wth
ComplexityofAlgorithms
Mod
ule
#6
– O
rder
s of
Gro
wth
Computertimeneeded
Mod
ule
#6
– O
rder
s of
Gro
wth
Computationalcomplexityclasses
• Tractableproblemsarepolynomialcomplexityorless.Saidtobelongtocomputa1onalcomplexityclassP,for“polynomial”
• Exponen1alandfactorialcomplexityaresaidtobe“intractable”.
• SpecialclassNP(non-determinis1cpolynomial)
Mod
ule
#6
– O
rder
s of
Gro
wth
NPproblems • Nopolynomial1mealgorithmforsolvingtheproblem
• Butcancheck/validateapoten1alsolu1oninpolynomial1me.
• E.g.factoringlargeintegers• Centralbelief:P≠NP
• ProvingthisisoneoftheMillenniumPrizeproblems(win$1million!!)
