Embed Size (px)
Citation preview
UNDERSTANDING PROGRAM EFFICIENCY: 1 (download slides and .py files and follow along!)
6.0001 LECTURE 10
6.0001LECTURE10 1
Today § Measuringordersofgrowthofalgorithms
§ Big“Oh”notaAon§ Complexityclasses
6.0001LECTURE10 2
WANT TO UNDERSTAND EFFICIENCY OF PROGRAMS § computersarefastandgeGngfaster–somaybeefficientprogramsdon’tmaLer?◦ butdatasetscanbeverylarge(e.g.,in2014,Googleserved30,000,000,000,000pages,covering100,000,000GB–howlongtosearchbruteforce?)
◦ thus,simplesoluAonsmaysimplynotscalewithsizeinacceptablemanner
§
§ separate!meandspaceefficiencyofaprogram§ tradeoffbetweenthem:◦ cansomeAmespre-computeresultsarestored;thenuse“lookup”toretrieve(e.g.,memoizaAonforFibonacci)
◦ willfocusonAmeefficiency
6.0001LECTURE10 3
howcanwedecidewhichopAonforprogramismostefficient?
WANT TO UNDERSTAND EFFICIENCY OF PROGRAMS ChallengesinunderstandingefficiencyofsoluAontoacomputaAonalproblem:
§ aprogramcanbeimplementedinmanydifferentways
§ youcansolveaproblemusingonlyahandfulofdifferentalgorithms
§ wouldliketoseparatechoicesofimplementaAonfromchoicesofmoreabstractalgorithm
6.0001LECTURE10 4
HOW TO EVALUATE EFFICIENCY OF PROGRAMS § measurewitha!mer
§ counttheoperaAons§ abstractnoAonoforderofgrowth
6.0001LECTURE10 5
TIMING A PROGRAM § useAmemodule
§ recallthat!imporAngmeansto
bringinthatclass def c_to_f(c):!
intoyourownfile
return c*9/5 + 32 !!
§ startclock t0 = time.clock()!§ callfuncAon c_to_f(100000)!
t1 = time.clock() - t0!§ stopclock Print("t =", t, ":", t1, "s,”)
6.0001LECTURE10 6
import time!
!
TIMING PROGRAMS IS INCONSISTENT § GOAL:toevaluatedifferentalgorithms§ runningAmevariesbetweenalgorithms§ runningAmevariesbetweenimplementa!ons§ runningAmevariesbetweencomputers§ runningAmeisnotpredictablebasedonsmallinputs
§ AmevariesfordifferentinputsbutcannotreallyexpressarelaAonshipbetweeninputsandAme
6.0001LECTURE10 7
COUNTING OPE§ assumethesestepstakeconstant!me:• mathemaAcaloperaAons• comparisons• assignments• accessingobjectsinmemor
• thencountthenumberofoperaAonsexecutedasfuncAonofsizeofinput
RATIONS def c_to_f(c):! return c*9.0/5 + 32 !!def mysum(x):! total = 0!
ange(x+1):!+= i!
for i in r total y return tot
mysumà1+3
al!
6.0001LECTURE10 8
xops
COUNTING OPERATIONS IS BETTER, BUT STILL… § GOAL:toevaluatedifferentalgorithms
§ countdependsonalgorithm
§ countdependsonimplementa!ons
§ countindependentofcomputers
§ nocleardefiniAonofwhichopera!onstocount
§ countvariesfordifferentinputsandcancomeupwitharelaAonshipbetweeninputsandthecount
6.0001LECTURE10 9
STILL NEED A BETTER WAY • AmingandcounAngevaluateimplementa!ons
• Amingevaluatesmachines
• wanttoevaluatealgorithm
• wanttoevaluatescalability• wanttoevaluateintermsofinputsize
6.0001LECTURE10 10
STILL NEED A BETTER WAY § GoingtofocusonideaofcounAngoperaAonsinanalgorithm,butnotworryaboutsmallvariaAonsinimplementaAon(e.g.,whetherwetake3or4primiAveoperaAonstoexecutethestepsofaloop)§ Goingtofocusonhowalgorithmperformswhensizeofproblemgetsarbitrarilylarge§ WanttorelateAmeneededtocompleteacomputaAon,measuredthisway,againstthesizeoftheinputtotheproblem§ Needtodecidewhattomeasure,giventhatactualnumberofstepsmaydependonspecificsoftrial
6.0001LECTURE10 11
NEED TO CHOOSE WHICH INPUT TO USE TO EVALUATE A FUNCTION § wanttoexpressefficiencyintermsofsizeofinput,soneedtodecidewhatyourinputis
§ couldbeaninteger--mysum(x)
§ couldbelengthoflist--list_sum(L)
§ youdecidewhenmulApleparameterstoafuncAon--search_for_elmt(L, e)
6.0001LECTURE10 12
DIFFERENT INPUTS CHANGE HOW THE PROGRAM RUNS § afuncAonthatsearchesforanelementinalistdef search_for_elmt(L, e):! for i in L:! if i == e:! return True! return False!
§ wheneisfirstelementinthelistàBESTCASE
§ wheneisnotinlistàWORSTCASE
§ whenlookthroughabouthalfoftheelementsinlistàAVERAGECASE
§ wanttomeasurethisbehaviorinageneralway
6.0001LECTURE10 13
BEST, AVERAGE, WORST CASES § supposeyouaregivenalistLofsomelengthlen(L) § bestcase:minimumrunningAmeoverallpossibleinputsofagivensize,len(L) • constantforsearch_for_elmt • firstelementinanylist
§ averagecase:averagerunningAmeoverallpossibleinputsofagivensize,len(L) • pracAcalmeasure
§ worstcase:maximumrunningAmeoverallpossibleinputsofagivensize,le• linearinlengthof
n(L) listforsearch_for_elmt
• mustsearchenArelistandnotfindit
6.0001LECTURE10 14
ORDERS OF GROWTH Goals:§ wanttoevaluateprogram’sefficiencywheninputisverybig§ wanttoexpressthegrowthofprogram’srun!measinputsizegrows§ wanttoputanupperboundongrowth–asAghtaspossible§ donotneedtobeprecise:“orderof”not“exact”growth§ wewilllookatlargestfactorsinrunAme(whichsecAonoftheprogramwilltakethelongesttorun?)§ thus,generallywewant!ghtupperboundongrowth,asfunc!onofsizeofinput,inworstcase
6.0001LECTURE10 15
MEASURING ORDER OF GROWTH: BIG OH NOTATION § BigOhnotaAonmeasuresanupperboundontheasympto!cgrowth,oiencalledorderofgrowth
§ BigOhorO()isusedtodescribeworstcase• worstcaseoccursoienandistheboLleneckwhenaprogramruns
• expressrateofgrowthofprogramrelaAvetotheinputsize
• evaluatealgorithmNOTmachineorimplementaAon
6.0001LECTURE10 16
EXACT STEPS vs O() def fact_iter(n):! """assumes n an int >= 0"""! answer = 1! while n > 1:! answer *= n! n -= 1! return answer!
§ computesfactorial§ numberofsteps: § worstcaseasymptoAccomplexity:
• ignoreaddiAveconstants• ignoremulAplicaAveconstants
6.0001LECTURE10 17
WHAT DOES O(N) MEASURE? § InterestedindescribinghowamountofAmeneededgrowsassizeof(inputto)problemgrows
§ Thus,givenanexpressionforthenumberofoperaAonsneededtocomputeanalgorithm,wanttoknowasymptoAcbehaviorassizeofproblemgetslarge
§ Hence,willfocusontermthatgrowsmostrapidlyinasumofterms
§ AndwillignoremulAplicaAveconstants,sincewanttoknowhowrapidlyAmerequiredincreasesasincreasesizeofinput
6.0001LECTURE10 18
SIMPLIFICATION EXAMPLES § dropconstantsandmulAplicaAvefactors§ focusondominantterms
: n2O(n2) + 2n + 2
O(n2) : n2 + 100000n + 31000
O(n) : log(n) + n + 4
O(nlogn) : 0.0001*n*log(n) + 300n
O(3n) : 2n30 + 3n
6.0001LECTURE10 19
TYPES OF ORDERS OF GROWTH
6.0001LECTURE10 20
ANALYZING PROGRAMS AND THEIR COMPLEXITY § combinecomplexityclasses
• analyzestatementsinsidefuncAons• applysomerules,focusondominantterm
LawofAddi!onforO():• usedwithsequen!alstatements• O(f(n))+O(g(n))isO(f(n)+g(n))• forexample, for i in range(n):! print('a')! for j in range(n*n):! print('b')!
isO(n)+O(n*n)=O(n+n2)=O(n2)becauseofdominantterm6.0001LECTURE10 21
ANALYZING PROGRAMS AND THEIR COMPLEXITY § combinecomplexityclasses
• analyzestatementsinsidefuncAons• applysomerules,focusondominantterm
LawofMul!plica!onforO():• usedwithnestedstatements/loops• O(f(n))*O(g(n))isO(f(n)*g(n))• forexample, for i in range(n):! for j in range(n):! print('a')!
isO(n)*O(n)=O(n*n)=O(n2)becausetheouterloopgoesnAmesandtheinnerloopgoesnAmesforeveryouterloopiter.
6.0001LECTURE10 22
COMPLEXITY CLASSES § O(1)denotesconstantrunningAme§ O(logn)denoteslogarithmicrunningAme§ O(n)denoteslinearrunningAme§ O(nlogn)denoteslog-linearrunningAme§ O(nc)denotespolynomialrunningAme(cisaconstant)§ O(cn)denotesexponenAalrunningAme(cisaconstantbeingraisedtoapowerbasedonsizeofinput)
6.0001LECTURE10 23
COMPLEXITY CLASSES ORDERED LOW TO HIGH
O(1) : O(log n) : O(n) : O(n log n): O
O
(nc) :
(cn) :
constant
logarithmic
linear
loglinear
polynomial
exponenAal
6.0001LECTURE10 24
COMPLEXITY GROWTH CLASS n=10 =100 =1000 =1000000
O(1) 1 1 1 1
O(logn) 1 2 3 6
O(n) 10 100 1000 1000000
O(nlogn) 10 200 3000 6000000
O(n^2) 100 10000 1000000 1000000000000
O(2^n) 1024 1267650600228229401496703205376
107150860718626732094842504906000181056140481170553360744375038837035105112493612249319837881569585812759467291755314682518714528569231404359845775746985748039345677748242309854210746050623711418779541821530464749835819412673987675591655439460770629145711964776865421676604298316
52624386837205668069376
Goodluck!!
6.0001LECTURE10 25
LINEAR COMPLEXITY § SimpleiteraAveloopalgorithmsaretypicallylinearincomplexity
6.0001LECTURE10 26
LINEAR SEARCH ON UNSORTED LIST def linear_search(L, e):! found = False! for i in range(len(L)):! if e == L[i]:! found = True! return found!
§ mustlookthroughallelementstodecideit’snotthere
§ O(len(L))fortheloop*O(1)totestife==L[i]◦ O(1+4n+1)=O(4n+2)=O(n)
§ overallcomplexityisO(n)–wherenislen(L)
6.0001LECTURE12 27
CONSTANT TIME LIST ACCESS § iflistisallints◦ ithelementat◦ base+4*i
§ iflistisheterogeneous◦ indirecAon◦ referencestootherobjects
…
…
6.0001LECTURE12 28
LINEAR SEARCH ON SORTED LIST def search(L, e):! for i in range(len(L)):! if L[i] == e:! return True! if L[i] > e:! return False! return False§ mustonlylookunAlreachanumbergreaterthane§ O(len(L))fortheloop*O(1)totestife==L[i]§ overallcomplexityisO(n)–wherenislen(L)§ NOTE:orderofgrowthissame,thoughrunAmemaydifferfortwosearchmethods
6.0001LECTURE12 29
LINEAR COMPLEXITY § searchingalistinsequencetoseeifanelementispresent§ addcharactersofastring,assumedtobecomposedofdecimaldigitsdef addDigits(s):!
val = 0!
for c in s:!
val += int(c)!
return val!
§ O(len(s))
6.0001LECTURE10 30
LINEAR COMPLEXITY § complexityoiendependsonnumberofiteraAonsdef fact_iter(n):!
prod = 1!
for i in range(1, n+1):!
prod *= i!
return prod!
§ numberofAmesaroundloopisn§ numberofoperaAonsinsideloopisaconstant(inthiscase,3–seti,mulAply,setprod)◦ O(1+3n+1)=O(3n+2)=O(n)
§ overalljustO(n)
6.0001LECTURE10 31
NESTED LOOPS § simpleloopsarelinearincomplexity
§ whataboutloopsthathaveloopswithinthem?
6.0001LECTURE10 32
QUADRATIC COMPLEXITY determineifonelistissubsetofsecond,i.e.,everyelementoffirst,appearsinsecond(assumenoduplicates)!def isSubset(L1, L2):! for e1 in L1:! matched = False! for e2 in L2:! if e1 == e2:! matched = True! break! if not matched:! return False! return True!
6.0001LECTURE10 33
QUADRATIC COMPLEXITY def isSubset(L1, L2):! for e1 in L1:! matched = False! for e2 in L2:! if e1 == e2:! matched = break! if not matched:! return False! return True!
outerloopexecutedlen(L1)Ames
eachiteraAonwillexecuteinnerloopuptolen(L2)Ames,withconstantnumberTrue!ofoperaAons
O(len(L1)*len(L2)) worstcasewhenL1andL2samelength,noneofelementsofL1inL2
O(len(L1)2)
6.0001LECTURE10 34
QUADRATIC COMPLEXITY findintersecAonoftwolists,returnalistwitheachelementappearingonlyonce!
def intersect(L1, L2):! tmp = []! for e1 in L1:! for e2 in L2:! if e1 == e2:! tmp.append(e1)! res = []! for e in tmp:! if not(e in res):! res.append(e)! return res!
6.0001LECTURE10 35
QUADRATIC COMPLEXITY def intersect(L1, L2):! tmp = []! for e1 in L1:! for e2 in L2:! if e1 == e2:! tmp.append(e res = []! for e in tmp:! if not(e in res):! res.append(e)! return res!
firstnestedlooptakeslen(L1)*len(L2)steps secondlooptakesatmostlen(L1)steps
1)! determiningifelementinlistmighttakelen(L1)steps ifweassumelistsareofroughlysamelength,then O(len(L1)^2)
6.0001LECTURE10 36
O() FOR NESTED LOOPS def g(n):! """ assume n >= 0 """! x = 0! for i in range(n):! for j in range(n):! x += 1! return x!
§ computesn2veryinefficiently§ whendealingwithnestedloops,lookattheranges§ nestedloops,eachitera!ngn!mes§ O(n2)
6.0001LECTURE10 37
THIS TIME AND NEXT TIME § haveseenexamplesofloops,andnestedloops
§ giverisetolinearandquadraAccomplexityalgorithms
§ nextAme,willmorecarefullyexamineexamplesfromeachofthedifferentcomplexityclasses
6.0001LECTURE10 38
MIT OpenCourseWarehttps://ocw.mit.edu
6.0001 Introduction to Computer Science and Programming in PythonFall 2016
For information about citing these materials or our Terms of Use, visit: https://ocw.mit.edu/terms.
