The Self-taught Programmer Cory Althoff...If you do want to become a software engineer, there is an increasing demand for good engineers and not enough qualified people to fill the

TheSelf-taughtProgrammer____________________________

CoryAlthoff

Copyright©2016byCoryAlthoffAllrightsreserved.Thisbookoranyportionthereofmaynotbereproducedorusedinanymannerwhatsoeverwithouttheexpresswrittenpermissionofthepublisherexceptfortheuseofbriefquotationsinabookreview.ISBN978-1-940733-01-2LibraryofCongressNumberwww.theselftaughtprogrammer.io

ThisbookisdedicatedtomyparentsAbbyandJamesAlthoffforalwayssupportingme.

PartIIntroductiontoProgramming

Chapter1.Introduction

"Mostgoodprogrammersdoprogrammingnotbecausetheyexpecttogetpaidorgetadulationbythepublic,butbecauseitisfuntoprogram."—LinusTorvalds

ImajoredinPoliticalScienceatClemsonUniversity.BeforeIchosethispathIconsideredComputerScience.IevenenrolledinanIntroductiontoProgrammingclassmyFreshmanyear,butquicklydroppedit.Itwastoodifficult.WhilelivinginSiliconValleyaftergraduationIdecidedIneededtolearntoprogram.Ayearlater,IwasworkingasasoftwareengineerIIateBay(aboveanentrylevelsoftwareengineer,butbelowaseniorsoftwareengineer).Idon’twanttogivetheimpressionthatthiswaseasy.Itwasincrediblychallenging.Inbetweenthrowingthingsatthewall,itwasalotoffuntoo.

IstartedmyjourneylearningtoprograminPython,apopularprogramminglanguage.Thisbook,however,isnotaboutteachingyouhowtoprograminaspecificlanguage(althoughitdoes).Thereareplentyofamazingbooks,classesandresourcesthatdothatalready.Thefocusiseverythingelsethosestandardresourcesdonotteachyou.It’saboutthethingsIhadtolearnonmyowninordertobecomeasoftwareengineer.Thisbookisnotmeantforsomeonelookingforacasualintroductiontoprogrammingsotheycanwritecodeasahobby.Thisbookiswrittenspecificallyforthoselookingtoprogramprofessionally.Whetheryourgoalistobecomeasoftwareengineer,anentrepreneurortouseyournewprogrammingskillsinanotherprofession,thisbookwaswrittenforyou.

Learningaprogramminglanguageisonlypartofthebattle.Thereareotherskillsyouneedinordertospeakthelanguageofcomputerscientists.IwillteachyoueverythingIlearnedonmyjourneyfromprogrammingnovicetoprofessionalsoftwareengineer.Iwrotethisbooktogiveaspiringprogrammersanoutlineofwhattheyneedtoknow.Asaself-taughtprogrammer,Ididn’tknowwhatIneededtolearn.Theintroductiontoprogrammingbooksareallthesame.TheyteachyouthebasicsofhowtoprogramineitherPythonorRubyandsendyouonyourway.ThefeedbackI’veheardfrompeoplefinishingthesebooksis,“WhatdoIdonow?Iamnotaprogrammeryet,andIdon’tknowwhattolearnnext.”Thisbookismyanswertothatquestion.

HowThisBookIsStructured

Thisbookisdividedintosixparts,basedonmovingthroughthefollowingstages:learningtoprogram,learningobject-orientedprogramming,learningtouseprograms(likeyouroperatingsystem)thatwillmakeyouabetterprogrammer,learningComputerScience,learningtoprogramforproductionandgettingajobandworkingonateam.

Manyofthesubjectscoveredinasinglechapterofthisbookcouldbe—andare—coveredbyentirebooks.Mygoalisnottocovereverydetailofeverysubjectyouneedtoknow.Mygoalistogiveyouamap—anoutlineofalloftheskillsyouneedtodevelopinordertoprogramprofessionally.

PartI:IntroductiontoProgramming.Youwillwriteyourfirstprogramasquicklyaspossible,hopefullytoday.IfyoualreadyknowhowtoprogramyoucanusethissectiontostartlearningPython.IfyoualreadyknowPython,skipit.PartII:IntroductiontoObject-orientedProgramming.Icoverthedifferentprogrammingparadigms—focussingonobject-orientedprogramming—andbuildagamethatwillshowyouthepowerofprogramming.Afterthissection,you’llbehookedonprogramming.PartIII:IntroductiontoProgrammingTools.Youlearntousedifferenttoolstotakeyourprogrammingproductivitytothenextlevel.Bythispointyouarehookedonprogrammingandwanttogetevenbetter.Youwilllearnmoreaboutyouroperatingsystem,howtocollaboratewithotherengineersusingversioncontrol,howtouseyourInteractiveDevelopmentEnvironmenttoboostyourproductivityandhowtoinstallandmanageotherpeople'sprograms.PartIV:IntroductiontoComputerScience.Nowthatyoucanprogram,youwillhaveallkindsofquestionsabouthoweverythingworks.Thissectioniswherealotofthosequestionsgetanswered.Icoveralgorithmsanddatastructures,networkprogrammingandcomputerarchitecture.PartV:ProgrammingforProduction.Youwilllearntoprogramforproduction(createcodethatisactuallyusedbyotherpeople).Icoverthesoftwaredevelopmentprocess,testingandbestprogrammingpractices.PartVI:LandaJob.Thefinalsectionisaboutgettingajobasasoftwareengineer,workingonateamandimprovingasaprogrammer.Iprovidetipsonhowtopassatechnicalinterview,workonateamaswellasadviceonhowtofurtherimproveyourskills.

Ifyoudon’thaveanyprogrammingexperience,youshouldtrytopracticeprogrammingonyourownasmuchaspossiblebetweeneachsection.Thereareadditionalresourcestoexploreprovidedattheendofeachsection.Don’ttrytoreadthisbooktooquickly.Instead,useitasaguideandpracticeforaslongasyouneedinbetweensections.

EndgameFirst

ThewayIlearnedtoprogramistheoppositeofhowComputerScienceisusuallytaught,andIstructuredthebooktofollowthisapproach.Traditionally,youspendalotoftimelearningtheory;somuchso,thatmanyComputerSciencegraduatescomeoutofschoolnotknowinghowtoprogram.Inhisblog,WhyCan’tProgrammers..Program?,JeffAtwoodwrites:“Likeme,theauthorishavingtroublewiththefactthat199outof200applicantsforeveryprogrammingjobcan’twritecodeatall.Irepeat:theycan’twriteanycodewhatsoever.”ThisledAtwoodtocreatetheFizzBuzzcodingchallenge,aprogramming

testusedinprogramminginterviewstoweedoutcandidates.Mostpeoplefailthechallenge,andthat’swhywespendsomuchofthisbooklearningtheskillsyouwilluseinpractice.Don’tworry,wealsolearnhowtopasstheFizzBuzztest.InTheArtofLearning,JoshWaitzkinofSearchingforBobbyFischerfame,describeshowhelearnedhowtoplaychessinreverse.Insteadofstudyingopeningmoves,hestartedlearningtheendgame(wherethereareonlyafewpiecesleftontheboard)first.Thisgavehimabetterunderstandingofthegame,andhewentontowinmanychampionships.Similarly,Ithinkitismoreeffectivetolearntoprogramfirst,thenlearntheorylater,onceyouaredyingtoknowhoweverythingworksunderthehood.ThatiswhyIwaituntilthefourthsectionofthebooktointroduceComputerSciencetheory.Whiletheoryisimportant,itwillbeevenmorevaluableonceyoualreadyhaveprogrammingexperience.

TheSelf-taughtAdvantage

Learninghowtoprogramoutsideofschoolisincreasinglycommon.A2015StackOverflow(anonlinecommunityofprogrammers)surveyfound48percentofrespondentsdidnothaveadegreeincomputerscience10.WhenIwashiredateBay,IwasonateamthatincludedprogrammerswithCSdegreesfromStanford,CalandDuke,aswellastwoPhysicsPhD’s.At25,itwasintimidatingtorealizethatmy21-year-oldteammatesknew10timesmoreaboutprogrammingandcomputersciencethanIdid.

Asintimidatingasitmightbetoworkwithpeoplewhohavebachelor ’s,master ’sandPhD’sinComputerScience,neverforgetyouhavewhatIliketocallthe“self-taughtadvantage.”Youarenotreadingthisbookbecauseateacherassignedittoyou,youarereadingitbecauseyouhaveadesiretolearn,andwantingtolearnisthebiggestadvantageyoucanhave.

WhyYouShouldProgram

Programmingcanhelpyourcareerregardlessofyourprofession.IfyouarereadingthisbookIassumeyouhavealreadydecidedyouwanttolearntoprogram.ButI’mstillgoingtocoverwhyyoushouldtogiveyouextramotivation.Learningtoprogramisempowering.Ilovecomingupwithnewideas,andIalwayshaveanewprojectIwanttoworkon.OnceIlearnedhowtoprogram,Icouldsitdownandbuildmyideaswithoutneedingtofindsomeonetodoitforme.

Programmingwillalsomakeyoubebetterateverything.Seriously.Therearen’tmanysubjectsthatdon’tbenefitfromfinelytunedproblem-solvingskills.Recently,IhadtodotheverytedioustaskofsearchingforhousingonCraigslist,andIwasabletowriteaprogram

todotheworkformeandemailmetheresults.Learningtoprogramwillfreeyoufromrepetitivetasksforever.

Ifyoudowanttobecomeasoftwareengineer,thereisanincreasingdemandforgoodengineersandnotenoughqualifiedpeopletofilltheavailablepositions.By2020,anestimatedonemillionprogrammingjobswillgounfilled45.Evenifyourgoalisn’ttobecomeasoftwareengineer,jobsinfieldslikescienceandfinancearebeginningtofavorcandidateswithprogrammingexperience.

StickingWithIt

Ifyoudon’thaveanyprogrammingexperienceandarenervousaboutmakingthisjourney,Iwantyoutoknowyouareabsolutelycapableofdoingit.Therearesomecommonmisconceptionsaboutprogrammersliketheyallarereallygoodatmath.Thisisn’ttrue,butitishardwork.Luckily,alotofthematerialcoveredinthisbookiseasiertolearnthanyouthink.

Inordertoimproveyourprogrammingskillsyoushouldpracticeprogrammingeveryday.Theonlythingthatwillholdyoubackisnotstickingwithit,solet’sgooversomewaystomakesureyoudo.IusedachecklisttomakesureIpracticedeverydayanditreallyhelpedmestayfocused.

Ifyouneedextrahelp,TimFerriss,aproductivityexpert,recommendsthefollowingtechniquetostaymotivated.Givemoneytofamilyorfriendswiththeinstructionsthatitiseithertobereturnedtoyouuponcompletingyourgoalwithinatimeframeofyourchoosing,ordonatedtoanorganizationyoudislike.

HowThisBookisFormatted

Thechaptersinthisbookbuildononeanother.Ifyouseesomethingyoudon’tunderstand,itmighthavebeencoveredinapreviouschapter.Itrynottore-explainconcepts,sokeepthisinmind.Importantwordsareitalicizedwhentheyaredefined.Thereisavocabularysectionattheendofeachchapterwhereeachitalicizedwordisdefined.

TechnologiesUsedInThisBook

Thisbookteachescertaintechnologiesinordertogiveyouasmuchpracticalprogrammingexperienceaspossible.InsomecasesIhadtochoosebetweenmanydifferentpopulartechnologies.InChapter20:“VersionControl”(forthosereaderswhoare

unfamiliarwithversioncontrol,Iwillexplainlater),wegooverthebasicsofusingGit,apopularversioncontrolsystem.IchoseGitbecauseIconsiderittheindustrystandardforversioncontrol.IusePythonforthemajorityofprogrammingexamplesbecauseitisapopularfirstprogramminglanguagetolearn,anditisaveryeasylanguagetoread,evenifyouhaveneverusedit.ThereisalsoahugedemandforPythondevelopersinjustabouteveryfield.But,Itrytobeastechnologyagnosticaspossible:focusingonconceptsinsteadoftechnologies.

Inordertofollowtheexamplesinthisbookyouwillneedacomputer.Youcomputerhasanoperatingsystem—aprogramthatisthemiddlemanbetweenthephysicalcomponentsofthecomputerandyou.WhatyouseewhenyoulookatyourscreeniscalledagraphicaluserinterfaceorGUI,whichispartofbyyouroperatingsystem.

Therearethreepopularoperatingsystemsfordesktopandlaptopcomputers:Windows,UnixandLinux.WindowsisMicrosoft’soperatingsystem.Unixisanoperatingsystemcreatedinthe1970s.Currently,themostpopularUnixoperatingsystemisApple’sOSX.Linuxisanopen-sourceoperatingsystemusedbythemajorityoftheworld’sservers.Aserverisacomputerorcomputerprogramthatperformstaskslikehostingawebsite.Open-sourcemeansthesoftwareisnotownedbyacompanyorindividual,insteaditismaintainedbyagroupofvolunteers.LinuxandUnixarebothUnix-likeoperatingsystems,whichmeanstheyareverysimilar.ThisbookassumesyouareusingacomputerrunningWindows,OSXorUbuntu(apopularversionofLinux)asyouroperatingsystem.

Vocabulary

FizzBuzz:aprogrammingtestusedinprogramminginterviewstoweedoutcandidates.operatingsystem:Aprogramthatisthemiddlemanbetweenthephysicalcomponentsofthecomputerandyou.open-source:Softwarethatisnotownedbyacompanyorindividual,butisinsteadmaintainedbyagroupofvolunteers.Windows:Microsoft’soperatingsystem.Unix:Anoperatingsystemcreatedinthe1970s.Apple’sOSXisaversionofUnix.Linux:Anopen-sourceoperatingsystemusedbythemajorityoftheworld’sservers.server:Acomputerorcomputerprogramthatperformstaskslikehostingawebsite.

Challenge

Createadailychecklistthatincludespracticingprogramming.

Chapter2.GettingStarted

“Agoodprogrammerissomeonewhoalwayslooksbothwaysbeforecrossingaone-waystreet.”—DougLinder

WhatisProgramming

Programmingiswritinginstructionsforacomputertoexecute.TheinstructionsmighttellthecomputertoprintHello,World!,scrapedatafromtheinternetorreadthecontentsofafileandsavethemtoadatabase.Theseinstructionsarecalledcode.Programmerswritecodeinmanydifferentprogramminglanguages.Inthepast,programmingwasmuchharder,asprogrammerswereforcedtousecryptic,low-levelprogramminglanguageslikeassemblylanguage.Whenaprogramminglanguageislow-levelitisclosertobeingwritteninbinary(0sand1s)thanahigh-levelprogramminglanguage(aprogramminglanguagethatreadsmorelikeEnglish),andthusishardertounderstand.Hereisanexampleofasimpleprogramwritteninanassemblylanguage:

global_start

section.text

_start:

movrax,1

movrdi,1

movrsi,message

movrdx,13

syscall

;exit(0)

moveax,60

xorrdi,rdi

syscall

message:

db"Hello",10

43

Hereisthesameprogramwritteninamodernprogramminglanguage:

print("Hello,World!")

Asyoucansee,programmerstodayhaveitmucheasier.Youwon’tneedtospendtimelearningcryptic,low-levelprogramminglanguagesinordertoprogram.Instead,youwilllearntouseaneasy-to-readprogramminglanguagecalledPython.

WhatisPython

Pythonisanopen-sourceprogramminglanguagecreatedbyDutchprogrammerGuidovanRossumandnamedaftertheBritishsketchcomedygroup,MontyPython’sFlyingCircus.OneofvanRossum’skeyinsightswasthatprogrammersspendmoretimereadingcodethanwritingit,sohecreatedaneasy-to-readlanguage.Pythonisoneofthemostpopularandeasiesttolearnprogramminglanguagesintheworld.Itrunsonallthemajoroperatingsystemsandcomputersandisusedineverythingfrombuildingwebserverstocreatingdesktopapplications.Becauseofitspopularity,thereisalargedemandforPythonprogrammers.

InstallingPython

Inordertofollowtheexamplesinthisbook,youneedtohavePython3(version3ofPython)installed.YoucandownloadPythonforWindowsandOSXathttp://python.org/downloads.IfyouareonUbuntu,Python3comesinstalledbydefault.MakesureyoudownloadPython3,notPython2.SomeoftheexamplesinthisbookwillnotworkifyouareusingPython2.

Pythonisavailableforboth32-bitand64-bitcomputers.Ifyouhaveanewcomputer,purchasedafter2007,itismostlikelya64-bitcomputer.Ifyouaren’tsureifyourcomputeris32-bitor64-bit,aninternetsearchshouldhelpyoufigureitout.

IfyouareonWindowsoraMac,downloadthe64-or32-bitversionofPython,openthefileandfollowtheinstructions.Youcanalsovisit

http://theselftaughtprogrammer.io/installpythonforvideosexplaininghowtoinstallPythononeachoperatingsystem.

Troubleshooting

Fromthispointforward,youneedtohavePythoninstalled.IfyouhavingproblemsinstallingPython,pleaseskipaheadtochapter11tothesectiontitled“GettingHelp”.

TheInteractiveShell

PythoncomeswithaprogramcalledIDLE(shortforinteractivedevelopmentenvironment);itisalsothelastnameofEricIdle,oneofthemembersofMontyPython’sFlyingCircus.IDLEiswhereyouwillbetypingyourPythoncode.Onceyou’vedownloadedPython,searchforIDLEinExplorer(PC),Finder(Mac)orNautilus(Ubuntu).Irecommendcreatingadesktopshortcuttomakeiteasytofind.

ClickontheIDLEicon,andaprogramwiththefollowinglineswillopenup(althoughthiscouldchangesodon’tworryifthemessageisabsentordifferent):

Python3.5.1(v3.5.1:37a07cee5969,Dec52015,21:12:44)

[GCC4.2.1(AppleInc.build5666)(dot3)]ondarwinType"copyright","credits"or"license()"formoreinformation.>>>

Thisprogramiscalledtheinteractiveshell.YoucantypePythoncodedirectlyintotheinteractiveshellandtheshellprintstheresults.Attheprompt(>>>)type:


andpressenter.IDLEmightrejectcodethatiscopiedfromKindle,othereBooksorwordprocessors

likeMicrosoftWord.Ifyoucopyandpastecodeandgetanunexplainableerrormessage,trytypingthecodedirectlyintothewindow.Youmusttypethecodeexactlyasitiswrittenintheexample,includingquotationmarks,parenthesesandanyotherpunctuation.

TheinteractiveshellwillrespondbyprintingHello,World!

Intheprogrammingworld,whenyouteachsomeoneanewprogramminglanguage,itistraditionthatthefirstprogramyouteachthemishowtoprintHello,World!So,congratulations!Youjustwroteyourfirstprogram.

SavingPrograms

Theinteractiveshellisusefulforquickcomputations,testingsmallbitsofcodeandwritingshortprogramsyoudon’tplanonusingagain.YoucanalsouseIDLEtosaveaprogramforreuse.StarttheIDLEapplication,click“File”(inthemenubaronthetopleftoftheIDLEeditor)thenselect“NewFile.”Thiswillopenupatexteditor,whichusuallyhasablankwhitebackground.Youcanwriteyourcodeinthistexteditor,thensaveittorunlater.

Whenyourunyourcode,theoutputofthecodeyouwrotewillappearintheinteractiveshell.Youneedtosaveyourchangeswhileeditingcodebeforerunningitagain.TypetheHello,World!programintothetexteditor:

Goto“File”againandselect“SaveAs.”Nameyourfile“hello_world.py”andsaveit.Pythonfileshavetoendwith.py.Onceyou’vesavedyourfile,click“Run”(again,inthemenubarinthetopleftcorneroftheIDLEeditor)andselect“RunModule.”Alternatively,youcanpresstheF5keycommand,theequivalentofselecting“RunModule”fromthemenubar.HelloWorld!willprintintheinteractiveshell,asifyouhadtypedthislineofcode.However,inthiscase,sinceyousavedyourprogram,youcanrunitasmanytimesasyoulike.

Theprogramyoucreatedissimplyafilewitha.pyextension,locatedonyourcomputerwhereveryousavedit.ThenameIchoseforthefile—“hello_world.py”—iscompletelyarbitrary,youcannamethefileanything.Likethisexample,writingprogramsin

Pythonsimplyinvolvestypingtextintofilesandrunningthemusingtheinteractiveshell.Easy,right?

RunningExamplePrograms

Throughoutthebook,Iwillgiveexamplesofcodeandtheresultsthatprintoutwhenyourunit.WheneverIdothis,youshouldenterthecodeandrunityourself.

Sometimesyoushouldtypethecodeintheinteractiveshell,andsometimesyoushouldtypethecodeintoanew.pyfile.Iwillexplainhowyouwillknowwheretorunyourcodeinthenextchapter.

Thereasonforthisisshortexamplesarebestrunusingtheshell,andthetexteditorisbetterforlongerprogramsyouwanttosaveandedit.Intheinteractiveshell,ifyoumakeamistakeinyourcode—atypoforexample—andthecodedoesn’twork,youhavetotypeeverythingagain.Usingthetexteditorletsyousaveyourwork,soifyoumakeamistake,yousimplyeditthemistakeandreruntheprogram.

Anotherreasonthedistinctionisimportantistheoutputofaprogramrunningfromafileversustheshellcanbeslightlydifferent.Ifyoutype100intotheinteractiveshellandpressenter,theinteractiveshellwilloutput100.Ifyoutype100intoa.pyfileandrunit,therewillbenooutput.Thisdifferencecancauseconfusion,sobemindfulofwhereyouarerunningaprogramfromifyoudonotgetthesameoutputastheexample.

Vocabulary

programming:Writinginstructionsforacomputertoexecute.code:Theinstructionsprogrammerswriteforacomputertoexecute.low-levelprogramminglanguage:Aprogramminglanguageclosertobeingwritteninbinary(0sand1s)thanahigh-levelprogramminglanguage.assemblylanguage:Atypeofdifficulttoreadprogramminglanguage.high-levelprogramminglanguage:AprogramminglanguagethatreadsmorelikeEnglishthanalow-levelprogramminglanguage.Python:Theeasytoread,open-sourceprogramminglanguageyouwilllearntouseinthisbook.CreatedbyGuidovanRossumandnamedaftertheBritishsketchcomedygroup,MontyPython’sFlyingCircus.

Challenge

TrytoprintsomethingotherthanHello,World!.

Chapter3.IntroductiontoProgramming“It’stheonlyjobIcanthinkofwhereIgettobebothanengineerandanartist.There’sanincredible,rigorous,technicalelementtoit,whichIlikebecauseyouhavetodoveryprecisethinking.Ontheotherhand,ithasawildlycreativesidewheretheboundariesofimaginationaretheonlyreallimitation.”—AndyHertzfeld

OurfirstprogramprintedHello,World!.Let’sprintitahundredtimes.Typethefollowingcodeintotheinteractiveshell(printneedstobeindentedbyexactlyfourspaces):

foriinrange(100):print("Hello,World!")

YourshellshouldprintHello,World!ahundredtimes.EventhoughyouwillprobablyneverneedtoprintHello,World!ahundredtimes,thisexamplequicklyshowsyouhowpowerfulprogrammingis.Canyouthinkofanythingelseyoucandoahundredtimessoeasily?Ican’t.Thatisthepowerofprogramming.

Examples

Throughoutthebook,youwillsee“>>”aftereachprogrammingexample.Thisrepresentstheoutputoftheprogram(printedintheinteractiveshell).Ellipses(...)mean“andsoon.”Thepreviousprogramwillbewrittenlikethisifitismeanttobetypedintotheinteractiveshell:

foriinrange(100):print("HelloWorld")

>>HelloWorld>>HelloWorld>>HelloWorld…

Andlikethisifitshouldberunfroma.pyfile:#https://goo.gl/rKQedq

foriinrange(100):print("HelloWorld")

>>HelloWorld

>>HelloWorld>>HelloWorld…

Thedifferencebetweenthetwoexamplesistheadditionof#https://goo.gl/8eou3Zatthetopoftheprogram.Thisiscalledacomment(explainedinthenextsection).TheURL(https://goo.gl/8eou3Z)willtakeyoutoawebpagethatcontainsthecodefromtheexamplesoyoucaneasilycopyandpasteitintotheIDLEtexteditorifyouarehavingproblemsgettingthecodetorun.Wheneveryouseeacommentlikethis,youshouldrunthecodeintheexamplefroma.pyfile.

Inbothexamples,thecolor-codedtextrepresentscode,andallthetextthatcomesafter“>>”representstheoutputoftheinteractiveshell.AnythingwritteninCourierNewfontrepresentssomeformofcodeorcodeoutput.Forexample,ifIrefertothewordforinthepreviousprogramitwillbewrittenintheCourierNewfont.

CourierNewisatypeoffix-width(non-proportional)fontoftenusedtodisplayprogrammingtextbecauseeachcharacterhasthesamewidth,soindentationandotherdisplaycharacteristicsofcodealignmentareeasiertoobserve.

Comments

Acommentisaline(orpartofaline)ofcodewritteninEnglish(oranotherlanguage)precededbyaspecialsymbolthattellstheprogramminglanguageyouareusingtoignorethatline(orpartofaline)ofcode.InPython,thepoundsymbolisusedtocreatecomments.

Acommentexplainswhatalineofcodedoes.Programmersusecommentstoclarifywhytheydidsomethingtomakethelineofcodeeasiertounderstandforwhoeverreadsit.Youcanwritewhateveryouwantinacomment,aslongasitisonlyonelinelong.Hereisanexampleofacommenttakingupanentirelineofcode,followedbyanexampleofacommenttakinguppartofalineofcode:

#Thisisacommentprint("Thecommentdoesnotaffectthiscode")

>>Thecommentdoesnotaffectthiscode

print("Thecommentdoesnotaffectthiscode")#Thisisacomment

>>Thecommentdoesnotaffectthiscode

Youshouldonlywriteacommentifyouaredoingsomethingunusualinyourcode,ortoexplainsomethingthatisnotobviousinthecodeitself.Usecommentssparingly—donotcommentoneverylineofcodeyouwrite—savethemforspecialsituations.Hereisanexampleofanunnecessarycomment:

print("Hello,World!")#thislineofcodeprintsHello,World!

Itisunnecessarybecauseitisveryclearwhatthislineofcodedoes.Hereisanexampleofagoodcomment:

d=math.sqrt(l**2+w**2)#lengthofthediagonal

Evenifyouunderstoodexactlyhowthiscodeworks(andyouwillbetheendofPartI),youstillmightnotknowhowtocalculatethelengthofadiagonalofarectangle,whichiswhythiscommentisuseful.

Printing

Wearenotlimitedtoprinting“Hello,World!”inourprograms.Wecanprintwhateverwe’dlikeaslongaswesurrounditwithquotes:

print("Python")

>>Python

print("Hola!")

>>Hola!

Lines

Pythonprogramsaredividedintolinesofcode.Takealookatthisprogram:

#line1#line2#line3

Therearethreelinesofcode.Itisusefultorefertoeachpieceofcodebythelineitison.InIDLEyoucangoto“Edit”andselect“Gotoline”tojumptoaspecificlineinyourprogram.Sometimesapieceofcodeislongandtakesupmorethanoneline.Whenthishappens,itisokforthecodetoextendtothenextline(whenextendedaccordingtoPython'srules):

print("""Thisisareallyreallyreallyreallyreallyreallyreallyreallylonglineofcode.""")

IwillexplaintherulesforextendinglinesofPythoncodeinPartV.

Keywords

ThePythonlanguagehasalistofwordswithspecialmeaning.Thesearecalledkeywords.forisakeywordwe’vealreadyseenthatisusedtoexecutecodemultipletimes.Wewilllearnmorekeywordsthroughoutthischapter.

Spacing

Let’stakeanotherlookatourprogramthatprintsHello,World!ahundredtimes:

#https://goo.gl/rKQedq

foriinrange(100):print("Hello,World!")

AsInotedearlier,printisindentedbyfourspaces.Wewillcoverwhyshortly,butithastodowithlettingPythonknowwhenblocksofcodebeginandend.Inthemeantime,pleasebeawarethatwheneveryouseeanindentinanexampleitisanindentoffourspaces.Withoutproperspacing,yourprogramwillnotwork.

Spacingisnotusedlikethisinotherprogramminglanguages.Otherapproachesincludeusingkeywordsorbracketsinsteadofspacing.PythonproponentsbelievetherequireduseofproperspacingmakesPythonlesstedioustoreadandwritethanotherlanguages,butthebestapproachtothisproblemisfrequentlydebatedamongstprogrammers.

DataTypes

DifferentkindsofdatainPythonaregroupedintodifferentcategories,ordatatypes.InPython,eachdatavalue,like2or"Hello,World!",iscalledanobject.WewilllearnmoreaboutobjectsinPartII,butfornowthinkofanobjectasadatavalueinPythonwiththreeproperties:anidentity,adatatypeandavalue.Theidentityofanobjectiswhereitisstoredinmemory,whichneverchanges(andwhichwewillbeignoringfornow).Thedatatypeofanobjectisthecategoryofdatatheobjectbelongsto,anddeterminesthepropertiestheobjecthas.Thisalsoneverchanges.Thevalueofanobjectisthedataitrepresents—thenumber2,forexample,hasavalueof2.

"Hello,World!"isanobjectwithadatatypecalledstr,shortforstring,andthevalue"Hello,World!".Whenwerefertoanobjectwithastrdatatype,wecallitastring.Astringis

asequenceofoneormorecharacterssurroundedbyquotes.Youcanusesinglequotesordoublequotes,butthequotesatthebeginningandendofagivenstringmustmatch:

“Hello,World!”>>Hello,World!

‘Hello,World!’>>Hello,World!

Acharactercanbeanysymbolfoundinaunicodecharactertableliketheinteractiveonefoundathttp://unicode-table.com/en(IexplainwhatunicodeisinPartIV).Stringsareusedtorepresenttext,andtheyhaveuniqueproperties.

Thenumbersweusedtodomathintheprevioussectionarealsoobjects—buttheyarenotstrings,theyhaveadifferentdatatype.Wholenumbershavethedatatypeint,shortforinteger.Numberslike2,3,4and10allhavethedatatypeint.Anotherwayofsayingthisistheyareallintegers.Likestrings,integershavetheirownproperties.Forexample,youcandividetwointegers,butyoucannotdividetwostrings.

Althoughwholenumbersareintegers,fractionalnumbers(numberswithadecimalpoint)haveadifferentdatatypecalledfloat.2.1,8.2and9.9999areallexamplesofobjectswiththefloatdatatype.Theyarecalledfloats.Likealldatatypes,floatshavetheirownpropertiesandbehaveinacertainway.Floatsbehavesimilarlytointegers:

2.2+2.2>>4.4

But,therearesomeimportantdifferencesbetweenfloatsandintegersyouwilllearnaboutlater.

ObjectswithabooldatatypehaveavalueofeitherTrueorFalseandarecalledbooleans:

True>>True

False>>False

ObjectswithadatatypeNoneTypealwayshavethevalueNone.ObjectswithaNoneTypedatatypeareusedtorepresenttheabsenceofavalue:

None>>None

Iwillexplainhowthedifferentdatatypesareusedinprogrammingthroughoutthischapter.

ConstantsandVariables

YoucanusePythontodomathjustlikeyouwoulduseacalculator.Youcanadd,subtract,divide,multiply,raiseanumbertoapowerandmuchmore.Remembertotypealloftheexamplesinthissectionintotheshell.

2+2>>4

2-2>>0

4/2>>2

2*2>>4

Aconstantisavaluethatneverchanges.Eachofthenumbersinthepreviousexampleisaconstant:thenumbertwowillalwaysrepresentthevalue2.Avariable,ontheotherhand,referstoavalue;butthatvaluecanchange.Avariableconsistsofanamemadeupofoneormorecharacters.Thatnamegetsassignedtoavalueusingtheassignmentoperator(the=sign).Unlikeaconstant,thevalueofavariablevaluecanchange.

Someprogramminglanguagesrequiretheprogrammertoincludevariable"declarations"thattelltheprogramminglanguagewhatdatatypethevariablewillbe.Pythonmakesitsimpler:youcreateavariablesimplybyassigningavaluetoitusingtheassignmentoperator.Hereisanexampleofcreatingavariable:

b=100b

>>100

Hereishowwecanchangethevalueofavariable:

x=100x

x=200x>>100>>200

Wecanalsousetwovariablestoperformarithmeticoperations:

x=10y=10z=x+yza=x-ya

>>20>>0

Oftenwhenprogramming,youwanttoincrement(increase)thevalueofavariable,ordecrement(decrease)thevalueofavariable.Becausethisissuchacommonoperation,Pythonhasaspecialsyntax—ashortcut—forincrementinganddecrementingvariables.Toincrementavariable,youassignthevariabletoitself,andontheothersideoftheequalssignyouaddthevariabletothenumberyouwanttoincrementby:

x=10x=x+1x>>11

Todecrementavariable,youdothesamething,butinsteadsubtractthenumberyouwanttodecrementby:

x=10x=10-1x

>>9

Theseexamplesareperfectlyvalid,butthereisashortermethodforincrementinganddecrementingvariablesyoushoulduseinstead:x=10x+=1x>>11

x=10x-=1x

>>9

Variablesarenotlimitedtostoringintegervalues—theycanrefertoanydatatype:

my_string=“Hello,World!”>>

my_float=2.2>>

my_boolean=True>>Youcannamevariableswhateveryou’dlike,aslongasyoufollowfourrules:

0.Variablescan’thavespaces.Ifyouwanttousetwowordsinavariable,putanunderscorebetweenthem:i.e.,my_variable=“Astring!”0.Variablenamescanonlycontainletters,numbersandtheunderscoresymbol.0.Youcannotstartavariablenamewithanumber.Althoughyoucanstartavariablewithanunderscore,ithasaspecialmeaningthatwewillcoverlater,soavoidusingituntilthen.0.YoucannotusePythonkeywordsforvariablenames.Youcanfindalistofkeywordshere:http://zetcode.com/lang/python/keywords

Syntax

Syntaxisthesetofrules,principles,andprocessesthatgovernthestructureofsentencesinagivenlanguage,specificallywordorder.70TheEnglishlanguagehasasyntax,andsodoesPython.Forexample,inEnglishyoumustendasentencewithaperiod.Pythonalsohasasetofrulesthatmustbefollowed,soitalsohassyntax.

InPython,stringsarealwayssurroundedbyquotes.ThisisanexampleofPython’ssyntax.ThefollowingisavalidPythonprogram:


ItisavalidprogrambecausewefollowedPython’ssyntaxbyusingquotesaroundourtextwhenwedefinedastring.IfweonlyusedquotesononesideofourtextwewouldviolatePython’ssyntax,andourcodewouldnotwork.Pythondoesnotlikewhenyouviolateitssyntax,andinthenextsectionyoufindoutwhathappenswhenyoudo.

ErrorsandExceptions

IfyouwriteaPythonprogramanddisregardPython’ssyntaxyouwillgetoneormoreerrorswhenyourunyourprogram.Whenthishappens,thePythonshellwillinformyouyourcodedidnotwork,andgivesyouinformationabouttheerrorthatoccurred.LookatwhathappensifyoutrytodefineastringinPythonwithaquoteononlyoneside:

my_string=“HelloWorld.

>>File"/Users/coryalthoff/PycharmProjects/se.py",line1my_string='d^SyntaxError:EOLwhilescanningstringliteral

Thisiscalledasyntaxerror.Syntaxerrorsarefatal.Aprogramcannotrunwithasyntaxerror.Whenyourunaprogramwithasyntaxerror,Pythonletsyouknowaboutitintheshell.Themessagewilltellyouwhatfiletheerrorwasin,whatlineitoccurredonandwhatkindoferroritwas.Althoughthiserrormaylookintimidating,errorslikethishappenallthetime.

Whenthereisanerrorinyourcode,youshouldgotothelinenumbertheproblemoccurredon,andtrytofigureoutwhatyoudidwrong.Inthisexample,youwouldgotoline1ofyourcode.Afterstaringatitforawhile,youwouldeventuallynoticethereisonlyonequote.Tofixit,addaquoteattheendofthestringandreruntheprogram.Fromthispointforward,Iwillrepresenttheoutputofanerrorlikethis:

>>SyntaxError:EOLwhilescanningstringliteral

Foreasierreading,Iwillonlyshowthelastlineoftheerror.Pythonhastwokindsoferrors:syntaxerrorsandexceptions.Anyerrorthatisnota

syntaxerrorisanexception.AZeroDivisionErrorisanexampleofanexceptionthatoccursifyoutrydividingbyzero.

Thedifferencebetweenasyntaxerrorandanexceptionisexceptionsarenotnecessarilyfatal(thereisawaytomakeaprogramrunevenifthereisanexceptionwhichyouwilllearnaboutinthenextchapter).Whenanexceptionoccurs,Pythonprogrammerssay“Python(oryourprogram)raisedanexception”.Hereisanexampleofanexception:10/0

>>ZeroDivisionError:integerdivisionormodulobyzero

Ifyouincorrectlyindentyourcode,yougetanIndentationError:

y=2x=1

>>IndentationError:unexpectedindent

Asyouarelearningtoprogram,youwillfrequentlygetsyntaxerrorsandexceptions(includingoneswedidnotcover),buttheywilldecreaseovertime.Remember,whenyourunintoasyntaxerrororexception,gotothelinewheretheproblemoccurredandstareatituntilyoufigureoutthesolution(orsearchtheinternetfortheerrororexceptionifyouarestumped).

ArithmeticOperators

EarlierweusedPythontodosimplearithmeticcalculationslike4/2.Thesymbolsweusedinthoseexamplesarecalledoperators.OperatorsinPythonaredividedintoseveralcategories,andtheoneswe’veseensofararecalledarithmeticoperators.HereisalistofsomeofthemostcommonarithmeticoperatorsinPython:

Operator Meaning Example Evaluatesto

** Exponent 2**2 4

% Modulo/remainder 14%4 2

// Integerdivision/flooredquotient 13//8 1

/ Division 13/8 1.625

* Multiplication 8*2 16

- Subtraction 7-1 6

+ Addition 2+2 4

Whentwonumbersaredividedthereisaquotientandaremainder.Thequotientistheresultofthedivisionandtheremainderiswhatisleftover.Themodulooperatorreturnstheremainderwhentwonumbersaredivided:

2%2>>0

10%3>>1

Therearetwodifferentoperatorsfordivision.Thefirstis//,whichreturnsthequotient:

14//3>4

Thesecondis/,whichreturnstheresultofthefirstnumberdividedbythesecondasafloatingpointnumber:

14/3>4.666666666666667

Youcanraiseanumberbyanexponentwiththeexponentoperator:2**2>>4

Thevalues(inthiscasenumbers)oneithersideofanoperatorarecalledoperands.Together,twooperandsandanoperatorformanexpression.Whenyourprogramruns,Pythonevaluateseachexpression,andreturnsasinglevalue.Whenyoutypeanexpressionlike2+2intothePythonshell,theexpressionisevaluatedto4.

Theorderofoperationsisasetofrulesusedinmathematicalcalculationstoevaluateanexpression.RememberPleaseExcuseMyDearAuntSally?Itisanacronymusedtohelpyouremembertheorderofoperationsinmathequations:parentheses,exponents,multiplication,division,additionandsubtraction.Parenthesesoutrankexponents,whichoutrankmultiplicationanddivision,whichoutrankadditionandsubtraction.Ifthereisatieamongoperators,likeinthecaseof15/3×2,youevaluatefromlefttoright:.Inthiscasetheansweristheresultof15dividedby3times2.Pythonfollowsthesameorderofoperationswhenitevaluatesmathematicalexpressions:

2+2*2>>6

(2+2)*2>>8

Inthefirstexample,2*2isevaluatedfirst,followedby2+2,becausemultiplicationtakesprecedenceoveraddition.Inthesecondexample,(2+2)isevaluatedfirst,becauseexpressionsinparenthesesarealwaysevaluatedfirst.

ComparisonOperators

ComparisonoperatorsareanothercategoryofoperatorsinPython.Comparisonoperatorsaresimilartoarithmeticoperators;theyareoperatorsusedinexpressionswithoperandsoneitherside.Unlikeexpressionswitharithmeticoperators,expressionswithcomparisonoperatorsevaluatetoeitherTrueorFalse.


> Greaterthan 100>10 True

< Lessthan 100<10 False

>= Greaterthanorequal 2<=2 True

<= Lessthanorequal 22/8 2.75

== Equal 3*5 15

!= Notequal 5-2 3

Anexpressionwiththe>operatorreturnsthevalueTrueifthenumberontherightisbiggerthanthenumberontheleft,andFalseifitisnot:

100>10>>True

Anexpressionwiththe<operatorreturnsthevalueTrueifthenumberontheleftissmallerthanthenumberontheright,andFalseifitisnot:

100<10>>False

Anexpressionwiththe>=operatorreturnsthevalueTrueifthenumberontherightisbiggerthanorequaltothenumberontheleft.OtherwisetheexpressionreturnsFalse:2>=2

>>True

Anexpressionwiththe<=operatorreturnsthevalueTrueifthenumberontherightissmallerthanorequaltothenumberontheleft.OtherwisetheexpressionreturnsFalse:2<=2

>>True

Anexpressionwiththe==operatorreturnsthevalueTrueifthetwooperandsareequal,andFalseifnot:

2==2>>True

1==2>>False

Whereasanexpressionwiththe!=operatorcheckreturnsTrueifthetwooperandsarenotequal,andFalseotherwise:

1!=2>>True

2!=2>>False

Earlier,weassignedvariablestonumbers,likex=100using=.Itmaybetemptingtoreadthisinyourheadas“xequals100,”butdon’t.Aswesawearlier,=isusedtoassignavaluetoavariable,nottocheckforequality.Whenyouseex=100,youshouldthink“xgetsonehundred.”Thecomparisonoperator==isusedtocheckforequality,soifyouseex==100,thenyoushouldthink“xequals100”.

LogicalOperators

Thelastcategoryofoperatorswewillcoverarecalledlogicaloperators.ExpressionswithlogicaloperatorsaresimilartoexpressionswithcomparisonoperatorsinthattheybothevaluatetoTrueorFalse.


and and TrueandTrue True

or or TrueandFalse True

not not notTrue False

ThePythonkeywordandtakestwoexpressionsandreturnsTrueifalltheexpressionsevaluatetoTrue.IfanyoftheexpressionsareFalse,itreturnsFalse.

1==1and2==2>>True

1==2and2==2>>False

1==1and2==1>>False

2==1and1==1>>False

Youcanusetheandkeywordmultipletimesinonestatement:1==1and10!=2and1%1!=3and2<10>>True

WhenPythonevaluatesastatementwiththeandkeyworditstopsevaluatingthestatementassoonasanexpressionevaluatestoFalse.PythondoesnotevaluatetheremainingexpressionsbecauseitisgoingtoreturnFalseregardlessofhowtherestoftheexpressionsevaluate(onlyoneexpressionhastoevaluatetoFalseforFalsetogetreturnedfortheentirestatement).Understandingthisbehaviorcanhelpyouwritemoreefficientcode.Forexample,ifoneofyourexpressionstakesalongtimetoevaluate,youshouldputitlast,becauseitwillbelesslikelytobeevaluatedthere.

ThekeywordortakestwoormoreexpressionsandevaluatestoTrueifanyoftheexpressionsevaluatetoTrue:1==1or1==2>>True1==1or2==2>>True

1==2or2==1>>False

2==1or1==2>>False

Likeand,youcanusemultipleorkeywordsinonestatement:1==1or1==2or1==3or1==4or1==5>>True

TheresultoftheexampleaboveisTruebecause1==1isTrueeventhoughalltherestoftheexpressionsevaluatetoFalse.Likeand,whenevaluatingstatementswithorinthem,assoonasoneexpressionevaluatestoTrue,PythonstopsevaluatingtherestoftheexpressionsbecauseitisgoingtoreturnTrueregardlessofhowtheyevaluate.

Youcanplacethekeywordnotinfrontofanexpression,anditwillchangetheresultoftheexpressionsevaluationtotheoppositeofwhatitwouldhaveotherwiseevaluatedto.IftheexpressionwouldhaveevaluatedtoTrue,itwillevaluatetoFalsewhenprecededbynot,andifitwouldhaveevaluatedtoFalse,itwillevaluatetoTruewhenprecededbynot.

not1==1>>Falsenot1==2>>True

ConditionalStatements

Thekeywordsif,elifandelseareusedinconditionalstatements.Conditionalstatementsareatypeofcontrolstructure:ablockofcodethatcanmakedecisionsbyanalyzingthevaluesofvariables.Aconditionalstatementiscodethatisabletoexecuteadditionalcodecircumstantially.Hereisanexampleinpseudocode(fakecodeusedtoillustrateanexample)tohelpclarifyhowthisworks:

If(expression)Then(code_area1)Else(code_area2)Thispseudocodeexplainsthatyoucandefinetwoconditionalstatementsthatworktogether.IftheexpressiondefinedinthefirstconditionalstatementisTrue,allofthecodedefinedincode_area1getsexecuted.IftheexpressiondefinedinthefirstconditionalstatementisFalse,allofthecodedefinedincode_area2getsexecuted.Thefirstpartoftheexampleiscalledanifstatement,andthelatteriscalledanelsestatement.Together,theyformanifelsestatement:awayforprogrammerstosay“ifthishappensdothis,otherwisedothat”.HereisanexampleofanifelsestatementinPython:

#https://goo.gl/uYp6ha

country="America"ifcountry=="America":print("Hello,America!")else:print("Hello,Canada!")

>>Hello,America!

Lines2and3formanifstatement.Anifstatementismadeupofalineofcodestartingwiththeifkeywordfollowedbyanexpression,acolon,anindentationandoneormorelinesofcodetobeexecutediftheexpressioninthefirstlineevaluatestoTrue.Lines3and4formanelsestatement.Anelsestatementstartswiththeelsekeyword,followedbyacolon,anindentationandoneormorelinesofcodetoexecuteiftheexpressionintheifstatementevaluatestoFalse.

Togethertheyformanifelsestatement.ThisexampleprintsHello,America!becausetheexpressionintheifstatementevaluatestoTrue.IfwechangethevariablecountrytoCanada,theexpressionintheifstatementwillevaluatetoFalse,theelsestatement’scodewillexecuteandourprogramwillprintHelloCanada!instead.

#https://goo.gl/bd4LVW

country="Canada"

ifcountry=="America":print("Hello,America!")else:print("Hello,Canada!")

>>HelloCanada!

Anifstatementcanbeusedonitsown:

#https://goo.gl/jOlzVl

country="America"ifcountry=="America":print("Hello,America!")

>>Hello,America!

Youcanhavemultipleifstatementsinarow:

#https://goo.gl/WBoKWA

x=2ifx==2:print("Thenumberis2.")ifx%2==0:print("Thenumberiseven.")ifx%2!=0:print("Thenumberisnotodd.")

>>Thenumberis2.>>Thenumberiseven.

EachifstatementwillexecuteitscodeonlyifitsexpressionevaluatestoTrue.Inthiscase,thefirsttwoexpressionsevaluatetoTrue,sotheircodeisexecuted,butthethirdexpressionevaluatestoFalse,soitscodeisnotexecuted.

Ifyoureallywanttogetcrazy,youcanevenputanifstatementinsideofanotherifstatement.Thisiscallednesting:

#https://goo.gl/S6Z9rp

x=10y=11

ifx==10:ify==11:print(x+y)

>>21

Inthiscasex+ywillonlyprintiftheexpressionsinbothifstatementsevaluatetoTrue.Anelsestatementcannotbeusedonitsown;itcanonlybeusedattheendofanifelsestatement.

Weusetheelifkeywordtocreateelifstatements.elifstandsforelseif,andelifstatementscanbeindefinitelyaddedtoanifelsestatementtoallowittomakeadditionaldecisions.

Ifanifelsestatementhaselifstatementsinit,theifstatementexpressiongetsevaluatedfirst.IftheexpressioninthatstatementevaluatestoTrue,itscodeisexecutedandnoothercodeisexecuted.However,ifitevaluatestoFalse,eachconsecutiveelifstatementisevaluated.AssoonasanexpressioninanelifstatementevaluatestoTrue,itscodeisexecutedandnomorecodeexecutes.IfnoneoftheelifstatementsevaluatetoTrue,thecodeintheelsestatementisexecuted.Hereisanexampleofanifelsestatementwithelifstatementsinit:

#https://goo.gl/L0OorN

country="Thailand"ifcountry=="Japan":print("Hello,Japan!")elifcountry=="Thailand":print("Hello,Thailand!")elifcountry=="India":print("Hello,India!")elifcountry=="China":print("Hello,China!")else:print("Hello,world!")

>>Hello,Thailand!

HereisanexamplewherenoneoftheexpressionsintheelifstatementsevaluatetoTrue,andthecodeintheelsestatementisexecuted:

#https://goo.gl/Qwb5OD

country="Mars"ifcountry=="America":print("Hello,America!")elifcountry=="Canada":print("Hello,Canada!")elifcountry=="Thailand":print("Hello,Thailand!")elifcountry=="Mexico":print("Hello,Mexico!")else:

print("Hello,world!")

>>Hello,World!

Finally,youcanhavemultipleifstatementsandelifstatementsinarow:

#https://goo.gl/tgcmXN

x=100ifx==10:print("10!")elifx==20:print("20!")else:print("Idon’tknowwhatxis!")

ifx==100:print("xis100!")

ifx%2==0:print("xiseven!")else:print("xisodd!")

>>Idon’tknowwhatxis!>>xis100!>>xiseven!

Statements

AstatementisatechnicaltermthatdescribesvariouspartsofthePythonlanguage.YoucanthinkofaPythonstatementasacommandoracalculation.ThiswillbecomemoreclearaswetakealookatexamplesofdifferentkindsofPythonstatements.Inthissectionwewillalsotakeadetailedlookatthesyntaxofstatements.Don'tworryifsomeofthisseemsconfusingatfirst.Spendasmuchtimeasnecessaryrereadingthissectionandgobacktotheearlierexamplesandcomparethemwithwhatyouarelearninghere.Itmighttakeawhiletounderstandthesyntaxofstatements,butitwillstarttomakemoresensethemoretimeyouspendpracticingPythonandwillhelpyouunderstandseveralprogrammingconcepts.

Pythonhastwokindsofstatements:simplestatementsandcompoundstatements.Simplestatementscanbeexpressedinonelineofcode,whereascompoundstatementsgenerallyspanmultiplelines(butcanbewritteninonelineinsomecircumstances).Herearesomeexamplesofsimplestatements:

print("Hello,World!")>>‘Hello,World!”

2+2>>4

Compoundstatementsgenerallyspanmorethanonelineofcode.You’vealreadyseenmultipleexamplesofcompoundstatements:ifstatements,ifelsestatementsandthefirstprogramwewroteinthischapterthatprinted“Hello,World!”onehundredtimesareallexamplesofcompoundstatements.

Compoundstatementsaremadeupofoneormoreclause.Aclauseconsistsoftwoormorelinesofcode:aheaderfollowedbyasuite(s).Aheaderisalineofcodeinaclausethatcontainsakeywordfollowedbyacolonandasequenceofoneormorelinesofindentedcode.Aftertheindentthereareoneormoresuites.Asuiteisjustalineofcodeinaclause.Thesuitesarecontrolledbytheheader.Ourprogramthatprints“Hello,World!”ahundredtimesismadeupofasinglecompoundstatement:

#https://goo.gl/rKQedqforiinrange(100):print("Hello,World!")

>>HelloWorld>>HelloWorld>>HelloWorld…

Thefirstlineoftheprogramistheheader.It’smadeupofakeyword—for—followedbyacolonandanindentedlineofcode.Aftertheindentationisasuite—print(“Hello,World!”).Inthiscase,theheaderusesthesuitetoprintHello,World!ahundredtimes.Thisiscalledaloop,whichyoulearnmoreaboutinChapter7.Thiscodeonlyhasoneclause.

Acompoundstatementcanalsobemadeupofmultipleclauses.Youalreadysawthiswithifelsestatements.Anytimeanifstatementisfollowedbyanelsestatement,theresultisacompoundstatementwithmultipleclauses.Whenacompoundstatementhasmultipleclauses,theheaderclausesworktogether.Inthecaseofanifelsecompoundstatement,whentheifstatementevaluatestoTrue,theifstatement’ssuitesexecuteandtheelsestatement’ssuitesdonotexecute.WhentheifstatementevaluatestoFalse,theifstatement’ssuitesdonotexecuteandtheelsestatement’ssuitesexecuteinstead.Thelastexamplefromtheprevioussectionhasthreecompoundstatements:

#https://goo.gl/tgcmXN

x=100ifx==10:print("10!")elifx==20:

print("20!")else:print("Idon’tknowwhatxis!")

ifx==100:print("xis100!")

ifx%2==0:print("xiseven!")else:print("xisodd!")

>>Idon’tknowwhatxis!>>xis100!>>xiseven!

Thefirstcompoundstatementhasthreeclauses,thesecondcompoundstatementhasoneclauseandthelastcompoundstatementhastwoclauses.

Onelastthingaboutstatements,statementscanhavespacesbetweenthem.Spacebetweenstatementsdoesnotaffectthecode.Sometimesspacesareusedbetweenstatementstomakecodemorereadable:

print("Michael")print("Jordan")

>>Michael>>Jordan

Vocabulary

comment:Aline(orpartofaline)ofcodewritteninEnglish(oranotherlanguage)precededbyaspecialsymbolthatletstheprogramminglanguageyouareusingknowitshouldignorethatline(orpartofaline)ofcode.keyword:AwordwithaspecialmeaninginPython.YoucanseeallofPython’skeywordsathttp://zetcode.com/lang/python/keywordsconstant:Avaluethatneverchanges.variable:Anameassignedtoavalueusingtheassignmentoperatorassignmentoperator:The=sign.increment:Increasingthevalueofavariable.decrement:Decreasingthevalueofavariable.datatype:Acategoryofdata.object:AdatavalueinPythonwiththreeproperties:anidentity,adatatypeandavalue.

string:Anobjectwithadatatypestr.Itsvalueisasequenceofoneormorecharacterssurroundedbyquotes.integer:Anobjectwithadatatypeint.Itsvalueisawholenumber.float:Anobjectwithadatatypefloat.Itsvalueisafractionalnumber.boolean:Anobjectwithadatatypebool.ItsvalueiseitherTrueorFalse.nonetype:AnobjectwithadatatypeNoneType.ItsvalueisalwaysNone.syntax:Thesetofrules,principles,andprocessesthatgovernthestructureofsentencesinagivenlanguage,specificallywordorder.70syntaxerror:Afatalprogrammingerrorcausedbyviolatingaprogramminglanguage’ssyntax.exception:Anonfatalprogrammingerror.operator:Symbolsusedwithoperandsinanexpression.arithmeticoperator:Acategoryofoperatorsusedinarithmeticexpressions.operand:Avalueoneithersideofanoperator.expression:Codewithanoperatorsurroundedbytwooperands.orderofoperations:Asetofrulesusedinmathematicalcalculationstoevaluateanexpression.comparisonoperator:AcategoryofoperatorsusedinexpressionthatevaluatetoeitherTrueorFalse.logicaloperator:AcategoryofoperatorsthatevaluatetwoexpressionsandreturneitherTrueorFalsecontrolstructure:Ablockofcodethatmakesdecisionsbyanalyzingthevaluesofvariablesconditionalstatement:Codethatisabletoexecuteadditionalcodecircumstantially.ifelsestatement:Awayforprogrammerstosay“ifthishappensdothis,otherwisedothat.”ifstatement:Thefirstpartofanifelsestatement.elsestatement:Thesecondpartofanifelsestatement.elifstatement:Statementsthatcanbeindefinitelyaddedtoanifelsestatementtoallowittomakeadditionaldecisions.statement:Acommandoracalculation.simplestatement:Astatementthatcanbeexpressedinonelineofcodecompoundstatement:Astatementsthatgenerallyspansmultiplelines(butcanbewritteninonelineinsomecircumstances).clause:Thebuildingblocksofcompoundstatements.Aclauseismadeupoftwoormorelinesofcode:aheaderfollowedbyasuite(s).header:Aheaderisalineofcodeinaclausecontainingakeywordfollowedbyacolonandasequenceofoneormorelinesofindentedcode.suite:Alineofcodeinaclausecontrolledbyaheader.

Challenge

Writeaprogramwithavariablecalledageassignedtoanintegerthatprintsdifferentstringsdependingonwhatintegerageis.

Chapter4.Functions

“Functionsshoulddoonething.Theyshoulddoitwell.Theyshoulddoitonly.”—RobertC.Martin

Afunctionisacompoundstatementthatcantakeinput,executeinstructions,andoptionallyreturnanoutput.Callingafunctionmeansgivingthefunctiontheinputitneedstosoitcanexecuteitsinstructionsandoptionallyreturnanoutput.FunctionsinPythonaresimilartofunctionsinmath.Ifyoudon’trememberfunctionsfromalgebra,hereisanexample:

#thefollowingisalgebra,notPythoncodef(x)=x*2

Thelefthalfoftheequationdefinesafunctionf—thattakesoneparameter—x.Aparameterisdatapassedyoupassintoafunctionwhenitscalled.Afunctioncanhaveoneparameter,multipleparametersornoparameters.Afunction’sparametersareitsinput.

Therighthalfoftheequationisthedefinitionofthefunction(theinstructionsitexecuteswhencalled).Therighthandsideoftheequationusesisabletousetheparameterthatwaspassedin(x)tomakeacalculationandreturntheresult(theoutput).Inthiscase,ourfunctiontakesaparameter(x);,multipliesitby2;andreturnstheresult.

InbothPythonandalgebra,youcallafunctionwiththefollowingsyntax:[function_name]([parameters_seperated_by_commas]).Youcallafunctionbyputtingparenthesesafterthefunctionname.TYouputtheparametersgoinsidetheparenthesiswitheachparameterseparatedbyacomma.Ifwecallthefunctionwejustdefinedabove(?)andpassintheparameter4,wegetthefollowingresult:

#thefollowingisalgebra,notPythoncodef(4)>>8

Ifwepassin10asaparameterweget20:

f(10)>>20

Inourfirstexample,wecalledourfunctionwith4asaparameter(x).Ourfunctionexecuteditscode,x*2,andreturnedtheresult—8.Inoursecondexample,wecalledourfunctionwith10asaparameter(x).Ourfunctionexecuteditscode,x*2,andreturnedtheresult—20.

RepresentingConcepts

Thisisabrupt,maybesomethinglike“Asyoumayhavefiguredoutbynow…”Wearenotlimitedtoprinting“Hello,World!”inourprograms.Wecanprintwhateverwe’dlike:

print(“Python!”)

>>Python!

Fromhereonout,IwillusethefollowingconventiontoexplainconceptsliketheideathatyoucanprintanythinginPython:print(“[what_you_want_to_print]”).Thebracketsandthetextinsideofthemrepresentthatyouneedtobereplacedbyapieceofcodesubstituteapieceofcodeinplaceofthem.Whenyouaretryingtofollowanexamplewritteninthisformatlikethis,donottypethebrackets.Thewordsinsideofthebracketsareahintforthecodeyouneedtoreplacethebracketswith.Everythingoutsideofthenotinbracketsrepresentsactualcodethatyoushouldtype.Thisformatisawayofexpressingthatyoucantypeprint(“[what_you_want_to_print]”)intoPython,replace[what_you_want_to_print]withwhateveryouwanttoprint,andPythonwillprintit:

print(“Idonotlikegreeneggsandham.”)print(“TheCatintheHat”)

>>Idonotlikegreeneggsandham>>TheCatintheHat

Programmingisfullofconventions:agreeduponwaysofdoingthings.Thisformatisanexampleofaconventionthatisusedintheprogrammingworldandwillbeusedthroughoutthebook.

DefiningFunctions

TocreateafunctioninPythonwechooseafunctionname,defineitsparameters,definewhatthefunctionwilldo,andwecanchoosetooptionallyreturnavalue(ifwedon’treturnavaluethefunctionwillreturnNone).Weusethefollowingsyntaxtodefineafunction:

def[function_name]([parameters_seperated_by_commas]):[function_definition]

Ourmathematicalfunctionf(x)=x*2lookslikethisinPython:

#https://github.com/calthoff/tstp/blob/master/part_I/functions/df_ex1.py

deff(x):returnx*2

>>

ThekeyworddeftellsPythonyouareabouttodefineafunction.isakeywordusedtodefineafunction.Whenyouuseit,Pythonknowsyouareabouttodefineone.Afterdef,youcannameyourfunctionanythingwhateveryou’dlike.Byconvention,youshouldneverusecapitallettersinayourfunctionname,sandiftherearetwowordsinyourfunctionnameyoushouldseparatethemwithanunderscore—like_this.Onceyou’venamedyourfunction,putparenthesesafterit.Insidetheparentheseis,youputyourparameter(s).Inthis(theprevious?)example,ourfunctiononlyhasoneparameter(x),butifyouwantyourfunctiontoacceptmorethanoneparameter,youmustseparateeachparameterinsidetheparentheseiswithacomma.Aftertheparentheseisyouputacolonandindentbyfourspaces(likeanyothercompoundstatement).Anycodeindentedfourspacesafterthecolonisthefunction’sdefinition.Inthiscase,ourfunction’sdefinitionisonlyoneline—returnx*2.returnisanotherkeyword.Itisusedtodefinethevalueafunctionoutputswhenyoucallit,referredtoasthevaluethefunctionreturns.

TocallafunctioninPython,weusethesyntaxwelearnedearlier:[function_name]().Hereisanexampleofcallingourfunctionwith2asaparameter:

f(2)>>

Youwillnoticetheconsoledidn’tprintanything.Ourfunctionworked,butitjustdidn’tprinttheresultbecausewedidn’ttellPythonto.Ifwewanttoprintthevalueourfunctionreturned,wecansaveourfunction’soutputinavariable:

#https://github.com/calthoff/tstp/blob/master/part_I/functions/df_ex3.pyresult=f(2)print(result)

>>4Youcansavetheresultyourfunctionreturnsinavariablewheneveryouneedtousethevaluelaterinyourprogram.Functionsarenotonlyjustusedtoreturnvalues.Returningavalueisoptional,asisincludingareturnstatementinyourfunction.Asidefromreturningvalues,functionsalsoencapsulatefunctionalityyouwanttoreuse.Forexample:#addgithub

defeven_odd(x):ifx%2==0:print(“even”)else:

print(“odd”)

even_odd(2)even_odd(3)

>>even>>odd

Wedidn’tdefineavalueforourfunctiontoreturn,butourfunctionisstilluseful.ItOurfunctiontestsifx%2==0andprintswhetherxisevenorodddependingontheresult.Remember,moduloreturnstheremainderwhenyoudividetwonumbers.Ifthereisnoremainderwhenyoudivideanumberby2(moduloreturns0)thenumberisbydefinitioneven.Ifthereisaremainder,thenumberisodd.Youmayneedtousethisfunctionalityinseveraldifferentplacesinyourprogram.Itwouldbepoorprogrammingtotypethecodeweusedinourfunction’sdefinitioneverytimeyouwanttousethisfunctionality.That’swhatfunctionsarefor.Youputfunctionalityinafunction,anditletsyoueasilyreusethatfunctionalitythroughoutyourprogram,withouthavingtodoanyextrawork.Thisisalittleconfusing-areyougoingtoexplainlateron?Youcouldmentionthisissomethingyou’llcoverhowtodoinChapter__.

Parameters

Sofar,we’veonlydefinedfunctionsthatacceptoneparameter.Afunctiondoesn’thavetotakeanyparameters.Hereisanexampleofafunctionthatdoesnottakeanyparameters:


deff():return1+1

result=f()print(result)

>>2

Hereisanexampleofafunctionthatacceptsmultipleparameters:


deff(x,y,z):returnx+y+z

result=f(1,2,3)print(result)

>>6

Therearetwotypesofparametersafunctioncanaccept.Theparameterswe’veseensofararecalledrequiredparameters.Whenafunctioniscalled,alloftherequiredparametersmustbepassedintothefunction,oryouwillgetanerrorinyourprogram.Thereisanotherkindofparameter—optionalparameters—thatletthecallerofthefunctionpassinaparameteriftheywantto,buttheydonothaveto.Iftheydonotpassinaparameter,adefaultvaluedefinedbythefunctionwillbeusedinstead.OYoudefineoptionalparametersaredefinedwiththefollowingsyntax:[function_name]([parameter_name]=[paramater_value]).Likerequiredparameters,optionalparameterstheymustbeseparatedbycommas.Hereisanexampleofafunctionthattakesanoptionalparameter:

#https://github.com/calthoff/tstp/blob/master/part_I/functions/op_ex1.py

deff(x=10):ifx==10:print(“xisten”)else:print(“xisnotten”)

default=f()pass_in=f(2)

print(default)print(pass_in)

>>‘xisten’I’mconfusedwhytherearesinglequotesoranyquotesatall.>>‘xisnotten’I’mconfusedwhytherearesinglequotesoranyquotesatall.

First,wecallourfunctionwithoutpassinginaparameter.Becausetheparameterisoptionalwedon’thavetopassitinthisisallowed,andxisassignedthevaluewedefinedinouroptionalparameter—10.Whenourfunctioniscalled,xisequalto10,andso‘xisten’(ifthereshouldbesinglequotesabove,leaveit,ifnottakethemout)prints.

Next—wecallourfunctionagain—butthistimewepassin2asaparameter.Thedefaultvalue10isignoredbecauseweprovidedavaluethistime,andsoxgets2and“xisnotten”(samethingaboutthequotes,evenifI’mjustmissingsomethingIthinktheyshouldbesinglequotes)prints.Youcandefineafunctionwithbothrequiredandoptionalparameters,butthereisonerule:allofyourrequiredparametersmustbedefinedbeforeyouroptionalparameters:

defrequired_optional(x,y=10)returnx+y

>>

PassingParameters

Whenyoudefineafunctionwithparameters,sometimesthoseparametershavetobeaspecificdatatypeinorderforthefunctiontowork.Howdoyoucommunicatethistowhoevercallsyourfunction?Whenyouwriteafunction,itisgoodpracticetoleaveacommentatthetopofthefunctionexplainingwhatdatatypeeachparameterneedstobe.WewilldiscussthisfurtherinPpartV.Whenyougiveafunctionparameterswhenyoucallit,itisreferredtoas“passing”thefunctionparameters.Thissentencemakessense,butisfairlyconfusing.Coulditbe:WhenyougiveafunctionparametersTHENyoucallit,itisreferredtoas“passing”thefunctionparameters.

pass

Wecanusethekeywordpasstocreateafunctionthatdoesnothing:

”””https://github.com/calthoff/tstp/blob/master/part_I/functions/pass_ex1.py”””

deff():pass

f()>>

passThepasskeywordisusefulwheneveryouwanttocreateafunction,butfinishthedefinitionlater.

NestedFunctions

Youcandefineafunctioninsideofafunction.Thisiscallednesting.Thefirstfunctioniscalledtheouterfunction,andthesecondfunctioniscalledtheinner(ornested)function,ornestedfunction.Hereisanexample:

deff():print("InnerFunction!")

defx():print("NestedFunction!")

x()

f()

>>InnerFunction!>>NestedFunction!

Wewillnotbecoveringwhythisisimportantinthisbookbecauseyoudonotneeditisnotimportanttousenestedfunctionswhenyouarelearningtoprogram.,butIwantedtoincludedthisexamplesoyouknowthatitispossible.

Scope

Variableshaveanimportantpropertycalledscopewedidn’tdiscusswhenwefirstcoveredthem.Whenyoudefineavariable,thevariable’sscopereferstowhatpartofyourprogramhasaccesstoit.Thisisdeterminedbywherethevariableisdefinedinyourprogram.Ifyoudefineavariableoutsideofafunction(oraclass,whichwelearnaboutinPartII)thevariablehasaglobalscope:thevariablecanbeaccessedanywhereinyourprogram.Ifyoudefineavariableinsideofafunction(orclass)ithaslocalscope:thevariablecannotbeaccessedanywhereinyourprogram—;itcanonlybeaccessedinthefunction(orclass)itwasdefinedin(oranynestedfunctionsorclasses).Herearesomeexamplesofvariableswithglobalscope:

x=1y=2z=3

>>

Thesevariableswerenotdefinedinsideofafunction(orclass)andthereforehaveaglobalscope.Thismeanswecanaccessthemanywhere—includinginsideofafunction:

x=1y=2z=3

deff():print(x)print(y)print(z)

f()

>>1

>>2>>3Ifwedefinethesesamevariablesinsideofafunction,wecanonlyaccesstheminsideofthatthefunctionwedefinedthemin(orafunctionnestedinsidethefunctionwedefinedthemin).Ifwetrytoaccessthemoutsideofthefunctiontheyweredefinedin,Pythonraisesanexception:

deff():x=1y=2z=3

print(x)print(y)print(z)

>>NameError:name'x'isnotdefined

Ifwesticktousingthesevariablesinsideourfunction,thereisnoproblem:

deff():x=1y=2z=3print(x)print(y)print(z)

f()>>1>>2>>3Ifyouwanttochangethevalueofaglobalvariableinsidealocalscope,youneedtousetheglobalkeywordfollowedbythevariableyouwanttochange:

#https://github.com/calthoff/tstp/blob/master/part_I/functions/scope_ex3.py

x=100

deff():globalxx+=1

print(x)f()

>>101

Thereasonprogramminglanguageshavescopeisbecausehavingnoscope(everyvariablecanbeaccessedanywhereinaprogram)causesproblems.Ifyouhavealargeprogram,andyouwriteafunctionthatusesthevariablex,youmightaccidentlychangethevalueofavariablecalledxthatwaspreviouslydefinedinyourprogram;whichwillchangethebehaviorofyourprogramandmaycauseanerrororunexpectedresults.Thelargeryourprogramgets,andthemorevariablesithas,themorelikelythisbecomes.However,withscope,ifyoudefineavariablexinsideofafunction,thereisaguaranteeyouwillnotaccidentallychangethevalueofanypreviouslydefinedvariablesoutsideofyourfunctionbecauseinordertochangethevalueofavariableoutsideofyourfunction,youmustexplicitlyusetheglobalkeyword.

Built-inFunctions

Pythoncomeswithalibraryofbuilt-infunctions.Theyperformallsortsofdifferentfunctionalityandarereadytousewithoutanyworkonyourpart.We’vealreadyseenoneexampleofabuilt-infunction—thefirstprogramwewroteusedtheprintfunctiontoprint“Hello,World!”.lenisanotherbuiltinfunction.Itreturnsthelengthofanobject—likeastring.Thelengthofastringisthenumberofcharactersinit.

len(“Monty”)>>5

len(“Python”)>>6

typeisanotherbuilt-infunction.Itreturnswhatdatatypeanobjectis:

type(“HelloWorld”)>><type'str'>type(100)>><type'int'>

type(1.0)>><type'float'>

Thebuilt-instrfunctiontakesanobjectandreturnsanewobjectwithastringdatatype.Forexample,wecanusestrtoconvertanintegertoastring.

str(100)>>'100'

inttakesanobjectandreturnsanewobjectwithanintegerdatatype:

int(“1”)>>1

Andfloattakesanobjectandreturnsanewobjectwithanintegerdatatype:

float(100)>>100.0

inputisabuilt-infunctionthatcollectsinformationfromthepersonusingourprogram.

“““https://github.com/calthoff/tstp/blob/master/part_I/introduction_to_programming/input_ex1.py”””

age=input("Howoldareyou?")age=int(age)ifage<21:print("Youareyoung!")else:print("Wowyouareold!")

>>Howoldareyou?

Theinputfunctiontakesastringasaparameteranddisplaysthestringtothepersonusingtheprogramintheshell.Theycanthentypearesponseintotheshell,andwecansavetheirresponseinavariable—inthiscasewesavetheresponseinthevariableage.

Nextweusetheintfunctiontochangeagefromastringtoanintegerbecauseinputcollectsdatafromtheuserasastring,andwewantourvariabletobeanintegersowecancompareittootherintegers.Oncewehaveaninteger,ourifelsestatementdetermineswhichmessagegetsprintedtotheuser,dependingonwhattheytypedintotheshell.Iftheusertypesanumbersmallerthan21,“Youareyoung!”prints.Iftheusertypesanumbergreaterthan21,“Wowyouareold!”prints.

ExceptionHandling

Whenyourelyonuserinputfromtheinputfunction,youdonotcontroltheinputtoyourprogram—theuserdoes,andthatinputcouldcauseanerror.Forexample,saywewriteaprogramtocollecttwonumbersfromauser,andprintouttheresultofthefirstnumberdividedbythesecondnumber:

“““https://github.com/calthoff/tstp/blob/master/part_I/introduction_to_programming/exception_handling_ex1.py”””

a=input(“typeanumber”)b=input(“typeanothernumber”)a=int(a)b=int(b)print(a/b)

>>typeanumber>>10>>typeanothernumber>>5

>>2Ourprogramappearstowork.However,wewillrunintoaproblemiftheuserinputs0asthesecondnumber:

a=input(“typeanumber”)b=input(“typeanothernumber”)a=int(a)b=int(b)print(a/b)>>typeanumber

>>10>>typeanothernumber>>0

>>ZeroDivisionError:integerdivisionormodulobyzero

Ourprogramworks—untiltheuserdecidestoenter0asthesecondnumber,inwhichcaseourprogramraisesanexception.Wecannotallowpeopletousethisprogramandhopetheywillnotenter0asthesecondnumber.Onewaytosolvethisistouseexceptionhandling,whichallowsyouto“catch”exceptionsiftheyoccuranddecidewhattodo.

Thekeywordstryandexceptareusedforexceptionhandling.Wecanchangeourprogramtouseexceptionhandlingsoifauserenters0asthesecondnumber,ourprogramprintsamessagetellingthemnottoenter0insteadofraisinganexception.

InPythonexceptionsareobjects—whichallowsustousetotheminourprograms.EachexceptioninPythonisanobject.Youcanseethefulllistofbuilt-inexceptionshere:https://docs.python.org/3/library/exceptions.html.Wheneveryouareinsituationwhereyouthinkyourcodemayraiseanexception,youcanuseacompoundstatementwiththekeywordstryandexcepttocatchtheexception.

Thetryclausedefinestheerrorthatcouldoccur.Theexceptclausedefinescodethatwillonlyexecuteiftheexceptiondefinedinyourtryclauseoccurs.Hereisanexampleofhowwecanuseexceptionhandlinginourprogramsoifauserenters0asthesecondnumberourprogramdoesn’tbreak:

“““https://github.com/calthoff/tstp/blob/master/part_I/introduction_to_programming/exception_handling_ex2.py”””

a=input(“typeanumber”)b=input(“typeanothernumber”)try:print(a/b)exceptZeroDivisionError:print(“bcannotbezero.Tryagain.”)

>>typeanumber>>10>>typeanothernumber>>0>>“bcannotbezero.Tryagain.”

Iftheuserentersanythingotherthan0,thecodeinourtryblockisexecutedandourexceptblockdoesn’tdoanything.Butiftheuserenters0,insteadofraisinganexception,thecodeinourexceptblockisexecutedandourprogramprints“bcannotbezero.Tryagain.”.

Docstrings

Docstringsarecommentsatthetopofafunctionormethodthatexplainwhatthefunctionormethoddoes,anddocumentswhattypesoftheparametersshouldbepassedtoit.Hereisanexample:

defadd(x,y):"""Returnsx+y.:paramx:intfirstintegertobeadded.:paramy:intsecondintegertobeadded.

:return:intsumofxandy."""returnx+y

Thefirstlineofthedocstringclearlyexplainswhatourfunctiondoes.Whenotherdevelopersreuseyourfunctionormethod,theydonotwanttohavetoreadthroughallofyourcodetofigureoutwhatitdoes.Therestofthelinesofthedocstringliststhefunction’sparameters,itsreturnvalue,andsomeadditionalinformation,includingthetypeforalloftheparametersandthereturnvalue.Docstringswillhelpyouprogramfaster,becauseifyouforgetwhatapieceofcodedoes,youcanquicklyfigureitoutbyreadingthedocstringinsteadofallofthecodeinafunction,classormethod.Itwillalsomakeitmucheasierforotherprogrammerstouseyourcode.InsomecasesI’veomitteddocstringsInormallywouldhaveincludedthemtomakemycodeasconciseaspossibleforeasyreading—butwheneverIamwritingcodeforproduction(codethatisactuallyusedbyotherpeople)—Iusedocstrings.

Challenge

Writeafunctionthatdoessomethinginteresting,anduseitseveraltimesinaprogram.

Chapter5.Containers

"Badprogrammersworryaboutthecode.Goodprogrammersworryaboutdatastructuresandtheirrelationships."—LinusTorvalds

Inchapter3,welearnedhowtostoreobjectsinvariables.Inthischapterwelearntostoreobjectsincontainers—specialobjectsthatcanstoreandretrieveotherobjects(likestrings).Inthischapter,wewillgooverthreecommonlyusedcontainers:lists,tuplesanddictionaries.

Lists

Alistisamutablecontainerthatstoresobjectsinaspecificorder.Whenacontainerismutableitmeanstheobjectsinthethecontainercanchange—objectscanbeaddedandremovedfromthecontainer.

[image]

ListsarerepresentedinPythonwithbrackets.Therearetwosyntaxestocreatealist.Wecancreateanemptylistwiththelistfunction:

new_list=list()new_list>>[]

Orwecancreateanemptylistwithbrackets:

new_list=[]new_list>>[]

Bothsyntaxescreateanewemptylist.Whenyoucreateanewlistwiththelistfunctionyoucanalsopassinobjectsyouwanttoaddtoyourlistasparameters:

my_list=list(“Apple”,“Orange”,“Pear”)my_list

>>['Apple','Orange','Pear']

Orlikethisusingthesecondsyntax:

my_list=[“Apple”,“Orange”,“Pear”]

my_list


Eachobjectinalistiscalledaniteminthelist.Inthisexampletherearethreeitemsinourlist:‘Apple’,‘Orange’and‘Pear ’.Listskeeptheiritemsinorder—theordertheitemsenteredthelist.Unlesswechangetheorderofourlist,‘Apple’willalwaysbethefirstitem,‘Orange’theseconditemand‘Pear ’thethirditem.‘Apple’isatthebeginningofthelist,and‘Pear ’isattheend.Wecanaddanewitemtotheendofourlistusingtheappendfunction:

my_list.append(“Banana”)my_list.append(“Peach”)my_list

>>['Apple','Orange','Pear',‘Banana’,’Peach’]

Listsarenotlimitedtostoringstrings—theycanstoreanydatatype:

new_list=[]new_list.append(True)new_list.append(100)

new_list.append(1.1)new_list.append(‘Hello’)

>>[True,100,1.1,‘Hello’]

Everyiteminalisthasapositioninthelist—calleditsindex.Youcanfigureouttheindexofanyiteminalistbystartingatthebeginningofthelistandcounting.Theonlytrickypartisyouhavetostartcountingatzero,becausethefirstiteminalisthasanindexofzero.Sothefirstiteminalistisatindexzero,theseconditeminalistisindexone,andsoon.Countingstartingatzerotakessomegettingusedto,sodon’tworryifitfrustratesyouatfirst.Youcanaccesseachiteminalistwithitsindexusingthesyntax[list_name][[index]].Iputindexintwobracketstorepresentthat[index]shouldbereplaced,butshouldbeinsidebrackets.

my_list=[“Apple”,“Orange”,“Pear”]my_list[0]my_list[1]my_list[2]

>>Apple>>Orange>>Pear

Youcanchangeaniteminalistbysettingtheindexoftheitemtoanewobject:

color_list=[“blue”,“green”,“yellow”]

color_listcolor_list[2]=“red”color_list

>>[“blue”,“green”,“yellow”]>>[“blue”,“green”,“red”]

Ifyoutrytoaccessanindexthatdoesn’texist,Pythonwillraiseanexception:

color_list=[“blue”,“green”,“yellow”]color_list[4]

>>IndexError:listindexoutofrange

Youcanremovethelastitemfromalistwithpop:

color_list=[“blue”,“green”,“yellow”]color_listitem=color_list.pop()

itemcolor_list

>>[“blue”,“green”,“yellow”]>>“yellow”>>[“blue”,“green”]

Youcannotpopfromanemptylist,ifyoudoPythonwillraiseanexception.Youcancheckifanitemisinalistusingthekeywordin:

color_list=[“blue”,“green”,“yellow”]“green”incolor_list

>>True

Addthekeywordnottocheckifanitemisnotinalist:

color_list=[“blue”,“green”,“yellow”]“black”notincolor_list

>>True

Youcangetthesizeofalist(thenumberofitemsinit)withthelenfunction:

len(color_list)>>3

Finally,youcangetarangeofitemsinalistwithslicing.Weslicealistwithastartindexandanendindexseparatedbyacoloninbracketsoutsideofourlist.Slicingreturnsa

newlist(a“slice”oftheoldone)madeupofeverythingbetweenthestartandendindex.Thesyntaxforslicingis[list_name][[start_index:end_index]].Hereisanexampleofslicingalist:

new_list=['Apple','Orange','Pear',‘Banana’,‘Peach’]new_list[0:3]


A“gotcha”withslicingisthestartindexincludestheitematthatindex,buttheendindexdoesn’tincludetheitemattheendindex,itonlyincludestheitembeforetheendindex.Thismeansifyouwanttoslicefrom“Apple”to“Pear”,youneedtoslicefromindex0,toindex3(insteadofindex2),becausetheitemattheendindexisnotincludedintheslice.

Tuples

Atupleisanimmutablecontainerthatstoresobjectsinaspecificorder.Whenacontainerisimmutableitmeansthecontentsofthecontainercannotchange.Thatmeansunlikealist,onceyouputanobjectintoatuple,youcannolongerchangeit.Onceyoucreateatupleyoucannotchangethevalueofanyoftheitemsinit,youcannotaddnewitemstoit,andyoucannotremoveitemsfromit.Tuplesarerepresentedwithparenthesis.Therearetwosyntaxestocreateatuple:

my_tuple=tuple()my_tuple

>>()

Andmy_tuple=()my_tuple>>()

Ifyouwantyourtupletocontainobjects,youmustaddthemtoyourtuplewhenyoucreateit.Hereishowyouadditemstoatupleusingthefirstsyntax:

my_tuple=tuple(“brown”,“orange”,“yellow”)my_tuple

>>(“brown”,“orange”,“yellow”)

Andthesecond:

my_tuple=(“brown”,“orange”,“yellow”)

my_tuple

>>(“brown”,“orange”,“yellow”)

Atuplewithoneiteminitstillneedsacommaaftertheitem:

(‘self_taught’,)

>>(‘self_taught’,)

Onceyou’vecreatedyourtuple,ifyoutrytoaddanobjecttoit,Pythonwillraiseanexception:

my_tuple=(“brown”,“orange”,“yellow”)my_tuple[1]=“red”

>>TypeError:'tuple'objectdoesnotsupportitemassignment

Youcan,however,accessdatafromatuplelikealist—youcanreferenceanindexandsliceatuple:

my_tuple=(“brown”,“orange”,“yellow”)my_tuple[0]

my_tuple[1:2]

>>yellow>>('yellow','orange')

Youcancheckifanitemisinatupleusingthekeywordin:

my_tuple=(“brown”,“orange”,“yellow”)“brown”inmy_tuple

>>True

Addthekeywordnottocheckifanitemisnotinatuple:

my_tuple=(“brown”,“orange”,“yellow”)“black”notinmy_tuple

>>True

Youmaybewonderingwhyyouwouldwanttouseadatastructurethatappearstobelikealist,butlesshelpful.Tuplesareusefulwhenyouaredealingwithvaluesyouknowwillneverchange,andyoudon’twantotherpartsofyourprogramtohavetheabilitytochangethosevalues.Agoodexampleisifyouareworkingwithgeographiccoordinates.YoumaywanttostorethelongitudeandlatitudeofNewYorkinatuplebecauseyouknowthe

longitudeandlatitudeofNewYorkisnevergoingtochange,andyouwanttomakesureotherpartsofyourprogramdon’thavetheabilitytoaccidentallychangethem.

Dictionaries

Dictionariesareanotherbuilt-incontainerforstoringobjects.Theyaremutable—butunlikelistsandtuples—theydonotstoreobjectsinaspecificorder.Instead,dictionariesareusedtomaponeobject(calledthekey)toanotherobject(calledthevalue).Dictionariesarerepresentedwithcurlybrackets.Therearetwosyntaxesforcreatingdictionaries:

my_dict=dict()my_dict

>>{}

And:

my_dict={}my_dict

>>{}

Youaddobjectstoadictionarybymappingakeytoavalue.Eachkeymappedtoavalueinadictionaryiscalledakeyvaluepair.Hereishowyoucreatekeyvaluepairswhenyoucreateadictionarywiththefirstsyntax:

my_dict=dict({“Apple”:“Red”,“Banana”:“Yellow”})my_dict

>>{'Apple':'Red','Banana':'Yellow'}

Andthesecond:

my_dict={“Apple”:“Red”,“Banana”:“Yellow”}my_dict

>>{'Apple':'Red','Banana':'Yellow'}Bothsyntaxeshaveakeyseparatedfromavaluebyacolon.Eachkeyvaluepairmustbeseparatedbyacomma.Unlikeatuple,ifyouhavejustonekeyvaluepair,youdonotneedacommaafterit.Onceyou’veaddedkeyvaluepairstoadictionary,youcanuseakeytolookupavalue.Youcanonlyuseakeytolookupavalue.Youcannotuseavaluetolookupakey:

my_dict[‘Apple’]

>>Red

Dictionariesaremutable,soonceyou’vecreatedoneyoucanaddmorekeyvaluepairswiththesyntax[my_dictionary][[key]]=[value]:

my_dictionary=dict()

my_dictionary[“programming”]=“awesome”my_dictionary[“programming”]

my_dictionary[“BillGates”]=“rich”my_dictionary[“BillGates”]

my_dictionary[“america_founded”]=1776my_dictionary[“america_founded”]

>>awesome>>rich>>1776

Youcanusetheinkeywordtocheckifakeyisinadictionary.Youcannotusetheinkeywordtocheckifavalueisinadictionary.

“BillGates”inmy_dictionary

>>True

Addthekeywordnottocheckifakeyisnotinadictionary:

“BillPlates”notinmy_dictionary

>>True

Finally,youcandeleteakeyvaluepairfromadictionarywiththekeyworddel

my_dictionarydelmy_dictionary['BillGates']

my_dictionary

>>{'america_founded':1776,'programming':'awesome','BillGates':'Rich'}

>>{'america_founded':1776,'programming':'awesome'}

Challenge

Lists,tuplesanddictionariesarejustafewofthecontainersbuilt-intoPython.TakesometimetolookupandreadaboutPythonsets.Whatisasituationwouldyouuseasetin?

Chapter6.StringManipulation

“Intheory,thereisnodifferencebetweentheoryandpractice.But,inpractice,thereis.”—JanL.A.vandeSnepscheut

Pythonhasbuilt-infunctionalityformanipulatingstrings,suchaschangingastring’scaseorsplittingastringintotwopartsatagivencharacter.Thisfrequentlycomesinhandy.Sayforexample,youhaveastringINALLCAPSandyouwanttochangeittolowercase.Luckily,withPython,wecaneasilyfixthisproblem.InthischapterwewilllearnmoreaboutstringsandgooversomeofPython’smostusefultoolsformanipulatingstrings.

TripleStrings

Ifastringismorethanoneline,youneedtoputitintriplequotes:

“““lineonelinetwolinethree”””

Ifyoutrytodefineastringthatspansmorethanonelinewithsingleordoublequotes,youwillgetasyntaxerror.

Indexes

Stringsareiterable.Youcanaccesseachcharacterinastringwithitsindex,justlikeyoucanaccesseachiteminatuple.Liketuples,thefirstcharacterinastringstartswithindex0andeachsubsequentindexisincrementedby1.

my_string=“LAX”my_string[0]my_string[1]my_string[2]

>>‘L’>>‘A’>>‘X’

Inthisexampleweusedtheindexes0,1,and2toaccesseachofthecharactersinthestring“LAX”.Ifwetrytoaccessanelementpastthelastelement,Pythonraisesanexception:

my_string=“LAX”my_string[3]

>>IndexError:stringindexoutofrange

StringsareImmutable

Strings,liketuples,areimmutable.Youcannotchangecharactersinastring.Ifyouwanttochangethecharactersinastring,youneedtocreateanewstring.

Methods

Inchapter4,welearnedaboutfunctions.Inthischapter,willbeusingaconceptsimilartofunctions—methods—tomanipulatestrings.WelearnmoreaboutmethodsinPartIIofthisbook—butfornowyoucanthinkofmethodsasfunctionsthatobjects“come”with.

Youcanpassparameterstoamethod,whichcanexecutecodeandreturnaresultjustlikeafunction.Unlikeafunction,amethodgetscalledonanobject.Forexampleifwehaveastring“Hello”,wecouldcall“Hello”.[method_name]()onourstring.Otherthanbeingcalledonanobject,youcanthinkofamethodasthesamethingasafunction(fornow).

ChangeCase

Youcanchangeastringsoeveryletterisuppercasebycallingtheuppermethodonit:

“““Ifcomputerprogrammingwereacountry,itwouldbethethirdmostdiverseforlanguagesspoken.”””.upper()

>>“““IFCOMPUTERPROGRAMMINGWEREACOUNTRY,ITWOULDBETHETHIRDMOSTDIVERSEFORLANGUAGESSPOKEN”””

43

Similarly,youcanchangeeveryletterinastringtolowercasebycallingthelowermethodonit:

“““AdaLovelace,thedaughteroftheEnglishpoetLordByron,isconsideredtobethefirstcomputerprogrammer.”””.lower()

>>“adalovelace,thedaughteroftheenglishpoetlordbyron,isconsideredtobethefirstcomputerprogrammer”

42

Youcanalsocapitalizethefirstletterofeverywordinasentencebycallingthecapitalizemethodonastring:

”””youcanbuildacomputerusinganythingthatcanimplementaNAND-gateandtheconceptofzero(i.e.,somethingandnothing).allTuring-completeprogramminglanguagesareequallypowerful(ignoringpracticalities).lispappearedin1958andisstillregardedasbeingamongthemorepowerfulprogramminglanguagestoday.”””.capitalize()

>>”””YoucanbuildacomputerusinganythingthatcanimplementaNAND-gateandtheconceptofzero(i.e.,somethingandnothing).AllTuring-completeprogramminglanguagesareequallypowerful(ignoringpracticalities).Lispappearedin1958andisstillregardedasbeingamongthemorepowerfulprogramminglanguagestoday.”””40

format

Sometimesyouwillwanttocreateastringusingvariables.Thisisdonewiththeformatmethod:year_started=“1989”“Pythonwascreatedin{}.”.format(year_started)>>‘Pythonwascreatedin1989.’

Theformatfunctionlooksforanyoccurrencesof{}inthestringandreplacesthemwiththevaluesyoupassintoformat.Youarenotlimitedtousing{}once,youcanputasmanyoftheminyourstringasyou’dlike:

#https://github.com/calthoff/tstp/blob/master/part_I/string_manipulation/format.py

year_started=“1989”

creator=“GuidovanRossum”country=“theNetherlands”my_string=“Pythonwascreatedin{}by{}in{}.”.format(year_started,creator,country)

print(my_string)>>‘Pythonwascreatedin1989byGuidovanRossuminthe

Netherlands.’

split

Stringshaveamethodcalledsplitusedtoseparateonestringintotwostrings.Youpassthesplitmethodthecharacterorcharactersyouwanttousetoseparatethestring—forexample,wecanpassinaperiodtoseparatethisquotebyDanielCoyleintotwodifferentstrings:

”””Practicedoesn’tmakeperfect.Practicemakesmyelin,andmyelinmakesperfect.”””.split(“.”)

>>["Practicedoesn'tmakeperfect",'Practicemakesmyelin,andmyelinmakesperfect','']

Theresultisalistwithtwodifferentstringssplitattheperiodintheoriginalstring.

join

Thejoinmethodletsyouaddnewcharactersinbetweeneverycharacterinastring:my_string=‘abc’join_result=‘+’.join(my_string)join_result

>>‘a+b+c’

Youcanturnalistofstringsintoasinglestringbycallingthejoinmethodonanemptystring(“”)andpassinginalist:

the_Fox=[“The”,“fox”,“jumped”,“over”,“the”,“fence”,“.”]

one_string=“”.join(the_Fox)one_string

>>Thefoxjumpedoverthefence.

replace

Thereplacemethodletsyoureplaceeveryoccurrenceofacharacter(s)withanothercharacter(s).Thefirstparameteristhecharacter(s)toreplaceandthesecondparameteristhecharacter(s)toreplaceitwith.

my_string=“Thecatjumpedoverthehat.”my_string=my_string.replace(“a”,“@”)my_string

>>“Thec@tjumpedovertheh@t.”

index

Wecangettheindexofthefirstoccurrenceofacharacterinastringwiththeindexmethod.Wepassinthecharacterwearelookingfor,andwegettheindexofthefirstoccurrenceofthecharacterinthestring:‘cat’.index(‘a’)>>1

in

TheinkeywordchecksifonestringisinanotherstringandreturnsTrueorFalse:

“Playboy”in”””ApicturefromPlayboymagazineisthemostwidelyusedforallsortsofimageprocessingalgorithms”””

>>True42

Addthekeywordnotinfrontofintocheckifonestringisnotinanotherstring:

“hello”notin”””Thecomputerviruswasinitiallydesignedwithoutanyharmfulintentions”””

>>True42

EscapingStrings

Whatifyouwanttousequotesinsideastring?Ifweusequotesinsideastringwegetasyntaxerror:

”””SunTzusaid"TheSupremeartofwaristosubduetheenemywithoutfighting.""“““

>>SyntaxError:invalidsyntaxWecansolvethiswithescaping,whichmeansputtingaspecialsymbolinfrontofacharacterthathasspecialmeaninginPython(inthiscasethespecialcharacterisaquote),toletPythonknowthatthisparticularcharacterismeanttobeacharacter,andnotthespecialPythonsymbolitusuallyrepresents.Thespecialsymbolweusetoescapeourquoteisabackslash.

“““SunTzusaid\"TheSupremeartofwaristosubduetheenemywithoutfighting.\"”””

>>'SunTzusaid"TheSupremeartofwaristosubduetheenemywithoutfighting."'

Newline

Wecanuse“\n”insideastringtorepresentanewline:

print(“line1\nline2”)>>‘line1’>>‘line2’

Concatenation

Wecanaddtwostringstogetherusingtheadditionoperator.Theresultwillbeonestringwiththecharactersfromthefirststringfollowedbythecharactersfromthenextstrings.Thisiscalledconcatenation:

“cat”+“in”+“the”+“hat”>‘catinthehat’

“cat”+“in”+“the”+“hat”>>‘catinthehat’

StringMultiplication

Wecanalsomultiplyastringbyanumberwiththemultiplicationoperator:

“cat”*3>>‘catcatcat’

Chapter7.Loops

ThesecondprogramwelearnedhowtowriteprintedHello,World!ahundredtimes.Wedidthisbyusingaloop.Loopsarecompoundstatementsthatletusexecutecodeoverandoveragainacertainnumberoftimes,oraslongasaconditionisTrue.Inthischapterwewillgoovertwokindsofloops—forloopsandwhileloops.

ForLoops

Forloopsexecuteasetofinstructionsacertainnumberoftimes.Yougiveaforloopanumbertostartat(wewillcallthisnumbera),anumbertostopat(wewillcallthisnumberz),asetofinstructions,andavariablethatkeepstrackofthenumberoftimestheinstructionshavebeenexecuted(wewillcallthisvariablei).Youcanthinkofaforloopasacircle.Itgoesroundandroundexecutinginstructions.Everytimetheloopexecutesitsinstructions,i(whichstartsatthevalueofa)getsincrementedby1.Whenibecomesequaltoz,thetheforloopstops.

Youdefineaforloopwiththesyntaxfor[variable_name]inrange(a,z):[code_to_execute]whereaisthenumbertostartat,zisthenumbertostopat,and[variable_name]isavariablenameofyourchoosingthatgetsassignedtowhatevernumbertheloopison(we’vebeenreferringtoitasi)andcode_to_executeisthecodeyoudefinethatisexecutedeachtimearoundtheloop.Thisalliseasiertounderstandwithanexample:

#https://github.com/calthoff/tstp/blob/master/part_I/loops/for_loops_ex1.py

foriinrange(0,10):print(i)

>>0>>1…>>9

Inthisexamplewechoseiasourvariable_name.ThisisthevariablenamepeopleinthePythoncommunityusuallyusewhentheywriteaforloop.Yourcode_to_executehasaccesstothevariablei.

Whenwerunthisprogram,ourprogramentersourforloop.astartsat0,zstartsat10andistartsatthesamevalueasa—0.Thefirsttimearoundtheloopiis0,andourforloopexecutesitsinstructions—print(i)—whichprints0(becauseiisequalto0).Thenexttimearoundtheloop,iisincrementedby1,soiisnow1,andso1getsprinted.Thenext

timearoundtheloop,iisincrementedby1again,andsonowiis2,andso2getsprinted.Eventually,iwillequal10.Whenthishappens,nothingwillprint,becauseiisequaltozandsotheloopends.

Whenaloopends,itsinstructionsstopsexecuting,andPythonmovesontothenextlineofcodeaftertheloop.Thatiswhy9isthelastnumberprinted.Ifthereweremorecodeafterourloop,Pythonwouldexecuteit,butinthisexamplethereisnot,andsotheprogramends.Thereisanotherforloopsyntaxwecanuseforiteration.Iterationmeansgoingonebyonethroughaniterable.Aniterableisanobjectthathasindexes.Someexamplesofiterablesarestrings,lists,tuplesanddictionaries.Iterationisdonewiththesyntaxforvariable_nameiniterable:[code_to_execute].Justlikethefirstsyntax,wechoseavariablenameanddefinethecodetobeexecutedeachtimearoundtheloop.Inthissyntax,insteadofourloopexecutingcodeuntiliisequaltoz,ourloopstartsatthefirstindexintheiterable,andstopsafterthelastindex.Alsovariable_namegetsassignedtotheitemattheindexweareat(intheiterable).Hereisanexample:


my_string=“Python”forcharacterinmy_string:print(character)

>>‘P’>>‘y’>>‘t’>>‘h’>>‘o’>>‘n’


my_list=[“a”,“b”,“c”]foriteminmy_list:print(character)>>‘a’>>‘b’>>‘c’

#addgithub

my_tuple=(“a”,“b”,“c”)foriteminmy_tuple:print(character)

>>‘a’

>>‘b’>>‘c’

#addgithub

my_dict={“self”:“taught”,“programming”:“wizard”}forkeyinmy_dict:print(key)

>>“self”>>“wizard”

Eachoftheseexamplesloopsthroughaniterable,andprintseachiteminit.Youwillnoticeweusedseveraldifferentvariablenamesforvariable_name.Inthissyntax,insteadofusingi,youwanttouseadescriptivevariablename.Inthefirstexample,weusedcharacter,becauseeachitematanindexinastringiscalledacharacter.Inthesecondandthirdexamples,weuseditem,becauseeachobjectinalistortupleiscalledanitem.Inthelastexample,weusedkey,becausewhenyouiteratethroughadictionarylikethis,youcanonlyaccesseachkeyinthedictionary,notthevalue—sowechoseavariablenametomakethisclear.

Beingabletoloopthroughaniterableisveryuseful.Iterablesareusedtostoredata,andyoucanuseforloopstoloopthroughyourdatatoeasilymakechangestoit,ormovethedatafromoneiterabletoanother.

WhileLoops

Whileforloopsexecutecodeacertainnumberoftimes,whileloopsexecutecodeaslongastheexpressioninitsheaderevaluatestoTrue.Thesyntaxforcreatingawhileloopiswhile[expression]:[code_to_execute].Likeaforloop,awhileloopgoesaroundlikeacircleexecutingcode.Thedifferenceisinsteadofexecutingcodeasetamountoftimes,awhileloopexecutescodeaslongastheexpressionwedefineinitsheaderevaluatestoTrue.

IfweuseanexpressionthatalwaysevaluatestoTrue,ourloopwillrunforever.Thisiscalledaninfiniteloop.Writinganinfiniteloopiseasy(bepreparedtopresscontrol-conyourkeyboardinthePythonshell.Itistheonlywaytostopaninfiniteloopfromrunning).

#https://github.com/calthoff/tstp/blob/master/part_I/loops/while_loops_ex1.py

whileTrue:print(“Hello,World!”)

>>HelloWorld...

BecauseawhilelooprunsaslongasitsexpressionevaluatestoTrue—andTruealwaysevaluatestoTrue—thisloopwillrunforever,continuouslyexecutingthecodewedefined.Inotherwords,ourprogramwillneverstopprinting“Hello,World!”.Nowlet’slookatawhileloopwithanexpressionthatwilleventuallyevaluatetoFalse:#https://github.com/calthoff/tstp/blob/master/part_I/loops/while_loops_ex2.py

x=10whilex>0:

print(‘{}’.format(x))x-=1print(“HappyNewYear!”)

>>10>>9>>8>>7>>6>>5>>4>>3>>2>>1>>‘HappyNewYear!’

OurwhileloopwillexecuteitscodeaslongastheexpressionwedefinedisTrue.Inthiscase,thatmeansitwillexecuteitscodeaslongasx>0.xstartsat10(wedefinedxbeforewecreatedourwhileloop).Thefirsttimethroughourloop,xis10,sox>0evaluatestoTrue.Ourwhileloop’scodeprintsxandthendecrementsxby1—xisnow9.Thenexttimearoundtheloopxgetsprintedandxbecomes8.Thiscontinuesuntilxbecomes0,atwhichpointx>0evaluatestoFalseandourloopends.Pythonthenexecutesthenextlineofcodeafterourloop—print(“HappyNewYear!”)—whichprints“HappyNewYear!”.

Break

Youcanprematurelyendafororwhileloopwiththekeywordbreak.Forexample,thefollowingloopwillrunonehundredtimes:

#https://github.com/calthoff/tstp/blob/master/part_I/loops/break.py

foriinrange(0,100):

print(i)

>>0>>1...

Butifweaddabreakstatementtothecodetheloopexecutes,thelooponlyrunsonce:

#https://github.com/calthoff/tstp/blob/master/part_I/loops/break_ex2.py

foriinrange(0,100):print(i)break

>>0

Theloopgoesaroundonceandprints0.Whenthethebreakkeywordisexecuted,theloopends.Thisisusefulinmanysituations.Forexample,wecanwriteaprogramthataskstheuserforinputuntiltheytype“q”toquit:

#https://github.com/calthoff/tstp/blob/master/part_I/loops/break_ex3.py

“““Ifyouareunfamiliarthereferenceinthisexample,gowatchMontyPythonandtheHoly

Grail!”””

questions=[“Whatisyourname?”,“Whatisyourfavoritecolor?”,“Whatisyourquest?”]n=0

whileTrue:print(“Typeqtoquit”)answer=input(questions[n])ifanswer==“q”:

breakn+=1ifn>2:

n=0

Eachtimethroughourinfiniteloop,ourprogramwillasktheuseraquestionfromourlistofquestions.Weusethevariablentokeeptrackofanumberwhichweuseasanindextogetaquestionfromourquestionslist.Whennbecomesgreaterthan2,we’verunoutofquestionsandwesetnbackto0whichwillaskthefirstquestioninthelist.Thiswillgoonindefinitely,unlesstheusertypesin“q”,inwhichcaseourprogramhitsthebreakkeywordandtheloopends,whichendsourprogram.

Continue

Youcanusethekeywordcontinuetostopexecutingafororwhileloop’scode,andjumptothetopofaloop.Sayforinstance,wewanttoprintthenumbersfrom1to5,exceptforthenumber3.Wecandothisbyusingaforloopwiththecontinuekeyword:

#https://github.com/calthoff/tstp/blob/master/part_I/loops/continue.py

foriinrange(1,6):ifi==3:continueprint(i)

>>1>>2>>4>>5

Inourloop,wheniequals3,ourprogramhitsthecontinuekeyword.Insteadofcausingourlooptoexitcompletely—likethebreakkeyword—thelooppersistsbutwegetjumpedtothetopofourloop,whichmeansanyoftheloop’scodethatwouldhaveexecutedthattimearoundtheloopgetsskipped.Inthiscasewheniisequalto3,everythingaftercontinueisskipped(inthiscaseprint(i)).Theresultis3isnotprinted.Hereisthesameexamplewithawhileloop:#addgithubi=1whilei<=5:ifi==3:continueprint(i)

NestedLoops

Youcancombineloopsinvariousways.Forexample,youcanhaveoneloopinsideofanotherloop.Youcanalsohavealoopinsidealoopinsidealoop.Thereisnolimittotheamountoftimesyoucandothis,althoughinpracticeyouwanttolimitthenumberoftimesyoudo.

Whenaloopisinsideanotherloop,thesecondloopissaidtobenestedinthefirstloop.Thefirstloopiscalledtheouterloop,andthenestedloopiscalledtheinnerloop.Whenyounesttwoloops,theinnerlooprunsitsfullcourseeachtimearoundtheouterloop.Hereisanexample:

#https://github.com/calthoff/tstp/blob/master/part_I/loops/nested_loops.py

foriinrange(1,3):print(i)forletterin[‘a’,‘b’,‘c’]:print(letter)>>1>>‘a’>>‘b’>>‘c’>>2>>‘a’>>‘b’>>‘c’

Thenestedforloopwilliteratethroughthelist“[a’,‘b’,‘c’]”howevermanytimestheoutsideloopruns—inthiscasetwice.Ifwechangedourouterlooptorunthreetimes,theinnerloopwoulditeratethroughthelistthreetimes.

Ifyouhavetwolistsofnumbersandwanttocreateanewlistwithallofthenumbersfromeachlistaddedtogetheryoucanusetwoforloops:

#https://github.com/calthoff/tstp/blob/master/part_I/loops/nested_loops_ex2.pylist1=[1,2,3,4]

list2=[5,6,7,8]added_up=[]foriinlist1:forjinlist2:added_up.append(i+j)

print(added_up)

>>[6,7,8,9,7,8,9,10,8,9,10,11,9,10,11,12]

Inthesecondforloopweusedthevariablejbecauseiisalreadyinusebythefirstloop.Youcanalsonestaforloopinsideawhileloopandviceversa:

#https://github.com/calthoff/tstp/blob/master/part_I/loops/nested_loops_ex3.pywhileinput('Continueyorn?')!='n':foriinrange(1,5):

print(i)

>>Continueyorn?y12345Continueyorn?y12345Continueyorn?n>>>

Thisprogramwillprintthenumbers0-4untiltheuserentersn.

Challenge

Writeaprogramthathasaninfiniteloop(withqtoquit),andeachtimethroughtheloop,itaskstheusertoguessanumberandtellsthemwhethertheirguesswasrightorwrong.

Chapter8.Modules

Imaginewewantedtobuildalargeprojectwithtenthousandlinesofcode.Ifwekeptallofthatcodeinonefile,ourprojectwouldbedifficulttonavigate.Everytimetherewasanerrororexceptioninourcode,wewouldhavetoscrollthroughafilewithtenthousandlinestofindthelineofthecodewithaproblem.Programmerssolvethisproblembydividinglargeprogramsupintomultiplepieces.Eachpieceisstoredinamodule—whichisanothernameforaPythonfile.PythonhasaspecialsyntaxthatletsyouusecodefromonePythonmoduleinanotherPythonmodule.Pythonalsocomeswithbuilt-inmodulesyoucanusethatgiveyouaccesstoextrafunctionality.

ImportingBuilt-inModules

Inordertouseamodule,youmustimportitfirst,whichmeansuseaspecialsyntaxtomakethecodeinthemoduleavailabletouseinyourprogram.Thisisdonewiththesyntaximport[module_name].importisakeywordforimportingmodulesandmustbefollowedbythemodulename.Fornow,weareonlygoingtolearnhowtoimportbuilt-inmodulesandmoduleslocatedinthesamefolderasthemoduleyouareimportingitfrom.InPartIII,welearnhowimportmodulesfromdifferentlocations.WecanimportPython’sbuilt-inmathmodulewiththefollowingsyntax:

importmath

>>

ThemathmoduleisamodulethatcomeswithPythonwhenyouinstallit.ItisaregularPythonfilewithabunchofmathrelatedfunctionality—itcontainsPythonfunctionsthatareusefulwhenyouaredoingmath.Onceyou’veimportedamodule,youcanuseanyofthecodefromit.Youcanaccessafunctioninthemodulewiththesyntax[path_to_module].[function_name]().Withthissyntax,youcanuseanyofthecode(suchasafunction)fromthemathmoduleinyourprogram:

importmath

math.fabs(-3)>>3

Thefabsfunctionreturnstheabsolutevalueoftheparameteryoupassin.Youmaybewonderinghowyouaresupposedtoknowthereisamathmodulewithafunctioncalledfabs

init.AlistofPython’sbuilt-inmodulescanbefoundathttps://docs.python.org/3/py-modindex.html.Ifyousearchforthemathmodule,thereisalinkthattakesyoutoapagethatlistseveryfunctioninthemathmodule,whateachfunctiondoes,andwhatparametersittakes.

Anotherbuilt-inmoduleistherandommodule.Hereisanexampleofimportingtherandommodule,andusingafunctionfromitcalledrandintthattakestwonumbersasparametersandreturnsarandomnumberbetweenthem.

#Theoutputofthisprogrammightnotbe52whenyourunit—it’srandom!

importrandom

random.randint(0,100)

>>52

Thereareothersyntaxesforimportingmodules,butingeneralimport[module_name]isthesyntaxyoushoulduse,andtheonewewillbeusingthroughoutthebook.Finally,youshoulddoalloftheimportsforyourprogramatthetopofyourfile.

ImportingModules

Inthissection,wearegoingtocreateamodule,andusethecodefromitinanothermodule.Firstcreateanewfolderonyourcomputercalledtstp,thencreateafilecalledhello.pyinit.Insidehello.pyaddthefollowingcode:

#https://github.com/calthoff/tstp/blob/master/part_I/modules/hello.py

defprint_hello():print(“Hello”)

Savethefile.Insidethetstpfolder,createanotherPythonfilecalledproject.py.Insideproject.pyaddthefollowingcode:

#https://github.com/calthoff/tstp/blob/master/part_I/modules/project.py

importhellohello.print_hello()

>>‘Hello’

Usingtheimportkeywordwecaneasilyusecodefromourfirstmoduleinoursecondmodule.

Challenge

Ichallengeyoutowritethreefunctionsinamodule,andusetheminanotherPythonprogram.

Chapter9.Files

Pythonmakesiteasytoworkwithfiles.Youcaneasilyreadandwritedatafromafile.Readingdatafromafilemeansaccessingthefilescontent.Writingdatatoafilemeansaddingnewcontenttothefile.Thisisusefulwheneveryouwanttousedatafromafileinaprogram,oroutputdatafromaprogramtoafile.Inthischapter,wewilllearnthebasicsofworkingwithfiles—includinghowtoreadandwritetofiles.

WorkingWithFiles

Theopenfunctiontakesastringrepresentingthepathtoafileandastringrepresentingthemodetoopenthefileinasparameters.Thepathtoafile—alsocalledafilepath—representsthelocationonyourcomputerafileresides,forexample:/Users/calthoff/my_file.txtisthefilepathtoafilecalledmy_file.txt.Eachwordseparatedbytheslashthatprecedesitisthenameofafolder.Togethertheyrepresentthelocationofthatfile.Ifafilepathonlyhasthenameofthefile,Pythonwilllookforthefileinwhateverfolderyouarerunningyourprogramin.

Thesecondparameterrepresentsthemodetoopenthefilein,whichdeterminestheactionsyouwillbeabletoperformonthefile.Hereareafewofthemodesyoucanopenafilewith:

“r”Opensafileforreadingonly.

“w”Opensafileforwritingonly.Overwritesthefileifthefileexists.Ifthefiledoesnotexist,createsanewfileforwriting.

“w+”Opensafileforbothwritingandreading.Overwritestheexistingfileifthefileexists.Ifthefiledoesnotexist,createsanewfileforreadingandwriting.

5

Onceyou’vepassedtheopenfunctionafilepathandmodeasparameters,opencreatesafileobject(WelearnaboutobjectsinPartII)wecanusetoreadorwritetoourfile(orbothdependingonthemodeyouchose).Hereisanexamplewritingtoafile:

my_file=open(“my_file.txt”,“w”)my_file.write(“HellofromPython!”)my_file.close()

>>

opencreatesanewfilecalledmy_file.txt(becausewepassedin“w”asourmode)inwhateverdirectoryyouranourprogramin.Wesaveourfileobjectreturnedbytheopenfunctionininthevariablemy_file.

Nowwecancallthewritemethodonourfileobject,whichacceptsastringasaparameterandwritesittothenewfilewecreated.Finally,weclosedourfilebycallingtheclosemethodonthefileobject.Thisisanimportantstep.Wheneveryouopenafileusingtheopenmethod,youneedtocloseitwiththeclosemethod.Ifyouhaveaprogramwhereyouusetheopenmethodonmultiplefiles,andyouforgettoclosethem,itcancauseproblemsinyourprogram.

Usingwith

Becauseforgettingtoclosefilesyouopenedcancauseproblems,thereisasecond,preferedsyntaxforopeningfiles.ThepreferredwaytoopenafileinPythonistouseacompoundstatementusingthewithkeywordandthesyntaxwithopen(‘my_file’,[mode])as[variable_name]:[your_code].Whenyouuseacompoundstatementusingwiththefileautomaticallyclosesafterthelastsuiteexecutesinthestatement.Thefileobjectgetssavedinavariableofyourchoosing([variable_name]).Hereistheexamplefromtheprevioussectionwrittenusingthepreferredsyntaxforopeningafile:

#https://github.com/calthoff/tstp/blob/master/part_I/files/using_with.pywithopen(‘my_file.txt’,‘w’)asmy_file:my_file.write(‘HellofromPython!’)

>>

Aslongasyouareinsideofthewithstatement,youcanworkwiththefileyouopenedusingthevariableyoucreated—inthiscasemy_file.AssoonasPythonleavesthewithstatement,itclosesthefileforyou.

ReadingFiles

Ifyouwanttoaccessthecontentsofafile(readthefile)youpassin“r”asthesecondparametertoopen.Thenyoucancallthereadmethodonyourfileobjectwhichreturnsaniterableyoucaniteratethroughtogeteachlineofthefile.

#https://github.com/calthoff/tstp/blob/master/part_I/files/reading_files.py

withopen(“my_file.txt”,“r”)asmy_file:forlineinmy_file.read():print(line)

>>HellofromPython!

Youcanonlycallreadonafileonce(eachtimeyourunyourprogram)togetsitscontents,soyoushouldsavethefilecontentsinavariableorcontainerifyouneedtousethefilecontentsagainlaterinyourprogram.Forexample,wecouldchangethepreviousexampletosavethefilecontentsinalist:

#

my_list=list()

withopen(“my_file.txt”,“r”)asmy_file:forlineinmy_file.read():my_list.append(line)

print(my_list)

>>HellofromPython!

Withthefilecontentssavedinalist,wecaneasilyaccessitlaterinourprogramwheneverweneedit.

CSVFiles

Pythoncomeswithabuilt-inmoduleforworkingwithCSVfiles.CSVstandsforcommaseparatedvalue.ItisafileformatcommonlyusedinMicrosoftExcel:aprogramforcreatingspreadsheets.AcommainaCSVfileiscalledadelimiter.EverypieceofdataseparatedbyacommarepresentsacellinExcel.EachlineoftheCSVfilerepresentsarowinExcel.Hereisanexampleofthecontentsofacsvfile:

#my_file.csvone,two,threefour,five,six

Youcouldloadthisfileintoexcelandone,twoandthreewouldeachgettheirowncellsinthefirstrowofthespreadsheet;andfour,fiveandsixwouldeachgettheirowncellsinthesecondrowofthespreadsheet.WecanuseawithstatementtoopenaCSVfileliketheexamplefromtheprevioussection,butinsidethewithstatementweneedtousethecsvmoduletoconvertourfileobject

intoacsvobject.ThecsvmodulehasamethodcalledwriterthatacceptsafileobjectandadelimiterandreturnsacsvobjectwithamethodcalledwriterowwecanusetowritetoourCSVfile:

#https://github.com/calthoff/tstp/blob/master/part_I/files/csv_files.py

importcsv

withopen(‘my_file.csv’,‘w’)ascsvfile:spamwriter=csv.writer(csvfile,delimiter=',’)spamwriter.writerow([‘one’,‘two’,‘three’])spamwriter.writerow([‘four ’,‘five’,‘six’])

>>Thewriterowmethodacceptsalistasaparameter.EveryiteminthelistgetswrittentotheCSVfileandeachitemisseparatedbythedelimiteryoupassedtothewritermethod(inthiscaseacomma).writerowonlywritesonerow,sowehavetocallittwicetocreatetworows.Whenyourunthisprogram,itwillcreateanewfilecalledmy_file.csvandwhenyouopenthefilewithatexteditor,itwilllooklikethis:

#my_file.csvone,two,threefour,five,six

IfyouloadthisfileintoExcel(orGoogleSheetsafreeExcelalternative),thecommasdisappear,butone,twoandthreewilleachhavetheirowncellinrowone;andfour,fiveandsixwilleachhavetheirowncellinrowtwo.

Wecanalsousethecsvmoduletoreadthecontentsofafile.ToreadfromaCSVfile,firstwepassin‘r ’toopenasasthesecondparameter.Thisopensthefileforreadingonly.Insteadofusingthewritermethodlikethepreviousexample,weusethereadermethod,butstillpassinthefilepathandacommaasthedelineator.

readerreturnsaniterablewecaniteratethroughtoprinteachrow.Wecancallthejoinmethodonacommatoaddacommainbetweeneachvaluetoprintthecontentsofthefilelikeitappearsintheoriginalfile:

#https://github.com/calthoff/tstp/blob/master/part_I/files/csv_files_ex2.py

importcsv

withopen(‘my_file.csv’,‘r ’)ascsvfile:spamreader=csv.reader(csvfile,delimiter=',')forrowinspamreader:print(','.join(row))

>>one,two,three

>>four,five,six

Challenge

Datapersistswhenitoutlivestheprogramthatcreatedit.Wecanusefilestopersistdatabywritingtheoutputofourprogramstoafile.Writeaprogramthatcollectsdatafromauser—andsavesittoafile—sothedatapersists.

Chapter10.BringingItAllTogether

Inthischapter,wearegoingtocombinetheconceptswe’velearnedsofartobuildatext-basedgame:theclassicgameHangman.Ifyou’veneverplayedHangman,here'showitworks.PlayerOnepicksasecretwordanddrawslinesrepresentingeachletterintheword(wewilluseanunderscoretorepresenteachline).PlayerTwotriestoguessthewordoneletteratatime.IfPlayerTwoguessalettercorrectly,thecorrespondingunderscoreisreplacedwiththecorrectletter.IfPlayerTwoguessesincorrectly,PlayerOnedrawsapieceofapictureofahangman(apersonhanging).IfPlayerTwocompletesthewordbeforethepictureofthehangmanisdrawn,theywin,ifnottheylose.InourprogramthecomputerwillbePlayerOne,andthepersonplayingthegamewillbePlayerTwo.AreyoureadytobuildHangman?

Hangman

Hereisthebeginningofourhangmancode:

#https://github.com/calthoff/tstp/blob/master/part_I/bringing_it_all_together/hangman.py

defhangman():word="caat"wrong_guesses=0stages=["","________","||","|0","|/|\","|/\","|"]letters_left=list(word)score_board=['__']*len(word)win=Falseprint('WelcometoHangMan')

Firstwecreateafunctioncalledhangmanwecancalltostartthegame.Thewordtoguessisstoredinthevariableword(youcanchangethewordtheplayerhastoguessbychangingthevariable).Weuseanothervariablewrong_guessestokeeptrackofhowmanyincorrectletterstheplayerhasguessed.

stagesisalistfilledwithstringswewillusetodrawourHangman.Wheneachstringinthestageslistisprintedonanewline,apictureofahangmanforms.letters_leftisalistmade

upofeachcharacterinword.Wewilluseittokeeptrackofwhichlettersarelefttoguessinourword.

scoreboardisalistofstringsusedtokeeptrackofthehintswedisplaytotheusere.g.,“c__t”ifthewordis“cat”.Theinitialvalueofscore_boardiscalculatedwith[‘__’]*len(word)whichreturnsalistmadeupofthenumberofunderscorestostartwith.Forexampleifthewordis“cat”score_boardstartsas[“__”,“__”,“__”].

Finally,wehaveawinvariablethatstartsasFalsetokeeptrackofwhethertheplayerhaswonthegameyetornot.TostartthegameoffweprintWelcometoHangman.

Whenyoubuildagame,younormallyusealoopthatcontinuesuntilthegameisover.Hereistheloopwewilluseinourgame:

whilewrong_guesses<len(stages)-1:print('\n')guess=input("Guessaletter")ifguessinletters_left:character_index=letters_left.index(guess)score_board[character_index]=guessletters_left[character_index]='$'else:wrong_guesses+=1print(''.join(score_board))print('\n'.join(stages[0:wrong_guesses+1]))if'__'notinscore_board:print('Youwin!Thewordwas:')print(''.join(score_board))win=Truebreak

Ourloopcontinuesaslongasthevariablewrong_guessesislessthanthelen(wrong_guesses)-1.wrong_guesseskeepstrackofthenumberofwrongletterstheuserhasguessed,soassoonastheuserhasguessedmorewronglettersthanthenumberofstringsthatmakeupthehangman,thegameisover.Thereasonwesubtract1isbecausethelengthfunctioninPythondoesnotcountfrom0,itcountsfrom1.Inordertocompensateforthis,wehavetosubtractone,tocompensateforthefactthatlengthcountsstartingfrom1insteadof0.Thereasonwewanttocountfrom0isbecausestagesisalist,andwearegoingtobeusingwrong_guessesasanindextogetthestringsfromstagesandindexesstartat0.

Insideourloop,weprintablankspacetomakeourgamelooknicewhenit’sprintedintheshell.Nextwecollecttheplayer ’sguesswiththebuilt-ininputfunctionandstorethevalueinthevariableguess.Ifguessisinletters_left(avariablefromthebeginningofourprogramthatkeepstrackofthelettersthathaven’tbeenguessedyet),theplayerguessedcorrectly.Iftheplayerguessedcorrectlyweneedtoupdateourscore_boardlist,whichweuselaterinthegametodisplaythescore.Iftheuserguessed“c”,wewanttochangeourscore_boardtolooklike

this:[“c”,“__”,“__”].Weusetheindexmethodonourletters_leftlisttogetthefirstindexoftheletterthatwasguessed.Weusethatindextoreplacetheunderscoreinscore_boardattheindexwiththecorrectlyguessedletter.Wehaveaproblemthough.Becauseindexonlyreturnsthefirstindexofthecharacterwearelookingfor,ourcodewillnotworkifword(thewordtheplayerisguessing)hasmorethanoneofthesamecharacter.Tocompensateforthis,wemodifyletters_leftbyreplacingthecharacterthatwasjustcorrectlyguessedwithadollarsignsothatthenexttimearoundtheloop,ifthereismorethanoneletterinthewordindexwillfindsecondoccurrenceofthelettersincethefirstoccurrenceoftheletterwasreplacedbyadollarsign.Ifontheotherhandtheplayerguessesanincorrectletter,wesimplyincrementwrong_guessesby1.

Nextweprintthescoreboardandprintourhangmanusingourscore_boardandstageslists.Toprintthescoreboard,allwehavetodoisprint''.join(score_board).

Printingthehangmanistrickier.Wheneachofthestringsinourstageslistisprintedonanewline,acompletepictureofahangmanisprinted.Wecaneasilycreatetheentirehangmanbyprinting'\n'.join(stages).Thisconnectseachstringinthestageslistwithablankspace(\n).Butwewanttoprintourhangmanatthestagewearecurrentlyat,whichweaccomplishbyslicingourstageslist.Westartatstage0andsliceupuntilwhateverstageweareat(representedbythevariablewrong_guesses)plusone.Thereasonweaddone,isbecausewhenyouareslicing,theendslicedoesnotgetincludedintheresults.Thisgivesusonlythestringsweneedtoprintthestageofthehangmanwearecurrentlyonwhichwethenprint.

ThelastthingwedoinourloopischeckiftheuserhaswonthegameIftherearenomoreunderscoresinthescore_boardlist,weknowtheuserhasguessedallthelettersandwonthegame.Iftheuserhaswon,weprintthattheywonandweprintthewordtheycorrectlyguessed.WealsosetthevariablewintoTruewhichisusedwhenwebreakoutofourloop.

Oncewebreakoutofourloop,iftheuserwon,wedonothing,andtheprogramisover.Iftheuserdidnotwin,thevariablewinwillbeFalse.Ifthatisthecaseweknowtheuserlostthegameandweprintthefullhangman,print“Youlose!”followedbythewordtheyincorrectlyguessed:

ifnotwin:print('\n'.join(wrong_guesses[0:stage]))print('Youlose!Thewordswas{}'.format(word))

hangman()

Hereisourcompletecode:

#https://github.com/calthoff/tstp/blob/master/part_I/bringing_it_all_together/hangman.py

defhangman():stage=0

wrong_guesses=["","________","||","|0","|/|\","|/\","|"]word="cat"score_board=['__']*len(word)win=Falseprint('WelcometoHangMan')whilestage<len(wrong_guesses)-1:print('\n')guess=input("Guessaletter")ifguessinword:score_board[word.index(guess)]=guesselse:stage+=1print((''.join(score_board)))print('\n'.join(wrong_guesses[0:stage+1]))if'__'notinscore_board:print('Youwin!Thewordwas:')print(''.join(score_board))win=Truebreakifnotwin:print('\n'.join(wrong_guesses[0:stage]))print('Youlose!')

hangman()

Challenge

Buildingtext-basedgamesisagreatwaytoimproveyourprogrammingability.Buildanothertext-basedgamethatinterestsyou.

Chapter11.Practice

“Thefoolwonders,thewisemanasks.”—BenjaminDisraeli

Ifthisisyourfirstprogrammingbook,Irecommendyouspendtimepracticingbeforemovingontothenextsection.Inthischapter,Iprovideexercisestohelpyougetadditionalpracticebeforemovingontothenextsection,resourcestocheckout,andwecoverhowtogethelpifyougetstuck.

Exercises

0.Createatextbasedgameofyourfavoritesport.0.Makeupyourowntextbasedgame.WhenIwasstartingoutIbuiltafantasybasedgamebasedonHeroesofMightandMagicIII0.Buildatextbasedmagic8ballprogramwheretheusercanshakeamagiceightballandgetpredictionsabouttheirfuture0.Createaprogramthataskstheuserwhatkindofmoodtheyareinandrecommendsasong.0.Buildaprogramthatprintsouttenbrandsandletstheusertypethemin.Whentheydo,theprogramshouldprintthebrand’strademarki.e.,theusercouldtypeNikeandtheprogramwouldprint“Justdoit.”

Read

0.http://programmers.stackexchange.com/questions/44177/what-is-the-single-most-effective-thing-you-did-to-improve-your-programming-skil0.https://www.codementor.io/ama/0926528143/stackoverflow-python-moderator-martijn-pieters-zopatista

GettingHelp

Ifyougetstuck,Ihaveafewsuggestions.Thefirstispostingaquestiononhttp://forum.theselftaughtprogrammer.io,itisaforumIsetupforpeoplereadingthisbook

togetanswerstoanyquestionstheyhave.IalsorecommendcheckingoutStackExchange—anamazingresourcefor

programmers.TherearetwowebsitesonStackExchangeyoushouldexplore.ThefirstisStackOverflow.Ifyouhaven’tusedityet,youwillsoonbefamiliarwithit.GooglealmostanyprogrammingquestionandananswerwillpopuponStackoverflow,whichmakesitagamechangerforlearningtoprogram.Learningtorelyonotherpeople'shelpwasanimportantlessonforme.Strugglingtofigurethingsoutisamajorpartofthelearningprocess,butatsomepointitbecomescounter-productive.Workingonprojectsinthepast,Iusedtocontinuetostrugglewaypastthepointwhereitwasproductive.Todayifthathappens,Iwillpostaquestiononline,iftheanswerisnotalreadyonthere.EverytimeI’vepostedaquestiononline,someonehasbeenabletoanswerit.Ican’tsayenoughabouthowhelpfulandfriendlytheprogrammingcommunityis.

OtherResources

linktowebsiteforotherpythonresources

PartIIIntroductiontoObject-orientedProgramming

Chapter12.ProgrammingParadigms

“Thereareonlytwokindsoflanguages:theonespeoplecomplainaboutandtheonesnobodyuses”—BjarneStroustrup

Aprogrammingparadigmisacertainwayofprogramming.Therearemanydifferentprogrammingparadigms.Youwon’tneedtolearnallofthematthebeginningofyourcareer—butit’simportanttoknowwhatsomeofthemostpopularparadigmsare.Theprogrammingparadigmswewillgooverinthischapterareimperativeprogramming,functionalprogrammingandobject-orientedprogramming—withafocusonobject-orientedprogramming—thesubjectoftheremainingchaptersofthissection.

State

Oneofthefundamentaldifferencesbetweendifferentprogrammingparadigmsisthehandlingofstate.Stateisthedatayourprogramhasaccessto.Programsstoredatainvariables—sostateisthevalueofaprogram’svariablesatagiventimetheprogramisrunning.

ImperativeProgramming

InPartI,welearnedtoprogramimperatively.Imperativeprogrammingcanbedescribedas“dothis,thenthat”.Animperativeprogramisasequenceofstepsmovingtowardasolution—witheachstepchangingtheprogram’sstate.Anexampleofimperativeprogrammingwouldbe:

x=2y=4z=8xyz=x+y+z

>>14

Eachstepoftheprogramchangestheprogram'sstate.Wegettoxyzbyfirstdefiningx,followedbyy,followedbyzandfinallydefiningxyz.

FunctionalProgramming

Functionalprogrammingisanotherpopularprogrammingparadigm.Itoriginatesfromlambdacalculus.Functionalprogramminginvolveswritingfunctionsthat—giventhesameinput—alwaysreturnthesameoutput.Infunctionalprogramming,youonlyprogramwithfunctions,youdonotuseclasses—afeatureofobject-orientedprogrammingwewilllearnaboutshortly.ThereisalotofjargoninfunctionalprogrammingandMaryRoseCookdoesagreatjobcuttingthroughitwithherdefinition,“Functionalcodeischaracterisedbyonething:theabsenceofsideeffects.Itdoesn’trelyondataoutsidethecurrentfunction,anditdoesn’tchangedatathatexistsoutsidethecurrentfunction.”61ShefollowsherdefinitionwithanexamplewhichIwillalsosharewithyou.Hereisanunfunctionalfunction:

a=0

defincrement():globalaa+=1

Hereisafunctionalfunction:

defincrement(a):returna+1

Thefirstfunctionisunfunctionalbecauseitreliesondataoutsideofitself,andchangesdataoutsideofthecurrentfunctionbyincrementingaglobalvariable.Thesecondfunctionisfunctionalbecauseitdoesnotrelyonanydataoutsideofitself,anditdoesnotchangeanydataoutsideofitselfeither.Functionalprogrammerswritefunctionsthiswaytoeliminatesideeffects—theunintendedconsequencesthathappenwhenyouareconstantlychangingthestateofyourprogram.

Object-orientedProgramming

Theobject-orientedprogrammingparadigminvolveswritingprogramswhereyoudefineandcreateobjectsthatinteractwitheachother.We’vebeenprogrammingwithobjectsthiswholetime—strings,integersandfloatsareallexamplesofobjects.Butyoucanalsodefineyourownobjectsusingclasses.Classesaretheblueprintusedtocreateobjects.Youcanthinkofaclassastheideaofanobject.Thinkofanorange.Anorangeisanobject.Afruitweighingbetween2to10ouncesistheideaofanorange—aclass.

WecanmodelorangesinPythonbydefiningaclasswecanusetocreateorangeobjects.Wedefineaclassusingtheclasskeywordfollowedbythenamewewanttogiveourclass.Aclassisacompoundstatementwithaheaderfollowedbysuites.Youwritesuitesafter

theheader,whichcanbesimplestatements,aswellascompoundstatementscalledmethods.Methodsarelikefunctions,buttheyaredefinedinsideofaclass,andcanonlybecalledontheobjecttheclasscancreate.WesawdifferentexamplesofthisinChapter5whenwecalledvariousmethodsonstrings.HereisanexamplehowwecanrepresentanorangeusingaclassinPython:#https://github.com/calthoff/tstp/blob/master/part_II/object_oriented_programming/orange_ex1.py

classOrange:print("Orangecreated!")

Westartedwiththeclasskeywordfollowedbythenameofourclass—inthiscaseOrangebecausewearemodelingoranges.Byconvention,classesinPythonalwaysstartwithacapitalletterandarewrittenincamelCase—whichmeansifaclassnameismadeupofmorethanoneword,thewordsshouldnotbeseparatedbyanunderscore(likeafunctionname),insteadeachwordshouldbecapitalizedLikeThis.

Afterourclassdefinitionwehaveasimplestatement—print(“Orangecreated!”).Thiscodewillexecutewhenwecreateanorangeobject.Withthisclassdefinition,wecancreateasmanyOrangeobjectsaswe’dlike:

orange=Orange()print(type(orange))print(orange)

>>Orangecreated!>><class'__main__.Orange'>>><__main__.Orangeobjectat0x101a787b8>

WecreatedanewOrangeobjectusingthesamesyntaxweusetocallafunction—[classname]().Thisiscalledinstantiatinganobject,whichmeanscreatinganewobject.“Orangecreated!”printsassoonasweinstantiateourOrangeobject.Whenweprinttype(orange),thetypefunctiontellsusourOrangeobjectisaninstanceoftheOrangeclasswejustcreated.WhenweprintourOrangeobject,PythonletsusknowitisanOrangeobject,andthengivesusitslocationinmemory.Whenyouprintanobjectlikethis,thelocationinmemoryprintedonyourcomputerwillnotbethesameastheexample,becausetheobject’slocationinmemorychangeseachtimetheprogramruns.

NowwearegoingtoaddamethodtoourOrangeclass.Youdefineamethodwiththesamesyntaxasafunction.Thereareonlytwodifferences:amethodmustbedefinedasasuiteinaclass,andamethodhastoacceptatleastoneparameter(exceptinspecialcasesIwon’tgointo).Youcannamethefirstparameterofamethodwhateveryou’dlike,butbyconventionthefirstparameterinamethodisalwaysnamedself,andI’veneverseenthisconventionbroken.

Thereasoneverymethodmusthaveafirstparameterisbecausewheneveramethodiscalledonanobject,Pythonautomaticallypassesthemethodtheobjectthatcalledit.Thisconceptexistsinmostprogramminglanguagesthatsupportobject-orientedprogramming,however,Pythonmakesthepassingofselfasaparameterexplicitwhereasmanyotherlanguagesmakeitimplicit.WhatImeanisthatPythonmakesyouexplicitlydefineselfineverymethodyoucreatewhereasinotherlanguagesselfisjustimpliedtohavebeenpassedtotheobject.Ifwedefineamethodandprintself,wewillseeitistheOrangeobjectwecalledourmethodon:


defprint_orange(self):print(self)

Orange().print_orange()

>>Orangecreated!>><__main__.Orangeobjectat0x101a787b8>

selfisusefulbecausewecanuseittodefineandaccessvariablesthatbelongtoourOrangeobject.Wedothisbydefiningaspecialmethodcalled__init__,whichstandsforinitialize.Whenyouinstantiateanobject,ifyou’vedefinedamethodcalled__init__,Pythonautomaticallycallsthe__init__methodforyouwhentheobjectiscreated.Inside__init__wecangiveourOrangeobjectvariableswiththesyntaxself.[variable_name]=[variable_value].Hereisanexample:


def__init__(self):self.color="orange"self.weight=10

defprint_orange(self):print(self)print(self.color)print(self.weight)

orange=Orange()orange.print_orange()

>>Orangecreated!>><__main__.Orangeobjectat0x10564dba8>>>orange>>10

Using__init__wecannowcreateorangesthatgetacolorandweightwheninitializedandwecanuseandchangethesevariablesinanyofourmethodsjustlikeregularvariables.InPython,anymethodsurroundedonbothsidesbyunderscoresiscalledamagicmethodwhichmeansitdoessomethingspecial.Theprint_orangemethodwasusedtoillustrateanexample,itwillnotbeamethodinourorangeaswecontinuetomodelit.Wecanchangeourclassdefinitionsothepersoncreatingtheobjectcanpassintheirownvariableswhentheycreateaneworange,insteadoftheweightandcolorstartingwithdefaultvalues.Hereisournewclassdefinition:

#https://github.com/calthoff/tstp/blob/master/part_II/object_oriented_programming/orange_ex3.py

classOrange:def__init__(self,weight,color,mold):"""allweightsareinoz"""self.weight=weightself.color=color

Nowwecancreateawidervarietyoforangesobjects:

orange=Orange(10,'orange',)

Wejustcreateda10oz(perthecomment“allweightareinoz”),orangecoloredorange.Wecanaccesstheorangesvariablesusingdotnotation:

print(orange.weight)

>>10

print(orange.color)

>>“orange”

Wecanalsochangeanyofthevaluesofourorangeobject:

orange.weight=100print(orange.weight)

>>100

Movingforwardwiththemodelingofourorange,thereismoretoanorangethanjustitsphysicalpropertieslikecolorandweight.Orangescanalsodothings,andweneedtobeabletomodelthataswell.Whatcananorangedo?Well,foronething,orangescangobadfrommold.WecanmodelamoldingorangebyaddingamoldvariabletoourOrangeclass,andcreatingamethodthatincrementsthemoldvariablewhenit’scalled:

#https://github.com/calthoff/tstp/blob/master/part_II/object_oriented_programming/orange_ex2.py

classOrange():def__init__(self):"""allweightsareinoz"""self.weight=6self.color='orange'self.mold=0

defrot(self,days,temperature):self.mold=days*(temperature*.1)

orange=Orange()print(orange.mold)orange.rot(10,98)print(orange.mold)

>>0>>98.0

Nowourorangeobjectswillbeabletorot.Wedefinedamethodthatacceptsthenumberofdaysit'sbeensincetheorangewaspicked,andtheaveragetemperatureduringthattimeasparameters.Withourmade-upformula,wecanincreasetheamountofmoldtheorangehaseverytimewecalltherotmethod;andourorangenowhastheabilitytorot.

Thatwasalottotakein,solet'sgooveritallonemoretime.InObject-orientedprogramming,weuseclassestomodelobjects.Theseobjectsgroupvariables(state)andmethods(foralteringstate)togetherinasingleunit—theobject.Classeshaveaspecialmethodcalled__init__thatPythoncallswhenanobjectiscreatedusingthesyntax[class_name]().ThisisanexampleofoneofPython’smagicmethods.ThefirstargumenttoanymethodiscalledselfbyconventionandexistsbecausePythonautomaticallypassestheobjectthatcalledthemethodintothemethod.

Challenge

Pickaprogrammingparadigmandresearchwhatproblemsitsolves.

Chapter13.TheFourPillarsofObject-orientedProgramming

"Gooddesignaddsvaluefasterthanitaddscost."—ThomasC.Gale

Therearefourmainconceptsinobject-orientedprogramming—oftencalledthefourpillarsofobject-orientedprogramming:inheritance,polymorphism,abstractionandencapsulation.Theseconceptsmustallbepresentinaprogramminglanguageinorderforittobeconsideredanobject-orientedprogramminglanguage.Python,JavaandRubyareallexamplesofobject-orientedlanguages.Notallprogramminglanguagessupportobject-orientedprogramming—forexampleHaskellisafunctionalprogramminglanguagethatdoesnotsupportobject-orientedprogramming.Inthischapter,wetakeadeeperlookatobject-orientedprogrammingbyexploringeachofthefourpillarsofobject-orientedprogramming.

Inheritance

Inheritanceinprogrammingissimilartogeneticinheritance.Ingeneticinheritanceyoucaninheritattributesfromyourparents,likeyoureyecolor.Similarlywhenyoucreateaclass,itcaninheritfromanotherclass(whichisthencalleditsparentclass)—givingthenewclassyoucreatedtheparentclass’svariablesandmethods.Inthissectionwewillmodelakidandadultusinginheritance.First,wedefineaclasstorepresentanadult:

“““https://github.com/calthoff/tstp/blob/master/part_II/more_object_oriented_programming/inheritance_ex1.py”””

classAdult():def__init__(self,name,height,weight,eye_color):"""heightisinfeet,weightinlbs."""self.name=nameself.height=heightself.weight=weightself.eye_color=eye_color

defprint_name(self):print(self.name)

tom=Adult("Tom",6,150,"brown")

print(tom.name)print(tom.height)print(tom.weight)print(tom.eye_color)tom.print_name()

>>Tom>>6>>150>>brown>>Tom

UsingthisclasswecancreateAdultobjectswithaname,height,weightandeyecolor.Inaddition,ourAdultobjectshaveamethodcalledprint_namethatprintstheparent’sname.

Wecanmodelahumanchildthatalsohasaname,height,weight,eyecolorandcanprintitsname;withanextramethodwedon’twantourAdultobjectstohavecalledprint_cartoon;usinginheritance.Youinheritfromaparentclassbyaddingparenthesistotheclassnameyouaredefining,andpassingintheclassnameyouwanttoinheritfromasaparameter.Hereisanexample:

“““https://github.com/calthoff/tstp/blob/master/part_II/more_object_oriented_programming/inheritance_ex2.py”””

classAdult():def__init__(self,name,height,weight,eye_color):#heightisinfeet,weightinlbs.self.name=nameself.height=heightself.weight=weightself.eye_color=eye_color

defprint_name(self):print(self.name)

classKid(Adult):defprint_cartoon(self,favorite_cartoon):print("{}'sfavoritecartoonis{}".format(self.name,favorite_cartoon))

child=Kid("Lauren",3,50,"blue")print(child.name)print(child.height)

print(child.weight)print(child.eye_color)child.print_name()child.print_cartoon('DuckTales')

>>brown>>Ricky>>DuckTales

BypassinginAdulttoourKidclass,ourKidclassinheritsthevariablesandmethodsofourAdultclass:whenwecreateaKidobjectwepassitaname,height,weightandeyecolor;andourKidobjectisabletousethemethodprint_name;allofwhichwasinheritedfromitsparentclass(withouthavingtodefineanyofitinourKidclass).Thisisimportantbecausenothavingtorepeatcodemakesourprogramsmallerandthereforemoremanageable.

AfterinheritingfromourAdultclass,allwehadtodowasdefineanewmethodcalledprint_cartooninourKidclasstocreateaKidclasswithallofthefunctionalityofourAdultclass,plusadditionalfunctionality;allwithoutaffectingourAdultclass.

Polymorphism

ThebestdefinitionI’vefoundofpolymorphismis“polymorphismistheability(inprogramming)topresentthesameinterfacefordifferingunderlyingforms(datatypes)”69Aninterfacereferstooneormorefunctionsormethods.Let’stakealookatasituationwherethisisthecase:

print('HelloWorld')print(200)print(200.1)

>>“HelloWorld”>>200>>200.1

Inthisexample,wewereabletopresentthesameinterface(theprintfunction)forthreedifferentdatatypes:astring,anintandafloat.Wedidn’tneedtocallthreeseparatefunctions—print_string,print_int,orprint_floatinordertoprintthesethreedifferentdatatypes—insteadPythonhasjustoneinterfaceforallofthem.Let’stakealookatanotherexample.Saywewanttowriteaprogramthatcandrawdifferentshapes:triangles,squaresandcircles.Eachoftheseshapesisdrawninadifferentway,sothemethodstodrawthemwouldallhavedifferentimplementations.InPython,wecancreatedifferentdrawmethodsforeachshapesothatTriangle.draw()willdrawatriangle,Square.draw()willdrawasquare,andCircle.draw()willdrawacircle.Eachoftheseshape

objectshasit’sowndrawinterfacethatknowshowtodrawitself.Whenwehaveashapeobject,weknowwecancallthedrawfunctiontodrawtheshape.Thesameinterfaceispresentedforallthedifferentshapedatatypes.

IfPythondidnotsupportpolymorphism—wewouldneedafunctionthatcreatesatriangle,andanotherfunctioncalleddraw_triangletodrawit;afunctiontocreateacircle,andafunctioncalleddraw_circletodrawit;etc.BecausePythonhaspolymorphism,everyshapecansimplybedrawnwithitsdrawmethod.Thismakesourshapeobjectsmucheasiertouseandexplain.Insteadofexplaining—wehavethreefunctionsrepresentingthreedifferentshapes,andanotherthreefunctionsthatdrawseachofthem;wecansimplytellwhoeverisusingthecode:wehavethreeshapes—ifyouwanttodrawone—callitsdrawmethod.

Abstraction

Weuseabstractioninobject-orientedprogrammingwhenwecreateaclassanddefineitsmethods.Saywecreateaclasstorepresentaperson.Whenwedefineourpersonclass—andthemethodsthatgowithit—wearecreatinganabstraction.Ourdefinitionofapersoncouldincludeeyecolor,haircolor,heightandethnicityaswellastheabilitytoread,writeanddraw.Wecouldhaveafivefootthreepersonwithblueeyes,blondehairunabletoread,writeordraw.Orwecouldhaveasixfootfivepersonwithbrowneyes,brownhairthatcanread,writeanddraw.Bothofthesefallintothecategoryofthepersonabstractionwe’vecreated.

Whenwedesignobject-orientedprograms,wecreateabstractionsofdifferentconceptsthatallworktogethertoformourprogram.Forexample,wemaycreateanabstractionofaperson,andanabstractionofagovernment,andmodelhowmanypeopleliveundereachgovernmentintheworld.Abstractionallowsustomodelobjectswithclearboundaries,andhavetheminteractwitheachother.InPart4welearnmoreaboutabstractionandhowPythonitselfisbuiltonmultiplelayersofit.

Encapsulation

Inobject-orientedprogramming,encapsulationhidesourcodesinternaldata.Whencodeisencapsulated,itmeanswhenitiscalled,thecallercannotaccessthecode'sinternaldata.Takealookatthemethodget_data:

classData:defget_data(self,index,n):data=[1,2,3,4,5]data.append(n)

Themethodhasavariablecalleddata.Whenwecallget_data,thereisnowayforustoaccessthisvariablebecauseofencapsulation.Iftherewasnoencapsulation,wemightbeabletoaccessthevariabledata—andappendntoit—likethis:

#warningthiscodedoesnotworkData.get_data.data.append(6)

Ifthiswasallowed,anyonecouldaccessthedatavariableinourget_datamethod.Insteadofrelyingonourimplementationoftheget_datamethod—theycouldappendntodatathemselves.Thisisnotaproblem—untilwechangetheimplementationofget_data.Whatifwedecidewantthevariabledatatobeatupleinsteadofalist?Ifwemakethischange,itwillbreakanyone’scodecallingappendonthevariabledata,becausetuplesdonothaveanappendmethod.Butbecauseofencapsulation,thisscenarioisnotpossible(whichiswhythecodedoesnotwork),andwecanbeassuredchangestotheinternalimplementationofourcodewon’tbreakourclient’scode(clientisatermforthepersonusingapieceofcode).

Composition

Whilecompositionisnotoneofthefourpillarsofobject-orientedprogramming,itisanimportantconceptrelatedtotherest.Compositionisusedtorepresent“hasa”relationships—itoccurswhenoneobjectstoresanotherobjectasavariable.Forexample,saywewanttorepresenttherelationshipbetweenadoganditsowner—thisisa“hasa”relationship—adoghasanowner.Firstwedefineourdogandpeopleclasses:

classDog():def__init__(self,name,breed,owner):self.name=nameself.breed=breedself.owner=owner

classPerson():def__init__(self,name):self.name=name

Whenwecreateourdogobject,wepassinapersonobjectastheownerparameter:

mick=Person("MickJagger")dog=Dog("Stanley","FrenchBulldog",mick)print(dog.owner)

>>MickJagger

NowourdogStanleyhasanowner—aPersonobjectnamedMickJagger—wecaneasilyreference.

Challenge

CreateanOrangeclassandobjectasmanytimesasyouhavetountilyoucandoitwithoutreferencingthischapter.

Chapter14.MoreObject-orientedProgramming

“Commentingyourcodeislikecleaningyourbathroom-youneverwanttodoit,butitreallydoescreateamorepleasantexperienceforyouandyourguests.”—RyanCampbell

Inthischapterwecoveradditionalconceptsrelatedtoobject-orientedprogramming.WhilesomeoftheseexamplesarespecifictoPython,themajorityoftheseconceptsarepresentinotherlanguagesthatsupportobject-orientedprogramming.

HowVariablesWork

Inthissection,wearegoingtolearnmoreaboutvariables.Variable“point”toanobject.

number=100

numberpointstoanintegerobjectwiththevalue100.

[illustrationofpointtoanobject]

number=101

Whenweassignanewvaluetonumber,itpointstoanewintegerobjectwiththevalue101,andtheoldintegerobjectwithavalueof100isdiscardedbecauseitisnolongerbeingused.Twovariablescanpointtothesameobject:x=100y=x

xpointstoanintegerobjectwithavalueof100.Whenweassignytox,ynowpointstothesameintegerobjectxpointsto:theybothpointtoanintegerobjectwithavalueof100.

[insertillustrationfromhttp://cdn.oreillystatic.com/en/assets/1/event/95/Python%20103_%20Memory%20Model%20_%20Best%20Practices%20Presentation.pdf]

Whatdoyouthinkthefollowingprogramwillprint?

x=10y=xx+=1print(x)

print(y)

Theansweris11and10.xpointstoanintegerobjectwithavalueof10,andwhenwecreatey,itpointstothesameintegerobject.Whenweincrementx,xpointstoanewintegerobject—withavalueof11,buttheintegerobjectwithavalueof10isnotdiscarded,becauseitisbeingused:ystillpointstotheintegerobjectwithavalueof10.Sowhenweprintx,11printsbecauseweassignedxtoanewintegerobject—11—butwhenweprinty—10printsbecausechangingthevalueofx,whichpointstotheintegerobject11,doesnotaffectthevalueofy.Hereisanotherexampletoillustratethispoint.Whatdoyouthinktheoutputofthiscodewillbe?

x=[1,2,3]y=xy[2]=100print(x)print(y)

Theoutputwillbe[1,2,100]twice.Thereasonisthatbothxandypointtothesamelistobject.Inthethirdline,wemakeachangetothatsinglelistobject,andwhenweprintbothvariables,itprintsthelistobjecttheybothpointto,withthechangesmadeinline3.

is

ThekeywordisreturnsTrueiftwoobjectsarethesameobject(theyarestoredinthesamelocationinmemory)andFalseifnot.

“““https://github.com/calthoff/tstp/blob/master/part_II/more_object_oriented_programming/is_example.py”””

classPerson:def__init__(self):self.name='Bob'

bob=Person()the_same_bob=bobprint(bobisthe_same_bob)

another_bob=Person()print(bobisanother_bob)

>>True>>False

Whenweusethekeywordiswithbobandthe_same_bob,theresultisTruebecausebothvariablespointtothesamePersonobject.WhenwecreateanewPersonobjectandcompareittotheoriginalbobtheresultisFalsebecausethevariablespointtodifferentPersonobjects.

None

Thebuilt-inconstantNoneisusedtorepresenttheabsenceofavalue:

x=Nonex>>None

WecantestifavariableisNoneusingconditionalstatements.

“““https://github.com/calthoff/tstp/blob/master/part_I/introduction_to_programming/none_ex1.py”””

x=10ifx:print("xisnotNone")else:print("xisNone:(")

>>xisnotNone

x=Noneifx:print("xisnotNone")else:print("xisNone:(")

>>xisNone:(

Whilethismaynotseemusefulnow,itwillbelater.

ClassesAreObjects

InPython,classesareobjects.ThisideacomesfromtheinfluentialprogramminglanguageSmallTalk.Thismeansthatwhenrunaprograminwhichyoudefineaclass—

Pythonturnsitintoanobject—whichyoucanthenuseinyourprogram:

classPterodactyl:pass

print(Pterodactyl)

>><class'__main__.Pterodactyl'>

Withoutanyworkonourpart,Pythonturnsourclassdefinitionintoanobjectwhichwecanthenuseinourprogram,byprintingitforinstance.

ClassVariablesvs.InstanceVariables

Classescanhavetwotypesofvariables—classvariablesandinstancevariables.Allofthevariableswe’veseensofarhavebeeninstancevariablesdefinedwiththesyntaxself.[variable_name]=[variable_value].Instancevariablesbelongtotheobjectthatcreatedthem.Inotherwordswecandothis:

classLiger:def__init__(self,name):self.name=name

connor=Liger("Connor")print(connor.name)

>>Connor

Classvariablesbelongtoboththeclassthatcreatedthemandtheobject.ThatmeanswecanaccessthemwiththeclassobjectPythoncreatesforeachclass:

classLiger:interests=["swimming","eating","sleeping"]

def__init__(self,name):self.name=name

print(Liger.interests)

>>['swimming','eating','sleeping']

InthisexamplewenevercreatedaLigerobject,yetwewereabletoaccesstheinterestsclassvariable.Thisisbecauseclassvariablescanbeaccessedbytheclassthatcreatedthem.Classvariablescanalsobeaccessedbytheobject:

classLiger:

interests=["swimming","eating","sleeping"]

def__init__(self,name):self.name=name

larry=Liger("Larry")print(larry.interests)

>>['swimming','eating','sleeping']

Classvariablesareusefulwhenyouwanteveryobjectinaclasstohaveaccesstoavariable.InthiscasethenameofeachLigercanbeunique,butallofourLigerstohaveaccesstothesamelistofinterests.

Privatevariables

Mostprogramminglanguageshavetheconceptofprivatevariablesandmethods:specialvariablesandmethodsthedesignerofaclasscancreatethattheobjecthasaccessto,buttheprogrammerusingtheobjectdoesnothaveaccessto.Onesituationprivatevariablesandmethodsareusefulinisifyouhavemethodorvariableinyourclasstheclassusesinternally,butyouplantochangetheimplementationofyourcodelateron(oryouwanttopreservetheflexibilitytohavetheoptionto),andthereforedon’twantwhoeverisusingtheclasstorelyonthosevariablesandmethodsusedinternallybecausetheymightchange(andwouldthenbreaktheircode).

UnlikeotherlanguagesPythondoesnothaveprivatevariables.Variablesthatarenotprivatearecalledpublicvariables,andallofPython’svariablesarepublic.Pythonsolvestheproblemprivatevariablesresolveanotherway—byusingconvention.InPython,ifyouhaveavariableormethodthecallershouldnotaccess,youprecedeitsnamewithanunderscore.Pythonprogrammersknowifamethodorvariablehasanunderscore,theyshouldn’tuseit,althoughtheyarestillabletoattheirownrisk.Hereisanexampleofaclassthatusesthisconvention:“““https://github.com/calthoff/tstp/blob/master/part_II/more_object_oriented_programming/private_variables_ex1.py”””

classPublicPrivateExample:def__init__(self):self.public_variable="callersknowtheycanaccessthis"self._dontusethisvariable="callersknowtheyshouldn'taccessthis"

defpublic_method(self):#callersknowtheycanusethismethod

pass

def_dont_use_this_method(self):#callersknowtheyshouldn'tusethismethodpass

Pythonprogrammersreadingthiscodewillknowself.public_variableissafetouse,buttheyshouldn’tuseself._dontusethisvariable,andiftheydo,theydosoattheirownriskbecausethepersonmaintainingthiscodehasnoobligationtokeepself.dontusethisvariablearoundbecausecallersareanotsupposedtobeaccessingit.Thesamegoesforthetwomethods:Pythonprogrammersknowpublic_methodissafetouse,whereas_dont_use_this_methodisnot.

OverridingMethods

Whenaclassinheritsamethodfromaparent,wehavetheabilitytooverrideit.Takealookatthefollowingexample:

“““https://github.com/calthoff/tstp/blob/master/part_II/more_object_oriented_programming/overriding_ex1.py”””

classMammal:def__init__(self,name):self.hunger=100self.tired=100self.name=name

defprint_result(self,amount,action):print("{}{}decreasedby{}.".format(self.name,action,

amount))

defeat(self,decrease):self.hunger-=decreaseself.print_result(decrease,'hunger')

defsleep(self,decrease):self.tired-=decreaseself.print_result(decrease,'tiredness')

classDolphin(Mammal):pass

classTiger(Mammal):defsleep(self,decrease):self.tired-=decreaseprint("Thetigerisreallytired!")

dolphin=Dolphin('dolphin')dolphin.eat(10)dolphin.sleep(10)

tiger=Tiger('tiger')tiger.eat(10)tiger.sleep(10)>>dolphinhungerdecreasedby10.>>dolphintirednessdecreasedby10.>>tigerhungerdecreasedby10.>>Thetigerisreallytired!

WecreatedtwoclassesthatinheritfromMammal.Thefirstclass,Dolphin,inheritsallofitsfunctionalityfromtheMammalparentclasswithoutmakinganychanges.ThesecondclassTigerdefinesamethodcalledsleep,withdifferentfunctionalitythanthesleepmethoditinheritedfromitsparentclass.Whenwecalltiger.sleep,thenewmethodwedefinedgetscalledinsteadoftheinheritedmethod.Otherthanthis,TigerandDolphinhaveallthesamefunctionalityinheritedfromtheparentclassMammal.

Super

Thebuilt-insuperfunction,letsuscallamethodaclassinheritedfromitsparent.Thesuperfunctionisusedwiththesyntaxsuper().[parent_method]([parameters])whereyoureplaceparent_methodwiththeparentmethodyouwanttocallandpassitanyparametersitneeds.Theparentmethodisthencalledandexecuted,andtherestofthecodeinthemethodsuperwascalledfromthenfinishesexecuting.HereisanexampleofhowwecancalltheMammalparentclass'ssleepmethodfromourTigerclasses’sleepmethod,inordertousethecodefromtheMammalclass’ssleepmethodfollowedbyadditionalfunctionality:

classTiger(Mammal):defsleep(self,decrease):super().sleep(decrease)print("Thetigerisreallytired!")

tiger=Tiger('tiger')

tiger.eat(10)tiger.sleep(10)

>>tigertirednessdecreasedby10.>>Thetigerisreallytired!

FirstweusethesuperkeywordtocalltheMammalparentclass’ssleepmethodandpassinthedecreasevariableasparameter.Mammal’ssleepmethodisexecutedandprints“tigertirednessdecreasedby10”.TheTigerclassessleepmethodthenexecutesthenewfunctionalityweaddedtotheTigerclassessleepmethodand“Thetigerisreallytired!”prints.Byusingthesuperkeywordwewereabletogiveachildclassthefunctionalityfromaparentclass’smethodwithouthavingtoretypethefunctionalityinthechildclass.Thisisimportantbecauseyoushouldalwaysavoidrepeatingcodewiththesamefunctionalityindifferentplacesinyourprogramwhenyoucan.

OverridingBuilt-inMethods

EveryclassinPythonautomaticallyinheritsfromaparentclasscalledObject.AllclassesinPythoninheritmethodsfromthisparentclass.Python’sbuilt-infunctionsusethesemethods(whichwelearnedarecalledmagicmethods)—indifferentsituations—likewhenweprintanobject:

classLion:def__init__(self,name):self.name=name

lion=Lion("Dilbert")print(lion)

>><__main__.Lionobjectat0x101178828>

WhenweprintourLionobject,Pythoncallsthe__repr__methodonourobject,whichitinheritedfromtheObjectparentclass.Itprintswhateverthe__repr__methodreturns.Wecanoverridethisbuilt-inmethodtochangewhathappenswhentheprintfunctionprintsourobject.:

“““https://github.com/calthoff/tstp/blob/master/part_II/more_object_oriented_programming/overriding_builtin.py”””

classLion:def__init__(self,name):self.name=name

def__repr__(self):

returnself.name

lion=Lion("Dilbert")print(lion)

>>Dilbert

Becauseweoverrodethe__repr__method,whenweprintourLionobject,theLionobject’sname—Dilbert—getsprintedinsteadofsomethinglike<__main__.Lionobjectat0x101178828>whichtheinherited__repr__methodwouldhavereturned.Notallmagicmethodsareinherited.Pythonexpressionslike2+2expecttheoperandstohaveamethodtheoperatorcanusetoevaluatetheexpression.Intheexample2+2,integerobjectshaveamethodcalled__add__whichiscalledwhentheexpressionisevaluated,but__add__isnotinheritedwhenyoucreateaclass.Wecancreateobjectsthatcanbeusedasoperandsinanexpressionwiththeadditionoperatorbydefiningan__add__methodinourclass:

classAlwaysPositive:def__init__(self,number):self.number=number

def__add__(self,other):returnabs(self.number+other.number)

x=AlwaysPositive(-20)y=AlwaysPositive(10)

print(x+y)>>10

OurAlwaysPositiveobjectscanbeusedasoperandsinanexpressionwiththeadditionoperatorbecausewedefinedamethodcalled__add__.Themethodmustacceptasecondparameter,becausewhenanexpressionwithanadditionoperatorisevaluated,__add__iscalledonthefirstoperandobject,andthesecondoperandobjectgetspassedinto__add__asaparameter.Theexpressionthenreturnstheresultof__add__.

Inthisexample,weaddedatwist.Weusedthefunctionabstoreturntheabsolutevalueofthetwonumbersbeingaddedtogether.Becausewedefined__add__thisway,twoAlwaysPositiveobjectsaddedtogetherwillalwaysreturntheabsolutevalueofthesumofthetwoobjects.

Challenge

Writeclassestomodelfourspeciesintheanimalkingdom.


“It'salltalkuntilthecoderuns.”—WardCunningham

Inthischapter,wearegoingtousewhatwe’velearnedaboutclassesandobjectstocreatethecardgameWar.Ifyou’veneverplayed,Warisagamewhereeachplayerdrawsacardfromthedeck,andtheplayerwiththehighestcardwins.WewillbuildWarbydefiningaclassrepresentingacard,adeck,aplayer,andfinally,aclasstorepresentthegameitself.TheseclasseswillworktogethertocreatethegameWar.

Cards

Firstwedefineourcardclass:

#https://github.com/calthoff/tstp/blob/master/part_II/war/war.py

classCard:suits=["spades","hearts","diamonds","clubs"]values=[None,None,"2","3","4","5","6","7","8","9","10","Jack","Queen","King","Ace"]

def__init__(self,value,suit):"""suitandvalueshouldbeintegers"""self.value=valueself.suit=suit

def__lt__(self,other):ifself.value<other.value:returnTrueifself.value==other.value:ifself.suit<other.suit:returnTrueelse:returnFalsereturnFalse

def__gt__(self,other):ifself.value>other.value:returnTrueifself.value==other.value:

ifself.suit>other.suit:returnTrueelse:returnFalsereturnFalse

def__repr__(self):returnself.values[self.value]+"of"+self.suits[self.suit]

OurCardclasshastwoclassvariables:suitsandvalues.suitsisalistofstringsrepresentingallofthedifferentsuitsacardcouldbe.valuesisalistofstringsrepresentingthedifferentnumericvaluesacardcouldbe.ThefirsttwoindexesofvaluesareNonesothatthestringsinthelistmatchupwiththeindextheyrepresent—inotherwordssothatthestring“2”invaluesispositionedatindex2.

Ourcardclassalsohastwoinstancevariables:suit,andnumber—eachrepresentedbyaninteger;andrepresentwhatkindofcardtheCardobjectis.Forexample,wecancreatea2ofheartsbycreatingaCardobjectandpassingintheparameters2and1(1representsheartsbecauseheartsisatindexoneinoursuitslist).LaterourCardobjectwillusethesevariablestoprintwhatkindofcarditis,byusingthemasindexesinthesuitsandvalueslists.

illustration

InordertoplayWar,weneedtocomparetwocardstoseewhichcardisbigger.Weaddedtheabilitytocomparetwocardsinanexpressionwiththecomparisonoperatorsgreaterthan(>)andlessthan(<)toourCardclassbydefiningthetwomagicmethodsusedbythesecomparisonoperators:__lt__and__gt__.BydefiningthesemethodswecandeterminewhathappenswhenourCardobjectsarecompared—justlikewedidwithourAlwaysPositiveexampleinthepreviouschapterexceptwithcomparisonoperatorsinsteadoftheadditionoperator.Thecodeinthesemethodslookstoseeifthecardisgreaterthanorlessthantheothercardpassedinasaparameter.However,wemusttakeonemorethingintoconsideration—whathappensifthecardshavethesamevalue—forexampleifbothcardsare10’s.Ifthissituationoccurs,wecomparethevalueofthesuitsandreturntheresult.Thisworksbecausetheintegersweusetorepresentthesuits,areinorder—withthemostpowerfulsuitgettingthehighestintegerandtheleastpowerfulsuitgettingthelowestinteger.Withthesemethodsdefined,wecancomparetocardsusingthegreaterthanandlessthanoperators:

card1=Card(10,2)card2=Card(11,3)print(card1<card2)

>>True

card1=Card(10,2)card2=Card(11,3)print(card1>card2)

>>False

Thelastthingwedoisoverridethemagicmethod__repr__,whichwelearnedinthepreviouschapterisusedbytheprintfunctiontoprintanobject.Overridingthe__repr__methodletsusprintthecardaCardobjectrepresents:

card=Card(3,2):printcard

>>3ofdiamonds

Deck

Nextweneedtodefineaclasstorepresentadeckofcards.Ourdeckwillcreatealistoffiftytwocards—fourofeachsuit—whenitisinitializedanditwillhaveamethodtoremoveacardfromthecardslist.Hereisourdeck:

fromrandomimportshuffle

classDeck:def__init__(self):self.cards=[]foriinrange(2,15):forjinrange(4):self.cards.append(Card(i,j))shuffle(self.cards)

defremove_card(self):iflen(self.cards)==0:returnreturnself.cards.pop()

WhenaDeckobjectisinitialized,thetwoforloopsin__init__createallofthecardsinafiftytwocarddeckandappendthemtoourcardslist.Thefirstloopisfrom2to15becausecardsstartwiththevalue2andendwiththevalue14(theace).Eachtimearoundtheinnerloop,anewcardwillbecreatedusingtheintegerfromtheouterloopasthevalue(i.e.,14foranace)andtheintegerfromtheinnerloopasthesuit(i.e.a2forhearts).Thisresultsinthecreationoffiftytwocards—oneforeverysuit.

Finally,weusetheshufflemethodfromtherandommoduletorandomlyrearrangetheitemsinourcardslisttomimicshufflingadeckofcards.Ourdeckhasjustonemethod,remove_card,whichreturnsNoneifourlistofcardsisempty,andotherwiseremovesandreturnsacardfromthecardslist.UsingourDeckclass,wecancreateanewdeckofcardsandprintoutallthecardsinthedeck:

deck=Deck()forcardindeck.cards:print(card)

>>4ofspades>>8ofhearts...

Player

Weneedaclassrepresentingeachplayerinthegamesowecankeeptrackoftheircardsandhowmanyroundsthey’vewon.Hereisourplayerclass:

classPlayer:def__init__(self,name):self.wins=0self.card=Noneself.name=name

OurPlayerclasshasthreeinstancevariables:wins tokeeptrackofhowmanyroundsthey’vewon,cardtorepresentthecurrentcardtheplayerisholding,andname tokeeptrackoftheplayer ’sname.

Game

Finally,wecancreatetheclassrepresentingthegameitself:

classGame:def__init__(self):name1=input("player1name")name2=input("player2name")self.deck=Deck()self.player1=Player(name1)self.player2=Player(name2)

defplay_game(self):cards=self.deck.cardsprint("beginningWar!")response=Nonewhilelen(cards)>=2andresponse!='q':

response=input ( 'qtoquit.Anyotherkeytoplay.' )player1_card=self .deck.remove_card()player2_card=self .deck.remove_card() print ( "{}drew{}{}drew{}".format( self.player1.name ,

player1_card , self .player2.name ,player2_card)) ifplayer1_card>player2_card:self .player1.wins+= 1print ( "{}winsthisround" .format( self .player1.name)) else :self .player2.wins+= 1print ( "{}winsthisround" .format( self .player2.name)) print ( "TheWarisover.{}wins" .format( self.winner( self.player1 ,self .player2)))

def winner ( self , player1 , player2): ifplayer1.wins>player2.wins: returnplayer1.name ifplayer1.wins<player2.wins: returnplayer2.name return "Itwasatie!"

Whenourgameobjectisinitialized,the__init__ methodiscalledandweusethe inputfunctiontocollectthenamesofthetwoplayersinthegamewhichwesaveinthevariablesname1 andname2 .WecreateanewDeck objectandstoreitintheinstancevariabledeck,andcreatetwoPlayer objectsusingthenameswecollectedfromtheuserandstorethemintheinstancevariablesplayer1 andplayer2.OurGameclasshasamethodcalled play_gameusedtostartthegame.Theheartofthemethodistheloopthatkeepsthegamegoingaslongastherearetwoormorecardsleftinthedeck,andaslongasthevariableresponse isnotequalto“q” .Beforeourloopwedefineresponseas None,andlaterwesetittotheinputoftheuser,whichistheonlysituationitcanbecome“q” andendthegame.Otherwise,thegameendswhentherearelessthantwocardsleftinthedeck.

Eachtimethroughtheloop,twocardsaredrawn.Thefirstcardisassignedtoplayer1andthesecondcardisassignedto player2.Nextweprintthenameofeachplayerandwhatcardtheydrew.Wecomparethetwocardstoseewhichcardisgreater,incrementthewinsinstancevariablefortheplayerthatwon,andprintamessagethatsayswhichplayerwon.

OurGameclassalsohasamethodcalledwinnerwhichtakestwoplayerobjects,looksatthenumberofroundstheywon,andreturnstheplayerthatwonthemostround.Whenourdeckrunsoutofcards,weprintamessagethatthewarisover,callthewinnermethod(passinginboth player1andplayer2 )andprintthenameoftheresult—theplayerthatwon.

War

Whenweputitalltogether,wehavethecardgameWar:

from randomimportshuffle

class Card :suits=["spades" , "hearts" , "diamonds" ,"clubs" ]values=[ None ,None , "2" , "3", "4", "5" ,"6" , "7", "8" ,"9" , "10" , "Jack", "Queen" , "King", "Ace" ]

def__init__ ( self ,value , suit): """suitandvalueshouldbeintegers""" self .value=value self .suit=suit

def__lt__ ( self , other): if self.value<other.value: returnTrueifself .value==other.value: ifself .suit<other.suit:returnTrueelse :returnFalsereturnFalse

def__gt__ ( self, other): if self.value>other.value: returnTrueifself .value==other.value: ifself .suit>other.suit:returnTrueelse :returnFalsereturnFalse

def__repr__ ( self ): return self .values[ self .value]+"of"+ self .suits[self .suit]

class Deck: def__init__ ( self ): self .cards=[] for iin range ( 2 , 15 ): forj in range (4 ):self .cards.append(Card(i , j))

shuffle( self .cards)

def remove_card ( self ): if len( self .cards)==0 : returnreturn self .cards.pop()

class Player : def __init__ ( self , name): self .wins= 0 self .card= None self .name=name

class Game: def __init__ ( self ):name1= input ( "player1name")name2= input ( "player2name") self .deck=Deck() self .player1=Player(name1) self .player2=Player(name2)

def play_game( self ):cards= self .deck.cards print ( "beginningWar!" )response=Nonewhile len(cards)>= 2andresponse!= 'q':response=input ( 'qtoquit.Anyotherkeytoplay.')player1_card=self .deck.remove_card()player2_card=self .deck.remove_card() print ( "{}drew{}{}drew{}" .format( self .player1.name , player1_card, self .player2.name ,player2_card)) ifplayer1_card>player2_card: self.player1.wins+= 1 print ( "{}winsthisround".format( self.player1.name)) else : self.player2.wins+= 1 print ( "{}winsthisround".format( self.player2.name)) print ( "TheWarisover.{}wins" .format( self .winner( self .player1, self .player2)))

def winner ( self, player1 , player2): if player1.wins>player2.wins: return player1.name if player1.wins<player2.wins:

return player2.name return"Itwasatie!"

game=Game()game.play_game()

>>"player1name"…

Chapter16.Practice

Exercises

0.Buildanobject-orientedtext-basedversionofBlackjack.0.Buildawebscrapertocollectdatafromanotherwebsite.0.FindaPythonprojectyouareinterestedinhostedonpip,(hintlookonGitHub),downloadit,anduseitinaprogram.

Read

0.https://julien.danjou.info/blog/2013/guide-python-static-class-abstract-methods0.http://stackoverflow.com/questions/2573135/python-progression-path-from-apprentice-to-guru

PartIIIIntroductiontoProgrammingTools

Chapter17.Bash

Inthissectionofthebookwelearntousetoolsthatwillmakeusmoreeffectiveprogrammers.InthischapterwelearntousethecommandlineshellBashthatcomeswithmanyversionsofLinux andOSX (whichareUnix-likeoperatingsystems).Acommandlineshellisaprogramthatletsyoutypeinstructionsintoitthatyourcomputerexecuteswhenyoupressenter.

OnceyouareabletouseBash,youwillbeabletouseUnix-likeoperatingsystemsmoreeffectively.ThisisimportantbecauseUnix-likeoperatingsystemsaresowidelyusedintheprogrammingworld:Linuxrunsonthemajorityoftheworld’swebservers,andmanyprogrammingjobswillrequireyoutobefamiliarwithUnix-likeoperatingsystems.TheexamplesinthischapterassumeyouareusingeitherUbuntuorOSX.IfyouareusingWindows,youcanfollowalongwiththeexamplesbygoingtotheselftaughtprogrammer.io/bash whereyouwillfindalinktoawebsitethatemulatesBash—youjusttypethecommandsintothegreenboxinthewebsite.

FindingBash

YoucanfindBashbysearchingforterminalfromtheicontitledSearchyourcomputerandonlineresourcesifyouareusingUbuntuorfrom Spotlightsearch ifyouareusingaMac.UsingBashBashissimilartothePythonShell.TheBashcommandlineshelltakesitsownprogramminglanguagecalledBashasinput.TheprogramminglanguageBashhascommands—whicharelikefunctionsinPython.Westartwithakeyword;typeaspace;typetheparameterwewanttogivethecommand(ifany);hittheenterkey;andBashreturnstheresult.OneofBash’scommandsisecho,whichissimilartothe printfunctioninPython.Hereisanexample:

$echoHello,World!>>Hello,World!

Wetypedthecommandecho intoBash,followedbyaspaceandHello,World! asaparameter.Whenwepressenter,Hello,World! printsintheBashcommandlineshell.Youcanalsouseprogramsyou’veinstalled—likePython—fromtheBashcommandlineshell.Enterthecommandpython3tousePythonfromtheshell:

$python3>>

NowyoucanexecutePythoncode:

print(“Hello,World!”)>>Hello,World!

Enterexit() toexitPython.

RelativevsAbsolutePaths

Anoperatingsystemismadeupofdirectoriesandfiles.Adirectoryisanotherwordforafolderonyourcomputer.Alldirectoriesandfileshaveapath.Apathisasanaddresswhereadirectoryorfileexistsinyouroperatingsystem.WhenyouareusingtheBashcommandlineshell,youarealwaysinadirectory,locatedataspecificpath.Youcanseethepathyouareatbyenteringthecommandpwd intheBashcommandlineshell:

pwd>>/Users/bernie

pwd standsforprintworkingdirectory(aworkingdirectoryisthedirectoryyouarecurrentlyin),andthepaththatprintsonyourcomputerwillbethenameofwhateverdirectoryyouarein.

Thefoldersinyouroperatingsystemareatree.AtreeisanimportantconceptinComputerSciencecalledadatastructure(coveredinPartIV).Inatree,thereisarootatthetop.Therootcanhavebranches,andeachoneofthebranchescanhavemorebranches,andthosebranchescanhavebranches.Thisgoesonindefinitely.

[addillustrationofatreedatastructure]

root/\homeetc/\berniebin/\testprojectsEverybranchonthetreeisadirectory,includingtheroot.Thetreeshowshowtheyareconnectedtoeachother.WhenyouareusingBash,atanygiventimeyouareatoneofthelocationsonthetree,andapathisawayofexpressingthatlocation.TherearetwowaysofexpressingalocationonUnix-likeoperatingsystems:absolutepathsandrelativepaths.Anabsolutepathisawayofexpressingalocationstartingfromtherootdirectory.Anabsolutepathismadeupofthenameoffoldersinthetree,inorder,separatedbybackslashes.Theabsolutepathtotheberniedirectory(fortheoperatingsystemillustratedabove)is

/home/bernie .Thefirstslashrepresentstherootdirectory;followedbythehomedirectory;followedbyanotherslashandtheberniedirectory.Theotherwayofspecifyingalocationonyourcomputeriscalledusingarelativepath.Insteadofstartingattherootdirectory,arelativepathisrelativetothedirectoryyouarein.Youroperatingsystemknowsapathisrelativewhenitdoesn’tbeginwithabackslash.Ifwewerelocatedinthehome directoryintheexampleabove,therelativepathtotheprojectsdirectorywouldbebernie/projects .Ifwewereinthehomedirectory,therelativepathtobernie issimplybernie .

Navigating

Youcanusethecd command,whichstandsforchangedirectory,tonavigatetoadirectoryusinganabsoluteorrelativepath.Enterthecd commandfollowedbytheabsolutepath/ tonavigatetotherootdirectory:

$cd/

Youarenowinyourrootdirectory,whichyoucanverifywiththecommand pwd :

$pwd>>/

Thecommand ls,whichstandsforlistdirectories,printsthefoldersinthedirectoryyouarein:

$ls>>bindevinitrd.imglost+found

...

Youcancreateanewdirectorywiththecommandmkdir .Directorynamescannothavespacesinthem.Usethemkdir commandtomakeanewdirectorycalled tstp :

$mkdirtstp>>

Verifythenewdirectoryexistswiththecommand ls.Nowusethe cd commandtoenterthetstp directorybypassingintherelativepathto tstp asaparameter:

$cdtstp

Nowusethe cdcommandfollowedbytwoperiodstoreturntothefolderyouwereinbeforeyouenteredthetstp directory:

$cd..

Deletethenewdirectoryusingthecommandrmdir ,whichstandsforremovedirectory:

$rmdirtstp

Usethecommand lstoverifythedirectorywasdeleted.

Flags

Commandshaveaconceptcalledflagsthatallowtheissuerofthecommandtochangethecommandsbehavior.Ifyouuseacommandwithoutanyflags,allofthecommandsflagsaresettofalse.Butifyouaddaflagtoacommand,theflagissettotrueandthebehaviorofthecommandchanges.The - and-- symbolsareusedtoattachflagstoacommand. --authorisanexampleofaflagyoucanaddtothels commandtoprinttheauthorofallofthedirectoriesandfilesthatgetlisted.ThisexampleisforLinux,onOSXyoucanusethesameflagbutyouneedtouseonedashinsteadoftwo.Hereisanexampleofusingthe--authorflagwiththelscommand:

$ls--author>>drwx------+13coryalthoff442BSep1617:25Pictures...

Whenyouaddthe --author flagtoyour ls command,thenameofeachdirectoryandfolderinyourcurrentdirectorywillprint—aswellassomeadditionalinformation,includingthenameofthepersonthatcreatedthem.

vim

Vimisacommandlinetexteditor.It'slikeMicrosoftWord,exceptyouuseitfromthecommandline.IfyouareusingUbuntu,firstinstallvimwiththecommand apt-getinstallvim .Makesuretoenter Y whenprompted.IfyouareusingaMac,itshouldcomewithvimIfyouareusingtheonlinebashshell,italreadyhasviminstalled.Youcancreateanewfilewithvimbygoingtothecommandlineandusingthecommandvim[nameofthefiletocreate] .Usethecommandvimself_taught.txt tocreateanewtextfilecalled self_taught.txt .Presstheiorinsertkeyandtypeaparagraphoftextintothefile.Anyparagraphoftextyoufindontheinternetwilldo.Thereasonyouhavetohittheiorinsertkeywhenyoufirstenterafileisbecausevimhasdifferentmodesoptimizedfordifferentactivities.vimstartsin,NormalMode,whichisnotmeantforaddingtexttothefile(itismeantforeasynavigation)—youcandeletetextinnormalmode,butyoucannotinsertnewtext.Onceyoupresstheiorinsertkeyandenternormalmode,youcanusevimlikeawordprocessor—trytypingintovim.

Sinceyoucan’tuseyourmousetomovethecursoraround,it’simportanttolearnafewshortcutstojumptodifferentlocationsinyourdocument,sothatyoudon’tendupusingthearrowkeysonyourkeyboard(becausethat’sslow).Topracticemovingaround,firstmakesureyouareinNormalMode( control-c ).Youcanmovetothebeginningofawordbypressingb andtheendofawordwithe .0willmoveyoutothebeginningofthelineyouareon,whilethedollarsign$ willmoveyoutotheendoftheline.H willmoveyoutothefirstlineofthepageandL willmoveyoutothelast. Youcandeleteentirelinesoftextinnormalmodebypressingthedkeytwice.Spendsometimeusingthesekeystogetfamiliarwithwithnavigatingthroughafileusingvim.

ToexitvimyouneedtofirstswitchtoNormalModebypressingcontrol-c .Nextpresstheshift keyandthenhitthecolon key(whilestillholdingtheshiftkey).Fromhereyoucantypeq! ifyouwanttoquitwithoutsavingyourchangestothefile,ortypexifyouwantsaveyourchangesandquit.Onceyou’vetypedoneoftheseoptions,presstheenterkeytoexit.vimisusefulinafewsituations:serversareusuallyonlyaccessedwithacommandlineshell,soifyouwanttomakechangestoafileonaserver,youneedtouseacommandlinetexteditor,andonceyougetgoodatusingvim,itisoftenfastertouseittomakechangesthanusingaconventionalwordprocessor.Trytypingvimtutor intheBashcommandlineshellandseewhathappens.

Touch

Youcanusethecommandtouch followedbyafilepathtoquicklycreateanewfile:

$touchpurple_carrots.txt>>

ViewAFileWithless

The less commandenablesyoutoviewfilesfromthecommandline.Pickafileonyourcomputerandusethecommandless[filepath] toviewthefilefromthecommandlineintheless program.Makesureyouareinthesamefolderyoucreatedthefileselftaught.txt inwithvimandpassthefilenameasaparameter:

$lessself_taught.txt>>whatevertextyouputinyourfile

Pressq toexit.

Users

Operatingsystemshavedifferentusers.Auserrepresentsapersonintheworld—someoneusingtheoperatingsystem—andeachuserhasdifferentactivitiestheyareallowedtodo—calledtheirpermissions—aswellastheirowndirectoriesandfilesthatotheruserscan’taccess.Userscanalsobeplacedintogroups,withanentiregroupgiven.Youcanseethenameoftheuseryouareonyouroperatingsystemwiththecommandwhoami :

$whoami>>cory

Normallyyoustartastheuseryoucreatedwhenyouinstalledyouroperatingsystem.Thisuserisnotthemostpowerfulusertheoperatingsystemhas.Therootuseristhehighestleveluser,whichmeansithasthehighestlevelofpermissions.Everysystemhasarootuser.Therootusercandoananything:forexampletherootusercancreateordeleteotherusers.

Forsecurityreasons,younormallydonotloginastherootuser.Insteadyouuseacommandcalledsudoinfrontofanothercommandtotemporarilyusethepoweroftherootusertoissuethecommand.sudo allowsyoutodomostthingsastherootuser,butnoteverything.Makesuretobecarefulusingsudo ,becauseusingcommandswithsudocanharmyouroperatingsystemifyoudon’tknowwhatyouaredoing.Wearenotgoingtocoverusingsudointhisbook,butI’vesetupatutorialonusing sudo at theselftaughtprogrammer.io/sudo youcanreadthroughonceyoufeelcomfortableusingtheBashcommandlineshell.

Permissions

Everydirectoryandfileonyourcomputerhasasetofpermissions.Thesepermissionsdefinewhatactionsausercanperformonthem.Therearethreetypesofpermissions:r ,rwandx,whichstandforread,writeandexecute.Readingafilemeansviewingit,writingafilemeanschangingit,andexecutingafilemeansrunningitasaprogram.Youcanviewafileordirectory’spermissionswiththecommandls-lah[name_of_file] .Usethecommandtouch tocreateanewfilecalledtstp :

$touchtstp>>

Nowwecanviewthefilespermissions:

$ls-lahtstp-rw-r--r--1coryalthoffstaff5BFeb2111:55test.py

Ourfilehasthreepermissionsthatapplytothreedifferentgroupsofusers—representedby -rw-r--r-- .Thefirstsetappliestotheownerofthefile,thesecondsetappliestothegroupassignedthefileandthethirdsetappliestoeveryone.Sointhisexample,theownerofthefilehaspermissiontoreadandwritethefile,whereaseveryoneelsecanonlyreadthefile.

Youcanaddandsubtractpermissionswiththecommandchmod(shortforchangemode).Youcanaddpermissiontoreadwiththecommand chmod+r ,permissiontoreadandwritewiththecommand chmod+rw ,andpermissiontoexecutewithchmod+x .Youcansubtractthesamepermissionswithchmod-r ,chmod-w andchmod-x ,respectively.

BashPrograms

Ifwewanttorunabashscriptfromthecommandline,weneedtogivethefileownerpermissiontoexecute.Createanewfilecalledhello_world.sh andtypeechoHello,World!init.Thereasonweusethe .shextensionistoletanyonewhoseesthisfileknowitisaBashscript.Thesyntax./[file_name] isusedtoexecuteafileinthedirectoryyouarein.Youexecuteafilewith/[file_name] andtheperiodmeanslookforthefileinthecurrentdirectory.Youcouldalsoreplacetheperiodwiththepathtothefile.Tryexecutingthefile(makesureyouareinthesamedirectoryasthefile):

$./hello_world.sh>>Permissiondenied

Thereasontheconsoleprintedpermissiondeniedisbecausewedonothavepermissiontoexecutethefile.Wecanchangethatwiththechmod command.

$ chmodu+xhello_world.sh$./hello_world.sh .>>Hello,World!

Intheexampleabove,weaddedu to+x toformu+x becauseustandsforuser,andweonlywantedtogiveourselvestheabilitytoexecutethefile.Ifyouarerepeatinganargumentfromthepreviouscommandline,inthiscasethefilename,youcanuse!$ torepresenttherepeatedargument.

Processes

HiddenFiles

Youroperatingsystem,anddifferentprogramsonyourcomputer,usehiddenfilestostoredata. HiddenfilesinUnix-likesystemsarefilesthatbydefaultarenotshowntousersbecausechangingthemcanaffecttheprogram(s)thatdependsonthem.Hiddenfilesstartwithaperiod—forexample—.hidden .Youcanseehiddenfilesusingthecommand lswiththeflag-a ,whichstandsforall.Createahiddenfilenamed .self_taughtwith touch.hiddenandtestifyoucanseeitwiththecommandsls andls-a .

EnvironmentalVariables

Youroperatingsystemcanstoreandusevariablescalledenvironmentalvariables.Youcancreateanewenvironmentalvariablefromthecommandlinewiththesyntaxexportvariable_name=[variable_value] .Inordertouseanenvironmentalvariable,youmustputadollarsigninfrontoftheenvironmentalvariable’sname.Forexample:

$exportx=100$echo$x>>100

Creatinganenvironmentalvariablefromthecommandlineisnotpermanent.Ifyouquitthecommandline,reopenit,andtype echo$x,theterminalwillnolongerprint100 .WecanmaketheenvironmentalvariablexpersistentbyaddingittoahiddenfileusedbyUnix-likeoperatingsystemscalled.profile locatedinyourhomedirectory.Gotoyourhomedirectorywiththecommand cd~ (~isashortcutforrepresentingyourhomedirectoryonUnix-likeoperatingsystems)andopenyour .profilefileusingvim withthecommandvim.profile .MakesuretoenterNormalMode,typeexportx=100intothefirstlineofthefile,andexitwith:x .Closeandreopenyourcommandline,andyoushouldstillbeabletoprinttheenvironmentalvariableyoudefined:

$echo$x>>100

Theenvironmentalvariablexgets 100aslongasit’sdefinedinyour .profilefile.

$PATH

WhenyoutypeacommandintotheBashcommandshell,itlooksforthecommandinallofthedirectoriesstoredinanenvironmentalvariablenamed$PATH .$PATH isastringofdirectorypathsseparatedbycolons.TheBashcommandshelllooksineachofthesedirectoriesforafilematchingthenameofthecommandyoutyped.Ifthefileisfound,the

commandisexecuted,ifnotthecommandshellprints“commandnotfound”.Useecho toprintyour$PATH environmentalvariable(your$PATHmaylookdifferentthanmine):

echo$PATH>>/usr/local/sbin:/user/local/bin:/usr/bin:...

Inthisexampletherearethreedirectorypathsin$PATH :/usr/local/sbin,/user/local/bin and/usr/bin .TheBashcommandlineshellwillbeabletoexecuteanycommandfoundinthesethreedirectories.Youcanseewhereacommandlike grepisinstalledbypassingitasaparametertothecommand which:

$whichgrep>>/usr/bin/grep

The grepcommandislocatedin /usr/bin ,oneofthelocationsinmyoperatingsystem’s$PATH environmentalvariable.

pipes

InUnix-likesystems,thecharacter | isknownasapipe.Youcanuseapipetousetheoutputofonecommandastheinputforanothercommand.Forexample,wecanpasstheoutputofthe ls commandastheinputoftheless commandtoopenlesswithallofthefilesinthecurrentdirectory:

$ls|less>>Applications...

Theresultisatextfilewiththeoutputof ls openedupintheprogramless (pressqtoquitless ) .Youarenotlimitedtopipingtwocommands—youcanchainmultiplecommandstogetherusingpipes.

c at

Youcanusetheversatile catcommandtodisplaythecontentsofafileandtocatenate,whichmeans“toconnectinaseries.”11 .Createanewfilecalledhello.txt andaddHello,World! asthefirstline.Nowusethecat commandtoprintthecontentsofthefile:

$cathello.txt>>echoHello,World!

Tousethecat commandtocatenatetwofiles,firstcreateafilecalled c1.txt andaddthetextBoy .Thencreateanotherfilecalledc2.txt andaddthetextMeetsWorld .Nowwecancatenatethembypassingbothfilesasparameterstothecatcommand ,followedbythegreaterthansymbol( >),andthenameofthenewfiletocreate:

$ catc1.txtc2.txt>combined.txt >>

Openupthenewlycreatedcombined.txt whichshouldsayBoyMeetsWorld .

RecentCommands

Youcanscrollthroughyourrecentcommandsbypressingtheupanddownarrowsinthecommandlineshell.Toseealistofallofyourrecentcommandsusethecommandhistory:

$history>>1.echoHello,World!>>2.pwd>>3.ls…

JumpAround

Whenyouaretypingacommandintheterminal,therewillbetimeswhereyouwanttoeditthecommandyou’vealreadytyped.Yourfirstinstinctwillbetousethearrowrightorleftkeytomovethecursortothepartyouwanttochange.Butthisisslow.Insteadyoushoulduseshortcutsthatwillgetyoutherefaster.

Type echohello,world! (withoutpressingEnter)intheterminalanduseescb tomovethecursorbackoneword,andescftomovethecursorforwardoneword.Youcanalsomovethecursortothebeginningofthelinewithcontrola ortheendofthelinewithcontrole .

TabComplete

Tabcompleteisafeaturethatwillhelpimprovethespeedyougetthingsdonefromthecommandlineshell.Ifyouareinthemiddleoftypingacommandyouandpressthetabbuttononyourkeyboard,thecommandlineshellwilltrytoautocompletethecommandforyou.

Tryitforyourselfbytypingechinthecommandlinefollowedbytab ; ech willautomaticallygetturnedintoecho.Youcanalsousetab tocompletefileordirectorypaths.Starttypingthepathofthedirectoryyouareinandfinishitoffbypressingt ab. Ifyoupresstabandnothinghappens,itisbecausetwocommandsorpathsarenamedsimilarly,andtheshelldoesn’tknowwhichtochoose.Forexampleifyouhaveadirectorynamedcar ,andanotherdirectorynamed candy ,andyoutypeca andtrytotabcomplete,nothingwillhappenbecausetheshellwon’tknowwhethertochoosecar orcandy .Ifyouaddann soyou’vetypecan,andpresstabcomplete,theshellwillautocompletecandybecausetheshellknows car isnotcorrect.

Wildcard

Awildcardisacharacterusedtomatchapattern.Twoexamplesofwildcardsareanasterisk andaquestionmark.Theasteriskwildcardmatcheseverythingeitherstartingorendingwithapattern,dependingifyouputitbeforeorafterthepattern.Asteriskwildcardsarecommonlyusedwiththecommandls .Thecommand ls*.txtwillshowanyfilesinthecurrentdirectorythatendwith.txt ,whereasls.txt* willshowanyfilesthatwith.txt .

Aquestionmarkwillmatchanysinglecharacter.Solst?o willshowanyfilesordirectoriesthatmatcht followedbyanycharacterfollowedbyo.Ifyouhaddirectoriesnamedtwoandtoo ,theybothwouldgetprinted.

OtherTools

Ifyourterminalgetscluttered,youcanclearitwiththecommandclear .Ifaprocessistakingtoolong,youcankillitwithcontrol+c .Anotherpowerfulcommandisgrep, usedtosearchfilesforpatternsandcoveredinthenextchapter.

TheOneWeekChallenge

Ichallengeyoutoonlyusethecommandlineforoneweek.Thatmeansnographicaluserinterfaceforanythingotherthanusingtheinternet!Youshouldcompletethischallengeduringaweekyouareactivelyworkingonaprogrammingproject.

Chapter18.RegularExpressions

quote

Aregularexpression isasequenceofcharactersusedtolookforapatterninastring.Inthischapterwearegoingtopracticeusingregularexpressionsfromthecommandlineusinggrep ,acommandforusingregularexpressionstosearchfiles.WewillalsolearntouseregularexpressionsinPython.

Setup

Togetstartedpracticingusingregularexpressions,createafilecalledzen.txt .Fromthecommandline(makesureyouareinsidethedirectorywhereyoucreated zen.txt)enterthecommand python3-c“importthis”.ThiswillprintoutTheZenofPython,apoemwrittenbyTimPeters:

TheZenofPythonBeautifulisbetterthanugly.Explicitisbetterthanimplicit.Simpleisbetterthancomplex.Complexisbetterthancomplicated.Flatisbetterthannested.Sparseisbetterthandense.Readabilitycounts.Specialcasesaren'tspecialenoughtobreaktherules.Althoughpracticalitybeatspurity.Errorsshouldneverpasssilently.Unlessexplicitlysilenced.Inthefaceofambiguity,refusethetemptationtoguess.Thereshouldbeone--andpreferablyonlyone--obviouswaytodoit.Althoughthatwaymaynotbeobviousatfirstunlessyou'reDutch.Nowisbetterthannever.Althoughneverisoftenbetterthan*right*now.Iftheimplementationishardtoexplain,it'sabadidea.Iftheimplementationiseasytoexplain,itmaybeagoodidea.Namespacesareonehonkinggreatidea--let'sdomoreofthose!

The- c flaginPythonexecutesPythonpassedinasastring,andimportthis printstheZenofPythonwhenexecuted.CopyandpastetheZenofPythonintothefilezen.txt .Ialso

recommendreadingitasitcontainssomegreatwisdom.Usethefunction exit()toexitPython.If youareusingtheonlinebashshell .

IfyouareusingaMac,setthefollowingenvironmentalvariablesintheterminal:

$exportGREP_OPTIONS='--color=always'$exportGREP_COLOR='1;35;40'

Thiswillmakegrep highlightthewordsmatchedintheterminal,whichhappensbydefaultonUbuntubutnotonOSX.Remember,settinganenvironmentalvariablefromtheterminalisnotpermanent,soifyouexittheterminalandcomebackyouwillhavetosettheenvironmentalvariablesagain.Addtheenvironmentalvariablestoyour .profilefileifyouwanttomakethechangepermanent.

SimpleMatch

Thecommandgrepacceptsaregularexpressionandthepathtoafiletolookfortheregularexpressioninasparameters.Fromthecommandline,inthedirectorywhereyoucreatedthefile zen.txt ,enterthefollowingcommand:

$grepBeautifulzen.txt>>Beautiful isbetterthanugly.

Beautiful istheregularexpressionand zen.txt isthepathtothefiletolookfortheregularexpressionin.Beautiful isasequenceofcharactersthatmatchtheword Beautiful .Thisisthesimplestkindofregularexpression.Your console printedthelineBeautifulisbetterthanugly .withBeautiful highlightedbecauseitisthewordtheregularexpressionmatched.

IgnoreCase

IfwechangeourregularexpressionfromBeautiful to beautiful,itwillnolongermatchanythingintheZenofPython.Enter grepbeautifulzen.py toseeforyourself.Ifwewantourregularexpressiontomatchthewordbeautiful regardlessofcase(whetherornotcharactersarecapitalized),wecanusetheflag -i :

$grep-ibeautifulzen.txt>>Beautiful isbetterthanugly.

Becauseweaddedtheflag-i toourcommand,grepignorescaseandhighlightsBeautifulagain.

OnlyReturnMatched

grepreturnstheentirelineofthefileamatchwasfoundon.Wecanreturntheexactwordmatchedbyusingtheflag-o :$grep-oBeautifulzen.txt>>BeautifulNormally,grep wouldhavereturnedBeautiful isbetterthanugly.butbecauseweaddedtheflag-o ,onlytheexactmatch,Beautiful ,getsreturned.

MatchBeginningandEnd

Regularexpressionshavecertaincharactersthatdon’tmatchthemselves,butinsteaddosomethingspecial.Forexamplethe characterisusedtolookformatchesonlyiftheyoccuratthebeginningofaline:

$grep^Ifzen.txt>> If theimplementationishardtoexplain,it'sabadidea.>> If theimplementationiseasytoexplain,itmaybeagoodidea.

Similarly,wecanusethedollarsigntoonlymatchthelinesthatendwithapattern:

$grepidea.$zen.txt>>Iftheimplementationishardtoexplain,it'sabad idea.>>Iftheimplementationiseasytoexplain,itmaybeagoodidea.

Thelinesmatchedbothendwithidea.ThelineNamespacesareonehonkinggreatidea--let'sdomoreofthose! wasignoredbecausealthoughitincludesthewordidea,itdoesnotendwiththewordidea.Youcanalsocombinethetwoanchormatcheswe’vecoveredintotheregularexpression^$tosearchforemptylinesinafile.

MatchMultipleCharacters

Youcanusebracketsinaregularexpressiontomatchanyofthecharactersinsidethebrackets.Inthisexample,insteadofmatchingtextfromourzen.txtfile,wearegoingtouseapipetopassinastringtogrep :

$echoTwoisanumberandtooisnot.|grep-it[ow]o>> Two isanumberandtoo isnot.

Remember,thepipesymbolpassestheoutputofonecommandastheinputofthenext.Inthiscasetheoutputofechoispassedastheinputofgrep.ThecommandechoTwoisanumberandtooisnot.|grep-it[ow]owillmatchboth twoandtoo becausetheregexislookingforat followedbyeitheran oora w followedbyano.

Repetition

Wecanaddrepetitioninourpatternswiththeasterisksymbol.Theasterisksymbolmeans“ Theprecedingitemwillbematchedzeroormoretimes.” 56Wemightwanttosay,matchtw followedbyanyamountofo’s .Theregularexpression greptwo* accomplishesthis:

$echotwotwoootwoonottoo.|grep-otwo*>>two>>twooo>>twoo

Addinga*after twomeanstheregularexpressionshouldmatchanythingwithtwo followedbyanynumberofo’s .

Range

Youcanmatcharangeoflettersornumbersbyputtingtherangeinsidebrackets.Forexample,wemightwanttoonlymatchthenumbersin122233hello334443goodbye939333 .Theregex[ [:digit:]] * willmatchallthenumbersinthestringbecauseitincludesarangeofnumbers( 0-9) ,followedby*whichtellstheregextomatchasmanynumbersinarowasitcan.

$echo122233hello334443goodbye939333|grep-o[0-9]*>> 122233>>33443>>939333

Si milarly,youcanmatcharangeofcharacters(allcharactersinthiscase)withtheregex[[:alpha:]]:

$echo122233hello334443goodbye939333|grep[ [:alpha:]]*

Escaping

Whatifwewanttomatchoneofthespecialcharacterswe’vebeendiscussing,likethedollarsign?Wecandothisbyescapingthecharacter.WecoveredescapinginthechapterManipulatingStrings:escapingmeansprefixingacharacterwithaspecialcharactertolettheprogramevaluatingthesyntaxknowyouwanttousetheactualcharacterandnotthespecialmeaningthecharacternormallyhas.Escapinginregularexpressionsisdonewithabackwardslash:

$echoIlove$|grep\\$>>Ilove$

Normally,thedollarsignhasthespecialmeaningofonlymatchingsomethingattheendofaline,howeverbecauseweescapedit,ourregexlooksforthedollarsign.

RegularExpressionsinPython

Pythonhasalibrarycalledre thatletsyouuseregularexpressionsinPythonprograms.ThesymbolsinPythonregularexpressionsarenotalwaysthesameasgrep ,buttheconceptofregularexpressionsisthesame.

#https://github.com/calthoff/tstp/blob/master/part_III/regular_expressions/reg_expr_ex1.py

import re

line="Matchthis."

matchObj=re.search( 'this' ,line)

ifmatchObj: print (matchObj.group())else: print ( "Nomatch!" )

>>this

re comeswithdifferentmethodslikesearch,whichreturnsthefirstoccurrenceofthepatternyouarelookingfor.Ifamatchisfound,anSRE_Matchobjectisreturned,andwecangetthematchbycallingthefunctiongroup() .Ifnomatchisfound, re.search() returnsNone.Wecanusethefunctionfindall() inthe re moduletofindeveryoccurrenceofapattern,insteadofjustthefirstmatch.

#https://github.com/calthoff/tstp/blob/master/part_III/regular_expressions/reg_expr_ex2.py

importre

line="""Thenumbers172canbefoundonthebackoftheU.S.$5dollarbillinthebushesatthebase

oftheLincolnMemorial."""

matchObj=re.findall( '\d+' ,line)

ifmatchObj: printmatchObjelse : print ( "Nomatch!" )

>>[‘172’,‘5’]

re.findall() returnsalistofallthestringsmatchingthepattern,andanemptylistifthereisnomatch .The+ symbolmatchestheprecedingcharacteroneormoretimes.Ifwewanttomatcheverythinginbetweentwounderscores,wecandoitwith:#https://github.com/calthoff/tstp/blob/master/part_III/regular_expressions/reg_expr_ex3.py

importre

line="""__yellow____red__and__blue__arecolors"""

matchObj=re.findall( '__.*?__',line)

ifmatchObj: printmatchObjelse : print ( "Nomatch!" )

Thetwounderscoresmatchtwounderscores,theperiodmatchesanycharacter,andthequestionmarkfollowedbyanasteriskmeanskeepmatchinganycharacteruntilthereisanotherdoubleunderscore.

HereisafunexampleofusingregularexpressionsinPythontocreatethegameMadLibs:“““https://github.com/calthoff/tstp/blob/master/part_III/lets_read_some_code/lets_read_some_code.py”””

importre

text="""

Giraffeshavearousedthecuriosityof__PLURAL_NOUN__sinceearliesttimes.Thegiraffeisthe

tallestofallliving__PLURAL_NOUN__,butscientistsareunabletoexplainhowitgotitslong

__PART_OF_THE_BODY__.Thegiraffe'stremendousheight,whichmightreach__NUMBER____PLURAL_NOUN__,comesfromitslegsand__BODYPART__.

"""

def mad_libs (mls): """ :param mls:Stringwithpartstheusershouldfilloutsurroundedbydoubleunderscores.Underscorescannotbeinsidehinte.g.,no__hint_hint__only__hint__.""" hints=re.findall( "__.*?__" ,mls) ifhints: forword inhints:new_word= input ( "entera{}" .format(word))mls=mls.replace(word , new_word ,1 ) print (' \n ' ) print (mls) else : print ("invalidmls" )

mad_libs(text)

ZenChallenge

WritearegularexpressiontofindeverywordintheZenofPythonendingwiththeletter y .

Chapter19.PackageManagers

“Everyprogrammerisanauthor.”― SercanLeylek

Packagemanagersareprogramsthatinstallandmanageotherprogramsonyouroperatingsystem.Anexampleofmanagingaprogramiskeepingtheprogramuptodatewhenanewversioncomesout.Inthischapterwearegoingtolearntouseseveralpackagemanagers.Packagemanagersareusefulbecauseprogrammersuseprogramstocreatenewprograms.Forexample,mostprogramsuseoneormoredatabases(welearntouseadatabaseinthelastchapterofthissection),andwhichareprogramsthemselves.Programmersusepackagemanagerstodownloadandkeeptheirdatabasesuptodate,aswellastoinstallandmanagethewidevarietyofotherprogramstheyuseintheircraft.Inthischapterwewilllearntousethreepackagemanagers:apt-get,Homebrew,andpip.apt-getisapackagemanagerforUbuntu,HomebrewisapackagemanagerforOSX,OneGetisapackagemanagerforWindowsandpipisapackagemanagerthatcomeswithPythonandisusedtodownloadandmanagePythonprograms.

Packages

Apackageissoftware“packaged”fordistribution—itincludesthefilesthatmakeuptheactualprogram,aswellasfileswithmetadata(dataaboutdata);suchasthesoftware’sname,versionnumber,anddependencies(programsthatneedtobedownloadedinorderforittorunproperly).Packagemanagersdownloadpackages,installthem—whichmeansdownloadinganydependenciesthepackagehas.

Apt-get

Apt-getisapackagemanagerthatcomeswithUbuntu.Youcannotuseapt-getfromtheonlineBashcommandlineshellemulator.YouneedtohaveUbuntuinstalledsoyoucanusethecommandsudo .IfyoudonothaveUbuntuinstalledonyourcomputer,youcanskipthissection.

Youinstallapackagewithsudoapt-getinstall[package_name].Hereisanexampleofinstallingapackagenamed aptitude:

$sudoapt-getinstallaptitude

>>Doyouwanttocontinue?[Y/N]Y

…>>Processingtriggersforlibc-bin(2.19-0ubuntu6.7)…MakesuretotypeY whenpromptedwith“DoyouwanttoContinue[Y/N]”.The aptitudepackageshouldnowbeinstalled.Youcanuseitbytypingaptitude intotheBashcommandlineshell:

$aptitude>>

ThiswillopenupAptitude,ahelpfulprogramthatshowsinformationaboutpackages.YoushouldseealistofallthepackagesinstalledonyouroperatingsystemundertheoptionInstalledPackages,aswellasalistofpackagesyou’veyettoinstall,underNotInstalledPackages.Youcanalsolistthepackagesthathavebeeninstalledwithapt-getwiththecommandaptlist--installed :

$aptlist--installed>>python3-requests...

Youcanremovepackagesusingaptitudewiththesyntaxapt-getuninstall[package_name] .Ifyouwanttoremoveaptitude fromyourcomputer,youcanuninstallitwithsudoapt-getuninstallaptitude .That’sallthereistoit—installingandremovingprogramsisassimpleasusingtwocommandswithapt-get .

Homebrew

HomebrewisapopularpackagemanagerforOSX.IfyouhaveaMac,youcaninstallHomebrewfromthecommandline.Gotohttp://brew.sh/ andcopyandpastetheprovidedscriptintheterminal .Onceinstalled,youshouldseealistofcommandswhenyoutypebrewintoyourterminal:

$brew>>Exampleusage:>>brew[info|home|options][FORMULA...]...

WecaninstallpackageswithHomebrewusingthesyntaxinstall[package_name].Usethefollowingcommandtoinstallapackagecalledcalc :

$brewinstallcalc...>>==>Pouringcalc-2.12.5.0.el_capitan.bottle.tar.gz

>>/usr/local/Cellar/calc/2.12.5.0:518files,4.4M

Acalculatorprogramcalled calc isnowinstalled.Youcanuseitbytypingcalc fromthecommandline(typequit toexit);

$calc>>2+2>>4

Thecommand brewlist printsthesoftwareyou’veinstalledusingHomebrew:

$brewlist>>calc

YoucanremovepackagesusingHomebrewwiththesyntaxbrewuninstall[package_name] .Ifyouwanttoremovecalc fromyourcomputer,youcanuninstallitwiththecommandbrewuninstallcalc .Thatisallyouneedtoknow—youarereadytouseHomebrewtomanageyourpackages.

OneGet

OneGetisthefirstpackagemanagertocomewithWindows:itwasfirstreleasedwithWindowsTen.IfyoudonothaveWindowsTen,youcandownloadOneGetbyfollowingtheinstructionsonitsGitHub page:https://github.com/OneGet/oneget .

WecaninstallpackagesonOneGetusingthecommand

pip

PipisusedtodownloadPythonpackages.PipwillinstallaPythonpackageforyou,andonceinstalled,youcanimporttheprogramyoudownloadedasamoduleinyourPythonprograms.First,checkPipisinstalledbygoingtoeithertheBashcommandlineshell,ortheCommandPromptifyouareusingWindows(whichyoucanfindbysearchingCommandPromptfromtheRunWindow)andtypingpip .

$pip>>Usage:pip<command>[options]

Commands:installInstallpackages.downloadDownloadpackages.

....

Alistofcommandsyoucanusewithpipshouldprint.PipcomeswithPythonwhenyoudownloadit,butinearlierversionsitdidn’tsoifnothinghappenswhenyouenterpip,google“installingpipwitheasy_install”.

Wecanuse pipinstall[package_name]toinstallanewpackage.YoucanfindallofthePythonpackagesavailablefordownloadat https://pypi.python.org/pypi .Therearetwowaystospecifyapackage:youcanjustusethepackagename,oryoucangivethepackagenamefollowedby == andtheversionnumber.Ifyouusethepackagename,themostrecentversionwillgetdownloaded.Sometimesyoudon’twantthemostrecentversion,youwantaspecificversion,whichiswhythesecondoptionexists.HereisanexampleofhowwecaninstallapackagecalledFlask,apopularPythonpackagethatletsyoueasilycreatewebsites,usingaversionnumber:

pipinstall Flask==0.10.1 >>

OnUnix-likesystemsyouneedtouse sudo:

sudo pipinstall Flask==0.10.1 >>

Whentheinstallationisfinished,theflaskmodulewillbeinstalledinaspecialfolderonyourcomputercalledsite-packages.Site-packagesisautomaticallyincludedinyourPythonpath,sowhenPythonimportsamodule,itlooksinsite-packagestoseeifit'sthere.

Now,ifyouwriteaprogram,youwillbeabletoimportandusetheFlask module.CreateanewPythonfileandaddthefollowingcode:

from flaskimport Flask

app=Flask(__name__)

@ app.route ( '/' )defindex (): return"Hello,World!"

app.run( port = '8000' )

Ifyougoto http://127.0.0.1:8000/inyourwebbrowser,youwillseeawebsitethatsays“Hello,World!”.ThisexamplewastakenfromFlask’stutorial,whichisavailableattheselftaughtprogrammer.io/flask ifyouareinterestedinlearningabouthowFlaskworks.Youcanviewthepackagesyou’veinstalledwithpipwiththecommand pipfreeze :

pipfreeze>>Flask==0.10.1

…

Youcanusethesyntaxpipfreeze>[filename] tosavenamesofallthepackagesyou’veinstalledwithpiptoafile.Createarequirementsfilewith:

pipfreeze>requirements.txt>>

Openrequirements.txt withatexteditortoseethenewfile.Thisfileisusefulbecauseyoucanusethesyntaxpipinstall[requirements_file] toinstallallthepackageslistedinarequirementsfile.ThiscommandisusefulforquicklydownloadingthedependenciesofaPythonprogramthathasnotbeenlistedonpypi,andthusisnotavailableonpip.

Finally,youcanuninstallprogramsyou’vedownloadedwithpipuninstall[package_name] .TouninstallFlask,usethefollowingcommand:

pipuninstallflask .…>>Proceed(y/n)?y...

Challenge

FindthreeprogramsthatinterestyouonUbuntuusingaptitudeandinstallthemusingapt-get.

Chapter20.VersionControl

“Iobjecttodoingthingsthatcomputerscando.”—OlinShivers

Writingsoftwareisateamsport.Whenyouareworkingonaprojectwithmorethanoneperson,youwillbothbemakingchangestothecodebase —thefoldersandfilesthatmakeupyoursoftware—andyouneedtokeepthosechangesinsync.Youcouldbothperiodicallyemaileachotherwithyourchanges,andcombinethetwodifferentversionsyourself,butthatwouldquicklybecometedious.Alsowhatwouldhappenifyoubothmadechangestothesamepartoftheproject?Whosechangesshouldbeused?Thesearethekindsofproblemsaversioncontrolsystem solves.Aversioncontrolsystemissoftwaredesignedtoletyoueasilycollaborateonprojectswithotherprogrammers.

Therearemanydifferentversioncontrolsystems.Git andSVNarebothpopularchoices.Versioncontrolsystemsareprograms,usuallyusedinconjunctionwithaservicethatstoresyoursoftwareonthecloud,likeGitHub .InthischapterwearegoingtouseGit,aversioncontrolsystem,toputsoftwareonGithub,awebsitethatstoresyourcodeonthecloud.

Repositories

ArepositoryisadatastructurecreatedbytheprogramGittomanageasoftwareproject.Adatastructureisawayoforganizingandstoringinformation:listsanddictionariesareexamplesofdatastructures(youwilllearnmoreaboutdatastructuresinPartIV).Repositoriesareusedtokeeptrackofallthechangesinasoftwareproject.

WhenyouareworkingonaprojectmanagedbyGit,therearemultiplerepositories(usuallyoneforeachpersonworkingontheproject).Atypicalsituationlookslikethis:everybodyworkingontheprojectalsohastheirownrepositoryontheircomputercalledalocalrepository whichkeepstrackofallthechangestheymakeontheirowncomputer;thereisalsoacentralrepository hostedonawebsitelikeGitHubwhichallofthelocalrepositoriescommunicatewithtostayinsyncwitheachother.Aprogrammerworkingontheprojectcanupdatethecentralrepositorywiththechangesthey’vemadeintheirlocalrepositoryortheycanupdatetheirlocalrepositorywiththenewestchangesotherprogrammershavemadetothecentralrepository.imageofrepositories communicatingThisisdonefromthecommandlineusingtheprogramGit.

YoucancreateanewrepositoryusingtheGitprogramfromthecommandlineoronGitHub’swebsite.Onceyoucreatearepository,youcanusetheGitprogramtomanageitandcommunicatewithacentralrepository.

GettingStarted

Togetstarted,youneedtocreateaGithubaccount:gotoGithub.com/jointocreateone.CreateanewrepositoryonGithub.LogintoyourGitHubaccountatgithub.comandclickonthe+ buttonatthetoprightcornerofthescreen.ClickCreaterepositoryfromthedropdownmenu.Givetherepositorythename hangman .Makeitpublic ,andcheckinitializetherepositorywithareadme .NowclickCreatearepository.Ifatanytimeyourunintotroubleanddosomethingwronginthissection,gotothesettingspageofyourrepository,deleteit,andstartover.YoualsoneedtoinstallGit.YoucaninstallGitusingyourpackagemanagerofchoice.OnGitHub,hitthebuttoninthetoprightcornerandselectYourProfile. image

Youwillseethenameofyourrepository: hangman .Clickonit.YouwillseeabuttonthatsaysCloneOrDownload.Whenyouclickonit,youwillseeHTTPS: followedbyalink.Youcanusethislinktodownloadyourrepositorytoyourcomputerusingthecommandgitclone[repository_url] .Therepositorywilldownloadinwhateverdirectoryouissuethecommandfrom.Copythelink,orpressthecopylinktoclipboardbuttonanduseitwiththe gitclonecommand:

$gitclone>>

Uselstoverifytherepositorydownloaded:

$ls>>my_project

PushingandPulling

TherearetwomainthingsyouwillbedoingwithGit. Thefirstisupdatingthecentralrepository withchangesfromyourlocalrepository.Thisiscalledpushing becauseyouarepushingnewdatatoyourcentralrepository.

Thesecondthingyouwillbedoingiscalledpulling .Pullingdatameansupdatingyourlocalrepositorywithallofthenewchangesfromthecentralrepository.

Thecommandgitremote-v showsyouwhatcentralrepositoryyourlocalrepositoryispushingandpullingfrom.The -v flagstandsforverbose,whichmeansthecommandwillusuallyprintoutextrainformation.Usethiscommandinsidetoseethecentralrepositoryyourlocalrepositoryispushingandpullingfrom:

$gitremote-v

>>originhttps://github.com/[username]/my_git_project.git(fetch)

>>originhttps://github.com/[username]/my_git_project.git(push)

Thefirstlineshowsthecentralrepositoryyourprojectwillpulldatafromandthesecondlineshowsthecentralrepositoryyourprojectwillpushdatato.Generally,youwillpushandpullfromthesamecentralrepository.

PushingExample

Inthissection,youwillmakeachangetothelocalrepositoryforyourhangmanprojectandpushthatchangetoyourcentralrepository.

MovethePythonfilewiththecodeweusedtocreateHangmanintothe hangmandirectorywecreatedinGettingStarted.Ourlocalrepositorynowhasafilethatdoesnotexistinourcentralrepository—ourlocalrepositoryisoutofsyncwithourcentralrepository.Wecanfixthisbypushingthechangeswemadeinourlocalrepositorytoourcentralrepository.

Pushingchangesfromyourlocalrepositorytoyourcentralrepositoryhappensinthreesteps.FirstyoustageyourfileswhichiswhereyoutellGitwhichfileshavechangesthatyouwanttopushtoyourcentralrepository.Onceyou’vereviewedeverything,youcommitthefiles.

,thenyoucommitthemandfinallyyoupushthem.InthefirststepyoutellGitwhatfilesyouwanttopushtoourcentralrepository.Thisiscalledstagingafile.Whenafileisstaged,wehavethechancetochangeourmindandunstageit.Thesyntaxgitadd[file_path] isusedtostageafile.Usethecommandgitaddhangman.py tostageourfile:

$gitaddhangman.py>>

Thecommand gitstatusshowsyouthecurrentstateofyourprojectinrelationtoyourrepository.Newfilesthatexistinyourlocalproject,butdonotexistinyourrepositoryareshowningreen,asaredeletedfiles.Filesthathavebeenmodifiedareinred.Afileismodifiedifitisinyourrepository,butlocalversionandtheoneinyourrepositorydiffer.Enterthecommandgitstatus :

$gitstatus>>OnbranchmasterChangestobecommitted:(use"gitresetHEAD<file>..."tounstage)

newfile:hangman.py

Youshouldseethefilehangman.pyinagreenfont.Youhavetostageeachfileyouwanttopushtoyourrepositorywiththecommandgitadd[file] .Ifyoustageafileandchangeyourmind,youcanunstageitwithoutmakingchangestoyourcentralrepository.Youcanunstageafilewiththesyntaxgitreset[file_path] .Unstagehangman.pywiththecommandgitresethangman.py.Usegitstatustoseetheresult,andadditagainwithgitaddhangman.py :

$gitresethangman.py$gitstatus>>

$gitaddhangman.py>>

Oncewe’vestagedourfiles,andeverythinglooksthewaywewant,wearereadytomovetothenextstep—committingourfilestoourcentralrepository.Whenyoucommitfiles,youwanttousethe-m flagsoyoucanpassalongamessage.Themessagewillbesavedalongwithyourcommitinyourrepositorytohelpyourememberwhatchangesyoumadeinthatcommit.

$gitcommit-m“myfirstcommit”

>>1filechanged,1insertion(+)createmode100644hello_world.py

ThefinalstepistoactuallypushyourchangestoGitHub.Thisisdonewiththecommandgitpushoriginmaster :

$gitpushoriginmaster>>Countingobjects:3,done.Deltacompressionusingupto4threads.Compressingobjects:100%(2/2),done.Writingobjects:100%(3/3),309bytes|0bytes/s,done.Total3(delta0),reused0(delta0)Tohttps://github.com/[your_username]/my_project.git0eb3a47..48acc38master->master

OnceyouenteryourGitHubusernameandpasswordfromthecommandline,yourchangeswillbepushedtoGitHub.IfyoulookatyourrepositoryonGitHub’swebsite,youwillseehangman.pyisnowinyourproject.

PullingExample

Inthissection,welearnhowtoupdateyourrepositorywithchangesfromthecentralrepository.First,wehavetomakeachangeinourcentralrepository.Usethecommand gitpulloriginmaster toupdateyourlocalrepositorywiththechangewemade:

$gitpulloriginmaster>>Fromhttps://github.com/calthoff/my_project>>*branchmaster->FETCH_HEAD

Gitappliedthechangesfromourcentralrepositorytoourlocalrepository,andtheyarenowinsync.YoucanviewtheREADME.md fileonyourcomputertoseethechange.

RevertingVersions

Whenyouuseversioncontrol,theentirehistoryofyourprojectissavedandavailableforyoutouse.Ifyoudecideyouwanttoreverttoaversionofyourprojectfrom10daysago,youareabletodoso.Youcanviewyourproject’shistoryofcommitswithgitlog ,whichshouldoutputsomethinglikethis:

$gitlog

commitaeb4ef3cf3aabdb9205ea9e96e8cab5c0f5ca7eaAuthor:CoryAlthoff<[email protected]>Date:ThuJan2113:52:022016-0800

Thestringofnumbersandlettersaftercommit isthecommitnumber.Wecanusethisnumbertorevertourprojecttoexactlyhowitwasatthattime.Wecantravelbackintimewiththecommandgitcheckout[oldcommit] .Inthiscasethecommandwouldbegitcheckoutaeb4ef3cf3aabdb9205ea9e96e8cab5c0f5ca7ea .

diff

Wecanusethecommandgitdiff toseethedifferencebetweentheversionofafileinourlocalproject,andtheversioninourrepository.Add x=100 tothesecondlineofourhello_world.py file.Enterthecommand:

$gitdiffhello_world.py>>diff--gita/hello_world.pyb/hello_world.pyindexb376c99..83f9007100644---a/hello_world.py+++b/hello_world.py@@-1+1,2@@

print('hello')+x=100

Githighlights+x=100 ingreenbecausethelinechanged,the+ istosignifytheline x=100wasadded.

TheOtherPullRequest

Confusingly,therearetwoconceptsnamedpullinversioncontrol.Wepreviouslytalkedaboutpullingdatafromyourrepository.Thereisalsoanunrelatedconceptcalledapullrequest.ApullrequesttakesplaceonGitHub.Ifyouareworkingonabranchofaproject,andyouwanttomergeitwiththemasterrepository,youwouldissueapullrequestonGitHubtomergethetwo.Thisgivesyourteammatesthechancetoreviewandcommentonyourchanges.Ifeverythinglooksgood,someoneonyourteamcanapprovethepullrequestandmergethetwobranches.

LearningMore

Youtypicallycreateanewbranchwhenyouwanttodevelopanewfeatureforyourprogramorfixabugorproblem.Whenyouarefinishedwithwhateveryouaredoingonyourbranch,younormallymergeyourchangeswiththemasterbranch,whichistheprocessofcombiningthetwobranches.Coveringmergingisoutsidethescopeofthisbook,butgit-scmhasagreattutorialonityoucancheckoutat:https://git-scm.com/book/en/v2/Git-Branching-Basic-Branching-and-Merging.

Challenge

Chapter21.SQLite

“Data!Data!Data!Ican’tmakebrickswithoutclay!”—SirArthurConanDoyle

Databasesareprogramsusedtopersistdata.Datapersistsifitoutlivestheprocessthatcreatedit46 .Mostoftheprogramswe’vebuiltsofarworkfinewithoutpersistinganydata,withonenotableexception—ourwebscrapingprogramwebuiltinpartIItocollectheadlinesfromGoogleNews.Alloftheheadlineswecollectarelostaftertheprogramstopsrunning.Butwhatifwewanttoanalyzetheheadlinesfromthelastyear?Thisiswherepersistencecomesin,andwhydatabasesareimportant.Databasesperformtwomainoperations—readandwrite.Whenyouwritetoadatabase,youaregivingitinformationtostore.Whenyoureadfromadatabase,youareretrievinginformationfromit.Inanincreasinglydata-centricworld,databasesarebecomingexceedinglyimportant.Inthischapter,wegooverthebasicsofdatabases.

NoSQLvs.SQL

RelationaldatabaseswerefirstproposedbyEdgarF.Coddin1970.RelationaldatabasesstoredatalikeanExcelspreadsheet—dataisstoredinrowsandcolumns.ThedataisstoredandretrievedusingthequerylanguageSQL,whichstandsforStructuredQueryLanguage.PostGreSQLandMySQLareexamplesofpopularrelationaldatabases.

Recentlyanewbreedofdatabases,calledNoSQLhavegainedpopularity.NoSQLliterallymeans“noSQL.”Inotherwords,thethingthesenewbreedofdatabaseshaveincommonistheyarenotrelationaldatabasesusingSQL.RedisisanexampleofapopularNoSQLdatabases.Redisisa“keyvaluestore”whichmeansyoucanstoredatalikeadictionaryinPython,i.e.,storeakeyandavalue.Soforexampleyoucouldstore“Monty”asakeyand“Python”asavalue;queryRedisfor“Monty”anditwillreturn“Python”.WhileNoSQLdatabasesareuseful,,inthischapterwearelearningaboutrelationaldatabases,becausetheyareveryimportantinthesoftwareindustry.Onceyouarefamiliarwithrelationaldatabases,IencourageyoutolearnmoreaboutNoSQL.

GettingStarted

WewillgetstartedwithSQLbyusingSQLite.SQLiteisalightweightdatabasethatcomesbuiltintoyouroperatingsystem,sowedon’thavetoinstallanything.Gotothecommandlineandtypethecommand“ sqlite self_taught.db”tocreateanewdatabase. Ifyou

arecomingbacktoYoucanopenSQLiteanytimewiththecommand“sqlite3”.Ifyoualreadycreatedthedatabase,openitwith“.openself_taught.db”.ExitSQLitewiththecommand“.exit”.

#explainrelationaldatabaseswhatisacolumn,whatisarow

DataTypes

CreateaTable

CREATETABLEcustomers (idintfirst_name textlast_nametext ,date_createddate ,PRIMARYKEY ( column1) );

Everytableinarelationaldatabasehastohavesomethingcalledaprimarykey—auniqueidentifierforeachrow.Inthiscaseourprimarykeyisourfirstcolumn—calledid.Wecansetourprimarykeytoautoincrement,whichmeanstheprimarykeywillbeanumberthatthedatabaseautomaticallyincrementsforyouwheneveryouaddnewdata.Ourcustomerstablealsohasarowforthecustomer’sfirstnameandlastname.RelationaldatabasesaremadeupoftablesthatstoredatalikeanExcelspreadsheet,incolumnsandrows.Let’screateourfirsttabl e.Typethefollowinginsidesqlite.MakesureyouareinsideSQLitewiththecommand“sqlite3”andyourcommandlinesayssqlite.CREATETABLEself_taught.bdfl(namestringprojectstringageintbirthdaydate);Ournewinourdatabaseself_taughtiscalled“bdfl”,whichstandsforbenevolentdictatorforlife.BDFLisatitlegiventothecreatorsofopensourceprogrammingprojectslikeLinusTorvalds,creatorofLinux,GuidovanRossumcreatorofPython,DavidHeinemeierHanssoncreatorofRubyonRailsorMattMullenwegcreatorofWordpress.Wewilluseourtabletostoredataaboutthesebdfls,suchastheirname,project,andbirthdays.

Constraints

#findsomewheretoput# CreatingarelationshipwithanothertableisdonewithSQL,whichmakessurethedatabaseisawareoftherelationship.Youcan’tjustenteranyintegerunderthecustomercolumn,yourdatabasewillonlyletyouenteravalidprimarykeyforacustomerinthecustomertable.Aconstraintisaruleyoucanapplytoacolumnthatisenforcedbythedatabase.Examplesofconstraintsare:notnull,unique,primarykeyandforeignkey.

Ifyouputtheconstraint“notnull”onacolumn,thatcolumncannotbe“null”whichislike“None”inPython.Thismeansthecolumnmusthavedatainit.Sayyouarecollectingthefirst,middleandlastnameofsubscribersforanewsletteronyourwebsite.Thetableinyourdatabasetocollectthisinformationmightlooklikethis:

subscribersfirst|middle|lastYouprobablywouldwanttoput“notnull”constraintsonthefirstcolumn,whileallowing“null”inthemiddleandlastcolumns.Thereasonbeingthateveryonehasafirstname,butnoteveryonehasamiddleorlastname.WhatifPrincesignedupforyournewsletter?Ifyouputanotnullconstraintonlast,Princewouldn’tbeabletosignup.

Constraintsareimportantbecausetheymakeguaranteesaboutdata.Inthiscase,ifwemightwanttocreateaprogramthatanalyzesthefirstnamesofallofoursubscribers,ourprogramwouldprobablyperformsomesortofoperationoneachstring.Ifthe“first”columngaveourprogramanullvalueinsteadofastring,andourprogramtreatedthenullvaluelikeastring,itwouldcauseanerrorinourprogram.Byaddinganotnullconstrainttoourfirstcolumn,ourprogramcannowbeassuredthateveryfirstnameitgetsfromthedatabaseisgoingtobeastringandourprogramcantreatitassuch.

Ifwearecollectingdataforanewsletter,wealsoneedtocollectourusers’emailaddresses.Wecandothisbyaddinganemailcolumntoourtable:

ALTERTABLEcustomerADDemailstringUNIQUE

Weaddeda“unique”constrainttothisnewcolumn,sothattheemailcolumnmustbeunique.Thismeansifanemailisusedinonerowofourtable,itcannotbeusedinanotherrow.Thismakessensebecauseeveryemailaddressintheworldisunique,andiftwosubscribershavethesameemail,thereisaproblem.Aforeignkeyletsyouenforcethatthedataenteredinthetableisaprimarykeyinanothertable.WealreadysawanexampleofusingaforeignkeyinourAmazonexample.HereishowwecouldchangeatabletoaddaforeignkeyP_IdtoamadeuptablecalledPersons:

ALTERTABLEcustomerADDFOREIGNKEY(P_Id)REFERENCESPersons(P_Id)

Acheckconstraintisusedtomakesuredataenteredinatablemeetscertainspecifications.Hereisanexampleofacheckconstraintwecouldaddtoouremailcolumn:

ALTERTABLEcustomerADDCHECKemailvarchar(255)

Thisenforcesthatallemailsenteredintoourdatabasehavetobelessthan255characters.

InsertDataTimetoinsertdataaboutourfirstBDFLintoourtable.EnterthefollowingSQLstatementsoneatatime:

INSERTINTObdfl(name,project,age)VALUES(GuidovanRossum,Python,1-31-1956);

INSERTINTObdfl(name,project,age)VALUES(DavidHeinemeierHansson,RubyonRails,10-15-1979);

INSERTINTObdfl(name,project,age)VALUES(LinusTorvalds,Linux,12-28-1969);

INSERTINTObdfl(name,project,age)VALUES(MattMullenweg,WordPress,1-11-1984);

INSERTINTObdfl(name,project,age)VALUES(DriesBuytaert,Drupal,11-19-1978);

INSERT INTObdfl(name,project,age)VALUES(LarryWall,Perl,9-27-1954);

QueryData

Queryingdatameanslookingupdatainyourdatabasegivenspecificparameters.Exampleoftheparametersyoumightgivearewhattablethedataisin,anddifferentattributesyouwantthedatatocontain.Let’sstartbyqueryingforeverythinginourbdfltablebyenteringSELECT*frombdfl intosqlite.InSQL,SELECT* meansselecteverything.Ifwedonotwanttoselecteverything,andjustwanttoselectthenameofthebdflofLinux,wecandothiswithSELECTnameFROMbfdflWHEREproject=“Linux”

orQuery

Youcanadd“or”toyourquerytoselectfromarowifeithertheconditionbeforetheor, ortheconditionaftertheor istrue.SelecteverythingfromourtablewheretheprojectisRubyonRailsorWordpresswiththestatementSELECT*FROMbfdlWHEREproject=“Linux”ORproject=”RubyonRails” .

andQuery

Adding“and”toyourquerywillonlyselectarowifbothconditionsaretrue.FirsttrySELECT*FROMbfdlWHEREbfdl.nameLIKED%.ThisreturnsanyrowwherethenamestartswithaD.ItshouldreturnthedatastoredforDavidHeinemeierHanssonandDriesBuytaert.Nowtry“” SELECT*FROMbfdlWHEREnameLIKE“D%”ANDwhereA.Date>=Convert(datetime,'2010-04-01') .ThiswillonlyreturnDavidHeinemeierHansson.WhileDriesBuytaertstartswith“D”,hisbirthdayisnotgreaterthanthedateweprovided.

Count

Countthenumberofrowsinatablewith:

SELECTCOUNT(*)FROMbfdl>>

CommunicatingwithDatabases

Sofar,we’velearnedtocommunicatewithadatabasebyusingSQLfromtheterminal.Butwhatifwewanttostoredatacollectedin adatabasefr omaprogramwe’vewritteninadatabase?ThereistoolwecanusetoabstractSQLawayentirelycalledanORM,orobjectrelationalmap.Anobjectrelationalmapissoftwarethatletsyouinteractwithyourdatabaseusingclassesinyourprogramminglanguage,insteadofusingSQL.

UsinganORMlibrary,suchastheORMthatcomeswiththePythonwebframeworkDjango,wecanrepresentadatabasetableasaclassinPython.InsteadofusingSQLstatementstoreadandwritefromourtable,wecanusemethodsfromthetableclasswe

defined.HereisanexampleofhowwecouldrepresentandqueryourAmazoncustomertableusingDjango’sORM:

fromdjango.dbimportmodels

classCustomer(models.Model):username=models.CharField()Thisexampledefinesanewdatabasetablecalled“customer.”Inthiscasewehavetwocolumns,id whichisourprimarykeyautomaticallycreatedbyourORMandnotreflectedinourcode,and usernamewhichisavariablesettoaCharFieldobjectwhichDjangousestocreateadatabasecolumnthatstoresstrings.Now,whenwewanttointeractwithourdatabase,wecandoitusingour Customerclass.Forexample,wecancreateanewuser,whichwillbestoredinMySQL,withthefollowingcode:

user1=Customer.objects.create(username=”eddie”)

Toqueryourdatabasewesimplyneedtousetheclasswecreated:

eddie=Customer.objects.get(username=”eddie”)printeddie.username

>>“eddie”

Withthiscode,wewereabletosuccessfullyreadandwritefromourdatabasewithoutanySQL.Django’sORMtranslatesourcodetoSQLforus,sowedon’thavetoworryaboutit.

Challenge


“Themagicofmythandlegendhascometrueinourtime.Onetypesthecorrectincantationonakeyboard,andadisplayscreencomestolife,showingthingsthatneverwerenorcouldbe”....—FrederickBrooks

Inthischapter,wewillseehowpowerfulprogrammingcanbebybuildingawebscraper:aprogramthatextractsdatafromawebsite.Learningtoscrapedatafromwebsitesispowerful.Itgivesyoutheabilitytoextractanydatayouwantfromthelargestcollectionofinformationthathaseverexisted.Seeingthepowerofwebscrapers,andhoweasytheyaretobuild,isoneofthereasonsIgothookeduponprogramming,andIhopeithasthesameeffectonyou.

HTML

Beforewebuildourwebscraper,weneedaquickprimeronHTML,orhypertextmarkuplanguage.HTMLisoneofthefundamentaltechnologiesusedtobuildwebsites,alongwithCSSandJavaScript.YoucanthinkofHTMLasitsownlittlelanguage,usedtogiveawebsitestructure.HTMLismadeupoftagsthatawebbrowserusestolayoutawebpage.Infact,youcanbuildanentirewebsiteusingonlyHTML.Itwon’tbeinteractiveorlookverygood,becauseJavaScriptiswhatmakeswebsitesinteractive,andCSSiswhatgivesthemstyle,butitwillbeawebsite.HereisanexampleofawebsitethatwilldisplaythetextHello,World!:



<htmllang= "en" ><head><metacharset= "UTF-8"><title>MyWebsite </title></head><body> Hello,World!<ahref= "https://news.ycombinator.com/" > here </a></body></html>

Takeaminutetostudythiscode.NowsavethisHTMLintoafileandopenthefilewithyourwebbrowser byclickingon thefile(youmayhavetorightclickandchangethedefault

programtoopenthefilewithtoawebbrowserlikeChrome).Onceyouopenthefilewithyourwebbrowser,youwillseeawebsitethatsaysHelloWorld!withalinktotheYCombinatorwebsite.

YourwebbrowserusesthedifferenttagsinourHTMLtofigureouthowtodisplaythiswebsite.Tagshaveabeginningtagandclosingtag,oftenwithsomethingliketextinbetween.Forexample,yourbrowserdisplaysthetextinbetweenthe <title></title> tagsinthetabofyourbrowser.Anythinginbetweenthe<body></body> tags,makesuptheactualwebsite.Anythinginside <a></a>tagsisalink.ThereisalotmoretoHTML,butthisisallyouneedtoknowinordertobuildyourfirstwebscraper.

ScrapeGoogleNews

NowwecanbuildascraperthatfetchesalloftheheadlinesfromGoogleNews.Wewilldothisbyextractingallofthe<a></a>tagsinGoogleNews’sHTML.AswesawinourHTMLexample,each<a></a>taghasavariableinitcalled href ,e.g.,<ahref=”theselftaughtprogrammer.io”></a> .Wearegoingtoextractallofthehrefvariablesfromallofthe<a></a> tagsonGoogleNews’swebsite.Inotherwords,wearegoingtocollectalloftheURLsGoogleNewsislinkingtoatthetimewerunourprogram.WewillusetheBeautifulSoup libraryforparsingourHTML(convertingHTMLtoPythonobjects),sofirstinstallitwith:

pipinstallbeautifulsoup4==4.4.1

OnceBeautifulSoupisinstalled,wecangetGoogleNews’sHTMLusingPython’sbuilt-inurllib2 libraryforworkingwithURLs.Startbyimportingurllib2andBeautifulSoup:

“““https://github.com/calthoff/tstp/blob/master/part_III/lets_read_some_code/lets_read_some_code.py”””

importurllib2from bs4importBeautifulSoup

Nextwecreateascraperclass

classScraper : def__init__ ( self ,site): self.site=site

defscrape ( self ): pass

Ourmethodtakesawebsitetoscrapefrom,andhasamethodcalledscrapewhichwearegoingtocallwheneverwewanttoscrapedatafromthewebsitewepassedin.

Nowwecanstartdefiningourscrapemethod.

defscrape ( self ):response=urllib2.urlopen(self .site) html=response.read()

The urlopen() functionmakesarequesttoGoogleNewsandreturnsaresponseobject,whichincludesGoogleNews’sHTMLinitasavariable.Wesavetheresponseinour responsevariableandassignthevariablehtml to response.read() whichreturnstheHTMLfromGoogleNews.AlloftheHTMLfromGoogleNewsisnowsavedinthevariablehtml .ThisisallweneedinordertoextractdatafromGoogleNews.However,westillneedtoparsetheHTML.ParsingHTMLmeansreadingitintoourprogramandgivingitstructurewithourcode,suchasturningeachHTMLtagintoaPythonobject,whichwecandousingtheBeautifulSouplibrary.First,wecreatea BeautifulSoup objectandpassinourhtmlvariableandthestring ‘html.parser ’asaparametertoletBeautifulSoupknowweareparsingHTML:

defscrape ( self ):response=urllib2.urlopen(self .site)html=response.read()soup =BeautifulSoup(html ,'html.parser' )

Movingforward,wecannowprintoutthelinksfromGoogleNewswith:

defscrape ( self ):response=urllib2.urlopen(self .site)html=response.read()soup=BeautifulSoup(html ,'html.parser' ) fortaginsoup.find_all( 'a' ):url=tag.get( 'href') if urland'html'inurl: print ( " \n "+url)

find_all() isamethodwecancallonBeautifulSoup objects.IttakesastringrepresentinganHTML tagasaparameter( ‘a’ representing<a></a> ),andreturnsaResultSet objectcontainingallthe Tag objectsfoundbyfind_all() .The ResultSet issimilartoalist—youcaniteratethroughit(weneversaveResultSet inavariable,itissimplythevaluereturnedbysoup.find_all('a') ),andeachtimethroughtheloopthereisanewvariable:tag, representingatagobject.Wecallthemethodget() onthetagobject,passinginthestring‘href’ asaparameter(hrefisthepartofanHTML<ahref=“url”></a>tagwhichholdstheURL),anditreturnsastringURLwhichwestoreinthevariable url.

ThelastthingwedoistochecktomakesureURL isnotNone with ifurl,becausewedon’twanttoprintthe url ifitisempty.Wealsomakesure‘html’ isintheurl ,becausewedon’twanttoprintGoogle’sinternallinks.Iftheurl passesbothofthesetests,weuse‘\n’toprintanewlineandthenprinttheurl.Hereisourfullprogram:

importurllib2

from bs4import BeautifulSoup

class Scraper : def__init__ ( self,site): self .site=site

defscrape ( self):response=urllib2.urlopen( self.site)html=response.read()soup=BeautifulSoup(html ,'html.parser' ) fortagin soup.find_all('a' ):url=tag.get( 'href') ifurland 'html'in url: print ( " \n "+url)

Scraper().scrape( 'https://news.google.com/')

Runthescraper,andyoushouldseearesultsimilartothis:https://www.washingtonpost.com/world/national-security/in-foreign-bribery-cases-leniency-offered-to-companies-that-turn-over-employees/2016/04/05/d7a24d94-fb43-11e5-9140-e61d062438bb_story.html

http://www.appeal-democrat.com/news/unit-apartment-complex-proposed-in-marysville/article_bd6ea9f2-fac3-11e5-bfaf-4fbe11089e5a.html

http://www.appeal-democrat.com/news/injuries-from-yuba-city-bar-violence-hospitalize-groom-to-be/article_03e46648-f54b-11e5-96b3-5bf32bfbf2b5.html

...

NowthatyouhaveallofGoogleNews’sheadlinesavailableinyourprogram,thepossibilitiesarelimitless.Youcouldwriteaprogramtoanalyzethemostusedwordsintheheadlines,andbuildawordcloudtovisualizeit.Youcouldbuildaprogramtoanalyzethesentimentoftheheadlines,andseeifithasanycorrelationwiththestockmarket.Asyougetbetteratwebscraping,alloftheinformationintheworldwillbeopentoyou,andIhopethatexcitesyouasmuchasitexcitesme.

Challenge

ModifytheGoogleScrapertosavetheheadlinesinafile.

Chapter23.Practice

Exercises

0.Buildascraperforanotherwebsite.0.WriteaprogramandthenreverttoanearlierversionofitusingPyCharm.0.Downloadpylintusingpip,readthedocumentationforitandtryitout.

Read

0.http://www.tutorialspoint.com/python/python_reg_expressions.htm

PartIVIntroductiontoComputerScience

DataStructures&Algorithms

“Iwill,infact,claimthatthedifferencebetweenabadprogrammerandagoodoneiswhetherheconsidershiscodeorhisdatastructuresmoreimportant.Badprogrammersworryaboutthecode.Goodprogrammersworryaboutdatastructuresandtheirrelationships.”—LinusTorvalds

Thischapterisalightintroductiontoalgorithmsanddatastructures,perhapsthemostimportantsubjectinComputerScience.ThetitleoftheinfluentialbookAlgorithms+DataStructures=Programssummarizestheirimportance.Mygoalinthischapteristointroduceyoutothesubject,andclarifysomethingsIfoundconfusingwhenIwaslearningaboutthem(whichIstillam).Inadditiontoreadingthischapter,youdefinitelyneedtoreadmoreaboutalgorithmsanddatastructuresoutsideofthisbook,andalsospendalotoftimepracticingtheconceptsintroducedhere.ManyoftheexamplesinthischaptercomefromtheamazingbookPythonAlgorithmsandDataStructures byBradMillerandDavidRanum.Itisoneofmyfavoritebooks,availableonlineforfreeat:http://interactivepython.org.

WhatAreAlgorithms&DataStructures?

Analgorithmisaseriesofstepsthatcanbefollowedtosolveaproblem.Theproblemcouldbeanything,likesortingorsearchingalist,ortraversingeatree. Adatastructureisawaytostoreandorganizeinformation.Datastructuresarefundamentaltoprogramming,andwhateverprogramminglanguageyouusewillcomewithbuilt-indatastructures.Commondatastructuresincludehashtables,stacks,andlists.Datastructurescomewithdifferenttradeoffs,withcertaindatastructuresbeingbettersuitedforspecifictasksthanothers.

BigONotation

InComputerScience,wesolveproblemsusingalgorithms.Butwhatifyoucomeupwithtwodifferentalgorithmstosolvethesameproblem?Howdoyoudecidewhichisbest?BigONotationgivesyouaframeworkfordecidingifonealgorithmisbetterthananotherby

lookingatthenumberofstepseachalgorithmtakes,andchoosingtheonethattakestheleastamountofsteps. WecanuseanequationlikeT(n)=ntodescribeanalgorithm .Thefollowingsectionsintroducesomealgorithmsyoushouldlearn.

Modulo

Themodulooperator“%”returnstheremainderoftwonumberswhenyoudividethem.Forexample“2%2”returns0,whereas“3%2”returns1.ModuloishelpfulinsolvingavarietyofproblemslikeFizzbuzz,apopularfirstinterviewquestiondesignedtoweedoutpeoplethatcannotprogram.ThequestionwasintroducedbyImranGhoryinhisblogpostUsingFizzBuzztoFindDeveloperswhoGrokCoding .Ifyouknowthecorrectapproachtotheproblem,itiseasytosolve,whereasifyoudon’t,itcanappearcomplicated.Theproblemisusuallygivenas:Writeaprogramthatprintsthenumbersfrom1to100.Butformultiplesofthreeprint“Fizz”insteadofthenumberandforthemultiplesoffiveprint“Buzz”.Fornumberswhicharemultiplesofboththreeandfiveprint“FizzBuzz”.

Thekeytosolvingthisproblemisusingmodulo.Tosolvethisproblemyousimplyneedtoiteratefrom1to100andcheckifeachnumberisdivisibleby3,5orboth.Hereisthesolution:

deffizz_buzz (): fori inrange(0,101): if i%3==0and i%5==0: print'FizzBuzz' elifi%3==0: print'Fizz' elifi%5==0: print'Buzz' else : printi

Westartbyiteratingthroughthenumbers1to100withaforloop.Thenwesimplycheckeachoftheconditions.Weneedtocheckifthenumberisdivisibleby3or5first,becauseifitis,wecanmovetothenextnumber. Thisisnottrue withbeingdivisiblebyeither5or3,because ifeitheraretrue,westillhavetocheckifthenumberisdivisiblebyboth.Wethencancheckifthenumberisdivisibleby3or5(inanyorder).Finally,ifnoneoftheconditionsaretrue,wesimplyprintthenumber.Hereisanotherproblemwhereusingmoduloisthekeytofiguringouttheanswer:

Rotateanarrayof nelementstotherightbyk steps.

Forexample,withn=7and k =3,thearray[1,2,3,4,5,6,7]isrotatedto[5,6,7,1,2,3,4],i.e.,youaremovingeveryelementinthelistkpositions.

Whenyousolvethisproblem,yourfirstinstinctmightbetosimplyaddntotheindexofeachnumberandmovethenumbertothenewposition.Theproblemisthisdoesnotwork.Thekeytosolvingthisproblemisonceagainmodulo.Saywehadalistofnumbersfrom1-12.Wecanthinkofthislistasaclock.Ifwestartat12o’clock,andgoaroundtheclocktwelvehours,wearebacktowherewestarted.

Wecanachievethisbytakingthecurrentindex,addingthekeynewindexandgettingtheremainder.Ifyoutakeourlistof12numbersandyouwanttocalculatewherethenumber12wouldbe(startingatindex11)ifweadded12totheindex,youcancalculatethatwith11+

BubbleSort

Asortingalgorithmisanalgorithmthattakesagroupofnumbersandputstheminacertainorder.Therearemanydifferentalgorithmssuchasselectionsort,insertionsort,shellsort,mergesort,andquicksort.Inthissectionwewillimplementbubblesort,asortingalgorithmthatisnotveryefficient—buteasytounderstandandusefulforunderstandingsortingalgorithms.Hereisanimplementationofbubblesort:

def bubble_sort(num_list): """ :param num_list:Listofnumbers :return :Sortedlistofnumbers""" fori in range ( len(num_list)- 1 ,0 , - 1 ): forj inrange (i): if num_list[j]>num_list[j+1 ]:temp=num_list[j]num_list[j]=num_list[j+ 1 ]num_list[j+ 1 ]=temp

my_list=[ 4 , 266, 9 ,24 ,44 , 54 , 41 , 89 , 20 ]bubble_sort(my_list)print (my_list)

>>[4,9,20,24,41,44,54,89,266]

Inthisalgorithm

SequentialSearch

Searchalgorithmsareusedtofindanumberinalistofnumbers.Sequentialsearchisasimplesearchalgorithmthatcheckseachnumberinthelistonebyonetoseeifitmatchesthenumberitislookingfor.Asequentialsearchisthewayyousearchadeckofcardsforaspecificcard.Yougoonebyonethrougheachcard,ifthecardisthecardyouarelookingfor,youstop.Ifyoumakeitthroughtheentiredeckwithoutfindingthecard,youknowthecardisn’tthere.Hereisanexampleofsequentialsearch:

defsequential_search (number_list,number): """:paramnumber_list:Listofintegers.:paramnumber:Integertolookfor.:return:Trueifthenumberisfoundotherwisefalse.""" found=False foriinnumber_list: ifi==number:found=Truebreak returnfound

printsequential_search(range(0,100),2)>>Trueprintsequential_search(range(0,100),202)>>False

First“found”issettofalse.Thenweloopthrougheverynumberinthelistandcheckifitisequaltothenumberwearelookingfor.Ifitequaltothenumberwearelookingfor,wesetfoundtoTrue,exitourloopandreturnTrue.Otherwise,wecontinuetothenextnumberinthelist.IfwegetthroughtheentirelistandfoundhasneverbeensettoTrue,w.WereturnfoundwhichwillstillbesettoFalse.SequentialsearchisanO(n)algorithm.Inthebestcasescenario,thenumberwearelookingforcouldbethefirstnumberinthelist,inwhichcaseouralgorithmwouldtakeonlyonestep.However,intheworstcasescenario,thenumberisnotinthelistandwehavetocheckeverysinglenumberinthelist,ornsteps.Ifwehaveamillionitemsinourlist,worstcasewehavetosearchthroughamillionitems.Ifwehaveabillionitems,worstcasewehavetosearchthroughabillionitems.Asourlistgrows,theworstcasescenarioforouralgorithmgrowsbythesizeofourlist,makingthisalgorithmO(n).

BinarySearch

Binarysearchisalogarithmicalgorithmusedtosearchfornumbersinalist,butthenumbershavetobeordered.Rememberthis:,inorderforanalgorithmtobelogarithmic,itneedstoeitherbedividingormultiplyingtoasolution.Anyguesseshowbinarysearchworks?Binarysearchworksbycontinuallycuttingthelistinhalf.Thealgorithmpicksanumberinthemiddleofalist,andlooksatwhetherit’stherightnumber.Ifitistherightnumber,thesearchiscomplete.Ifit’snottherightnumber,thealgorithmthrowsawayhalfthelist .Ifthenumberwastoobig,itthrowsawayeverythingabovethenumberitselected.Ifthenumberwastoosmall,itthrowsawayeverythingbelowthenumberitselected.Imagewehaveanorderedlistfrom1to10,andwewanttosearchforthenumber3.Ourlistlookslikethis:

[1,2,3,4,5,6,7,8,9,10]

Ouralgorithmwouldfirstpickthenumber5becauseit’sinthemiddleofthelist.Since5isnotthenumberwearelookingfor,and3issmallerthanfive,ouralgorithmwouldthrowouteverythingabove5.Nowourlistlookslikethis:

[1,2,3,4,5]

Ourlistwouldnowpickthenumberthree,sinceit’sinthemiddleofthelist.Since3isthenumberwearelookingfor,ouralgorithmwouldstopandreturnthatthenumber3wasfoundinourlist.Noticeouralgorithmonlytooktwostepstofigureoutthreewasinourlist.Ifwesearchedthroughthelistlinearly,onebyone,lookingforthenumberthree,itwouldtakesusthreesteps.Hereisanexampleofabinarysearchalgorithmsearchingforthenumber3:

defbinary_search (number_list,number): """Logarithmicbinarysearchalgorithm.:paramnumber_list:Listoforderedintegers.:paramnumber:Integertosearchforinthepassedinlist.:return:Trueifthenumberisfound,otherwiseFalse.""" first=0last=len(number_list)-1number_found=False

whilefirst<=lastandnotnumber_found:middle=(first+last)/2 ifnumber_list[middle]==number:number_found=True else : if number<number_list[middle]:last=middle-1 else :first=middle+1 returnnumber_found

binary_search([1,2,3,4,5,6,7,8,9,10],3)

Weuseawhileloopthatcontinuesaslongasthevariablefirstisnotgreaterthanorequaltothevariablelast,andthevariablenot_trueisFalse.

Wecalculatethemiddleindexofthelistbyaddingthefirstindexofthelistwiththelastindexofthelistanddividingthembytwo.Thereasonwesubtractonefromlastisbecausethelengthofalistiscalculatedstartingfromone,whereasindexesstartatzero.Wechecktoseeifthemiddlenumberisthenumberwearelookingforbylookingupthemiddlenumberinourlistwithnumber_list[middle].Ifthemiddlenumberisthenumberwearelookingfor,not_trueissettoTrue,andweexittheloop.Otherwise,wechecktoseeifthenumberwearelookingforisbiggerorsmallerthanthemiddlenumber.Ifourmiddlenumberissmallerthanthenumberwearelookingfor,wechangelasttothemiddleindex-1,whichonthenextloopwillsplitthelistinhalfwitheverythingsmallerthanourcurrentmiddlenumber,whereasifourmiddlenumberisbiggerthanthenumberwearelookingfor,wechangefirsttomiddle+1,dividingthelistinhalfsoitcontainseverythingbiggerthanourmiddlenumber.

Recursion

Recursionisnotoriousasoneofthetoughestconceptsfornewprogrammerstograsp.Ifitisconfusingtoyouatfirst,don’tworry,it’sconfusingtoeveryone.Recursionisamethodofsolvingproblemsbybreakingtheproblemupintosmallerandsmallerpiecesuntilitcanbeeasilysolved.Thisisachievedwithafunctionthatcallsitself.Anyproblemthatcanbesolvedrecursivelycanalsobesolvediteratively,howeverincertaincases,recursionoffersamoreelegantsolution.Arecursivealgorithmmustfollowthethreelawsofrecursion:“1.Arecursivealgorithmmusthavea basecase.2.Arecursivealgorithmmustchangeitsstateandmovetowardthebasecase.3.Arecursivealgorithmmustcallitself,recursively.”

Let’sgooveranexampleofarecursivefunctionthathasallthree,afunctiontoprintoutthelyricstothepopularchildren’ssong“99BottlesofBeerontheWall” 19 :

defbottles_of_beer (bob): """Userecursiontoprintthebottlesofbeersong.:parambob:Integernumberofbeersthatarestartonthewall.""" if bob<1: print"Nomorebottlesofbeeronthewall.Nomorebottlesofbeer." return tmp=bob

bob-=1print"{}bottlesofbeeronthewall.{}bottlesofbeer.Takeonedown,passitaround,{}bottlesofbeeronthewall.".format(tmp,tmp,bob)bottles_of_beer(bob)

bottles_of_beer(99)>>99bottlesofbeeronthewall.99bottlesofbeer.Takeonedown,passitaround,98bottlesofbeeronthewall.>>98bottlesofbeeronthewall.98bottlesofbeer.Takeonedown,passitaround,97bottlesofbeeronthewall.…>>Nomorebottlesofbeeronthewall.Nomorebottlesofbeer.

Inthisexample,thebasecaseis:

ifbob<1:print"Nomorebottlesofbeeronthewall.Nomorebottlesofbeer."return

Thebasecaseofarecursivealgorithmiswhatfinallystopsthealgorithmfromrunning.Ifyouhavearecursivealgorithmwithoutabasecase,itwillcontinuetocallitselfforever,andyouwillgetaruntimeerrorsayingthemaximumrecursiondepthhasbeenexceeded.Theline:

bob-=1

satisfiesoursecondrulethatwemustmovetowardourbasecase.Inourexample,wepassedinthenumber“99”toourfunctionastheparameterbob.Sinceourbasecaseisbobbeinglessthan1,andbecausebobstartsat99,withoutthislinewewillneverreachourbasecase,whichwillalsogiveusamaximumrecursiondepthruntimeerror.Ourfinalruleissatisfiedwith:

bottles_of_beer(bob)

Withthisline,wearecallingourfunction.Thefunctionwillgetcalledagain,butwithoneimportantdifference.Insteadofpassingin99likethefirsttimethefunctionwascalled,thistime98willbepassedin,becauseofrulenumbertwo.Thethirdthefunctioniscalled97willbecalled.Thiswillcontinuetohappenuntileventuallybobisequalto0,andwehitourbasecase.Atthatpointweprint“Nomorebottlesofbeeronthewall.Nomorebottlesofbeer.”andreturn,signalingthealgorithmtostop.Let’sgooveronemorerecursivealgorithm.Sayyouaregiventhefollowingproblem:

Givenanon-negativeintegernum,repeatedlyaddallitsdigitsuntiltheresulthasonlyonedigit.Forexample:

Givennum=38,theprocessislike:3+8=11,1+1=2.Since2hasonlyonedigit,returnit.

Onewaywecansolvethisproblemisusingrecursion:

def add_digits (number):

"""

:param number:Int

:return :Singledigitint"""

number=str (number)

iflen(number)==1 :

return int (number)the_sum= 0

forcin number:

the_sum+=int (c)

returnadd_digits(the_sum)

printadd_digits( 99 )>>9

Inthisexample,ourfunctionacceptsanumberasaparameter,calculatesitssumandcallsitselfuntilithitsthebasecasewhichisanumberwithonlyonedigit.Ifwepassinthenumberninetynine,ithastwodigitssoitfailsthebasecase.Wethencalculatethatthe_sumequals18,sowecalladd_digitsagainandpassin18.Onceagain18doesnotpassourbasecasesowecalculatethesumofthedigitswhichthistimeis9,andcalladd_digitswith9asaparameter.Thethirdtimearound,ournumberis9andsinceit’sonlyonedigit,itdoessatisfyourbasecaseandwereturntheanswer.

AbstractDataTypes

Whenlearningaboutdatastructures,youwillcomeacrossthetermabstractdatatype.Irememberwhatanabstractdatatypeisbythinkingoftherelationshipbetweenanabstractdatatypeandadatastructureassimilar(althoughnotexactlythesame)astherelationshipbetweenaclassandanobject.

Ifyouwanttomodelanorangeinobject-orientedprogramming,aclassrepresentstheideaofanorange,whereastheobjectrepresentstheactualorange.Similarly,anabstractdatatyperepresentstheideaofacertaintypeofdatastructure.Anexampleofanabstractdatatypeisalist.Alistisanabstractdatatypethatcanbeimplementedseveralwayswithdatastructuressuchasanarray,oralinkedlist.

Nodes

NodeisatermusedfrequentlyinComputerScience.Youcanthinkofanodeasapointonagraph.Theinternetismadeupofroutersthatcommunicatewitheachother.Eachrouterisanodeinthenetwork.Nodesareusedinseveraldatastructures,includinglinkedlists,treesandgraphs.

Stacks

Astackisalast-in-first-outdatastructure.Thisisbestenvisionedasastackofdishes.Sayyoustackfivedishesontopofeachother.Inordertogettothelastdishinthestack,youhavetoremovealloftheotherdishes.Thisishowastackdatastructureworks.Youputdataintoastack.Everypieceofdataislikeadish,andyoucanonlyaccessthedatabypullingoutthedataatthetopofthestack.HereisanexampleofastackimplementedinPython:

classStack:def__init__(self):self.items=[]

defisEmpty(self):returnself.items==[]

defpush(self,item):self.items.append(item)

defpop(self):returnself.items.pop()

defpeek(self):returnself.items[len(self.items)-1]

defsize(self):returnlen(self.items)

Thetwomostimportantmethodsinthedefinitionarepushandpop.Pushletsyouputdataontopofthestack,andpopletsyoutakeitoffthestack.

Sowhatarestacksusedfor?Wellfirstofall,stacksareimportantforunderstandingrecursion.Recursionisafundamentalpartofprogrammingwegooverinalatersection.Mostnewprogrammersstrugglewithrecursion,butthekeytounderstandingrecursionistodeeplyunderstandhowastackworks.

Furthermore,stacksareusedtoreversethings.Whateveryouputonastackcomesoutinreverseorderwhenyoutakeitoff.Forexample,let’ssayyouwanttoreverseastring.Wecouldreversethestringbyfirstputtinginonastack,andthentakingitoff,likethis:

fromcollectionsimportstackass

my_string=“Hello”stack=s()

forcinmy_string:stack.push(c)

new_string=“”forcinstack:new_string+=cprintnew_string>>>olleH

Inthisexamplewewentthrougheachcharacterintheword“Hello”,andputitonourstack.Thenweweiteratedthroughourstack,andtookeverythingwejustputonthestack,offofit,andsavedtheorderinthevariablenew_string.Bythetimewegettothelastline,ourwordisreversedandourprogramprints“olleH”.I’mgoingtoshareanotherexampleofusingastack.It’sarareexampleofaquestionI’vebeenaskedtosolveinaninterview,andactuallyhaveusedonthejob.Theproblemis,writeaprogramthattestsastringforbalancedparentheses.So,forexample,“(hello)”wouldpass,but“(hello”wouldfail.“()()()”wouldpass,but“()(”wouldfail.Thislookseasyatfirst,untilwegetsomethinglikethis“((()((()((()((((()))))”.Howareyousupposedtokeeptrackofalltheparenthesis?Thekeytosolvingthisproblemistouseastack.Everytimewecomeacrossanopenparen,weputitonastack.Ifwecomeacrossaclosedparen,wepullanopenparenoffthestack.

defbalanced_paren (expression):stack=[]for cin expression: ifc=='(' :stack.append(c) elif c==')': iflen(stack)<1 :return False

stack.pop() iflen(stack)== 0 : return True returnFalse

Iftheparenthesisarebalanced,ourstackwillbeemptyafterourloop,andwecanreturn“True.”Onethingweneedtowatchoutfor,iftheparenthesisareunbalanced,wewilltrytopopfromanemptystackwhichwillcauseanerror.Thatiswhywhenwecomeacrossaclosedparen,wehavetofirstmakesurethestackisnotemptybeforewepopitoffthestack.Ifwecomeacrossaclosedparen,andthestackisempty,weknowtheparenthesisarenotbalancedandwecanreturn“False.”Ifattheendofthelooptherearestillopenparenthesisonthestack,wecanalsoreturn“False.”

LinkedLists

Alinkedlistismadeupofaseriesofnodes,witheachnodepointingtothenextnodeinthelist.MyfriendStevegaveagreatmetaphorforthinkingaboutlinkedlists.ImagineyouareintheMafiaandwanttogiveordersinsuchawaythatnooneknowswhoyouare.Youcouldsetupastructurewhereyougiveanorderanonymouslytothenextpersondownthechainofcommand.Youknowwhotheyarebuttheydon’tknowyou,theyonlyknowthenextpersontheyshouldgivetheorderto.Thepersontheygivethenextordertodoesn’tknowthem,butonlyknowsthenextpersontoreceivetheorder.Thischainofinformationiswhatasinglylinkedlistis.Inasinglylinkedlist,allofthenodesinthelistonlyknowaboutthenextnode.Theydon’tkeeptrackofthenodebehindthem.Youcangettoeverypieceofdatainalinkedlistbystartingattheheadofthelist,andmovingonebyonetoeachnextnode.Adoublylinkedlististhesamethingexcepteachnodekeepstrackofthenodebehindit,inadditiontokeepingtrackofthenextnode.Alinkedlistcanalsobeorderedorunordered.Inthissection,wewillimplementanunorderedsinglyanddoublylinkedlist:

classNode:"""Classrepresentingonenodeinalinkedlist."""def__init__(self,data):self.data=Node(data)self.next=None

classLinkedList:"""Classrepresentingalinkedlistdatastructure"""def__init__(self,head):self.head=head

defadd(self,data):

"""Addanewnodetothelinkedlist."""previous_head=self.headself.head=Node(data)self.head.next=previous_head

Ourlinkedlistclassissimple.Itstorestheheadofthelinkedlistandhasamethodcalled“add”toaddanewnodetothelist.Sincethelistisunordered,wedon’tcarewhereweputthenextnode,soit’seasiesttoputitattheheadofthelist.Themethodaddstoresthecurrentheadintheprevious_headvariable,sowedon’tlosetrackofit,createsanewnodewiththepassedindata,andsetsthenewnodeastheheadofthelist.Thenodethatusedtobetheheadisthensetasthenextnodeafterthenewhead.Nowwecancreatealinkedlistandadddatatoit:

linked_list=LinkedList()linked_list.add(1)linked_list.add(2)linked_list.add(3)

Togetthedatafromourlinkedlist,westartwiththehead,andvisiteverynodeuntilwehitanextnodethatequalsnone.

node=linked_list.headwhilenode:printnode.datanode=node.next

>>3>>2>>1

Wecanchangeoursinglylinkedlisttoadoublylinkedlistbykeepingchangingournodeclasstokeeptrackofthenodebehindit.

class Node : """Classrepresentingonenodeinalinkedlist.""" def __init__ ( self , data): self .data=data self . next=None self .previous=None

class LinkedList: """Classrepresentingalinkedlistdatastructure""" def __init__ ( self , data): self .head=Node(data)

def add ( self, data): """Addanewnodetothelinkedlist."""

previous_head=self .head self.head=Node(data)previous_head.previous= self .head self.head. next =previous_head

Ournewlinkedlististhesameasourpreviousone,exceptnoweverynodeknowsthenodeinfrontandbackofit.

Arrays

Anarrayisanimplementationofthelistabstractdatatype.Everypieceofdatainanarrayhastobethesametype.Soforexample,youcanhaveanarraymadeupofstringsorintsbutyoucannothaveanarraymadeupofboth.Python’sbuiltinlistdatastructureisimplementedinternallyinCasanarrayofpointers.

BinaryTrees

Atreeisanotherdatastructurethatgetsitsnamefromlookinglike,youguessedit,atree.Agreatexampleofatreedatastructureisthefilesystemonyourcomputer,whichisimplementedusingatree.Therearemanydifferentkindsoftreessuchasredandblacktrees,AVLtreesandbinarytrees.Inthissection,wewillbuildabinarytree.Abinarytreeismadeupofnodescontainingdata.Eachnodecanhavealeftchildandarightchild.Inthesamewayalinkedlistpointstothenextelementinthelist,anodecanpointtotwoothernodescalledtheleftandrightchild.Likeadoublylinkedlist,theleftandrightchildkeeptrackoftheirparent.HereisanexampleofabinarytreeimplementedinPython:

classNode : """Binarytreenode.""" def __init__(self,value):self.value=valueself.left_child=Noneself.right_child=None

classBinaryTree : """Thisclassrepresentsabinarytreedatastructure.""" def __init__(self,root): """:paramroot:Binarytreenode.

""" self.root=root

Ourbinarytreeissimplymadeupofaroot. Seenextsectionforatreewithchildren.

BreadthFirst&DepthFirstSearch

Ifwewanttovisiteverynodeinabinarytree,therearetwosearchalgorithmswecanuse:breadthfirstsearch,anddepthfirstsearch.Ifyouthinkofatreeofbeingmadeupofrowsandcolumns,inabreadthfirstsearchwevisiteachrowonebyone,whereasindepthfirstsearchwevisiteachcolumnonebyone.Wewillusethebinarytreefromthepreviousexampletocreateourtreetotraverse:

tree=BinaryTree(“a”)tree.left=TreeNode(“b”)tree.right=TreeNode(“c”)tree.left.left=TreeNode(“d”)tree.right.right=TreeNode(“e”)

Nowwecanwriteafunctionthattakesatreeasaparameteranddoesabreadthfirstsearchofthetree,printingoutthevalueofeachnodeitvisits:

defbreadth_first (tree):"""Breadthfirstsearchofbinarytreeprintouteachnode.:param root:BinaryTree"""current_level=[root]next_level=[]whilecurrent_level: fornode incurrent_level:print node.valif node.left_child:next_level.append(node.left_child)if node.right_child:next_level.append(node.right_child)current_level=next_levelnext_level=[]

Weusethelistcurrent_leveltokeeptrackofallthenodesinthelevelofthetreewearecurrentlyin.Whenthereisnomorecurrentlevel,ouralgorithmstops.Ourtreelookslikethis:

a

/\bc/\de

“current_level”willstartas[a],become[b,c],become[d,e]andfinallybecomeanemptylist[]atwhichpointouralgorithmisfinished.Weareabletodothisbykeepingtrackofthenextlevelofnodesinourlistnext_level.

Beforeourwhileloop,weaddtherootofourtreetocurrent_level,thentellourwhilelooptocontinue,aslongascurrent_levelisn’tempty.Inourwhileloop,weiteratethrougheverynodeincurrent_level,printingouteachnode.Thenwecheckifthenodehasanychildren.Ifitdoes,weaddthosechildrentoournext_levellist.Attheendofourforloop,weswitchthetwolists,sothecurrent_levellistissettothenext_levellist,andthenext_levellistbecomesempty.Thisiswhatallowsustomovefromoneleveltothenext.Eventually,whenwereachthelastlevel,thenodeswillnothaveanychildren,next_levelwillbeempty,current_levelwillbesettonext_level,andbecauseit’semptythealgorithmwillstop.

Adepthfirstsearchsearchesthetreeverticallyinsteadoftraversingacrossithorizontally.Ourtreefromthepreviousexamplewouldbesearched“a”,“b”,“d”,“e”,“c”.Wecanimplementdepthfirstsearchusingrecursion:

HashTables

InthechapterContainers,wecoveredPython’sbuiltindictionarydatatype.Dictionariesarehelpfulbecausetheycanstorekeysandvaluesandareincrediblyfastatgettingandsettingdata.Torecap,dictionariesmapkeystovalues.Forinstanceyoucouldaddthekey“super_computer”toadictionarywiththevalue“Watson”withthefollowingcode:

my_dictionary={}my_dictionary[“super_computer”]=“Watson”

Nowwecanretrievethekey“super_computer”in with:

printmy_dictionary[“super_computer”]>>>Watson

Theamazingthingaboutdictionariesistheycansetandgetdatainconstant time.Itdoesn’tmatterhowmanyrowsofdatawehaveinourdictionary.Wecouldhaveonebillionrows,andstilladdandretrievethevaluefor“super_computer”toourdictionaryinO(1)time.

Internally,Pythonusesahashtabletoimplementitsdictionary.AhashtableisadatastructurethatusesalistandahashfunctiontostoredatainO(1)time.Whenyouaddavaluetoahashtable,itusesahashfunctiontocomeupwithanindexinthelisttostorethedata.Whenyouretrievedatafromahashtable,itusesthesamehashfunctiontofindtheindexsoit

canretrieveitfromthelist.Inthisexample,ourhashtableisonlygoingtostorenumbers.Thehashfunctionwillreturntheresultofthenumbermoduloeleven.Soforexample,ourhashfunctionforonewouldreturnone,sowestorethenumberoneatindexoneinourlist.Ourhashfunctionforthenumberfivewouldreturnfive,andsowewouldstorethenumberfiveatindexfiveinourlist.Hereisanexampleofahashtable: class HashTable :

"""Hashtabledatastructure""" def__init__( self ): self .list=[ None]* 11

@ staticmethod defhash (n): """ :paramn:int :return :returnindexinlisttostorenumber.""" returnn%11

defset ( self ,n ,v): """ :paramn:int :paramv:canbeanytype.""" self .list[self .hash(n)]=v

defget ( self ,n): """ :paramn:int :return :intvaluefromlist""" returnself .list[hash (n)]

hash_table=HashTable ()hash_table.set( 1 ,'Disrupted')hash_table.set( 5 ,'HubSpot' )print (hash_table.get( 1))print (hash_table.get( 5))

Thisisanoversimplifiedexamplethatclearlyhasproblems.However,thegoalistoillustratehowahashtableworks. bc

Challenge

ChapterX.RelationalDatabaseDesign

Whenyoucreateusearelationaldatabase,youhavetodesignthedifferenttablesyourdatabasewillhave,howthetablesrelatetoeachother,whatcolumnstheywillhave,andwhatconstraintsareputonthosecolumns.Together,thismakesupyourdatabaseschema.Inthissection,wearegoingtotodesignaschematostoredataforawebsitelikeAmazon.com.First,weneedtothinkaboutthedataAmazonneedstostore.Thefirstthingthatcomestomindisproducts;Amazonclearlymusthaveadatabasewheretheystorealloftheirproducts.Amazonalsohastokeeptrackofcustomers—youdon’thavetoregisteranewaccounteverytimeyouordersomethingonAmazon,sotheymuststoretheircustomersinformationaswell.Acustomermightordermorethanoneproduct,soAmazonmustalsohaveawaytostoreorders.Let’sstartbydesigningatabletoholddataaboutproducts:product_______________________id|name|price1|ThePragmaticProgrammer|14.99Ourproducttablehasacolumncalledidthatservesastheprimarykey,andcolumnsforthenameoftheproductandtheprice.Ourprimaryidisanintegerthatautoincrements,thenamecolumnacceptsstrings,andthepricecolumnaccepts integers .Thedatashowninthetablesuchas1,“ThePragmaticProgrammer,”and14.99arenotpartofthedatabasedesign,butareanexampleofhowdatawouldlookinourtable.Thisconventionisusedthroughoutthischapter.

Notethatwhenyoudesignadatabaseschema,youwanttopickanamingconventionandsticktoit.Inthefollowingthiscasewewilluselowercaselettersandanunderscoreinbetweenwords.Nowweneedatabletokeeptrackofourcustomers.Ourcustomertableisverysimilartoourproducttable:

Customer

id|first_name|last_name______________1|Steve|Smith

Ourcustomertablehasaprimarykey,andtwocolumnsthatacceptstrings.Thisisallweneedtokeeptrackofourcustomers.

Wearegoingtokeeptrackofourordersusingtwodifferenttables.Thefirsttablewillmapanorderidtoaspecificcustomer,andthesecondtablewillkeeptrackoftheproductsineachorder.Hereisourfirsttableshownalongwiththecustomertable:

customerid|first_name|last_name______________1|Steve|Smith

order_______id|customer1|1

Ourordertablehasaprimarykeycalledidandacolumncalledcustomer.Ourcustomercolumnisdifferentthantherestofthecolumnswe’veseensofarbecauseitusesaconstraintcalledaforeignkey( coveredinthechapterSQL ).Thecustomercolumnofourordertableacceptsanintegerthatrepresentstheprimarykeyofacustomerinourcustomertable.Inourexample,thefirstentryintheordertablecustomercolumnis1.Ifwelookuptherowwith1asitsprimarykeyininourcustomertablewewouldgettherow“ 1|Steve|Smith”.Byusingaforeignkey,we’vesuccessfullylinkedourordertabletoourcustomertable.Thisiscalledcreatingarelationship.Imagineifwedecidedtoputtheinformationfromour“customer”tableinthe“order”tableinstead:

orderid|username|order_____________1|Steve|NoSQLDistilled2|Cory|ThinkPython3|Steve|TheTalentCode

Theproblemwiththisdesignisthatdataisduplicatedinourtable.TheusernameSteveisrepeatedtwice.IfweneededtochangeSteve’susernameto“Steven,”wemightaccidentallyonlychangethenameinthefirstrow,andforgettochangeitinthethird.Thiswouldcorruptourdata:

orderid|username|order_____________1|Steven|NoSQLDistilled2|Cory|ThinkPython3|Steve|TheTalentCode

Inouroriginaldesignthisisnotpossible.Takeanotherlookatourpreviousdesign:

customerid|username

_________8|Cory9|Steve

ordersid|username|order_________________1|9|NoSQLDistilled2|8|ThinkPython3|9|TheTalentCode

Whenweneedtochangeausername,weonlyhavetochangeitinoneplace—thecustomertable.Oncewechangethenameinourcustomertable,anyonelookingupusernamewithaforeignkeyof9willseethecustomer ’susernameisSteve.Thereisnochanceofaccidentallycorruptingthedatabecauseitonlyexistsinonelocation.

Tablescanhavethreetypesofrelationships:onetoone,onetomanyandmanytomany.Thisisanexampleofaonetoonerelationship.Youcreatebothaonetoonerelationshipandamanytoonerelationshipusingaforeignkey.Thedifferenceis,inaonetoonerelationship,bothtablescanhaveforeignkeystoeachother,althoughlikeinthiscase,theydon’thaveto.Inaonetomanyrelationship,onlythemanysidehasaforeignkeylinkingittotheone.Thisisnotsomethingyourdatabaseknowsabout,butratheraconstructinventedtohelpyoudesigndatabases.

Inthisexample,acustomercanhavemanyorders,butanordercannothavemanycustomers.Anotherexampleofamanytoonerelationshipisaclassroom.Ateachcanhavemanyclasses,butaclasscannothavemanyteachers.Inaonetoonerelationshiphowever,therelationshipcangobothways.Onepersonhasonepassport,andonepassporthasoneperson.Thefinalrelationshiptablescanhaveiscalledmanytomany.Inordertodothatweneedtocreateajunctiontable,whichweneedtodoinordertocompleteourAmazondesign.Ourfinaltableorder_itemwillkeeptrackofproducts:

order_item__________id|order_id|product_id1|1|1

Thistablehas id asaprimarykey,andtwoforeignkeys—order_id andproduct_id linkingthetabletoourorderandproducttable.Ourdesigniscomplete,wecanstoreandlookupalloftheinformationweneedtofulfillanorder.Hereareallofourtablestogether:

product_______________________id|name|price1|ThePragmaticProgrammer|14.99

customer

id|first_name|last_name______________1|Steve|Smith

order_______id|customer1|1

order_item__________id|order_id|product_id1|1|12|1|1

Ifwearereadytoshipanorder,wecangetalloftheinformationweneedbylookingattheorder_itemtableandusingthatinformationtoqueryothertables.Firstwewouldselectalltherowsfromourorder_idwithanorder_idof1.Thenwewouldlookupalltheproductsusingtheproduct_idineachrow.Finally,wewouldusetheorder_idkeytolookupthenameoftheforeigncustomerkeyinourordertable,andlookupthecustomer'snameinthecustomertableusingthatinformation.

Normalization

Oneofthechallengesyoufacewhenworkingwithadatabaseismaintainingdataintegrity,whichmeans“assuringtheaccuracyandconsistencyofdataoveritsentirelife-cycle” 49 .Normalizationandreferentialintegrityaresomeoftheconceptsthathelpensuredataintegrity.

Datanormalizationistheprocessofdesigningarelationaldatabaseinordertoreducedataredundancy,whichcanleadtoinaccuratedata.Whiletherearemanyrulesfordatanormalization,therearethreespecificrulesthateverydatabaseshouldfollow.Eachoftheserulesiscalleda“normal“form.” Ifthefirstruleisfollowed,thedatabaseisin"firstnormalform"or1nf.Ifallthreerulesarefollowed,thedatabaseisin"thirdnormalform"or3nf. 52Inordertoreacheachsuccessivelevelofnormalization,allofthepreviousrulesmustbefollowed.Inotherwords,iftherulefor2nfissatisfied,but1nfisnot,thedatabaseisnotconsidered2nf.

Toreachthefirstnormalform,youneedtoavoidduplicatingdatainmultiplerow,avoidstoringmorethanonepieceofinformationinarowandthetablemusthaveaprimarykey.Hereisanexampleofstoringduplicatedata:

t-shirt

____________color|colorblue|blue

Andanexampleofstoringmorethanonepieceofdatainonerow:

t-shirt______colorblue,large

Inthisexampleweareusingacommatostoretwopiecesofdatainonecolumn—“blue”and“large.”Thisissomethingyoushouldneverdo.Furthermore,neitheroftheseexamplesare1nfbecausetheydonothaveaprimarykey.

Hereisanexampleofatablethatis1nf:

t-shirtprimary_key=id_____________id|color1blue

Inorderforatabletobe2nf,allnonprimarykeycolumnsmustrelatetotheprimarykey.Let’slookatanexamplethatviolates2nf:

t-shirtprimary_key=itemprimary_key=color__________item|color|price|taxt-shirtred19.99.90t-shirtblue18.00.78poloyellow321.4pologreen401.8poloorange432

Thistableisnot2nfbecausethetwocolumnsthatarenotprimarykeys,priceandtaxrelatetoitem,butdonotrelatetocolor.

dealership________id|location|available1Portland|Yes

53

Normalizationisanimportantpartofdatabasedesign.Whilethereevenmorenormalizationruleswedidnotcover,itisimportanttoalwaysnormalizeyourdatabaseto3nf.Tohelpyouremembertherules,programmersoftenusethephrase“Thedatadependsonthekey[1NF],thewholekey[2NF]andnothingbutthekey[3NF]sohelpmeCodd ”(Codd,mentionedearlier,isthecreatorofrelationaldatabases)tohelpthemremembertherulesofofnormalization.

ReferentialIntegrity

Referentialintegrityisanotherwayofensuringdataintegrity.Itisameasureofconsistencyinadatabase.Ifforexamplewehavethefollowingtables:

customerid|username_________8|Cory9|Steve

orderid|username|order_________________1|9|NoSQLDistilled

andwedeletethesecondrowfromthecustomertable:

customerid|username_________8|Cory

orderid|username|order_________________1|9|NoSQLDistilled

Ourusernamecolumninourordertablereferencesaforeignidthatnolongerexists.Thisisaviolationofreferentialintegrity.Fortunately,yourdatabasemanagesreferentialintegrityforyou.Ifyoutrytodothisinarelationaldatabase,itwon’tletyou,youwillgetanerror.

Indexing

Youcanindexacolumninatableinordertomakereadsfaster.Indexesworklikeatelephonebooksortedbylastname.Ifyouwerelookingthroughsuchatelephonebook,youwouldn’tlookthrougheverysingleentry,youwouldimmediatelyskiptothesectionofthephonebookthatmatchesthelastnameofthepersonyouwerelookingfor.Thisiswhatanindexdoes.Whenyouindexacolumninadatabase,internallythedatabaseduplicatesthedatainthecolumn,butarrangesitinaspecificorder —alphabeticallyforexample,thatallowsittolookupdatafaster.HereisanexampleofcreatinganindexwithSQL:

CREATEINDEXmy_indexONtable_name(customers)

Thiswillcauseourdatabasetointernallyduplicateallofthedatainourcustomertableandarrangeitalphabetically.Nowwecanlookupcustomersmuchfaster.Thedrawbacktocreatinganindexisthatduplicatingdataandorganizingithasacost—itincreasesthetimeittakestowritetoyourdatabase.

Challenge

ChapterX.ComputerArchitecture

“Thereare10kindsofpeopleintheworld—thosewhounderstandbinaryandthosewhodon't.”—Anonymous

Nowthatw e’vecoveredthefundamentalsofprogramming—andsometoolstoprogrammoreeffectively—wearegoingtogooversomeofthebasicsofComputerScience.ComputerScienceisusuallytaughtasabstruselyaspossible—soI’veattemptedtomakeitasfriendlyaspossible—whilefocusingonthemostpracticalparts.Inthischapterwetakealookunderthehoodofwhatwe’velearnedsofar—weexplorehowPython,youroperatingsystemandcomputerwork.

HowYourComputerWorks

Computerscanonlyunderstandbinary,soinordertounderstandhowacomputerworks,youshouldhaveabasicunderstandingofbinary.Whenyounormallycount,youcountinbaseten,whichmeanswerepresenteverynumberintheworldusingonlytendigits.The“base”ofacountingsystemisthenumberofnumbersusedtorepresentallthenumberintheworld.Inthebase10countingsystem,oncewegetoverthenumbernine,werecombinenumbersfromonetotentocreatenewnumbers:

0 123456789 11 1 21 314151617181920

Basetwoisasystemofcountingjustlikebaseten.However,insteadofcombiningexistingnumbersaftertennumbers,basetwostartsdoingitaftertwonumbers0->0

1->110->211->3

0and1arethesameasbaseten.However,oncewegetto2,we’vegonepasttwonumbers,andweneedtocombineourfirsttwonumberstocreateanewnumber.Henceoneandzeroare“10”iscombinedtorepresent2.Eachnumberstartingfromtheleftrepresentswhetherornotthereisapowerof2.Soforexample,“ 10”meanstherearezero2**0’sand12**1:

102+0=2

Yourcomputerismadeupofhardware—aCPU,memoryandinput/outputdevices.Allcomputingisdonewiththesephysicalpiecesofhardware.Hardwareonlyunderstandsbinary.Thatmeansnomatterwhatprogramminglanguageyouuse,nomatterwhatoperatingsystemyouuse,everyinstructionacomputereverexecutesisinbinary.

[explainrepresentingthingsinbinary]Thischartshowshoweachcharacterismappedtoanumber.Forexample,97ismappedtotheletter“a”.AprogramusingASCIIwouldstorean“a”as1100001,or97inbinary.Whenthesameprogramneedstoretrievetheletter“a”,itlooksup97intheASCIItableandseesitrepresentstheletter“a”.YoucanusePythontoeasilygetthenumberacharactermapstoinASCII.Thefunction“ord”takesacharacterandreturnsthenumberitmapsto:

ord(“a”)>>97

ord(“z”)>>122

AtahighlevelacomputerismadeupofaCPUandmemory .TheCPUisthepartofacomputerthatexecutestheinstructionsprovidedbyaprogram.AcomputercanhaveoneormoreCPUS:acomputerwithmultipleCPUsissaidtohaveamulti-coreprocessor.ACPUhasaclockgeneratorthatproduces“clockcycles.”InaCISC,orComplexInstructionSetprocessor,eachinstructiontakesoneormore“clockcycles,”whereasRISC,orReducedInstructionSetComputingprocessors,arefaster,abletoexecutemultipleinstructionsperclockcycle.Clockspeedreferstothespeedamicroprocessorexecutesinstructions.

pythonversionofjavaCPUprograminStructuredComputerOrganization

Computershavetwotypesofmemory,RAMandROM.RAMstandsforrandomaccessmemoryandisvolatile,meaningitiserasedwhenacomputerturnsoff.Allarithmeticandlogicoperationstakeplaceinram.ROMstandsforreadonlymemory.Itisnotvolatile;itpersistsafteracomputeristurnedoffandisusedforthemostfundamentalpartsofanoperatingsystemneededwhenacomputerstartsup.Everythinginmemoryisstoredinbinary,sequencesofzerosandones.Thezerosandonesarestoredinchipsmadeupofmillionsorbillionsoftransistors.Eachtransistorcanbeturnedonoroffusingaflowofelectricity.Whenatransistoristurnedoffitrepresentsazeroinbinary,andwhenatransistoristurnedonitrepresentsaone.Memoryisnotwhereeverythingonacomputerisstored,despitethenamesoundinglikeitis.MemoryisusedincombinationwiththeCPUtoexecutetasks,likearithmeticoperations.Whenyoustoreinformationinadatabaseforexample,itiswrittentodiskstorage,notmemory.

I/O

TheCPUcombinedwithmemorymakeupisthebrainofacomputer.I/Oisthetransferofdatatoorfromthebrainbyotherdevices.ReadingandwritingdatafromdiskstorageisanexampleofI/O.

Aswelearnedearlier,computersonlyunderstandbinary.Sohowdocomputersworkwithcharacters?TheanswerischaracterencodingschemeslikeASCIIandUnicode.Toacomputer,charactersdonotexist.CharactersarerepresentedusingencodingschemeslikeASCIItomapcharacterstobinarynumbers.

HowProgrammingLanguagesWork

Machinecodeiscodemadeupentirelyofbinary(orhexwhichisbase16butignorethatfornow)thatcanbeexecuteddirectlybyyourcomputer ’shardware.YoucanprogrameveryPythonprogramwe’vewrittensofarinbinary.

ComputerScientistsuseaconceptcalledabstractiontomanagecomplexity.Whensomethingisanabstraction,itmeansyoucanuseitwithoutneedingtounderstandhowitworks.Whenyouprogram,youareprogrammingbeneathseverallayersofabstraction:youcanprograminPythonwithoutknowinghowPythoncommunicateswithyouroperatingsystem,howyouroperatingsystemcommunicateswithyourcomputer ’shardware,orhowyourcomputer ’shardwareexecutesbinary.

Programminglanguagesarebuiltonabstractions.Machinecodeisthelowestlevelprogramminglanguage—thatmeansithasnoabstractionsbeneathit.Aprogramminglanguagebecomeshigher-levelthefurtherawayitgetsfrommachinecode(thedirectinstructionsexecutedbytheCPU).Thehigher-levelaprogramminglanguageis—themoreabstractionsithasbeneathit.Themoreabstractionsaprogramminglanguagehasbeneathitthesloweritruns.IfyoucanwriteanyPythonprograminmachinecode—andtheprogramswillrunfaster—youmightbewondering,“Whydon’tprogrammerswritealloftheirprogramsinmachinecode?”Thereasonprogrammersdon’twriteprogramsinmachinecodeisbecauseitisinsanelytedious.Peopledon’tthinklikecomputers—theydon’tthinkpurelyinnumbers—whichledtotheideaofwritingaprograminmachinecodethatcouldtranslateamorehumanreadablelanguageintomachinecode,sohumanscouldwriteinalanguagethatwasmorenaturaltothem,andthisnewprogram(calledanassembler)wouldautomaticallytranslateassemblycodeittomachinecode(calledanassembler)—thisprogramminglanguageiscalledAssembly,anditisthefirstabstractionabovemachinecodeandtheprogramthattranslatesassemblycodetomachinecode.Hereisanexampleofaprogramaddingtwonumbersandstoringtheminavariableinmachinecode:

addmachinecodeexample

HereisthesameprogramwritteninAssembly:

addassemblycodeexample

Asyoucansee,theAssemblylanguagecoderepresentsthewayhumanthinksmuchthanthemachinecode;however,whileassemblycodeiseasiertoread,theinstructionsstillareonetoonewithmachinecode;thatis,everylineofassemblycodetranslatestoexactlyonelineofmachinecode—soprogramminginassemblyisstilltediousbecauseitrequirestheprogrammertowritesomanylinesofcodetogetanythingdone.Thisledprogrammerstodevelopevenhigherlevelprogramminglanguages—likeC—withinstructionsthatarenotonetoonewithmachinelanguageandthuslettheprogrammerwriteprogramswithfewerlinesofcode.Ccodeisfirsttranslatedintoassemblycode,andthenitistranslatedintomachinecode.HerethesameprogramaddingtwonumbersandstoringtheminavariablefromthepreviousexamplesinC:

intxx=2+2

ThisissimilartohowwewouldwritethisprograminPython,althoughtherearetwostepsinsteadofone.InC,unlikePython,UnlikePython—inC—youhavetodeclarewhattypeavariableisbeforeyouuseit.Thisisaconceptcalledstatictyping,whichwecoverlater.Althoughthiscodelooksfamiliar,CisalowerlevellanguagethanPython,whichisoneabstractionlevelaboveit.Ccodegetsconvertedtoassembly,andthentomachine,codebyaprogramcalledacompiler.Thedifferencebetweenanassemblerandacompileristhatacompilerincludesextrafunctionalitytooptimizeperformancewhenittranslatesahigherlevellanguagetomachinecode(compilersareusedineveryprogramminglanguagenotjustC).Buttheconceptisstillthesame—ahighlevellanguageistranslatedtomachinecode.

WhenyouprograminC,youhavetomanagethememoryyourprogramusesyourself;whenyouprograminPython—yourprogram’smemoryismanagedforyou,andyouhavenoaccesstoit.WhenyouprograminPython,youcancreatealist,andappendasmanyitemstothatlistasyou’dlike.WhenyouprograminC,thisisnotpossible.WhenyoudefineanarrayinC,youhavetoallocateacertainamountofmemorytoit.Youcan’tjustdefineanarray—youhavetoincludehowmanyitemswillbeinthatarray.Ifyoudefineanarraythatcanholdtenintegers—anddecideyouactuallyneedeleven—youhavetocreateabrandnewarray.InPython,aswe’veseen,yousimplycreatealistandappendasmanyitemstoitasneeded.

Pythonachievesthisbyaddingonemorelayerofabstraction.PythoncodedoesnotgetcompiledtomachinecodethewayCcodedoes.Instead,whenyourunaPythonprogram,itisexecutedintwosteps.First,thePythoncompiler(writteninC)translatesPythoncodetobytecode—aspecialkindofcodeconsistingonlyofnumbers—butmeanttobeexecutedbyavirtualmachine(softwarethatemulateshardware).Python’svirtualmachineprogramisthenexecutedbythehardware ,anditgoeslinebylinethroughthebytecodeandexecuteseachinstruction.Thisdesignofferstwoimportantadvantages.Becauseofthis,youcanwriteaprograminPythonwithoutworryingaboutmanagingmemory,an dwhyyouca

ThereareseveraldifferentimplementationsofPython.TheversionofPythonweareusingiscalledCPython.

WhenyouareprogramminginC,youhavetocompileyourprogrambeforeitwillrun.TheCcompilertakesyourcodeandtranslatesitintomachinecodeyourcomputercanunderstand.Oncecompiled,younolongerneedanysortofprogramtorunyourcode,youcanexecuteitdirectly.

Pythoncode,however,mustalwaysberunusingthe“python”program.ThisisbecausePythoniswhatisoftencalledan“interpreted”language,atermusedtodifferentiateitfromalanguagelikeC,whichiscalleda“compiled”language.Thisisconfusing,becausewhilePythonhasaninterpreter,italsohasacompiler.WhenyourunaPythonprogram,itscompilertranslatesyourcodetosomethingcalledbytecode,aspecialkindofcodethatislikebinarybutmeanttobeconsumedbyavirtualmachine.Atruntime,Python’svirtualmachinetranslatesthebytecodeintomachinecodeandexecutesitlinebyline.

Thesetwoapproachesbothhaveadvantagesanddisadvantages.OneadvantageofC’sapproachisspeed—compilingdirectlytomachinecodemakesC’sprogramsrunfasterthanPythonprograms.C’sapproachalsoallowsforvariablestobestaticallytyped.Thiseliminatesaclassoferrors,butalsohasdrawbackssuchasgivingprogrammerslessflexibility.Python’sapproachisadvantageousbecauseitallowsittobeplatform-independent.Python’suseofaninterpreteralsoallowsitsvariablestobedynamicallytyped,whichmakesprogramminginPythonmuchmoreflexiblethanC.

Programminglanguagescanbeeitherdynamicallyorstaticallytyped.PythonandRubyareexamplesofdynamicallytypedlanguages.Inadynamicallytypedlanguage,youcandeclareavariableasonetype,andlaterchangethatsamevariabletoavalueofadifferenttype.Forexample,thisisallowedinPython:

x=5x=“HelloWorld”

Inastaticallytypedlanguage,tryingtochangeavariablefromonetypetoanotherwillcauseanerrorbecauseinastaticallytypedlanguage;onceyoudeclareavariable’stype,youcannotchangeit.UnderstandingthedifferencebetweenstaticallyanddynamicallytypedlanguageswillpayoffwithhoursoffunarguingwithyourfriendswithComputerSciencedegreesaboutwhetherornottheformerisbetterthanthelatter.

Yourcomputerismadeupofphysicalhardware.ThishardwareiswhatrunsourPythonprogramsanditonlyunderstandsonething—binary.Binaryisacountingsystem.Countinginbinaryisnodifferentthanwhenyounormallycount—whichiscalledcountinginbaseten—exceptthereareonlytwodigitsinsteadoftendigits.Whenwecountinbaseten,westartatzero,andwhenwegettoninewesayohno!Weranoutofdigits.Wesolvethisbytakingadigitwealreadyhave—zero—andputtingitafterthefirstdigit(one)tocreatethenumberten.Binaryworksthesameway.Zeroinbinaryiszero.Oneinbinaryisone.Inbinary,azerooroneiscalledabit—shortforbinarydigit.Inbinary,afterthenumberone,werunoutof

digits,andlikebaseten,wereusedigitswealreadyhave.Weaddazerototheendofthenextnumber—two—whichbecomes10.Wedothesamethingforthree—whichbecomes11.Countingthiswayiscalledcountinginbasetwo.

(binary)base2base10

001110211310041015110611171000810019101010101111

Icoverhowtoconvertanumberinbase10toitsbinaryequivalentlaterinthebook.Fornow,justunderstandingwhatbinaryiswillsuffice,andallowsustoexploreafundamentalprogrammingconcept—datatypes,ortypesforshort.

Atthebeginningofthissectionwelearnedcomputersonlyunderstandbinary.Pythonneedstostorebothintegersandstringsinyourcomputersmemory(thepartofyourcomputerthatsavesdataandcanonlystorebinary)—sohowcanPythonstoreintegersandstringsinmemory?Wealreadywentoverhowthenumbertwoisrepresentedinbinary—soitiseasytounderstandhowPythonstoresintegerslike2inyourcomputer ’smemory—itjustconvertsthemtobinary.ButhowdoesyourPythonrepresentastringlike“z”whenittalkstoyourcomputer?Pythonrepresents“z”justlikethenumbertwo—inbinary(everythingisinbinary!).Toyourcomputer“z”is:0111101001111010isalsothenumber122inbinary.Sinceeverythingmustberepresentedinbinary,Pythonrepresentsstrings(like“z”or“a”)withbinarynumbers,andithasatablethatmapseachbinarynumbertoacharacterinthealphabet.Thistableiscalledaunicodetable.Youcancheckoutacoolexampleofaunicodetablehere: http://unicode-table.com/en/#0046 (clickonaletterandthenclickonthelinkinthepopuptoseethebinary).01111010ismappedto“z”.So01111010canrepresenteitherthenumber122or“z”.Whenyouputquotesaround“z”,thequotesletPythonknowyouarerepresentingastringandnotanothertypelikeaninteger.Ittakestheletterinquotesandlooksupintheunicodetablewhichbinarynumberrepresents“z”andthenusesthatwhenittalkstoyourcomputer.Thatiswhytypeissoimportant.Acomputercanonlyunderstandbinaryandsoprogramminglanguageslike

Pythonhavetodifferentiatebetweendifferenttypesofdata—likestringsandintegers—soitcanknowhowtorepresentthemtoyourcomputer.

HowYourOperatingSystemWorks

Anoperatingsystemisthesoftwarethatmanagesacomputer ’shardware,allowingyoutouseyourcomputer.YourcomputerhasalimitedamountofresourcessuchasmemoryandCPU,andyouroperatingsystemdeterminestheresourceseachprogramreceives,alongwithcreatingastructureformanagingfiles,managingdifferentusersandmanagingothercommonoperationsneededbyprogrammers.Thekernelisthemostfundamentalpartofanoperatingsystem,responsibleforallocatingresourceslikeCPUandmemorytodifferentprocesses.Processesareprogramsthatareexecuting.Thekernelassignsmemoryandastacktoeachnewprogramwhenitstartsrunning.Thestateofthecurrentprocessissavedinadatastructurecalledaprocesscontrolblock.

Thekernelcannotbeaccesseddirectly,sothereisanotherlayerofsoftwarebuiltontopofthekernelinordertoaccessitcalledtheshell(becauseitisashellaroundthekernel).WelearnedhowtousetheshellinthechapterTheCommandLineinPartIII.Operatingsystemshaveotherresponsibilitiesotherthanresourcesharing,butinordertolimitthesizeofthekernel’scode,otheroperatingsystemjobs(calleddaemons)arerunalongsideuserprograms.Whenthekernelswitchesfromoneprocesstoanother,itiscalledacontextswitch.

InterruptsInteruptsThefollowingexplanationonStackOverflowhelpedmeunderstandthedifference

betweenconcurrencyandparallelism:“Concurrencyiswhentwoormoretaskscanstart,run,andcompleteinoverlappingtimeperiods.Itdoesn’'tnecessarilymeanthey'lleverbothberunningatthesameinstant.E.g.,multitaskingonasingle-coremachine.Parallelism iswhentasksliterallyrunatthesametime,e.g.,onamulticoreprocessor.”RichieHindle4.

Howcanasinglecoreprocessorrunmultipletasksatonce?

Challenge

ChapterX.NetworkProgramming

Inthischapter,welookintohowcomputerscommunicatewitheachotherovernetworks.Anetworkisagroupofcomputersconnectedthroughsoftwareandhardwarethatallowsthemtoexchangemessages.54 .TheInternetisanexampleofanetwork.Inthischapterwewillexplorethefoundationoftheinternet—theclientservermodelandtheTCP/IPprotocol.Thenwewilldivedeeperintothesesubjectsbybuildingbothaclientandaserver.

Client-ServerModel

TheInternetcommunicatesusingtheclient-servermodel.Intheclient-servermodel,thereisaserveractivelylisteningforrequests(likeGoogle),sentbyaclient(yourwebbrowser).Clientssendrequeststoserversaskingfortheresourcestheyneedtorenderawebpage,andifeverythinggoeswell,theserverrespondsbysendingtheresourcestothebrowser.RequestsaremadeusingHTTP,orhypertexttransferprotocol,whichwecoverinthenextsection.WhenIsayresources,ImeantheHTML,JavaScriptandCSSfilesthebrowserneedstodisplayawebsitealongwithanyimages.WhenyouvisitGoogle’swebsite,youareseeingtheclient-servermodelinaction.Googlewaitsforyoutosendarequest,andrespondswiththeresourcesyourwebbrowserneedstodisplayGoogle’swebsitetoyou.

TrygoingtoGoogleinyourbrowserandcopyandpastetheURLintoawordprocessor.Youwillseethereisaslashaddedtotheendoftheurl.ThatisbecausewhenyougotoawebsitelikeGoogle,youarereallygoingto“ http://www.google.com/ ”.The“/”isreferencingtherootpageofthewebsite,whichyouwillrecallfromtheCommandLinechapterishowishowyoureferencetherootofanoperatingsystem.Sowhenyougoto“http://www.google.com/ ”,youarerequestingGooge’srootpage,whereasifyougoto“http://www.google.com/news ”youarerequesting“/news”andGooglewillrespondwithdifferentresources.However,beforeanyofthiscanhappen,yourwebbrowserneedstotranslate“http://www.google.com ”intoanIPaddress.ThisiswheresomethingcalledtheDNS,ordomainnamesystemcomesin.TheDNSisagianttablethatmapsallofthedomainsintheworldtotheirIPaddresses,maintainedbydifferentinternetauthoritiessuchasthe InternetAssignedNumbersAuthority.Anipaddressisauniquenumberthatrepresentseachcomputerontheinternet.TocommunicatewithGoogle,yourbrowserneedstogetitsIPaddress,whichitdoesbylookingitupintheDNS.Atalowlevel,allthiscommunicationhappensthroughsockets.Socketsarethefunctionsthatgiveprogramsaccesstoacomputer ’snetworkhardware.Socketsarecreatedbyyouroperatingsystemasadatastructure,allowingcomputerstoestablishconnectionswitheach

other.Aserveropensapassivesocket,andaclientopensanactivesocket.Apassivesocketstaysopenindefinitely,listeningforconnections,whereasanactivesocketrequestsdatafromapassivesocketandthencloses.Torecap,theclient-servermodelworksasfollows—auserentersadomainnameintothebrowser,andthebrowserlooksupthedomain’sIPaddressintheDNS.ThebrowsersendsanhttprequesttotheIPaddressitlookedup,andtheserverrespondswithanhttp requestlettingthebrowserknowitreceiveditsrequestandthensendstheresourcesthewebbrowserneedstodisplaytherequestedwebpagetoyou.

TCP/IP

Thecommunicationintheclient-servermodelfollowstheTCP/IPprotocol.Aprotocolisanagreeduponwayofdoingthings,usedtostandardizeaprocess.ProtocolsarenotlimitedtoComputerScience.IfyouweretomeettheQueenofEngland,therewouldbeaprotocolinplace—asetofruleseverypersonhastofollowwhenmeetingher.Youwouldn’tjustwalkuptoherandsay“Heybro!”Youwouldaddressheracertainway,speakpolitely,sticktocertainsubjectsetc.Thatisaprotocol.

ComputerscommunicatingovertheInternetuseaprotocolcalledTCP/IP.Imagineaninternetwithoutanagreeduponprotocol.Withnostandardforcommunicating,everytimetwocomputersneededtopassdatatooneanother,theywouldhavetonegotiatethetermsoftheircommunication.Nothingwouldevergetdone.LuckilywehaveprotocolslikeTCP/IPthatensurecommunicationhappensseamlessly.

TCP/IPiswhatiscalledaprotocolstack.Itisismadeupoffourlayers,witheachlayerusingitsownprotocol.Eachlayerisaprogramresponsibleforaccomplishingatask,andcommunicatingwiththelayersaboveandbelowit.WhiletheInternetcoulduseoneprotocol(insteadofastack),thebenefitofusingaprotocolstackseparatedintolayersisthatyoucanmakechangestoonelayerwithoutneedingtochangetheothers.Thinkaboutthepostoffice.Someoneatthepostofficeacceptspackages,thensomeoneelsesortsthemandpassesthemofftosomeonewhodeliversthem.Eachpersonhastheirownprotocolforaccomplishingtheirtask(andtheyallcommunicatewitheachother).Ifthedeliveryguydecidestodeliverpackagesusingdronesinsteadofatruck,thechangeinprotocoldoesn’taffectthepersonwhoacceptspackagesorthepersonwhosortsthem.ThisisthesamereasonwhyTCP/IPusesaprotocolstack,sochangestooneprotocolwon’taffecttheothers.

ThefourlayersofTCP/IParetheApplicationLayer,theTransportLayer,theInternetLayerandtheNetworkLayer.

[tcp/ippicture]

Let’stakealookatanexampleofdatamovingthroughTCP/IPbyonceagainthinkingaboutmail.ThinkoftheApplicationLayerasaprogramcontainingaletter.Whenyoutypeaurl

intoyourwebbrowser,theApplicationLayerwritesamessageontheletterthatlookssomethinglikethis:

[pictureofletterwithhttponit]

TheinformationontheletterisanHTTPrequest.Again,HTTPisaprotocolthatserversandclientsusetosendmessagestoeachother.TheHTTPrequestItcontainsinformationsuchastherequestedresource,thebrowsertheclientisusingandafewmorepiecesofinformation.

Theletteristhenpassedtothenextlayer,theTransportLayer.YoucanthinkoftheTransportLayerputtingtheletterinanenvelope.Outsidetheenvelope,theTransportLayerputsmoreinformation:

[pictureofenvelopewithwritingonit]

Theinformationincludesthedomainnametosendtherequestto,thedomaintherequestiscomingfrom,theportnumbertheserverison,andsomethingcalledachecksum.Dataisnotsentacrossthenetworkallatonce,itisbrokenupintopacketswhicharesentoneatatime.TheTransportLayerkeepsuseschecksumtomakesureallthepacketsgetdeliveredproperly.NowtheTransportLayerpassestheenveloptotheInternetLayerwhichtakestheenvelopeandputsitinanevenbiggerenvelop,withmoreinformationwrittenonit:[ pictureofenvelopwithwritingonit ]TheinformationwrittenontheInternetLayerenveloponlycontainstheinformationtherouterneedstodeliverthedatatotheserveritissendingthedatato.ItcontainstheIPaddressoftheserverandtheIPaddressofthecomputermakingtherequest.ItalsocontainstheTTL,whichstandsfortimetoliveand( explainttl) Atthispoint,theenvelopisconsideredapacket.Thisfinalenvelopeissenttothebottomlayer,theNetworkLayer,whichuseshardwareandsoftwaretophysicallysendthedata.ThedataisreceivedbytheNetworkLayerontheserverscomputersandtheenvelopispassedinreverseorderuptheprotocolstackwitheachlayerremovinganenvelopuntiltheletterisrevealedattheApplicationLeveloftheserver.TheserverthengoesthroughthesameprocessthroughtheTCP/IPstack,sendinganHTTPrequestbacksignallingthattherequestwaseithervalidorinvalid.Iftherequestwasvalid,itstartssendingtheresourcestheclientneeds.

Itisimportanttorememberthatdatadoesnotgetsentallatonce,itgetsbrokendownintopackets.Thebottomlayerofthestack,theNetworkLayermaysendthousandsofpacketstoaclient,

Challenge

Chapter#TK.BringingItAllTogether

CreateaServer

InthissectionwearegoingtousethePythonsocketlibrarytocreateasimplewebserverandclient,usingPython’sbuilt-inlibraryforcreatingandmanagingsockets.Wearegoingtocreateaserverthatlistensforrequestsandrespondstothemwiththedata,eandaclientwecanusetomakethoserequests.

Awebservercreatesasocket,bindsittoaport,andthenrunsaninfiniteloop,respondingtorequestsastheycomethroughthesocket;whereasaclientsimplyopensupasocketand,connectstoaservertogettheinformationitneeds.WewillstartbybuildingaserverinPython.Thefirststepistoimportthesocketanddatelibraries:

importsocketimportdatetime

Firstwegettoday’sdate:today=str(datetime.datetime.today())

Nowwecancreateasocket:

s=socket.socket(socket.AF_INET,socket.SOCK_STREAM)

socket.AF_INETisanaddressfamilyspecifyingwhataddressesinyoursocketcancommunicatewith.AF_INETisusedforcommunicatingovertheinternet.ThereareotheraddressfamilieslikeAF_BLUETOOTHthatcanbepassedinforcommunicatingoverBluetooth.“socket.SOCK_STREAM”meanswewanttouseTCPtoensuredelivery.#factcheck

NextwebindoursockettoTCPport8888 :

s.bind(“”,8888)

Andsetthelengthofthequeue(aqueueisusedbecausemultiplerequestscancomeinatthesametimeandadatastructureisneededtoprocessthem):

s.listen(10)

Nowwecancreatetheserver ’sinfiniteloopwhichwaitsforaconnectionandsendsthedatebackasaresponse:

while True:connect,address=s.accept()resp=(connect.recv(1024)).strip()#limitrequestto1024bytesconnect.send(“receivedhttprequest”)#Closetheconnectionwhenwearefinished:connect.close()

Hereisourfullserver:

import socketimportdatetime

today=datetime.datetime.today()s=socket.socket(socket.AF_INET,socket.SOCK_STREAM)s.bind(("",8888))s.listen(5)

whileTrue:connect,address=s.accept()resp=(connect.recv(1024)).strip()connect.send(today)connect.close()

Youcantestthisserverbyrunningtheprogramandgoingtolocalhost:8888inyourbrowser.Youshouldseethedatewhenyoudo.

CreateaClient

Nowletscreateaclienttomakerequeststoourserver.Justlikecreatingawebserver,westartoutbycreatingasocket:

importsockets=socket.socket(socket.AF_INET,socket.SOCK_STREAM)

Toconnecttoourserver,wegetthenameofourlocalmachineandsetthevariableporttotheportourserverusessowecanuseitlater:

#getnameofourlocalmachinehost=socket.gethostname()

#setportport=8888

Nowwecanconnecttoourhostnameatport8888bypassinginatuplewithourhostnameandport:

s.connect((hostname,port))

Savetheresponseandclosethesocket:

msg=s.recv()s.close()

Printthemessagewereceived:

print(“{}”.format(msg))That’sallthereistoit.We’vebuiltafunctioningclient.Whenyourunourclient,youwillgetthedatefromourserver.

Challenge

Chapter#TK .Practice

Exercises

Read

0.0.http://stackoverflow.com/questions/2794016/what-should-every-programmer-know-about-security

PartVProgrammingforProduction

Chapter#TK . Testing

“Ifdebuggingistheprocessofremovingbugs,thenprogrammingmustbetheprocessofputtingthemin.”-EdsgerDijkstra

Whenyoustartbuildingsoftwareotherpeoplearegoingtouse,thecodeintheproductpeopleendupusingiscalledproductioncode.Whenyouputsoftwareintoproduction,itmeansyouputitlivewherepeopleareusingit.PartVofthisbookisabouthowyoushouldprogramwhenyourgoalistoputsomethingintoproduction;inthissectionofthebookwelearnaboutthesoftwaredevelopmentprocess,withafocusontesting.Wealsolearnsomebestprogrammingpractices.

TheWaterfallSoftwareDevelopmentProcess

Asoftwaredevelopmentprocessisawayof “splittingofsoftwaredevelopmentworkintodistinctphases(orstages)containingactivitieswiththeintentofbetterplanningandmanagement,” 63TheWaterfallmodelisa“ sequential (non-iterative)designprocess,usedinsoftwaredevelopmentprocesses,inwhichprogressisseenasflowingsteadilydownwards(likeawaterfall)throughthephasesofconception,initiation,analysis,design,construction,testing,production/implementationandmaintenance.” 64 Itismadeupoffivephases:PlanningandRequirementsAnalysis,DefiningRequirements,SystemDesign,ImplementationandDeployment,SystemTestingandSystemMaintenance.

InthefirstphaseoftheSystemsDevelopmentCycleyoudeterminewhatproblemyouwanttosolve.Youlookathowthenewsystemaffectsyourcurrentpriorities,youanalyzetheresourcesyouwouldneedtobuildthenewsystem,andyouthinkaboutthesystem’srequirements.Youcandoafeasibilitystudyatthisstage,lookingatwhetherthesystemisoperationally,economically,andtechnicallyfeasible,amongotherconsiderations.

IntheDefiningRequirementsphaseyoudefineanddocumentthesystemrequirements.Thisisdonebyworkingwiththeprojectstakeholders— “ anindividual,group,ororganization,whomayaffect,beaffectedby,orperceiveitselftobeaffectedbyadecision,activity,oroutcomeofaproject”. 61

IntheSystemDesignphase,differentdesignapproachesarediscussed,andafinalapproachisagreeduponandoutlinedinadocumentcalledaDesignDocumentSpecificationwhichoutlinesthe“ designapproachfortheproductarchitecture.” 62 Adesignapproach“clearlydefinesallthearchitecturalmodulesoftheproductalongwithitscommunicationanddataflowrepresentationwiththeexternalandthirdpartymodules(ifany).Theinternaldesignofallthemodulesoftheproposedarchitectureshouldbeclearlydefinedwiththeminutestof

thedetailsinDDS.”62 Inotherwords,youdecidehowyouaregoingtoarchitectthesystemyouarebuilding,andcommitittotheDDS.

Thisiswherethemagichappens.Afterthreestagesofplanning,codingbeginsintheImplementationandDeploymentphase.Theproductisbuiltinthisphase,followingtheDesignDocumentSpecification.

TheSystemTestingPhaseiswheretheproductistestedforbugsandtestedtomakesureitmeetstherequirementsoutlinedintheDefiningRequirementsstage.Thenextchapterofthisbookcoverstestingindetail.Oncetheproducthasbeentested,itisputintoproductionandreleasedtothepublic.Duringthisphasetheproductisliveandthesoftwareteamperformsanynecessarymaintenance.

OtherSoftwareDevelopmentProcesses

Thewaterfallmethodisoneofmanysoftwaredevelopmentprocesses—liketheincrementalmodel:“amethodofsoftwaredevelopmentwheretheproductisdesigned,implementedandtestedincrementally(alittlemoreisaddedeachtime)untiltheproductisfinished.”65TherearealsovariousimplementationsofthepopularAgilemethodologywhichyoucanfindoutmoreaboutbyreadingtheManifestoforAgileSoftwareDevelopment:http://agilemanifesto.org.

Whilenotasoftwaredevelopmentprocess,inthenextchapter,weutilizeTestDrivenDevelopment,adevelopment techniquewhereyoumustfirstwriteatest thatfailsbeforeyouwritenewfunctionalcode.”66 TestDrivenDevelopmenthelpsyouwritebetterprogramsbyforcingyoutothinkclearlyaboutwhatyouaredesigning.

Testing

Inthischapterwewillfocusontesting,partofthesoftwaredevelopmentprocess. Testingaprogrammeanscheckingthattheprogram“meetstherequirementsthatguideditsdesignanddevelopment,respondscorrectlytoallkindsofinputs,performsitsfunctionswithinanacceptabletime,issufficientlyusable,canbeinstalledandruninitsintendedenvironments,andachievesthegeneralresultitsstakeholdersdesire.”57 Inordertoachievethis,wewriteprogramstotestourprograms.Inmostcases,testingisnotoptional;youshouldconsidereveryprogramyouintendtoputintoproductionincompleteuntilyouhavewrittentestsforit,unlessyouhaveaverygoodreasonnotto.Ifyouwriteaquickprogramtodosomethinglikemanipulateafile,andyouneveraregoingtouseitagain,testingmightbeawasteoftime.Butifyouarewritingaprogramthatotherpeoplearegoingtobeusing,you

shouldwritetests.Assomeonesmartoncesaid,“Untestedcodeisbrokencode.”Inthischapter,wearegoingtogooversomeofthefundamentalsoftesting.Funfact—thewordcomputer“bug”originatedfromanincidentin1942,whereGraceMurrayHopperfoundandremovedamothstuckinaMarkAikenRelayCalculator.

Assertions

Assertionsarethefoundationoftests.AnassertionisastatementthataprogrammerexpectstobeTrue(inwhichcaseitdoesn’tdoanything)andraisesanexceptionifitis False .Pythonhasabuilt-inassert keywordforcreatingassertions.Hereisanexample:

x=1assertx==1>>

x=1assertx==2>>Traceback(mostrecentcalllast):File"/Users/coryalthoff/PycharmProjects/self_taught/st.py",line

2,in<module>assertx==1

AssertionError

Inthefirstexample,theconditionfollowing assert isTrue ,sonoerrorisraised.Inthesecondexample,theconditionfollowingassertisFalse ,andanAssertionError israised.Assertionsareusedtocheckwhetherornotatest(whichwewilllearnhowtowriteshortly)passed.IfanAssertionError israised,thetestfailed,otherwiseitpassed.

TypesofTests

Testingisusuallydoneinfourdifferentphases:unittesting,integrationtesting,systemtesting,andacceptancetesting.Inthissection,wewillbrieflyexploreeachofthesephases.

Theunittestingphaseinvolveswritingunitteststhattestindividualpiecesofcodesuchasfunctions,methodsandclasses.Eachunittesttestsoneaspectofapieceofcodewithanassertion.Sayforexampleyouhaveafunctionthatprintswhateverstringyoupassit.Oneunittestmightpassthefunctionthestring“Hello” andtestthat “Hello” wasprinted;whereasanotherunittestmightpassthefunctionanintegertomakesurethefunctionisabletoproperlyhandleaninputthatisnotastring.Unittestsshouldtestthegeneral-usecaseforallofyourfunctionsclassesandmethods;checkingwhathappenswhentheyreceiveinputvaluesyouweren’texpecting,andtestboundaryconditions—whenthingslikealistgetbigorfull.

Youcanwriteyourunittestswithinaunittestingframework—aprogramforcreatingandstructuringunittests,likePython’sbuilt-inunittestingframeworkcalledunittest,whichcomeswithdifferentassertionmethodsthatletyoueasilytestdifferentconditions.Integrationtestingisperformedafterunittesting.Whileunittestingtestseachindividualpieceofcodeinamodule,integrationtestingteststomakesurethedifferentmodulesinaprojectworkwitheachother.Forexample,sayyouarebuildingabankingapplicationandyouhavetwomodules:transfer—fortransferringmoney—andbalance—forshowingthecustomer'sbalance.Anintegrationtestmightcheckthatwhenthetransfermodulemoves1,000dollarsfromacustomer'saccount,andthebalancemodulecorrectlyreducesthecustomer ’sbalanceby1000dollars.59Systemtestingteststheentiresystem. Therearemanydifferenttypesoftestsusedinsystemstesting:graphicaluserinterfacetesting—whichteststhepartofproductyoucansee;usabilitytesting—whichteststhatpeoplecanfigureouthowtousetheproduct;andsoftwareperformancetesting—whichteststoseehowtheproductperformsunderaheavyworkloadsuchasalargenumberofusers;amongothers.Finallythelastphaseofthetestingcycleisacceptancetesting—whichcheckstomakesurethesoftwaremeetstherequirementsagreeduponbytheprojectstakeholders.Acceptancetestingisnotdoneprogrammatically—itisdonebypeoplewhomakesurethetheproductrequirementslaidoutintherequirementsdocumentaremet.Therestofthischapterwillfocusonunittesting,becauseasanewsoftwareengineeronateam,youwillberesponsibleforwritingunittests—butyoumostlikelywillnotberesponsibleforintegration,systemsandacceptancetesting.

TDD

Aswelearnedinthepreviouschapter,TDDstandsfortestdrivendevelopment,andisasoftwaredevelopmenttechniquethathelpsyoudesignbetterprograms.Whenyoufollowtestdrivendevelopment,youwriteyourunittestsbeforeyouwriteyourprogram.FollowingTDDforcesyoutobreakoutofthepatternofputtingwritingunittestsoffuntiltheendofyourdevelopmentcycle,andthendecidingnottowritethem.Italsoguaranteesyouwillhaveunitteststhroughoutyourdevelopmentcycle.Lastly,TDDhelpsyoutodesignbettersoftwarebyforcingyoutothinkaboutthedesignrequirementsofyourprogrambywritingyourtestsfirst.

Inthissectionwearegoingtolearntowriteunittestsbycreatingastack usingTDDandPython’sunittesttestingframework.Wewillstartourdevelopmentprocessbywritingunitteststhatwillfail(butwouldworkifourstackwasproperlydesigned),andthenwritingcodetomakethetestspass.

Tobettervisualizethetestsweneedtocreate,wWewillstartbydefiningastackthatdoesn’tdoanythingtobettervisualizethetestsweneedtocreate.InthissectionIassumeyou

rememberhowastackworks(Ifyouforgethowastackworks,pleaserevisitthechapterDataStructures&Algorithms).

importunittest

classStack : def__init__ ( self): self .stack=[]

defpush ( self, item): pass

def pop( self): pass

def peak( self ): pass

def is_empty ( self ): pass>>>

We’vedefineda Stack class,however,noneofthemethodsinourclassactuallydoanything.Beforewedefinethemethodsforour Stack class,wearegoingtowriteallofourtests .

classStackTests(unittest.TestCa se):

defsetUp( self ): self.stack=Stack()

deftearDown( self): delself.stack

deftest_is_empty( self ): self.assertTrue( self.stack.is_empty())

deftest_push( self): self.stack.push( 100 ) self.assertFalse( self.stack.is_empty())

deftest_peak( self): self.stack.push( 'test' ) self.assertEqual( self .stack.peak(),'test' )

deftest_pop( self): self.stack.push( 10.1 )

self .stack.pop() self .assertTrue( self.stack.is_empty())

deftest_pop_value( self): self .stack.push( 'test_value')value=self .stack.pop() self .assertEqual(value ,'test_value' )>>>

Whenyouwriteunittestswiththeunittest framework,youstartbydefiningaclassthatinheritsfrom unittest.TestCase—inthisourclassiscalled StackTests .Theunittestframeworkusestheclassthatinheritsfrom unittest.TestCase toruntests;witheachmethodyoudefineinyourclass—aslongasitstartswith test —runsasanisolatedtest.Atestthatpasseswillnotraisean AssertionError ,andatestthatfailswill.Ifyourunourprogram,youwillseetheresultsofeachtest( theyallwillfailwithAssertionErrors ).

Hereisamoredetailedexplanationofthetestingcodeyouwrote.Thefirsttwomethods setUp andtearDowndonotstartwithtestbecausetheyarenottests:theyaremethodsinheritedfrom unittest.TestCase usedtohelpsetupourtests.

defsetUp( self):self .stack=Stack()

deftearDown( self ):delself .stack

setUp runsbeforeeachtest,and tearDown runsaftereachtest.InthiscaseweusesetUptocreateanew Stackobjectbeforeeachtest,anduse tearDowntodeleteitaftereachtest.Wedothisinordertomakesurewehaveabrandnew Stack objectineachtest,s othetestsdon’tinterferewitheachother.

Inthefirsttestwedefine,test_is_empty() ,usingtheassertionmethodassertTrue()whichtakesaparameterandraisesanAssertionError iftheparameterevaluatestoFalsebecauseifwedon’tputanythinginourstack,itshouldbeempty.

deftest_is_empty( self ):self .assertTrue( self.stack.is_empty())

Ournexttestis test_push() .Itcallspush() onourstackandpassesin100 .Weusetheassertionmethod assertFalseandpassinself.stack.is_empty() becauseafterpushingsomethingtoourstack,thestackshouldnolongerbeempty: self.stack.is_empty() shouldbeFalse .

deftest_push( self):self .stack.push( 100 )self .assertFalse( self.stack.is_empty())

Inournexttest, test_peak(),wepush ‘test’ontothe Stack anduseassertEqual() tocheckthatself.stack.peek() returnsthevaluewepushedtoourstack,inthiscaseself.stack.peek()and‘test’ shouldbeequal.

def test_peak(self ): self .stack.push( 'test' ) self .assertEqual( self.stack.peak() ,'test' )

Ournexttestis test_pop().Wepush 10.1 toourstack,call self.stack.pop() andusetheassertionmethodassertTrue() tocheckthatourpop() methodsuccessfullyremovedanitemandourstackisnowempty.

def test_pop(self ): self .stack.push( 10.1 ) self .stack.pop() self .assertTrue( self.stack.is_empty())

Ourfinaltest, test_pop_value(), pushesthestring ‘test_value’ toourstackandusestheassertionmethodassertEqual() tocheckthat pop() returns‘test_value’ .

def test_pop_value( self): self .stack.push( 'test_value')value=self .stack.pop() self .assertEqual(value,'test_value' )

Runourtests.Y ouwillbenotifieda llfivetestsfailed byraisingfiveAssertionErrors.ThisisbecausewehavenotdefinedanyofthemethodsinourStackclass.Ourmethodsdon’tdoanythingandsoallofourtestsfail.OneoftheadvantagesofTDDisthatwritingyourtestsfirsthelpsclarifyyourthinking.WenowknowexactlyhowweneedtodefineeachmethodinourStack classinordertopasseachtest.Hereiswhatweneedtodo:

class Stack: def __init__ ( self ): self .stack=[]

def push( self,item): self .stack.append(item)

def pop(self ): return self.stack.pop()

def peek( self): return self.stack[- 1]

def is_empty(self ): if len ( self .stack)>0:

returnFalse returnTrue

>>>Nowthatwe’vedefinedthemethodsinourStack ,whenyourunourtestsagain,thetestswillallpass.We’vetestedourclass,ourmethodsandgeneralusecases;andnowweneedtotestunexpectedusecases,badinputvalues,andboundaryconditions.Anexampleofanunexpectedusecaseiscallingpoponanemptystack,whichwillcauseanerror.Tofixthis,firstweshouldwriteanewtest.

classStackTests (unittest.TestCase):

defsetUp( self ): self.stack=Stack()

deftearDown ( self ): delself.stack

deftest_empty_pop( self ): withself.assertRaises( IndexError ): self.stack.pop()

Ournewtest test_empty_pop usestheassertionmethodself.assertRaises() withthe withstatementtotestifanexceptionisraised.Thecodeinsidethewith statementisexpectedtoraisetheexceptionpassedtoassertRaises.Iftheexceptionisraisedthetestpasses;iftheexceptionisnotraisedanAssertionError israisedandthetestfails.Nowwecanfixourstacktohandleemptypops:

classStack : def__init__ ( self ): self.stack=[]

defpush( self,item): self.stack.append(item)

defpop ( self): ifnotself.stack: raiseIndexError ( "Cannotpopfromemptystack") returnself.stack.pop()

defpeek( self ): returnself.stack[- 1]

defis_empty ( self ): iflen ( self .stack)>0 : returnFalse returnTrue

>>>

Weneedtothinkaboutbadinputvalues.Inthisexample,therearenobadinputvaluesbecauseanyobjectinPythoncanbeaddedtoalist,andthereforecanbeaddedtoourstack.However,youalwayswanttotakethetimetoatleastthinkaboutanyinputvaluesthatcouldbreakyourprogram.Finally,weshouldthinkaboutboundarycases:the“behaviorofasystemwhenoneofitsinputsisatorjustbeyonditsmaximumorminimumlimits.”58 InPython,thereisnolimittohowmanyobjectswecanputinalist,thereforethesizeofourstackisonlylimitedbytheamountofmemorythecomputerthatcreatedithas.However,someprogramminglanguageslimitthenumberofobjectsyoucanputinalist,andifthatwerethecaseinPython,wewouldhavetowriteatestforthatcondition.

WritingGoodTests

Goodtestsarerepeatable.Thatmeanswhenyourunyourtests , theyshouldworkinanyenvironment—ifyouwriteatestonOSX,itshouldalsoworkonWindowswithouthavingtomakeanychangestotothetest.Anexampleofviolatingthiswouldbeincludinghardcodeddirectorypathsinyourtest.WindowsandOSXusedifferentslashesfordirectorypaths—soyourtestinaWindowsenvironmentwouldnotworkinanOSXenvironment.Thismeansthetestisnotrepeatable,andneedstoberewritten.Testsshouldalsorunquickly.Testsneedtorunoften;trynottowriteteststhattakealongtimetorun.Finallyyourtestsshouldbeorthogonal—onetestshouldnotaffecttheother.

CodeCoverage

Codecoverageisthethetotalnumberoflinesofcodeinyourprojectcalledduringyourtestsdividedbythetotalnumberoflinesofcodeinyourproject.Codecoveragedoesnotmeasuretheefficiencyofyourtests,butisusefulforfindinguntestedpartsofyourcode.Yougenerallywanttohavecodecoverageabove80%.Ifyourcodecoverageislow,itmeansyouhavenottestedenoughofyourcode.TheprofessionalversionofPyCharm(theonethatcostsmoney)isintegratedwithatoolforanalyzingcodecoverage.IfyouusePyCharm’sfreeCommunityEdition,youareinluckbecausethetooltheprofessionalversionisintegratedwithiscalledcoverage.pyandisfreetouse.Thedocumentationforcoverage.pyisavailableat:https://coverage.readthedocs.io .

TestingSavesTime

It’seasytogetlazyandskiptesting,justifyingyourlazinessbysayingyoudon’thavetimetowritetests.Counterintuitively,takingthetimetowritetestswillsaveyouasubstantialamountoftimeinthelongrun.Thereasonisbecauseifyoudon’twritetests,youwillenduptestingyoursoftwaremanually—runningyourprogramyourselfwithvariousdifferentinputsandunderdifferentconditionstoseeifanythingbreaksasopposedtotestingyourprogramprogrammatically.Whileyoushouldtestyourentireprogrambytestingitmanuallyandlookingforbugs,youdon’twanttosolelyrelyonthis.Spendingyourtimemanuallytestinginputsandconditionswhenyoucouldeasilyautomatetheprocessisahugewasteoftime.Finally,ifyoucomebacktotheprojectinamonth,youwon’tbeabletorememberthedifferenttestsyouweremanuallyrunning. Testingisanimportantpartofprogramming.Mostsoftwareteamshaveatleastonepersondedicatedtotesting.GettingintothehabitoffollowingTDDwillimproveyourcodebymakingsureyoualwayswritetestsandthereforedecreasethenumberoferrorsinyourcode,andbyhelpingyoutothinkcarefullyabouthowyoudesignyourprograms.

Challenge

WriteunittestsfortheHangmanprogramwebuiltinPartI.

Wheneveryouarefollowingprogramminginstructions,andsee$,itmeanswhateverfollowsisacommandthatyoushouldtypeintothecommandline(noneedtotypethedollarsign). Programminglanguageshaveconventionsaswell.Conventionsarerulesgenerallyfollowedbythecommunityusingthelanguage.

Chapter#TK.BestProgrammingPractices

“Alwayscodeasiftheguywhoendsupmaintainingyourcodewillbeaviolentpsychopathwhoknowswhereyoulive.”-MartinGolding

Inthischapterwewillcoverafewgeneralprogrammingprinciplesthatwillhelpyouwriteproduction-readycode.Manyoftheseprinciplesoriginatedintheexcellentbook“ThePragmaticProgrammer”byAndyHuntandDaveThomas,abookthatdramaticallyimprovedthequalityofmycode.

WriteCodeAsALastResort

Yourjobasasoftwareengineeristowriteaslittlecodeaspossible.Whenyouhaveaproblemyouneedtosolve,yourfirstthoughtshouldnotbe“HowcanIsolvethis?”Itshouldbe,“HassomeoneelsesolvedthisproblemalreadyandcanIusetheirsolution?”Ifyouaretryingtosolveacommonproblem,chancesaresomeoneelsehasalreadysolvedit.Startbylookingonlineforasolution.Onlyafteryou’vedeterminednooneelsehasalreadysolvedtheproblemshouldyoustartsolvingityourself.

DRY

DRYisaprogrammingprinciplethatstandsforDon’tRepeatYourself.FollowingDRYiseasy:ifyouarewritingcodeandfindyourselfrepeatingthesamecode—stop.Donotrepeatyourself.Ifyoufindthatyouarecopyingpiecesofcode,pastingthemsomewhereelseinyourprogram,andmakingsmallchangestoittocreatenewcode—stop.Donotrepeatyourself.Wewillillustratethiswithaprogramthatmakeschangestoalistofwords.

defcapitalize_item(word , word_list): forindex , itemin enumerate(word_list): ifitem==word:word_list[index]=word_list[index].capitalize()

def change_letter (word , word_list , old_letter ,new_letter): forindex , itemin enumerate(word_list): ifitem==word:word_list[index]=word_list[index].replace(old_letter , new_letter)

words=['Programming' , 'is' , 'fun' ]upper( 'Programming' , words)print(words)change_letter( 'fun' , words , 'u' ,'$' )print(words)

>>['PROGRAMMING','is','fun']>>['PROGRAMMING','is','f$n']

BothfunctionsusePython’sbuilt-inenumerate function,whichtakesalistasaparameter,andallowsyoutouseafor looptoeasilycapturethecurrentindexofthelist,aswellasthecurrentiteminthelist.“Index”referstoanitem’spositionwithinalist.Ourprogramworks—butifyoulookcloselyatthecode—youwillseebothofourfunctionslookforawordinalisttoreplaceitwithsomethingnew.Insteadofhavingcodetosearchforawordinbothfunctions,weshouldcreateonefunctionthatreturnstheindexofthewordwearelookingfor(Pythonhasabuilt-infunctionindex()thatfindstheindexofastringinalist,butforthesakeofthisexamplewearenotusingit).Hereishowweshouldrefactorourcodetoavoidrepeatingourselves:

def find_index(word ,word_list):for index ,iteminenumerate (word_list):if item==word: return index

def upper(word ,word_list):index=find_index(word ,word_list)word_list[index]=word_list[index].upper()

def change_letter(word ,word_list ,old_letter ,new_letter):index=find_index(word ,word_list)word_list[index]=word_list[index].replace(old_letter ,new_letter)

words=['Programming','is','fun']upper('Programming',words)print(words)change_letter('fun',words,'u','$')print(words)

>>>['PROGRAMMING','is','fun']['PROGRAMMING','is','f$n']>>>

Bycreatinganewfunctionthatreturnstheindexofaword,wearenolongerr epeatingcode.Ifwedecidetochangethewaywesearchforanindex,weonlyneedtochangeourfind_indexfunction,insteadofchangingthecodetofindanindexinmultiplefunctions.

Orthogonality

OrthogonalityisanotherimportantprinciplepopularizedbythebookthePragmaticProgrammer .TheauthorsAndyHuntandDaveThomasexplain,“Incomputing,thetermhascometosignifyakindofindependenceordecoupling.Twoormorethingsareorthogonalifchangesinonedonotaffectanyoftheothers.Inawell-designedsystem,thedatabasecodewillbeorthogonaltotheuserinterface:youcanchangetheinterfacewithoutaffectingthedatabase,andswapdatabaseswithoutchangingtheinterface.” 16Putthisinpracticebyrememberingthatasmuchaspossible,“ashouldnotaffectb”.Ifyouhavetwomodules—moduleaandmoduleb—moduleashouldnotmakechangestothingsinmoduleb,andviceversa.Ifyoudesignasystemwhereaaffectsb;whichaffectsc;whichaffectsd;thingsquicklyspiraloutofcontrolandthesystembecomesunmanageable.

EveryPieceOfDataShouldHaveOneRepresentation

Thisisbestexplainedwithanexample.Sayyouarebuildingaproject,andinthatprojectyouareusingTwitter ’sAPI(applicationprogramminginterface)—aprogramthatgivesyouaccesstodatafromawebsitelikeTwitter.TwitterprovidesamoduleyoucandownloadonpiptoquerytheirAPIfordata.InordertouseTwitter ’sAPI,youhavetoregisterforanAPIkey—astringyousendtoTwitterwhenyouusetheirAPIsotheycanverifyitisyou.

Onceyou’veobtainedyourAPIkeyfromTwitter,youstartusingtheAPIintwofunctions.Thefirstfunctiongetsdatafromcelebrities,andthesecondfunctiongetsdatafromnoncelebrities.BothfunctionsneedtheAPIkeyinordertouseTwitter ’sAPI.Hereisanexampleofwhatthiscouldlooklike:

#WARNINGthiscodedoesnotactuallywork

import twitter_apiimportcelebrity_listimportregular_list

defcelebrity_data ():api_key='11330000aazzz22'return twitter_api.get_data(api_key , celebrity_list)

def people_data ():api_key= '11330000aazzz22' returntwitter_api.get_data(api_key , regular_list)

Thisisamadeupexample thatwon’tactuallywork,butthegistisweareusingamadeupmodulecalledtwitter_api tocallthefunctionget_datawithourapikeyandalistofpeople.AfewmonthsgobyandyouendupgettinganewAPIkeyfromTwitter.Yougotothecelebrityfunctionandchangethevariableapi_key tothenewAPIkey.It’sbeenalongtimesinceyouwrotethiscode,andyoucompletelyforgettheAPIkeyisusedinthesecondfunction.YouputyourcodeintoproductionandaccidentlyleavingtheoldAPIkeyinthesecondfunction;everythingbreaks,andyougetfired.Youcould'veavoidedyourtragicfatebyfollowingtherulethateverypieceofdatashouldhaveonerepresentation.ThecorrectwaytohandlethissituationistomakeaPythonfilewithavariablecalledapi_key.Thisiscalledaconfigurationfile.Yourprogramshouldimportapi_key fromthisfile,andbothfunctionsshoulduseit.Thisway,thepieceofdata(theAPIkey)isonlyrepresentedonce.Thatway,nomatterhowmanyplacestheAPIkeyisused,ifyouhavetochangetheAPIkey,youonlyneedtochangeitinoneplace,theconfigurationfile,andthepreviouslydiscusseddisasterwillbeav erted.

FunctionsShouldDoOneThing

Everyfunctionyouwriteshoulddoonething,andonethingonly.Ifyoufindyourfunctionsgettingtoolong,askyourselfifthefunctionyouarewritingisaccomplishingmorethanonetask.Limitingfunctionstoaccomplishingonetaskoffersseveraladvantages:yourcodewillbeeasiertoreadbecausethenameofyourfunctionwilldescribeexactlywhatitdoes;andifyourcodeisn’tworkingitwillbeeasiertodebugbecauseeveryfunctionisresponsibleforaspecifictask,soyoucanquicklyisolateanddiagnosethefunctionthatisn’tworking.AsRyanSingersays,“Somuchcomplexityinsoftwarecomesfromtryingtomakeonethingdotwothings.”

UseDummyData

WhileIwasateBayIwasgivenanassignmenttofixanerrorinourcode.TheprogramIwasdebuggingprocessedalargetextfileandtookfiveminutestorun.Iwouldmakeachangetotheprogramtotrytogetsomeinformationaboutwhatwaswrong,runtheprogram,andwaitfiveminutesfortheresults.Iwasnotmakinganyprogress,becauseIhadtowaitfiveminuteseverytimeImadeachange,andthatquicklyaddedup.Ifinallytookthetimetosubstitutethelargetextfilewithdummydata—fakedatamyprogramcouldusebutwouldonlytakeafewsecondstoprocess.ThiswayIcouldstilllookforthebuginthe

program,butmuchfaster.Takingthetimetosetupdummydata—evenifittakesyoutwentyminutes—w illquicklypayoffbyshorteningyourdebugcycle.

IfIt’sTakingTooLongYouAreProbablyMakingaMistake

Ifyouarenotworkingonsomethingobviouslycomplexlikeworkingwithalargeamountsofdata,andyourprogramistakingaverylongtimetoload,assumeyouaredoingsomethingwron g.

Logging

Loggingisthepracticeofrecordingdatawhenyoursoftwareruns.Youcanuseloggingtohelpdebugyourprogram,andtogainadditionalinsightintowhatwhathappenedwhenyourprogramran.Pythoncomeswithagreatloggingmodulethatletsyoulogeithertotheconsoleorafile.

Whensomethinggoeswronginyourprogram,youdon’twantittogounnoticed—youshouldloginformationaboutwhathappenedsoyoucanreviewitlater.Loggingisalsousefulforcollectingandanalyzingdata.Forexample,youmightsetupawebserversettologdata—includingthedateandtime—everytimeitreceivesarequest.Youcouldstoreallofyourlogsinadatabase,andcreateanotherprogramtoanalyzethatdataandcreateagraphdisplayingthetimesofdayyourwebsiteisvisitedthemost.

Goodprogrammersuselogging,summarizednicelybyHenrikeWarnewhenhesaid“Oneofthedifferencesbetweenagreatprogrammerandabadprogrammeristhatagreatprogrammeraddsloggingandtoolsthatmakeiteasytodebugtheprogramwhenthingsfail.”YoucanlearnhowtousePython’sloggingmoduleathttps://docs.python.org/3/howto/logging.html.

DoThingsTheBestWayTheFirstTime

Ifyouareinasituationwhereyouareprogrammingandyouthink,“Iknowthereisabetterwayofdoingthis,butI’minthemiddleofcodinganddon’twanttostopandfigureouthowtodoitbetter.”Stop.Doitbetter.

FollowConventions

Takingthetimetolearntheconventionsofthenewprogramminglanguageyouaretryingtolearnwillhelpyoureadcodewritteninthenewlanguagefaster.Pep8isasetofguidelinesforwritingPythoncode,andyoushouldreadit.It’savailableat:https://www.python.org/dev/peps/pep-0008.UseaPowerfulIDE

AnIDE,orInteractiveDevelopmentEnvironment,isaprogramyouusetowriteyourcode.Thusfar,we’vebeenusingIDLE,theIDEthatcomeswithPython.However,IDLEisjustoneoptionofmanydifferentIDEsavailable,andIdonotrecommendusingitlongterm,becauseitisnotverypowerfulcomparedtootherIDEs.Forexample,ifyouopenupaPythonprojectinabetterIDE,therewillbeadifferenttabsforeachPythonfile.InIDLE,youhavetoopenanewwindowforeachfile,whichinbigprojectsthisquicklygetstediousandit’sdifficulttonavigatebackandforthbetweenfiles.

IuseanIDEcalledPyCharmcreatedbyJetBrains.Theyofferafreeversionaswellasaprofessionalversion.Eitheronewillwork.SublimeisanotherpopularIDE.InthischapterIwillbegoingoversomeofthefeaturesIuseinJetBrainsIDEthatincreasesmyproductivity.BecauseanyIDEisliabletochangeitscommandsatanytime,therearenoexamplesinthischapter.InsteadIdescribesomeofthefeaturesPyCharmhas,togiveyouanideaofwhatanIDEiscapableof,soyouwon’twastetimedoingthingsmanuallythatyoucanquicklydowithanIDE.Iputatutorialattheselftaughtprogrammer.io/ide soIcankeepituptodate. Ifyouseeavariable,functionorobjectbeingusedandyouwouldliketoseeitsdefinition,PyCharmhasashortcuttojumptothethecodethatdefinedit(evenifitisinadifferentfile).Thereisalsoashortcuttojumpbacktothepageyoustartedfrom.PyCharmhasafeaturethatsaveslocalhistorywhichhasdramaticallyimprovedmyproductivity.PyCharmwillautomaticallysaveanewversionofyourprojecteverytimeitchanges.ThismeansyoucanusePyCharmaslocalversioncontrolsystembutwithouthavingtopushtoarepository.Youdon’thavetodoanything,ithappensautomatically.BeforeIknewaboutthisfeature,Iwouldoftensolveaproblem,changethesolution,andthendecideIwantedtogobacktotheoriginalsolution.IfIdidn’tpushtheoriginalsolutiontoGitHub,theoriginalsolutionwaslonggone,andIwouldhavetorewriteitagain.Withthisfeature,youcansimplyjumpbackintime10minutes,andreloadyourprojectexactlyhowitwas.Ifyouchangeyourmindagain,youcanjumpbackandforthbetweenthedifferentsolutionsasmanytimesasyouwant.Inyourworkflowyouareprobablycopyingandpastingcodealot,movingitfromonelocationonapagetoanother.InPyCharm,insteadofcopyingandpasting,youcanmovecodeupanddownonthepageyouareon.

PyCharmisintegratedwithpopularversioncontrolsystemslikeGitandSVN.Insteadofhavingtogotothecommandline,youcanuseGitfromPycharm.ThefewertripsyouhavetomakebackandforthbetweenyourIDEandthecommandline,themoreproductiveyouwillbe.PyCharmalsohasabuiltincommandlineandPythonShell.

InPycharmsProversion,thereisabuiltintoolforconnectingtorelationaldatabases,suchasMySQL.Ifyouareusingarelationaldatabase,thisisahugetimesaver.RememberingSQLsyntaxisdifficult,havingautocompleteforyourSQLisagamechanger.PyCharmhasapowerfuldebugger thatletsyoupausetheexecutionofaprogramatcertainpoints,checkthestatusofdifferentvariables,andwalkthroughaprogramstepbystep.

CodeReviews

Acodereviewiswhensomeonereadsyourcodeandgivesyoufeedback.Youshoulddoasmanycodereviewsasyoucan—especiallyasaself-taughtprogrammergettingstarted.Evenifyoufollowallofthebestpracticeslaidoutinthischapter,youarestilldoingthingswrong—youneedsomeonemoreexperiencedthanyoutoreadoveryourcodeandtellyouthemistakesyouaremakingsoyoucanfixthem.CodeReviewonStackExchangeisawebsitewhereyoucangetacodereviewfromacommunityofprogrammers.AnyonecangoonCodeReviewandposttheircode.OthermembersoftheStackExchangecommunityreviewyourcodeandgiveyoufeedbackaboutwhatyoudidwellandofferhelpfulsuggestionsonhowyoucanimprove.

Security

Securityisaneasysubjectfortheself-taughtprogrammertoignore.Youprobablywon’tbeaskedaboutsecurityinyourinterview,andsecurityisnotimportantfortheprogramsyouwritewhileyouarelearningtoprogram.However,onceyougetyourfirstjobprogramming,youaredirectlyresponsibleforthesecurityofthecodeyouwrite.Thelastthingyouwantisforyourcompany'sdatabasetogethackedbecauseyoumadeamistake.Bettertolearnsomepracticaltipstokeepyourcodesafenow.InthechapterTheCommandLinewelearnedtousesudotoissueacommandastherootuser.Imentionedyoushouldneverrunaprogramassudo.Youshouldalsodisablerootloginsifyouaremanagingthesystem.Thereareseveralreasonsyoushoulddothis:everyhackerisawarethereisarootaccountandsoitisaneasytargetwhenattackingasystem(henceitshouldbedisabled),ifyouareloggedinasrootyoucanirreversiblydamagethesystemyouarerunning,andifyourunaprogramasrootanditgetstakenoverbyahacker,thecompromisedprogramnowhasrootaccess.

Thereareseveralkindsofmaliciousattacksthatrelyonexploitingprogramsthatacceptuserinput,soyoushouldalsoassumealluserinputismaliciousandprogramaccordingly.AnSQLinjectionisatypeofattackthatoccurswhenausersubmitsinputtoa

programandaddsSQLtoit,allowingthemtoexecuteSQLinyourdatabase.Forinstance,youmighthaveacarwebsite,andonthatwebsiteyouwanttoreturndetailsaboutaspecificcarauserenters.Youexpecttheusertoprovideyouwiththenameofacar,suchas“NissanLeaf”.Onceyoureceivethenameofthecarfromtheuser,youwritemorecodetoqueryaMySQLtablecalledcarstodisplaydetailsaboutthecar.Youusethefollowingquery:

SELECT*FROMcarWHEREname=query_data;

Thevariablequery_datarepresentsthedatayoucollectedfromtheuser.Iftheuserenterssomethinglike“NissanLeaf”likeyouexpected,thisworksfine.Theproblemistheusercanenteranythingtheywant.Amalicioususercouldentersomethinglikethisintothewebsite:

“Ferrari”;DROPTABLEcars;

ThiswouldcauseyourprogramtoqueryMYSQLwiththefollowing,validSQL:

SELECT*FROMcarWHEREname=“Ferrari”;DROPTABLEcars

ThequerywouldgetallofthedataforacarnamedFerrari,thenitwoulddeleteyourdatabasetable—wipingoutallofyourdata.Tocombatthis,youassumealluserinputismalicious,andinsteadofwritingrawSQL—youuseanORMlikeSQLAlchemyoranotherlibrarythatprotectsagainstSQLinjections.Youdon’twanttowriteasolutiontothisproblemyourselfbecauseyoucouldtooeasilymakeamistake,whichsomeonecouldexploit.

Afinalstrategyforkeepingyoursoftwaresecureistominimizeyourattacksurface—thedifferentareasofyourprogramwhereattackerscouldextractdataorattackyoursysteminsomeway.Bymakingyourattackareaassmallaspossible,youcanreducethelikelihoodofvulnerabilitiesinyourprogram.Herearesomestrategiesforminimizingyourattacksurface:avoidstoringconfidentialdataifyoudon’thaveto,giveusersthelowestlevelofaccessyoucan,useasfewthirdpartylibrariesaspossible(thelesscodethelessamountofpossibleexploits),andgetridoffeaturesthatarenolongerbeingused(lesscodelessexploits).

Avoidinglogginginastherootuseronyoursystem,nottrustinguserinputandminimizingyourattacksurfaceareimportantstepstomakingsureyourprogramsaresecure.However,thesearejuststartingpoints.Youshouldalwaystaketimetotrytothinklikeahacker.Howwouldahackertrytoexploityourcode?Thiscanhelpyoufindvulnerabilitiesyouotherwisewould’veoverlooked.Thereisalotmoretolearnaboutsecuritythanwecancoverinthisbook,somakeanefforttoalwaysbethinkingandlearningaboutsecurity.BruceSchneiersaiditbest—“Securityisastateofmind”.

Challenge

Reviewaprogramyou’vewrittenandseeifyoufollowedtheprogrammingpracticesoutlinedinthischapter.

Chapter#TK.BringingItAllTogether

“Idon'tcareifitworksonyourmachine!Wearenotshippingyourmachine!”—VidiuPlaton

Congratulationsonmakingittothefinalcodingexerciseofthisbook.Inthischapterwearegoingtocreateaprogramthatscrapesalyricspageonlyrics.wikia.comandcreatesawordcloudwiththelyrics.Ifyou’veneverseenawordcloud,itisanimagederivedfromapieceoftextinwhichthesizeofeachwordcorrespondstohowmanytimesitappearedinthetext.[pictureofawordcloud]WewilldevelopourprogramusingtheWaterfallmodel andtestdrivendevelopment.

WordCloudLyrics

WebeginoursoftwaredevelopmentprocessinthePlanningandRequirementsAnalysisphase.First,wedefinetheproblemwewanttosolve:ourinabilitytoscrapeasonglyricswebsiteandturnthelyricsintoawordcloud.Weneverwanttowritecodeunlesswehaveto,sothefirstthingwedoischeckifsomeonehasalreadysolvedthisproblem.SomeGooglesearchingrevealsthisproblemhasbeensolved,andthecodeisavailableonGitHub.Normallywewouldusetheavailablesolutionifitmeetsourrequirements,andthesoftwaredevelopmentprocesswouldend,butforthesakeoflearningwecanforgetwesawthesolution,andconcludenoonehassolvedthisproblem.

NextweentertheDefiningRequirementsphase,wherewegooverourproject’srequirements,andtheresourceswewillneedtocompleteit.Ourprojectrequirestwomainpiecesoffunctionality:theabilitytoscrapedatafromawebsite,andtheabilitytocreateawordcloud.We’vealreadyscrapeddatafromGoogleNews,soweknowwehavethetoolsandexpertisetoscrapesonglyricsfromawebsite.Whatwedon’tknowishowtobuildawordcloud.Buildinganicelookingwordcloudisnoteasy,soifsomeonehasnotalreadysolvedthisproblemforus,itisgoingtosignificantlyincreasethetimeneededtofinishthisproject.Luckily,aGooglesearchrevealsthereisaPythonmodulecalledword_cloudthatletsyoueasilycreatebeautifulwordclouds,sowecanconcludeweshouldbeabletocompletethisprojectinareasonableamountoftime.Wealsohavetodefinetherequirementsforourproject—weneedtobeabletopassourprogramaurlthathassonglyricsfromlyrics.wikia.com,andourprogramneedstogenerateawordcloud.Wefinishthisphasebywritingourrequirementsdowninarequirements.txtdocument.

WearenowinthethirdphaseoftheWaterfallModel—SystemDesign—wherewedecidehowoursystemwillbearchitected.Ourprogramwillconsistoftwofunctions—onetogetthelyricsfromthewebsite,andtheothertocreatethewordcloud,whichwillalsobethemainfunctionweusetorunourprogram.

Afterdesigningoursystem,wearereadytoentertheImplementationandDeploymentphaseandstarttocode.SincewearefollowingTestDrivenDevelopment,wewillstartbywritingourtests.Weknowourprogramneedstodotwothings:getlyricsfromlyrics.wikia.com,andcreateawordcloud;sowewillstartbywritingatestforeachofthesepiecesoffunctionality.

importunittestimportos

defget_lyrics ():pass

defcreate_wordcloud ( wiki_url):pass

classTestBIAT (unittest.TestCase):def test_lyrics ( self ): """Testthatastringgetsreturnfromget_lyrics()""" self .assertIsInstance(get_lyrics() ,str )

def test_wordcloud_creation ( self ): """Testthatanewfileiscreatedwhencreate_wordcloud()is

called""" filecount_before= len (os.listdir())create_wordcloud( 'http://lyrics.wikia.com/wiki/The_Beatles:Lucy_In_The_S

ky_With_Diamonds' ) self .assertEqual(filecount_before+1 , len (os.listdir()))

Ourfirsttestchecksthatget_lyrics() functionreturnsastringbecausewewanttoscrapethelyricsfromthesiteandreturnthemasastringforourotherfunctiontouse.Oursecondtestchecksthattest_wordcloud_creation() createsanewfile.Whenwecreateawordcloudwearegoingtosaveitasafileinthefolderwhereweranourprogram,whichiswhywetesttomakesurecreate_wordcloud()createsanewfileinthecurrentdirectorywhenit’scalledwiththeurlofalyrics.wikia.comlyricspage.Wecheckthatanewfileiscreatedusingtheosmodule—abuilt-inPythonlibrarythathasdifferentfunctionalityforinteractingwithyouroperatingsystem.os.list.dir()returnsalistofallthefilesinthecurrentdirectory.Inourassertionintest_wordcloud_creation() ,wecheckthenumberoffilesinthecurrentdirectorybeforewecallcreate_wordcloud() +1isthesameasthenumberoffilesaftercreate_wordcloud() wascalled.Thisisbecauseafterwecallcreate_wordcloud() ,thereshouldbethesamenumberoffilesastherewere,onlyonemore(thenewfilethatwascreated).Whenwerunthesetests,theybothshouldfailwithanAssertionError .Nowallwehavetodoiswritethecodetomakethempass.First,weneedtoinstallthelibrarieswearegoingtousewithpip.Thefirsttwoarerequirementsforthewordcloud library.

$pip3installnumpy$pip3installpillow$pip3installwordcloud

YoushouldalreadyhaveBeautifulSoupinstalledfromwhenwebuiltourfirstscraper,butifnot:

$pip3installBeautifulSoup4

Westartourprogrambyimportingthelibrariesweinstalled:

fromwordcloud importWordCloudfrombs4 importBeautifulSoupimportrequestsfromos importpath

Nowwecanwriteafunctiontoscrapesonglyrics:

defget_lyrics (wiki_url , tag ,tag_name):"""Takesaurlforalyricspageoflyrics.wikia.com,atagandatagnamesearchesforthattagintheurlsHTMLthathasthetag_namepassedin.Returnsthesonglyricsfound.:param wiki_url:stringlyrics.wikialyricsurle.g.http://lyrics.wikia.com/wiki/The_Beatles:Girl.:param tag:stringHTMLtagtolookforlyricsine.g."div":param tag_name:stringnameofHTMLtagtolookforlyricsine.g.lyricbox:return :stringsonglyrics"""response=requests.get(wiki_url)soup=BeautifulSoup(response.text , 'html.parser')if soup.find_all(tag ,tag_name): return soup[ 0 ].text

Ourget_lyrics() functionshouldlookfamiliar.ItissimilartoourGoogleNewsscraper:wegettheHTMLfromthelyricswebsiteandpassitintoaBeautifulSoupobject;thenwepasstheparametertagandtag_nameintosoup.find_all() whichlooksforanHTMLtagwiththetag andtag_namewepassedin.Wereturntheresult—thesonglyrics(iftheywerefound).Wepasstag andtag_nameintoourfunctioninsteadofpassingitintosoup.find_all() directlybecausewearegettingdatafromalivewebsite,andthetagwearelookingforislikelytochange,sowedon’twanttohardcodethatdataintoourfunction,isnteadwepassitinsothecallerofthefunctioncandecidethecorrecttag andtag_nametopassin.Atthetimeofthiswriting,thesonglyricsareheldintheHTMLtag<divclass=“lyricbox”></div>.IftheHTMLtageverchanges,IwillupdateitonGitHub,sowatchoutforthatifthisprogramstopsworking.Hereishowwewouldcallourcurrentfunction:

get_lyrics( 'http://lyrics.wikia.com/wiki/The_Beatles:Lucy_In_The_Sky_With_Diamonds' , 'div' ,'lyricbox')

Nowwecancodeafunctiontocreateourwordcloud:

def create_wordcloud(wiki_url , file_name ,tag , tag_name): """Takesaurlforalyricspageoflyrics.wikia.comandcreatesawordcloudfromthelyrics. :param wiki_url:stringlyrics.wikialyricsurle.g.http://lyrics.wikia.com/wiki/The_Beatles:Girl.""" lyrics=get_lyrics(wiki_url , tag, tag_name)wordcloud=WordCloud().generate(lyrics)image=wordcloud.to_image()image.show()image.save(path.dirname(__file__)+ '/wordcloud.jpg' )

create_wordcloud( ' http://lyrics.wikia.com/wiki/The_Beatles:Lucy_In_The_Sky_With_Diamonds ',‘wordcloud.jpg’,‘div’,‘lyricbox’ )

Westartbygettingthelyricsfromthe wiki_urlwepassedinbycallingourget_lyrics()functionandpassingitthewiki_url .Fromhereonout,theT heWordCloud moduledoesallthehardworkforus.Wecreatea WordCloud objectandpassitthelyrics.NowwecanusethemethodstheWordCloud objecthastocreateourwordcloud.Thetricktoknowingwhatmethodsexistandfiguringouthowtousethemistoreadthe Wordcloudmodule’sdocumentation,availableat: https://github.com/amueller/word_cloud.

Firstwecallto_image() onourWordCloud object,andthencall show() ontheresulttocreateourwordcloud.Thiscreatesawordcloudthatwillpopuponourdesktop,butwealsowanttosavethewordcloudasafile,sowecallsave() —whichtakesapathtowhereweshouldsavethefileasaparameter,andsavesthefilethere;soifwewanttonameourfile“my_file.txt”andsaveittoourdesktop,wewouldpassinsomethinglike“/users/desktop/my_file.txt”.Inthiscase,wewanttosaveourfileinthefolderwhereourprogramisrunning,sowepassin path.dirname(__file__) ( path isafunctionfromtheosmodulewhichreturnsthepathwhereourprogramisrunning),andconcatenatethatwiththefile_name passedtotocreate_wordcloud() asaparameter—resultinginsomethinglike“users/program/word_cloud.jpg”(whichiswhereourwordcloudwillbesaved).That’sallthereistooit,ourfunctionnowcreatesabeautifulwordcloudbasedonthesonglyricsfromtheurlwepassin.

Whenyouruntheprogram,itshouldcreateawordcloudthatpopsuponyourdesktop(ifnotchecktheversionpostedonGitHubbecausethetagsmayhavechanged),savethefiletothefolderwheretheprogramisrunning,andwhenyourunourtests,theyshouldallpass.Hereisourfinishedcode:

from wordcloudimportWordCloudfrom bs4importBeautifulSoupimportrequestsfrom osimportpath

def get_lyrics (wiki_url ,tag , tag_name):

"""Takesaurlforalyricspageoflyrics.wikia.com,atagandatagnamesearchesforthattagintheurlsHTMLthathasthetag_namepassedin.Returnsthesonglyricsfound. :param wiki_url:stringlyrics.wikialyricsurle.g.http://lyrics.wikia.com/wiki/The_Beatles:Girl. :param tag:stringHTMLtagtolookforlyricsine.g."div" :param tag_name:stringnameofHTMLtagtolookforlyricsine.g.lyricbox :return :stringsonglyrics""" response=requests.get(wiki_url)soup=BeautifulSoup(response.text , 'html.parser' ) ifsoup.find_all(tag , tag_name): return soup[ 0 ].text

def create_wordcloud(wiki_url , file_name , tag , tag_name): """Takesaurlforalyricspageoflyrics.wikia.comandcreatesawordcloudfromthelyrics. :param wiki_url:stringlyrics.wikialyricsurle.g.http://lyrics.wikia.com/wiki/The_Beatles:Girl.""" lyrics=get_lyrics(wiki_url , tag, tag_name)wordcloud=WordCloud().generate(lyrics)image=wordcloud.to_image()image.show()image.save(path.dirname(__file__)+ '/wordcloud.jpg' )

create_wordcloud( 'http://lyrics.wikia.com/wiki/The_Beatles:Lucy_In_The_Sky_With_Diamonds' , 'word_cloud.jpg','div' , 'lyricbox' )

Nowthatwe’vewrittenourprogram,thenextphaseisSystemTesting.We’vealreadywrittenunittestsusingTestDrivenDevelopment,butwestillneedtowriteintegrationtestsandsystemtests,andhavepeopletesttheprogramasusers.Inthiscase,wearecreatingaproductforotherprogrammerstouse,soweshouldgetsomeprogrammerstotryoutourprogram,andseeiftheyareabletoeasilyuseit.Wewon’twriteanymoretestsinthisexample,buttrytothinkabouthowyouwouldapproachthis.ThefinalstepinourdevelopmentprocessisImplementationandDeployment.Sincewearebuildingaproductforotherprogrammers,puttingourprogramintoproductionmeansuploadingittoGitHubandPyPi.Oncewedothat,itisavailableforotherprogrammerstouse,anditisuptoustocontinuouslycheckinonthecodeandprovidemaintenanceforthecodeifanythingbreaks.IfyouareinterestedinlearninghowtouploadyourcodetoPyPiso

itisavailableonpip,followtheinstructionshere:http://peterdowns.com/posts/first-time-with-pypi.html.

Practice

Exercises

0.FindaprojectonGitHub,readthroughitandthinkaboutthequalityofthecode.

Read

0.http://googletesting.blogspot.com/2016/06/the-inquiry-method-for-test-planning.html

PartVILandaJob

ChapterX.YourFirstProgrammingJob

“Bewareof‘therealworld.’Aspeaker ’sappealtoitisalwaysaninvitationnottochallengehistacitassumptions.”—EdsgerW.Dijkstra

WelcometoPartVI,thefinalpartofthisbook,dedicatedtohelpingyouwithyourcareerasasoftwareengineer.Gettingyourfirstprogrammingjobrequiresextraeffort,butifyoufollowtheadviceinthischapter,youshouldhavenoproblem.Luckily,onceyoulandyourfirstprogrammingjobandgetsomeexperience,whenitcomestimetolookforyournextjob,recruiterswillstartreachingouttoyou .

ChooseaPath

Onethingtokeepinmindisthatprogrammingjobsarelumpedintospecificdomains:eachwiththeirownsetoftechnologiesandskillsets.Ifyoulookatprogrammingjobads,theheadlinewillbesomethinglike“PythonBackendProgrammerWanted.”Thismeanstheyarelookingforsomeonethatprogramsthebackendofawebsite,andisalreadyfamiliarwithPython.Ifyougotothejobdescription,therewillbealistoftechnologiestheidealcandidatewillbefamiliarwith,alongwithskillstheyshouldhave.Whileit’sfinetobeageneralist(aprogrammerthatdabblesineverything)whileyouarelearningtoprogram,anditispossibletogetajobasageneralistprogrammer,youprobablyshouldfindaspecificareaofprogrammingyouenjoyandstartbecominganexpertinit.Thiswillmakegettingajobsignificantlyeasier.

Webandmobiledevelopmentaretwoofthemostpopularprogrammingpaths.Applicationdevelopmentisoftensplitintotwoparts—thefrontendandthebackend.Thefrontendofanapplicationisthepartthatyoucansee—liketheinterfaceofawebapp.Thebackendiswhatyoucan’tsee—thepartthatprovidesthefrontendwithdata.Somecompanieswillhaveateamdedicatedtothefrontend,andateamdedicatedtothebackend.Othercompaniesonlyhirefullstackdevelopers—programmersthatcanworkonboththefrontandbackend.However,thisonlyappliestoapplicationdevelopment(buildingwebsitesorapps).Thereareallkindsofotherprogrammingareasyoucanworkinlikesecurity,platformengineering,anddatascience.Tolearnmoreaboutthedifferentareasofprogramming,gotositeslistingprogrammingjobs,andreadthroughthedescriptions.ThePythonJobBoardisagoodplacetostart:https://www.python.org/jobs.Lookattherequirementsforthedifferentjobs,aswellasthetechnologiestheyuse—thiswillgiveyouanideawhatyouneedtolearntobecompetitiveforthetypeofjobyouwant.

GettingInitialExperience

Inordertogetyourfirstprogrammingjobyouneedexperience.Buthowdoyougetprogrammingexperienceifnoonewillhireyouwithoutit?Thereareafewwaystosolvethisproblem.Onesolutionistofocusonopensource.Youcaneitherstartyourownopensourceproject,orcontributetothethousandsofopensourceprojectsonGitHub.Anotheroptionistodofreelancework;youcancreateaprofileonsiteslikeUpwork,andstartgettingsmallprogrammingjobsrightaway.Togetstarted,Irecommendfindingsomeoneyouknowthatneedssomeprogrammingworkdone.HavethemsignupforanUpworkaccountandofficiallyhireyoutheresotheycanleaveyouagreatreviewforyourwork.UntilyouhaveatleastonegoodreviewonasitelikeUpwork,itisdifficulttolandjobs.Oncepeopleseethatyou’vesuccessfullycompletedatleastonejob,gettinghiredforjobsbypeopleyou’venevermetbecomeseasier,becauseyou’veestablishedsomecredibility .

GettinganInterview

Onceyou’vegainedprogrammingexperiencethrougheitheropensourceorfreelanceworkit’stimetostartinterviewing.I’vefoundthemosteffectivewaytogetaninterviewistofocusonLinkedIn.Ifyoudon’thaveaLinkedInaccount,createonetogetstartednetworkingwithpotentialemployers.Addasummaryaboutyourselfatthetopofofyourprofile,andmakesuretohighlightyourprogrammingskills.Forexample,alotofpeopleaddsomethinglike“Languages:Python,JavaScript”atthetopoftheirprofile,whichhelpsrecruiterssearchingforthosekeywordsfindthem.Makesuretoaddyouropensourceorfreelancingexperienceasyourmostrecentjob.

Onceyourprofileiscomplete,startconnectingwithtechnicalrecruiters—therearetonsoftechnicalrecruitersonLinkedIn,sofindthemandsendthemarequesttoconnect.Theyarealwayslookingfornewtalent,sotheywillbeeagertoconnectwithyou.Oncetheyacceptyourinvitation,reachoutandaskiftheyhaveanyopenpositionsavailable.

TheInterview

Ifarecruiterthinksyoulooklikeagoodfitfortheroletheyarehiringfor,theywillsendyouamessageonLinkedInaskingtosetupaphonescreen.Thephonescreenwillbewiththerecruiter,soitisusuallynon-technical(althoughI’vehadrecruitersaskmesometechnicalquestionsthey’vememorizedtheanswertoduringthefirstinterview).The

conversationisaboutthetechnologiesyouknow,yourpreviousexperienceandseeingifyouwouldfitintothecompany’sculture.

Ifyoudowell,youwilladvancetothesecondround—atechnicalphonescreen—whereyouspeakwithoneormoremembersoftheengineeringteamandtheyaskyouthesamequestionsaboutyourexperienceandskillsasthefirstinterview.Thistimehowever,thequestionsarefollowedbyatechnicaltestoverthephone.Theengineerswillgiveyoutheaddressofawebsitewhereyoucaneditcode;theywillpostaprogrammingquestion;andaskyoutosolveit.Ifyoumakeitpastthesecondround—andthisnerve-rackingprocesshasn’tcausedyoutoabandonthisbookandquitprogrammin g—youwillhaveathirdinterview.Thethirdinterviewisonsite,inpersonatthecompany’soffice.Thethirdroundisalotlikethefirsttwo.Youmeetwithdifferentengineersontheteam,theyaskyouaboutyourskillsandexperience,andtherearemoretechnicaltests.Sometimesyouevenstayforlunchtoseehowyouinteractwiththeteam.Thethirdroundiswherethefamouswhiteboardcodingtestshappen.Ifthecompanyyouareinterviewingfordoeswhiteboarding,youwillbeaskedseveralprogrammingproblemsandaskedtosolvethemonawhiteboard.Irecommendbuyingawhiteboardandpracticing thisbeforehandbecausesolvingaprogrammingproblemonawhiteboardismuchharderthansolvingitonacomputer.Ofcoursenotallcompaniesfollowthisexactformula,youmayrunintodifferentvariationsofit,butingeneral,thisiswhatyoushouldexpect.

HackingTheInterview

Themajorityofprogramminginterviewsfocusontwosubjects—datastructuresandalgorithms.Thatmeanstopassyourprogramminginterview,youknowexactlywhatyoumustdo—getverygoodattwospecificarea sofComputerScience.Fortunately,thiswillalsohelpyoutobecomeabetterprogrammer.

Youcannarrowdownthequestionsyoushouldfocusonevenfurtherbythinkingabouttheinterviewfromtheinterviewer's’perspective.Thinkaboutthesituationyourinterviewersareisin;theysaysoftwareisneverfinished,andit’strue.Yourinterviewersmostlikelyhasalotofworktheyneedtogetdone,anddon’twanttodedicatealotofhistimetointerviewingcandidates.Comingupwithgoodprogrammingquestionsishard.Aretheygoingtospendtheirvaluabletimecomingupwithoriginalprogrammingquestions?Probablynot.Theyareprobablygoingtogoogle“programminginterviewquestions”andaskoneofthefirstonestheyfind.Thisleadstothesameinterviewquestionscomingupoverandoveragain—andtherearesomegreatresourcesouttheretopracticethem.LeetcodeisoneIrecommendyoucheckout.AlmosteveryquestionI’veeverbeenaskedinaprogramminginterviewisonLeetcode.

ChapterX.WorkingonaTeam

“Youcan’thavegreatsoftwarewithoutagreatteam,andmostsoftwareteamsbehavelikedysfunctionalfamilies.”-JimMcCarthy

Comingfromaself-taughtbackground,youareprobablyusedtoprogrammingalone.Onceyoujoinacompany,youwillneedtolearnhowtoworkonateam.Evenifyoustartyourowncompany,eventuallyyouwillneedtohireadditionalprogrammers,atwhichpointyouwillalsoneedtolearntoworkasateam.Programmingisateamsportandlikeanyteamsport,youneedtogetalongwithyourteammates.Thischapterprovidessometipsforsuccessfullyworkinginateamenvironment.

MastertheBasics

Whenyouarehiredbyacompany,youareexpectedtobecompetentintheskillscoveredinthisbook.Itisnotenoughtosimplyreadthisbook—youneedtomastertheskillsinit.Startasideprojectwithafriendanduseversioncontrol.Writenewtestseveryday.Scheduleprogramstorun.Ifyourteammateshavetoconstantlyhelpyouwiththebasics,theywillbecomefrustrated.

Don’tAskWhatYouCanGoogle

Asanew,self-taughtmemberofaprogrammingteam,youwillhaveplentytolearn,andyouwillneedtoaskalotofquestions.Thisisagreatwaytolearn,butyouwanttomakesureyouareonlyaskinggoodquestions.Ageneralruleistoonlyaskaquestionifyou’vespentatleastfiveminutestryingtoGoogletheansweryourself.Whileaskingquestionsisapositivething,ifyouasktoomanyquestionsyoucouldhaveeasilyfiguredoutyourself,youwillannoyyourteammates.Makesureyouareonlyaskinggoodquestionsbytryingtofindtheanswerforatleastfiveminutesbeforeyouaskit.

ChangingCode

Byreadingthisbook,you’vedemonstratedyouarethetypeofpersonwhoisconstantlylookingtoimprove.Unfortunately,noteveryoneonyourteamwillshareyourenthusiasmfor

becomingabetterprogrammer.Manyprogrammersdon’thavethedesiretokeeplearning,andtheyarefinedoingthingssuboptimally.Thiscanbeespeciallyprevalentinstartups,whereshippingcodefastisoftenmoreimportantthanshippinghighqualitycode.Ifyoufindyourselfinthissituation,youshouldlistentoWalterWhiteand“treadlightly.”Programmerscangettheiregoshurtveryeasilywhenyouchangetheircode.Evenworse,ifyouspendalotoftimefixingotherpeople’scode,youwillnothaveenoughtimetocontributetonewprojects,anditmaylooklikeyouarenotworkinghardenough.Thebestdefenseforthisistocarefullyquestionanycompanyyoujoinabouttheirengineeringculture.Ifyoustillfindyourselfinthissituation,itisbesttolistentoEdwardYourdon,“Ifyouthinkyourmanagementdoesn’tknowwhatit’sdoingorthatyourorganisationturnsoutlow-qualitysoftwarecrapthatembarrassesyou,thenleave.”

Evenifyouworkatagreatcompany,therearestilltimeswhenyouwillneedtofixotherpeople’scode,whichyoushouldapproachdelicately.MikeCoutermarshwroteanarticleonMediumcalled“Jr.Developers#5:HowtoImproveCodeWithoutAnyoneHatingYou”whichIrecommendyoureadforadviceonthesubject.Asanewprogrammerenteringyourfirstprogrammingjob,it’simportantforyoutogetalongwithyourteam—alienatingyourteammemberscanhappenmucheasierthanitmayseem—somakesuretoalwaysstaycognizantofhowyourteammateswillreactifyouwanttomakechangestotheircode.

ImposterSyndrome

Everyonethatprogramsfeelsoverwhelmedattimesand,nomatterhowhardyouworktherearegoingtobethingsyoudon’tknow.Asaself-taughtprogrammer,itisespeciallyeasytofeelinadequatebecausesomeoneaskedyoutodosomethingyou’veneverheardof,orbecauseyoufeelliketherearesomanyconceptsinComputerScienceyoustilldonotunderstand.Remember—thishappenstoeveryone—notjustyou.

IwassurprisedwhenmyfriendwithaMastersdegreeinComputerSciencefromStanfordtoldmehefeelsthiswayaswell.Hesaideveryoneinhisprogramdealtwithimpostersyndrome,andhenoticedtheyreactedinoneoftwoways:theyeitherstayedhumbleandwerealwayswillingtoadmitwhentheydidn’tknowsomething—andtriedlearnit—ortheypretendedtheykneweverything(whentheydidn’t)andstifledtheirlearning.Makesureyourememberyougottowhereyouarebyworkinghard,andit’sokifyoudon’tknoweverything,nobodydoes.Juststayhumble,relentlesslystudyanythingyoudon’tunderstand,andyouwillbeunstoppable .

FurtherLearning

“Thebestprogrammersarenotmarginallybetterthanmerelygoodones.Theyareanorder-of-magnitudebetter,measuredbywhateverstandard:conceptualcreativity,speed,ingenuityofdesign,orproblem-solvingability.”—RandallE.Stross

ThearticleABC:AlwaysBeCoding byDavidByttowgivesgreatadviceonhowtogetajobasasoftwareengineer.Thearticleissummarizedinthetitle—alwaysbecoding.IfyoucombineABCwithanewacronymImadeup—ABL—alwaysbelearning—youaresuretohaveanexceptionalcareer.Inthischapter,IamgoingtogooversomeprogrammingresourcesI’vefoundhelpful.

TheClassics

Thereareafewprogrammingbooksthatareconsideredmustreads:ThePragmaticProgrammer byAndyHuntandDaveThomas;DesignPatternsbyErichGamma,JohnVlissides,RalphJohnson,andRichardHelm(designpatternsareanimportantsubjectwedidn’tgetachancetocover);CodeComplete bySteveMcConnell;Compilers:Principles,Techniques,andTools,byAlfredAho,JeffreyUllman,MonicaS.LamandRaviSethi;andIntroductiontoAlgorithms byTheMITPress.Ialsohighlyreccomend ProblemSolvingwithDataStructuresandAlgorithms ,afree,interactive,excellentintroductiontoalgorithms,mucheasiertounderstandthanMIT’sIntroductiontoAlgorithms.

OnlineClasses

Onlinecodingclassesareapopularwaytolearntoprogram.Courseraisoneofthemostpopular,butIalsolovethelesserknownCodeschool.Ihighlyrecommendyoutakesomeoftheirclasses.Notonlydotheyhavegreatclassesondifferentprogramminglanguages,buttheyalsohaveclassesonGit,SQLandothersubjectsdiscussedinthisbook.

HackerNews

HackerNewsisaplatformforuser-submittednewsonthetechnologyincubatorYCombinatorswebsitefoundathttps://news.ycombinator.com.Itispopularwithprogrammersandwillhelpyoukeepuptodatewiththenewesttrendsandtechnologies.

Other

Thisarticleisgreatforfindingnewthingstolearn:http://matt.might.net/articles/what-cs-majors-should-know.

NextSteps

Firstofall—thankyouforpurchasingthisbook.Ihopeit’shelpedyoubecomeabetterprogrammer.Theprogrammingcommunityhasgivenmesomuchsupport,andbywritingthisbook,IhopeI’vemanagedtohelpyouinyourjourneythewaysomanypeoplehelpedme.Nowthatyou’vefinished,it’stimeforyoutogetdowntobusiness.Wheredoyougofromhere?Datastructuresandalgorithms,algorithmsalgorithms.GetonLeetCodeandpracticethosealgorithms.Thenpracticethemsomemore!InthischapterIgivesomefinalthoughtsonhowyoucancontinuetoimproveasaprogrammer(onceyoufinishedpracticingyouralgorithms).

FindaMentor

Findingamentorwillhelpyoutakeyourprogrammingskillstothenextlevel.Oneofthehardthingsaboutlearningtoprogramisthattherearesomanythingsyoucandosuboptimally,withoutknowingit.Imentionedearlieryoucanhelpcombatthisbydoingcodereviews:amentorcandocodereviewswithyouandhelpyouimproveyourcodingprocess,recommendbooks,andhelpyouwithprogrammingconceptsyouarehavingtroubleunderstanding.

StrivetoGoDeep

Thereisaconceptinprogrammingcalleda“blackbox”thatreferstosomethingyouuse,butdonotunderstandhowitworks.Whenyoufirststartprogramming,everythingisablackbox.Oneofthebestwaystogetbetteratprogrammingistotrytoopenupeveryblackboxyoucanfindandtrytounderstandhowitworks.Oneofmyfriendstoldmeitwasamajor“aha”momentforhimwhenherealizedthecommandlineitselfisaprogram.OpeningupablackboxiswhatIcallgoingdeep.Writingthisbookhelpedmegodeep.TherewerecertainconceptsIthoughtIunderstood,onlytofindoutIcouldn’texplainthem.Ihadtogodeep.Don’tjuststopatoneexplanation,readalltheexplanationsonthetopicyoucanfind.Askquestionsandreaddifferingopinionsonline.

Ifindoneofthemosthelpfulquestionstobe“Whatproblemdoesthissolve?”Forexample,welearnedaboutObject-orientedprogramminginthisbook.Butwhywasobject-

Orientedprogramminginvented?Whatproblemdoesitsolve?Arethereothersolutions?Pursuingtheanswerstothesetypesofquestionswillhelpyoubecomeabetterprogrammer.

Anotherwaytogodeepistobuildthingsyouwanttounderstandbetter.Havingtroubleunderstandingdatabases?Buildasimpledatabaseinyourfreetime.WhenIwashavingtroubleunderstandingcompilers—Ibuiltone.Takingthetimetodoaprojectlikethisiswellworththeinvestment,becauseitwillimproveyourunderstandingofwhateveryouarestrugglingwith.

OtherAdvice

Ioncecameacrossaforumdiscussingwaystobecomeabetterprogrammer.Thetopvotedanswerwasasomewhatsurprising:“Dothingsotherthanprogramming.”I’vefoundthistobetrue—readingbookslikeTheTalentCode byDanielCoylehasmademeabetterprogrammer;becausehelaysoutexactlywhatyouneedtodotomasteranyskill.Anothergoodbookaboutlearningaskillis MasterybyGeorgeLeonard.Keepyoureyeoutforthingsoutsideofprogrammingyoucanbringtoyourprogramminggame.ThelastpieceofadviceIwillleaveyouwithistospendasmuchtimeasyoucanreadingotherpeople’scode.Readingotherpeople’scodeisoneofthebestwaystoimproveasaprogrammer.Whenyouarelearning,youneedtomakesuretostrikeabalancebetweenwritingcode,andreadingcode.Readingotherpeople’scodeisgoingtobedifficultatfirst,butitisimportanttodoitbecauseyoucanlearnsomuchfromotherprogrammers.

IhopeyouenjoyedthisbookasmuchasIenjoyedwritingit.Pleasefeelfreetoemailmeatcory@theselftaughtprogrammer.ioforanyreason.Ialsohaveaprogrammingnewsletteryoucansignupforattheselftaughprogrammer.ioandaforumwhereyoucangetintouchwithmeandacommunityofpeoplelearningtoprogram.Ifyoulikedthisbook,pleasealsoconsiderleavingareviewonAmazon ,ithelpsgetthisbookinthehandsofmorepeople,andIreallyappreciateeveryreviewIreceive.Bestofluckontherestofyourjourney.Andremember—ABL!

Acknowledgements

Parents,Pam,Randee,Anzar,CoverDesigner,Lauren,Antoine,Torrey,JinChun.

Glossary

Biography

Citations0.http://stackoverflow.com/questions/466790/assembly-code-vs-machine-code-vs-object-code0.0.http://man7.org/linux/man-pages/man1/ls.1.html0.0.http://www.webopedia.com/TERM/C/clock_speed.html0.0.http://stackoverflow.com/questions/1050222/concurrency-vs-parallelism-what-is-the-difference0.0.http://berb.github.io/diploma-thesis/original/022_webapps.html0.0.http://stackoverflow.com/questions/11828270/how-to-exit-the-vim-editor0.0.http://superuser.com/questions/666942/why-it-is-not-recommend-to-use-root-login-in-linux0.0.http://programmers.stackexchange.com/questions/37294/logging-why-and-what0.0.http://stackoverflow.com/questions/10925478/how-to-read-api-documentation-for-newbs0.0.http://www.infoworld.com/article/2908474/application-development/stack-overflow-survey-finds-nearly-half-have-no-degree-in-computer-science.html0.0.http://www.merriam-webster.com/dictionary/catenate0.0.DesignPatternskindlelocation5460.0.DesignPatternskindlelocation6500.0.DesignPatternskindlelocation8240.0.https://hungred.com/wp-content/uploads/2009/05/ascii-table-cheat-sheet1.png0.0.ThePragmaticProgrammerkindlelocation8300.0.https://automatetheboringstuff.com/chapter0/0.

0.http://kevinlondon.com/2015/07/26/dangerous-python-functions.html0.0.http://interactivepython.org/runestone/static/pythonds/Recursion/TheThreeLawsofRecursion.html0.0.https://automatetheboringstuff.com/chapter1/0.https://www.quora.com/What-are-some-interesting-facts-about-computer-programming0. http://www.dkfindout.com/us/explore/eight-cool-facts-about-computer-coding/0.http://thenextweb.com/insider/2016/02/26/8-facts-every-computer-programmer-should-know/#gref0.http://cs.lmu.edu/~ray/notes/x86assembly/0.http://www.tutorialspoint.com/python/python_files_io.htm0.http://www.wsj.com/articles/computer-programming-is-a-trade-lets-act-like-it-1407109947?mod=e2fb0.https://en.wikipedia.org/wiki/Persistence_(computer_science)0.https://en.wikipedia.org/wiki/Column_family0.http://stackoverflow.com/questions/2570756/what-are-database-constraints0.https://en.wikipedia.org/wiki/Data_integrity0.https://www.sitepoint.com/understanding-the-observer-pattern/0.http://codedx.com/how-to-minimize-your-softwares-attack-surface/0.https://support.microsoft.com/en-us/kb/2838780.http://www.slideshare.net/jagaarj/database-design-normalization0.http://www.tcpipguide.com/free/t_WhatIsNetworking.htm0.Readwrite.com0.http://www.gnu.org/software/grep/manual/grep.html0.https://en.wikipedia.org/wiki/Software_testing0.https://en.wikipedia.org/wiki/Boundary_case0.http://www.guru99.com/integration-testing.html0.https://en.wikipedia.org/wiki/Systems_development_life_cycle#cite_note-10.ProjectManagementInstitute,20130.http://www.tutorialspoint.com/sdlc/sdlc_overview.htm0.https://en.wikipedia.org/wiki/Software_development_process0.https://en.wikipedia.org/wiki/Waterfall_model0.https://en.wikipedia.org/wiki/Incremental_build_model0.http://agiledata.org/essays/tdd.html0.http://www.agilenutshell.com/0.https://maryrosecook.com/blog/post/a-practical-introduction-to-functional-programming0.http://stackoverflow.com/questions/1031273/what-is-polymorphism-what-is-it-for-and-how-is-it-used0.https://en.wikipedia.org/wiki/Syntax

Documents

The Self-taught Programmer Cory Althoff...If you do want to become a software engineer, there is an increasing demand for good engineers and not enough qualified people to fill the