Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
TableofContentsTheNode.jsHandbook
Introduction
IntroductiontoNode
AbriefhistoryofNode
HowtoinstallNode
HowmuchJavaScriptdoyouneedtoknowtouseNode?
DifferencesbetweenNodeandtheBrowser
v8
Basics
RunNode.jsscriptsfromthecommandline
HowtoexitfromaNode.jsprogram
Howtoreadenvironmentvariables
Nodehostingoptions
CommandLine
UsetheNodeREPL
Passargumentsfromthecommandline
Outputtothecommandline
Acceptinputfromthecommandline
Nodemodulesandnpm
ExposefunctionalityfromaNodefileusingexports
npm
Wheredoesnpminstallthepackages
Howtouseorexecuteapackageinstalledusingnpm
Thepackage.jsonfile
Thepackage-lock.jsonfile
Findtheinstalledversionofannpmpackage
Howtoinstallanolderversionofannpmpackage
HowtoupdatealltheNodedependenciestotheirlatestversion
Semanticversioningrules
Uninstallingnpmpackages
2
Globalorlocalpackages
npmdependenciesanddevDependencies
npx
Workingwiththeeventloop
Theeventloop
nextTick
setImmediate
Timers
Asynchronousprogramming
Callbacks
Promises
async/await
TheNodeEventEmitter
Networking
HTTP
HowHTTPRequestswork
BuildanHTTPserver
MakingHTTPrequests
Axios
Websockets
HTTPS,secureconnections
FileSystem
Filedescriptors
Filestats
Filepaths
Readingfiles
Writingfiles
Workingwithfolders
Someessentialcoremodules
Thefsmodule
Thepathmodule
Theosmodule
Theeventsmodule
3
Thehttpmodule
Miscellaneous
Streams
WorkingwithMySQL
Differencebetweendevelopmentandproduction
4
TheNode.jsHandbook
TheNodeHandbookfollowsthe80/20rule:learnin20%ofthetimethe80%ofatopic.
Ifindthisapproachgivesawell-roundedoverview.ThisbookdoesnottrytocovereverythingunderthesunrelatedtoNode.Ifyouthinksomespecifictopicshouldbeincluded,tellme.
YoucanreachmeonTwitter@flaviocopes.
Ihopethecontentsofthisbookwillhelpyouachievewhatyouwant:learnthebasicsNode.js.
ThisbookiswrittenbyFlavio.Ipublishwebdevelopmenttutorialseverydayonmywebsiteflaviocopes.com.
Enjoy!
TheNode.jsHandbook
5
IntroductiontoNodeThispostisagettingstartedguidetoNode.js,theserver-sideJavaScriptruntimeenvironment.Node.jsisbuiltontopoftheGoogleChromeV8JavaScriptengine,andit'smainlyusedtocreatewebservers-butit'snotlimitedtothat
OverviewThebestfeaturesofNode.js
FastSimpleJavaScriptV8AsynchronousplatformAhugenumberoflibraries
AnexampleNode.jsapplicationNode.jsframeworksandtools
OverviewNode.jsisaruntimeenvironmentforJavaScriptthatrunsontheserver.
Node.jsisopensource,cross-platform,andsinceitsintroductionin2009,itgothugelypopularandnowplaysasignificantroleinthewebdevelopmentscene.IfGitHubstarsareonepopularityindicationfactor,having46000+starsmeansbeingverypopular.
IntroductiontoNode
6
Node.jsisbuiltontopoftheGoogleChromeV8JavaScriptengine,andit'smainlyusedtocreatewebservers-butit'snotlimitedtothat.
ThebestfeaturesofNode.js
Fast
OneofthemainsellingpointsofNode.jsisspeed.JavaScriptcoderunningonNode.js(dependingonthebenchmark)canbetwiceasfastthancompiledlanguageslikeCorJava,andordersofmagnitudefasterthaninterpretedlanguageslikePythonorRuby,becauseofitsnon-blockingparadigm.
Simple
Node.jsissimple.Extremelysimple,actually.
JavaScript
Node.jsrunsJavaScriptcode.ThismeansthatmillionsoffrontenddevelopersthatalreadyuseJavaScriptinthebrowserareabletoruntheserver-sidecodeandfrontend-sidecodeusingthesameprogramminglanguagewithouttheneedtolearnacompletelydifferenttool.
Theparadigmsareallthesame,andinNode.jsthenewECMAScriptstandardscanbeusedfirst,asyoudon'thavetowaitforallyouruserstoupdatetheirbrowsers-youdecidewhichECMAScriptversiontousebychangingtheNode.jsversion.
V8
IntroductiontoNode
7
RunningontheGoogleV8JavaScriptengine,whichisOpenSource,Node.jsisabletoleveragetheworkofthousandsofengineersthatmade(andwillcontinuetomake)theChromeJavaScriptruntimeblazingfast.
Asynchronousplatform
Intraditionalprogramminglanguages(C,Java,Python,PHP)allinstructionsareblockingbydefaultunlessyouexplicitly"optin"toperformasynchronousoperations.IfyouperformanetworkrequesttoreadsomeJSON,theexecutionofthatparticularthreadisblockeduntiltheresponseisready.
JavaScriptallowstocreateasynchronousandnon-blockingcodeinaverysimpleway,byusingasinglethread,callbackfunctionsandevent-drivenprogramming.Everytimeanexpensiveoperationoccurs,wepassacallbackfunctionthatwillbecalledoncewecancontinuewiththeprocessing.We'renotwaitingforthattofinishbeforegoingonwiththerestoftheprogram.
Suchmechanismderivesfromthebrowser.Wecan'twaituntilsomethingloadsfromanAJAXrequestbeforebeingabletointerceptclickeventsonthepage.Itallmusthappeninrealtimetoprovideagoodexperiencetotheuser.
Ifyou'vecreatedanonclickhandlerforawebpageyou'vealreadyusedasynchronousprogrammingtechniqueswitheventlisteners.
ThisallowsNode.jstohandlethousandsofconcurrentconnectionswithasingleserverwithoutintroducingtheburdenofmanagingthreadsconcurrency,whichwouldbeamajorsourceofbugs.
Nodeprovidesnon-blockingI/Oprimitives,andgenerally,librariesinNode.jsarewrittenusingnon-blockingparadigms,makingablockingbehavioranexceptionratherthanthenormal.
IntroductiontoNode
8
WhenNode.jsneedstoperformanI/Ooperation,likereadingfromthenetwork,accessadatabaseorthefilesystem,insteadofblockingthethreadNode.jswillsimplyresumetheoperationswhentheresponsecomesback,insteadofwastingCPUcycleswaiting.
Ahugenumberoflibraries
npmwithitssimplestructurehelpedtheecosystemofnode.jsproliferateandnowthenpmregistryhostsalmost500.000opensourcepackagesyoucanfreelyuse.
AnexampleNode.jsapplicationThemostcommonexampleHelloWorldofNode.jsisawebserver:
consthttp=require('http')
consthostname='127.0.0.1'
constport=3000
constserver=http.createServer((req,res)=>{
res.statusCode=200
res.setHeader('Content-Type','text/plain')
res.end('HelloWorld\n')
})
server.listen(port,hostname,()=>{
console.log(`Serverrunningathttp://${hostname}:${port}/`)
})
Torunthissnippet,saveitasa server.jsfileandrun nodeserver.jsinyourterminal.
ThiscodefirstincludestheNode.js httpmodule.
Node.jshasanamazingstandardlibrary,includingafirst-classsupportfornetworking.
The createServer()methodof httpcreatesanewHTTPserverandreturnsit.
Theserverissettolistenonthespecifiedportandhostname.Whentheserverisready,thecallbackfunctioniscalled,inthiscaseinformingusthattheserverisrunning.
Wheneveranewrequestisreceived,the requesteventiscalled,providingtwoobjects:arequest(an http.IncomingMessageobject)andaresponse(an http.ServerResponseobject).
Those2objectsareessentialtohandletheHTTPcall.
Thefirstprovidestherequestdetails.Inthissimpleexample,thisisnotused,butyoucouldaccesstherequestheadersandrequestdata.
IntroductiontoNode
9
Thesecondisusedtoreturndatatothecaller.
Inthiscasewith
res.statusCode=200
wesetthestatusCodepropertyto200,toindicateasuccessfulresponse.
WesettheContent-Typeheader:
res.setHeader('Content-Type','text/plain')
andweendclosetheresponse,addingthecontentasanargumentto end():
res.end('HelloWorld\n')
Node.jsframeworksandtoolsNode.jsisalow-levelplatform,andtomakethingseasierandmoreinterestingfordevelopersthousandsoflibrarieswerebuiltuponNode.js.
Manyofthoseestablishedovertimeaspopularoptions.Hereisanon-comprehensivelisttotheonesIconsiderveryrelevantandworthlearning:
Express,oneofthemostsimpleyetpowerfulwaystocreateawebserver.Itsminimalistapproach,unopinionated,focusedonthecorefeaturesofaserver,iskeytoitssuccess.Meteor,anincrediblypowerfulfull-stackframework,poweringyouwithanisomorphicapproachtobuildingappswithJavaScript,sharingcodeontheclientandtheserver.Onceanoff-the-shelftoolthatprovidedeverything,nowintegrateswithfrontendlibsReact,VueandAngular.Canbeusedtocreatemobileappsaswell.koa,builtbythesameteambehindExpress,aimstobeevensimplerandsmaller,buildingontopofyearsofknowledge.Thenewprojectbornoutoftheneedtocreateincompatiblechangeswithoutdisruptingtheexistingcommunity.Next.js,aframeworktorenderserver-siderenderedReactapplications.Micro,averylightweightservertocreateasynchronousHTTPmicroservices.Socket.io,areal-timecommunicationenginetobuildnetworkapplications.
IntroductiontoNode
10
AbriefhistoryofNodeAlookbackonthehistoryofNode.jsfrom2009totoday
Believeitornot,Node.jsisjust9yearsold.
Incomparison,JavaScriptis23yearsoldandthewebasweknowit(aftertheintroductionofMosaic)is25yearsold.
9yearsissuchalittleamountoftimeforatechnology,butNode.jsseemstohavebeenaroundforever.
I'vehadthepleasuretoworkwithNodesincetheearlydayswhenitwasjust2yearsold,anddespitethelittleinformationavailable,youcouldalreadyfeelitwasahugething.
Inthispost,IwanttodrawthebigpictureofNodeinitshistory,toputthingsinperspective.
Alittlebitofhistory2009201020112012201320142015201620172018
AlittlebitofhistoryJavaScriptisaprogramminglanguagethatwascreatedatNetscapeasascriptingtooltomanipulatewebpagesinsidetheirbrowser,NetscapeNavigator.
PartofthebusinessmodelofNetscapewastosellWebServers,whichincludedanenvironmentcalledNetscapeLiveWire,whichcouldcreatedynamicpagesusingserver-sideJavaScript.Sotheideaofserver-sideJavaScriptwasnotintroducedbyNode.js,butit'soldjustlikeJavaScript-butatthetimeitwasnotsuccessful.
OnekeyfactorthatledtotheriseofNode.jswastiming.JavaScriptsinceafewyearswasstartingbeingconsideredaseriouslanguage,thanksforthe"Web2.0"applicationsthatshowedtheworldwhatamodernexperienceonthewebcouldbelike(thinkGoogleMapsor
AbriefhistoryofNode
11
GMail).
TheJavaScriptenginesperformancebarraisedconsiderablythankstothebrowsercompetitionbattle,whichisstillgoingstrong.Developmentteamsbehindeachmajorbrowserworkhardeverydaytogiveusbetterperformance,whichisahugewinforJavaScriptasaplatform.V8,theenginethatNode.jsusesunderthehood,isoneofthoseandinparticularit'stheChromeJSengine.
Butofcourse,Node.jsisnotpopularjustbecauseofpureluckortiming.ItintroducedmuchinnovativethinkingonhowtoprograminJavaScriptontheserver.
2009Node.jsisbornThefirstformofnpmiscreated
2010ExpressisbornSocket.ioisborn
2011npmhits1.0BigcompaniesstartadoptingNode:LinkedIn,UberHapiisborn
2012Adoptioncontinuesveryrapidly
2013FirstbigbloggingplatformusingNode:GhostKoaisborn
2014Bigdrama:IO.jsisamajorforkofNode.js,withthegoalofintroducingES6supportandmovefaster
2015
AbriefhistoryofNode
12
TheNode.jsFoundationisbornIO.jsismergedbackintoNode.jsnpmintroducesprivatemodulesNode4(no1,2,3versionswerepreviouslyreleased)
2016TheleftpadincidentYarnisbornNode6
2017npmfocusesmoreonsecurityNode8HTTP/2V8introducesNodeinitstestingsuite,officiallymakingNodeatargetfortheJSengine,inadditiontoChrome3billionnpmdownloadseveryweek
2018Node10ESmodules.mjsexperimentalsupport
AbriefhistoryofNode
13
HowtoinstallNodeHowyoucaninstallNode.jsonyoursystem:apackagemanager,theofficialwebsiteinstallerornvm
Node.jscanbeinstalledindifferentways.Thisposthighlightsthemostcommonandconvenientones.
Officialpackagesforallthemajorplatformsareavailableathttps://nodejs.org/en/download/.
OneveryconvenientwaytoinstallNode.jsisthroughapackagemanager.Inthiscase,everyoperatingsystemhasitsown.
OnmacOS,Homebrewisthede-factostandard,and-onceinstalled-allowstoinstallNode.jsveryeasily,byrunningthiscommandintheCLI:
brewinstallnode
OtherpackagemanagersforLinuxandWindowsarelistedinhttps://nodejs.org/en/download/package-manager/
nvmisapopularwaytorunNode.ItallowsyoutoeasilyswitchtheNodeversion,andinstallnewversionstotryandeasilyrollbackifsomethingbreaks,forexample.
ItisalsoveryusefultotestyourcodewitholdNodeversions.
Seehttps://github.com/creationix/nvmformoreinformationaboutthisoption.
Mysuggestionistousetheofficialinstallerifyouarejuststartingoutandyoudon'tuseHomebrewalready,otherwise,Homebrewismyfavoritesolution.
Inanycase,whenNodeisinstalledyou'llhaveaccesstothe nodeexecutableprograminthecommandline.
HowtoinstallNode
14
HowmuchJavaScriptdoyouneedtoknowtouseNode?IfyouarejuststartingoutwithJavaScript,howmuchdeeplydoyouneedtoknowthelanguage?
Asabeginner,it'shardtogettoapointwhereyouareconfidentenoughinyourprogrammingabilities.
Whilelearningtocode,youmightalsobeconfusedatwheredoesJavaScriptend,andwhereNode.jsbegins,andviceversa.
IwouldrecommendyoutohaveagoodgraspofthemainJavaScriptconceptsbeforedivingintoNode.js:
LexicalStructureExpressionsTypesVariablesFunctionsthisArrowFunctionsLoopsLoopsandScopeArraysTemplateLiteralsSemicolonsStrictModeECMAScript6,2016,2017
Withthoseconceptsinmind,youarewellonyourroadtobecomeaproficientJavaScriptdeveloper,inboththebrowserandinNode.js.
Thefollowingconceptsarealsokeytounderstandasynchronousprogramming,whichisonefundamentalpartofNode.js:
AsynchronousprogrammingandcallbacksTimersPromisesAsyncandAwaitClosuresTheEventLoop
HowmuchJavaScriptdoyouneedtoknowtouseNode?
15
LuckilyIwroteafreeebookthatexplainsallthosetopics,andit'scalledJavaScriptFundamentals.It'sthemostcompactresourceyou'llfindtolearnallofthis.
Youcanfindtheebookatthebottomofthispage:https://flaviocopes.com/javascript/.
HowmuchJavaScriptdoyouneedtoknowtouseNode?
16
DifferencesbetweenNodeandtheBrowserHowwritingJavaScriptapplicationinNode.jsdiffersfromprogrammingfortheWebinsidethebrowser
BoththebrowserandNodeuseJavaScriptastheirprogramminglanguage.
BuildingappsthatruninthebrowserisacompletelydifferentthingthanbuildingaNode.jsapplication.
Despitethefactthatit'salwaysJavaScript,therearesomekeydifferencesthatmaketheexperienceradicallydifferent.
AsafrontenddeveloperthatwritesNodeappshaveahugeadvantage-thelanguageisstillthesame.
Youhaveahugeopportunitybecauseweknowhowharditistofully,deeplylearnaprogramminglanguage,andbyusingthesamelanguagetoperformallyourworkontheweb-bothontheclientandontheserver,you'reinauniquepositionofadvantage.
Whatchangesistheecosystem.
Inthebrowser,mostofthetimewhatyouaredoingisinteractingwiththeDOM,orotherWebPlatformAPIslikeCookies.ThosedonotexistinNode,ofcourse.Youdon'thavethedocument, windowandalltheotherobjectsthatareprovidedbythebrowser.
Andinthebrowser,wedon'thavealltheniceAPIsthatNode.jsprovidesthroughitsmodules,likethefilesystemaccessfunctionality.
AnotherbigdifferenceisthatinNode.jsyoucontroltheenvironment.Unlessyouarebuildinganopensourceapplicationthatanyonecandeployanywhere,youknowwhichversionofNodeyouwillruntheapplicationon.Comparedtothebrowserenvironment,whereyoudon'tgettheluxurytochoosewhatbrowseryourvisitorswilluse,thisisveryconvenient.
ThismeansthatyoucanwriteallthemodernES6-7-8-9JavaScriptthatyourNodeversionsupports.
SinceJavaScriptmovessofast,butbrowserscanbeabitslowandusersabitslowtoupgrade,sometimesontheweb,youarestucktouseolderJavaScript/ECMAScriptreleases.
YYoucanuseBabeltotransformyourcodetobeES5-compatiblebeforeshippingittothebrowser,butinNode,youwon'tneedthat.
AnotherdifferenceisthatNodeusestheCommonJSmodulesystem,whileinthebrowserwearestartingtoseetheESModulesstandardbeingimplemented.
DifferencesbetweenNodeandtheBrowser
17
Inpractice,thismeansthatforthetimebeingyouuse require()inNodeand importinthebrowser.
DifferencesbetweenNodeandtheBrowser
18
v8V8isthenameoftheJavaScriptenginethatpowersGoogleChrome.It'sthethingthattakesourJavaScriptandexecutesitwhilebrowsingwithChrome.V8providestheruntimeenvironmentinwhichJavaScriptexecutes.TheDOMandtheotherWebPlatformAPIsareprovidedbythebrowser.
V8isthenameoftheJavaScriptenginethatpowersGoogleChrome.It'sthethingthattakesourJavaScriptandexecutesitwhilebrowsingwithChrome.
V8providestheruntimeenvironmentinwhichJavaScriptexecutes.TheDOM,andtheotherWebPlatformAPIsareprovidedbythebrowser.
ThecoolthingisthattheJavaScriptengineisindependentbythebrowserinwhichit'shosted.ThiskeyfeatureenabledtheriseofNode.js.V8waschosenforbeingtheenginechosenbyNode.jsbackin2009,andasthepopularityofNode.jsexploded,V8becametheenginethatnowpowersanincredibleamountofserver-sidecodewritteninJavaScript.
TheNode.jsecosystemishugeandthankstoitV8alsopowersdesktopapps,withprojectslikeElectron.
v8
19
OtherJSenginesOtherbrowsershavetheirownJavaScriptengine:
FirefoxhasSpidermonkeySafarihasJavaScriptCore(alsocalledNitro)EdgehasChakra
andmanyothersexistaswell.
AllthoseenginesimplementtheECMAES-262standard,alsocalledECMAScript,thestandardusedbyJavaScript.
ThequestforperformanceV8iswritteninC++,andit'scontinuouslyimproved.ItisportableandrunsonMac,Windows,Linuxandseveralothersystems.
InthisV8introduction,IwillignoretheimplementationdetailsofV8:theycanbefoundonmoreauthoritativesites(e.g.theV8officialsite),andtheychangeovertime,oftenradically.
V8isalwaysevolving,justliketheotherJavaScriptenginesaround,tospeeduptheWebandtheNode.jsecosystem.
Ontheweb,thereisaraceforperformancethat'sbeengoingonforyears,andwe(asusersanddevelopers)benefitalotfromthiscompetitionbecausewegetfasterandmoreoptimizedmachinesyearafteryear.
CompilationJavaScriptisgenerallyconsideredaninterpretedlanguage,butmodernJavaScriptenginesnolongerjustinterpretJavaScript,theycompileit.
Thishappenssince2009whentheSpiderMonkeyJavaScriptcompilerwasaddedtoFirefox3.5,andeveryonefollowedthisidea.
JavScriptisinternallycompiledbyV8withjust-in-time(JIT)compilationtospeeduptheexecution.
Thismightseemcounter-intuitive,butsincetheintroductionofGoogleMapsin2004,JavaScripthasevolvedfromalanguagethatwasgenerallyexecutingafewdozensoflinesofcodetocompleteapplicationswiththousandstohundredsofthousandsoflinesrunninginthebrowser.
v8
20
Ourapplicationsnowcanrunforhoursinsideabrowser,ratherthanbeingjustafewformvalidationrulesorsimplescripts.
Inthisnewworld,compilingJavaScriptmakesperfectsensebecausewhileitmighttakealittlebitmoretohavetheJavaScriptready,oncedoneit'sgoingtobemuchmoreperformantthatpurelyinterpretedcode.
v8
21
RunNode.jsscriptsfromthecommandlineHowtorunanyNode.jsscriptfromtheCLI
TheusualwaytorunaNodeprogramistocallthe nodegloballyavailablecommand(onceyouinstallNode)andpassthenameofthefileyouwanttoexecute.
IfyourmainNodeapplicationfileisin app.js,youcancallitbytyping
nodeapp.js
RunNode.jsscriptsfromthecommandline
22
HowtoexitfromaNode.jsprogramLearnhowtoterminateaNode.jsappinthebestpossibleway
TherearevariouswaystoterminateaNode.jsapplication.
Whenrunningaprogramintheconsoleyoucancloseitwith ctrl-C,butwhatIwanttodiscusshereisprogrammaticallyexiting.
Let'sstartwiththemostdrasticone,andseewhyyou'rebetteroffnotusingit.
The processcoremoduleisprovidesahandymethodthatallowsyoutoprogrammaticallyexitfromaNode.jsprogram: process.exit().
WhenNode.jsrunsthisline,theprocessisimmediatelyforcedtoterminate.
Thismeansthatanycallbackthat'spending,anynetworkrequeststillbeingsent,anyfilesystemaccess,orprocesseswritingto stdoutor stderr-allisgoingtobeungracefullyterminatedrightaway.
Ifthisisfineforyou,youcanpassanintegerthatsignalstheoperatingsystemtheexitcode:
process.exit(1)
Bydefaulttheexitcodeis 0,whichmeanssuccess.Differentexitcodeshavedifferentmeaning,whichyoumightwanttouseinyourownsystemtohavetheprogramcommunicatetootherprograms.
Youcanreadmoreonexitcodesathttps://nodejs.org/api/process.html#process_exit_codes
Youcanalsosetthe process.exitCodeproperty:
process.exitCode=1
andwhentheprogramwilllaterend,Nodewillreturnthatexitcode.
Aprogramwillgracefullyexitwhenalltheprocessingisdone.
ManytimeswithNodewestartservers,likethisHTTPserver:
constexpress=require('express')
constapp=express()
app.get('/',(req,res)=>{
res.send('Hi!')
})
HowtoexitfromaNode.jsprogram
23
app.listen(3000,()=>console.log('Serverready'))
Thisprogramisnevergoingtoend.Ifyoucall process.exit(),anycurrentlypendingorrunningrequestisgoingtobeaborted.Thisisnotnice.
InthiscaseyouneedtosendthecommandaSIGTERMsignal,andhandlethatwiththeprocesssignalhandler:
Note: processdoesnotrequirea"require",it'sautomaticallyavailable.
constexpress=require('express')
constapp=express()
app.get('/',(req,res)=>{
res.send('Hi!')
})
constserver=app.listen(3000,()=>console.log('Serverready'))
process.on('SIGTERM',()=>{
server.close(()=>{
console.log('Processterminated')
})
})
Whataresignals?SignalsareaPOSIXintercommunicationsystem:anotificationsenttoaprocessinordertonotifyitofaneventthatoccurred.
SIGKILListhesignalsthattellsaprocesstoimmediatelyterminate,andwouldideallyactlikeprocess.exit().
SIGTERMisthesignalsthattellsaprocesstogracefullyterminate.Itisthesignalthat'ssentfromprocessmanagerslike upstartor supervisordandmanyothers.
Youcansendthissignalfrominsidetheprogram,inanotherfunction:
process.kill(process.pid,'SIGTERM')
OrfromanotherNode.jsrunningprogram,oranyotherapprunninginyoursystemthatknowsthePIDoftheprocessyouwanttoterminate.
HowtoexitfromaNode.jsprogram
24
HowtoreadenvironmentvariablesLearnhowtoreadandmakeuseofenvironmentvariablesinaNode.jsprogram
The processcoremoduleofNodeprovidesthe envpropertywhichhostsalltheenvironmentvariablesthatweresetatthemomenttheprocesswasstarted.
HereisanexamplethataccessestheNODE_ENVenvironmentvariable,whichissettodevelopmentbydefault.
Note: processdoesnotrequirea"require",it'sautomaticallyavailable.
process.env.NODE_ENV//"development"
Settingitto"production"beforethescriptrunswilltellNodethatthisisaproductionenvironment.
Inthesamewayyoucanaccessanycustomenvironmentvariableyouset.
Howtoreadenvironmentvariables
25
NodehostingoptionsANode.jsapplicationcanbehostedinalotofplaces,dependingonyourneeds.Thisisalistofallthevariousoptionsyouhaveatyourdisposal
Hereisanon-exhaustivelistoftheoptionsyoucanexplorewhenyouwanttodeployyourappandmakeitpubliclyaccessible.
Iwilllisttheoptionsfromsimplestandconstrainedtomorecomplexandpowerful.
Simplestoptionever:localtunnelZeroconfigurationdeployments
GlitchCodepen
ServerlessPAAS
ZeitNowNanoboxHerokuMicrosoftAzureGoogleCloudPlatform
VirtualPrivateServerBaremetal
Simplestoptionever:localtunnelEvenifyouhaveadynamicIP,oryou'reunderaNAT,youcandeployyourappandservetherequestsrightfromyourcomputerusingalocaltunnel.
Thisoptionissuitedforsomequicktesting,demoaproductorsharingofanappwithaverysmallgroupofpeople.
Averynicetoolforthis,availableonallplatforms,isngrok.
Usingit,youcanjusttype ngrokPORTandthePORTyouwantisexposedtotheinternet.Youwillgetangrok.iodomain,butwithapaidsubscriptionyoucangetacustomURLaswellasmoresecurityoptions(rememberthatyouareopeningyourmachinetothepublicInternet).
Anotherserviceyoucanuseishttps://github.com/localtunnel/localtunnel
Zeroconfigurationdeployments
Nodehostingoptions
26
Glitch
Glitchisaplaygroundandawaytobuildyourappsfasterthanever,andseethemliveontheirownglitch.comsubdomain.Youcannotcurrentlyhaveaacustomdomain,andthereareafewrestrictionsinplace,butit'sreallygreattoprototype.Itlooksfun(andthisisaplus),andit'snotadumbeddownenvironment-yougetallthepowerofNode.js,aCDN,securestorageforcredentials,GitHubimport/exportandmuchmore.
ProvidedbythecompanybehindFogBugzandTrello(andco-creatorsofStackOverflow).
Iuseitalotfordemopurposes.
Codepen
Codepenisanamazingplatformandcommunity.Youcancreateaprojectwithmultiplefiles,anddeployitwithacustomdomain.
ServerlessAwaytopublishyourapps,andhavenoserveratalltomanage,isServerless.Serverlessisaparadigmwhereyoupublishyourappsasfunctions,andtheyrespondonanetworkendpoint(alsocalledFAAS-FunctionsAsAService).
Toverypopularsolutionsare
ServerlessFrameworkStandardLibrary
TheybothprovideanabstractionlayertopublishingonAWSLambdaandotherFAASsolutionsbasedonAzureortheGoogleCloudoffering.
PAASPAASstandsforPlatformAsAService.Theseplatformstakeawayalotofthingsyoushouldotherwiseworryaboutwhendeployingyourapplication.
ZeitNow
Zeitisaninterestingoption.Youjusttype nowinyourterminal,andittakescareofdeployingyourapplication.Thereisafreeversionwithlimitations,andthepaidversionismorepowerful.Yousimplyforgetthatthere'saserver,youjustdeploytheapp.
Nodehostingoptions
27
Nanobox
Nanobox
Heroku
Herokuisanamazingplatform.
ThisisagreatarticleongettingstartedwithNode.jsonHeroku.
MicrosoftAzure
AzureistheMicrosoftCloudoffering.
CheckouthowtocreateaNode.jswebappinAzure.
GoogleCloudPlatform
GoogleCloudisanamazingstructureforyourapps.
TheyhaveagoodNode.jsDocumentationSection
VirtualPrivateServerInthissectionyoufindtheusualsuspects,orderedfrommoreuserfriendlytolessuserfriendly:
DigitalOceanLinodeAmazonWebServices,inparticularImentionAmazonElasticBeanstalkasitabstractsawayalittlebitthecomplexityofAWS.
SincetheyprovideanemptyLinuxmachineonwhichyoucanwork,thereisnospecifictutorialforthese.
TherearelotsmoreoptionsintheVPScategory,thosearejusttheonesIusedandIwouldrecommend.
BaremetalAnothersolutionistogetabaremetalserver,installaLinuxdistribution,connectittotheinternet(orrentonemonthly,likeyoucandousingtheVultrBareMetalservice)
Nodehostingoptions
28
Nodehostingoptions
29
UsetheNodeREPLREPLstandsforRead-Evaluate-Print-Loop,andit'sagreatwaytoexploretheNodefeaturesinaquickway
The nodecommandistheoneweusetorunourNode.jsscripts:
nodescript.js
Ifweomitthefilename,weuseitinREPLmode:
node
Ifyoutryitnowinyourterminal,thisiswhathappens:
❯node>
thecommandstaysinidlemodeandwaitsforustoentersomething.
Tip:ifyouareunsurehowtoopenyourterminal,google"Howtoopenterminalon".
TheREPLiswaitingforustoentersomeJavaScriptcode,tobemoreprecise.
Startsimpleandenter
>console.log('test')
test
undefined
>
Thefirstvalue, test,istheoutputwetoldtheconsoletoprint,thenwegetundefinedwhichisthereturnvalueofrunning console.log().
WecannowenteranewlineofJavaScript.
UsethetabtoautocompleteThecoolthingabouttheREPListhatit'sinteractive.
Asyouwriteyourcode,ifyoupressthe tabkeytheREPLwilltrytoautocompletewhatyouwrotetomatchavariableyoualreadydefinedorapredefinedone.
UsetheNodeREPL
30
ExploringJavaScriptobjectsTryenteringthenameofaJavaScriptclass,like Number,addadotandpress tab.
TheREPLwillprintallthepropertiesandmethodsyoucanaccessonthatclass:
ExploreglobalobjectsYoucaninspecttheglobalsyouhaveaccesstobytyping global.andpressing tab:
UsetheNodeREPL
31
The_specialvariableIfaftersomecodeyoutype _,thatisgoingtoprinttheresultofthelastoperation.
DotcommandsTheREPLhassomespecialcommands,allstartingwithadot ..Theyare
.help:showsthedotcommandshelp.editor:enableseditormore,towritemultilineJavaScriptcodewithease.Onceyouareinthismode,enterctrl-Dtorunthecodeyouwrote..break:wheninputtingamulti-lineexpression,enteringthe.breakcommandwillabortfurtherinput.Sameaspressingctrl-C..clear:resetstheREPLcontexttoanemptyobjectandclearsanymulti-lineexpression
UsetheNodeREPL
32
currentlybeinginput..load:loadsaJavaScriptfile,relativetothecurrentworkingdirectory.save:savesallyouenteredintheREPLsessiontoafile(specifythefilename).exit:existstherepl(sameaspressingctrl-Ctwotimes)
TheREPLknowswhenyouaretypingamulti-linestatementwithouttheneedtoinvoke.editor.
Forexampleifyoustarttypinganiterationlikethis:
[1,2,3].forEach(num=>{
andyoupress enter,theREPLwillgotoanewlinethatstartswith3dots,indicatingyoucannowcontinuetoworkonthatblock.
...console.log(num)
...})
Ifyoutype .breakattheendofaline,themultilinemodewillstopandthestatementwillnotbeexecuted.
UsetheNodeREPL
33
PassargumentsfromthecommandlineHowtoacceptargumentsinaNode.jsprogrampassedfromthecommandline
YoucanpassanynumberofargumentswheninvokingaNode.jsapplicationusing
nodeapp.js
Argumentscanbestandaloneorhaveakeyandavalue.
Forexample:
nodeapp.jsflavio
or
nodeapp.jsname=flavio
ThischangeshowyouwillretrievethisvalueintheNodecode.
Thewayyouretrieveitisusingthe processobjectbuiltintoNode.
Itexposesan argvproperty,whichisanarraythatcontainsallthecommandlineinvocationarguments.
Thefirstargumentisthefullpathofthe nodecommand.
Thesecondelementisthefullpathofthefilebeingexecuted.
Alltheadditionalargumentsarepresentfromthethirdpositiongoingforward.
Youcaniterateoverallthearguments(includingthenodepathandthefilepath)usingaloop:
process.argv.forEach((val,index)=>{
console.log(`${index}:${val}`)
})
Youcangetonlytheadditionalargumentsbycreatinganewarraythatexcludesthefirst2params:
constargs=process.argv.slice(2)
Passargumentsfromthecommandline
34
Ifyouhaveoneargumentwithoutanindexname,likethis:
nodeapp.jsflavio
youcanaccessitusing
constargs=process.argv.slice(2)
args[0]
Inthiscase:
nodeapp.jsname=flavio
args[0]is name=flavio,andyouneedtoparseit.Thebestwaytodosoisbyusingtheminimistlibrary,whichhelpsdealingwitharguments:
constargs=require('minimist')(process.argv.slice(2))
args['name']//flavio
Passargumentsfromthecommandline
35
OutputtothecommandlineHowtoprinttothecommandlineconsoleusingNode,fromthebasicconsole.logtomorecomplexscenarios
BasicoutputusingtheconsolemoduleCleartheconsoleCountingelementsPrintthestacktraceCalculatethetimespentstdoutandstderrColortheoutputCreateaprogressbar
BasicoutputusingtheconsolemoduleNodeprovidesa consolemodulewhichprovidestonsofveryusefulwaystointeractwiththecommandline.
Itisbasicallythesameasthe consoleobjectyoufindinthebrowser.
Themostbasicandmostusedmethodis console.log(),whichprintsthestringyoupasstoittotheconsole.
Ifyoupassanobject,itwillrenderitasastring.
Youcanpassmultiplevariablesto console.log,forexample:
constx='x'
consty='y'
console.log(x,y)
andNodewillprintboth.
Wecanalsoformatprettyphrasesbypassingvariablesandaformatspecifier.
Forexample:
console.log('My%shas%dyears','cat',2)
%sformatavariableasastring%dor %iformatavariableasaninteger
Outputtothecommandline
36
%fformatavariableasafloatingpointnumber%Ousedtoprintanobjectrepresentation
Example:
console.log('%O',Number)
Cleartheconsoleconsole.clear()clearstheconsole(thebehaviormightdependontheconsoleused)
Countingelementsconsole.count()isahandymethod.
Takethiscode:
constx=1
consty=2
constz=3
console.count(
'Thevalueofxis'+x+'andhasbeenchecked..howmanytimes?'
)
console.count(
'Thevalueofxis'+x+'andhasbeenchecked..howmanytimes?'
)
console.count(
'Thevalueofyis'+y+'andhasbeenchecked..howmanytimes?'
)
Whathappensisthatcountwillcountthenumberoftimesastringisprinted,andprintthecountnexttoit:
Youcanjustcountapplesandoranges:
constoranges=['orange','orange']
constapples=['justoneapple']
oranges.forEach(fruit=>{
console.count(fruit)
})
apples.forEach(fruit=>{
console.count(fruit)
})
Outputtothecommandline
37
PrintthestacktraceTheremightbecaseswhereit'susefultoprintthecallstacktraceofafunction,maybetoanswerthequestionhowdidyoureachthatpartofthecode?
Youcandosousing console.trace():
constfunction2=()=>console.trace()
constfunction1=()=>function2()
function1()
Thiswillprintthestacktrace.Thisiswhat'sprintedifItrythisintheNodeREPL:
Trace
atfunction2(repl:1:33)
atfunction1(repl:1:25)
atrepl:1:1
atContextifyScript.Script.runInThisContext(vm.js:44:33)
atREPLServer.defaultEval(repl.js:239:29)
atbound(domain.js:301:14)
atREPLServer.runBound[aseval](domain.js:314:12)
atREPLServer.onLine(repl.js:440:10)
atemitOne(events.js:120:20)
atREPLServer.emit(events.js:210:7)
CalculatethetimespentYoucaneasilycalculatehowmuchtimeafunctiontakestorun,using time()and timeEnd()
constdoSomething=()=>console.log('test')
constmeasureDoingSomething=()=>{
console.time('doSomething()')
//dosomething,andmeasurethetimeittakes
doSomething()
console.timeEnd('doSomething()')
}
measureDoingSomething()
stdoutandstderrAswesawconsole.logisgreatforprintingmessagesintheConsole.Thisiswhat'scalledthestandardoutput,or stdout.
console.errorprintstothe stderrstream.
Outputtothecommandline
38
Itwillnotappearintheconsole,butitwillappearintheerrorlog.
ColortheoutputYoucancolortheoutputofyourtextintheconsolebyusingescapesequences.Anescapesequenceisasetofcharactersthatidentifiesacolor.
Example:
console.log('\x1b[33m%s\x1b[0m','hi!')
YoucantrythatintheNodeREPL,anditwillprint hi!inyellow.
However,thisisthelow-levelwaytodothis.Thesimplestwaytogoaboutcoloringtheconsoleoutputisbyusingalibrary.Chalkissuchalibrary,andinadditiontocoloringitalsohelpswithotherstylingfacilities,likemakingtextbold,italicorunderlined.
Youinstallitwith npminstallchalk,thenyoucanuseit:
constchalk=require('chalk')
console.log(chalk.yellow('hi!'))
Using chalk.yellowismuchmoreconvenientthantryingtoremembertheescapecodes,andthecodeismuchmorereadable.
ChecktheprojectlinkIpostedaboveformoreusageexamples.
CreateaprogressbarProgressisanawesomepackagetocreateaprogressbarintheconsole.Installitusing npminstallprogress
Thissnippetcreatesa10-stepprogressbar,andevery100msonestepiscompleted.Whenthebarcompleteswecleartheinterval:
constProgressBar=require('progress')
constbar=newProgressBar(':bar',{total:10})
consttimer=setInterval(()=>{
bar.tick()
if(bar.complete){
clearInterval(timer)
}
},100)
Outputtothecommandline
39
Outputtothecommandline
40
AcceptinputfromthecommandlineHowtomakeaNode.jsCLIprograminteractiveusingthebuilt-inreadlineNodemodule
HowtomakeaNode.jsCLIprograminteractive?
Nodesinceversion7providesthe readlinemoduletoperformexactlythis:getinputfromareadablestreamsuchasthe process.stdinstream,whichduringtheexecutionofaNodeprogramistheterminalinput,onelineatatime.
constreadline=require('readline').createInterface({
input:process.stdin,
output:process.stdout
})
readline.question(`What'syourname?`,(name)=>{
console.log(`Hi${name}!`)
readline.close()
})
Thispieceofcodeaskstheusername,andoncethetextisenteredandtheuserpressesenter,wesendagreeting.
The question()methodshowsthefirstparameter(aquestion)andwaitsfortheuserinput.Itcallsthecallbackfunctiononceenterispressed.
Inthiscallbackfunction,weclosethereadlineinterface.
readlineoffersseveralothermethods,andI'llletyoucheckthemoutonthepackagedocumentationIlinkedabove.
Ifyouneedtorequireapassword,it'sbesttonowechoitback,butinsteadshowinga *symbol.
Thesimplestwayistousethe readline-syncpackagewhichisverysimilarintermsoftheAPIandhandlesthisoutofthebox.
AmorecompleteandabstractsolutionisprovidedbytheInquirer.jspackage.
Youcaninstallitusing npminstallinquirer,andthenyoucanreplicatetheabovecodelikethis:
constinquirer=require('inquirer')
varquestions=[{
type:'input',
Acceptinputfromthecommandline
41
name:'name',
message:"What'syourname?",
}]
inquirer.prompt(questions).then(answers=>{
console.log(`Hi${answers['name']}!`)
})
Inquirer.jsletsyoudomanythingslikeaskingmultiplechoices,havingradiobuttons,confirmations,andmore.
It'sworthknowingallthealternatives,especiallythebuilt-inonesprovidedbyNode,butifyouplantotakeCLIinputtothenextlevel,Inquirer.jsisanoptimalchoice.
Acceptinputfromthecommandline
42
ExposefunctionalityfromaNodefileusingexportsHowtousethemodule.exportsAPItoexposedatatootherfilesinyourapplication,ortootherapplicationsaswell
Nodehasabuilt-inmodulesystem.
ANode.jsfilecanimportfunctionalityexposedbyotherNode.jsfiles.
Whenyouwanttoimportsomethingyouuse
constlibrary=require('./library')
toimportthefunctionalityexposedinthe library.jsfilethatresidesinthecurrentfilefolder.
Inthisfile,functionalitymustbeexposedbeforeitcanbeimportedbyotherfiles.
Anyotherobjectorvariabledefinedinthefilebydefaultisprivateandnotexposedtotheouterworld.
Thisiswhatthe module.exportsAPIofferedbythe modulesystemallowsustodo.
Whenyouassignanobjectorafunctionasanew exportsproperty,thatisthethingthat'sbeingexposed,andassuch,itcanbeimportedinotherpartsofyourapp,orinotherappsaswell.
Youcandosoin2ways.
Thefirstistoassignanobjectto module.exports,whichisanobjectprovidedoutoftheboxbythemodulesystem,andthiswillmakeyourfileexportjustthatobject:
constcar={
brand:'Ford',
model:'Fiesta'
}
module.exports=car
//..intheotherfile
constcar=require('./car')
Thesecondwayistoaddtheexportedobjectasapropertyof exports.Thiswayallowsyoutoexportmultipleobjects,functionsordata:
ExposefunctionalityfromaNodefileusingexports
43
constcar={
brand:'Ford',
model:'Fiesta'
}
exports.car=car
ordirectly
exports.car={
brand:'Ford',
model:'Fiesta'
}
Andintheotherfile,you'lluseitbyreferencingapropertyofyourimport:
constitems=require('./items')
items.car
or
constcar=require('./items').car
What'sthedifferencebetween module.exportsand exports?
Thefirstexposestheobjectitpointsto.Thelatterexposesthepropertiesoftheobjectitpointsto.
ExposefunctionalityfromaNodefileusingexports
44
npmAquickguidetonpm,thepowerfulpackagemanagerkeytothesuccessofNode.js.InJanuary2017over350000packageswerereportedbeinglistedinthenpmregistry,makingitthebiggestsinglelanguagecoderepositoryonEarth,andyoucanbesurethereisapackagefor(almost!)everything.
IntroductiontonpmDownloads
InstallingalldependenciesInstallingasinglepackageUpdatingpackages
VersioningRunningTasks
IntroductiontonpmnpmisthestandardpackagemanagerforNode.js.
npm
45
InJanuary2017over350000packageswerereportedbeinglistedinthenpmregistry,makingitthebiggestsinglelanguagecoderepositoryonEarth,andyoucanbesurethereisapackagefor(almost!)everything.
ItstartedasawaytodownloadandmanagedependenciesofNode.jspackages,butithassincebecomeatoolusedalsoinfrontendJavaScript.
Therearemanythingsthat npmdoes.
Yarnisanalternativetonpm.Makesureyoucheckitoutaswell.
Downloadsnpmmanagesdownloadsofdependenciesofyourproject.
Installingalldependencies
Ifaprojecthasa packages.jsonfile,byrunning
npminstall
itwillinstalleverythingtheprojectneeds,inthe node_modulesfolder,creatingitifit'snotexistingalready.
Installingasinglepackage
Youcanalsoinstallaspecificpackagebyrunning
npminstall<package-name>
Oftenyou'llseemoreflagsaddedtothiscommand:
--saveinstallsandaddstheentrytothe package.jsonfiledependencies--save-devinstallsandaddstheentrytothe package.jsonfiledevDependencies
ThedifferenceismainlythatdevDependenciesareusuallydevelopmenttools,likeatestinglibrary,while dependenciesarebundledwiththeappinproduction.
Updatingpackages
Updatingisalsomadeeasy,byrunning
npmupdate
npm
46
npmwillcheckallpackagesforanewerversionthatsatisfiesyourversioningconstraints.
Youcanspecifyasinglepackagetoupdateaswell:
npmupdate<package-name>
VersioningInadditiontoplaindownloads, npmalsomanagesversioning,soyoucanspecifyanyspecificversionofapackage,orrequireaversionhigherorlowerthanwhatyouneed.
Manytimesyou'llfindthatalibraryisonlycompatiblewithamajorreleaseofanotherlibrary.
Orabuginthelatestreleaseofalib,stillunfixed,iscausinganissue.
Specifyinganexplicitversionofalibraryalsohelpstokeepeveryoneonthesameexactversionofapackage,sothatthewholeteamrunsthesameversionuntilthe package.jsonfileisupdated.
Inallthosecases,versioninghelpsalot,and npmfollowsthesemanticversioning(semver)standard.
RunningTasksThepackage.jsonfilesupportsaformatforspecifyingcommandlinetasksthatcanberunbyusing
npmrun<task-name>
Forexample:
{
"scripts":{
"start-dev":"nodelib/server-development",
"start":"nodelib/server-production"
},
}
It'sverycommontousethisfeaturetorunWebpack:
{
"scripts":{
npm
47
"watch":"webpack--watch--progress--colors--configwebpack.conf.js",
"dev":"webpack--progress--colors--configwebpack.conf.js",
"prod":"NODE_ENV=productionwebpack-p--configwebpack.conf.js",
},
}
Soinsteadoftypingthoselongcommands,whichareeasytoforgetormistype,youcanrun
$npmrunwatch
$npmrundev
$npmrunprod
npm
48
WheredoesnpminstallthepackagesHowtofindoutwherenpminstallsthepackages
Readthenpmguideifyouarestartingoutwithnpm,it'sgoingtogoinalotofthebasicdetailsofit.
Whenyouinstallapackageusing npm(oryarn),youcanperform2typesofinstallation:
alocalinstallaglobalinstall
Bydefault,whenyoutypean npminstallcommand,like:
npminstalllodash
thepackageisinstalledinthecurrentfiletree,underthe node_modulessubfolder.
Asthishappens, npmalsoaddsthe lodashentryinthe dependenciespropertyofthepackage.jsonfilepresentinthecurrentfolder.
Aglobalinstallationisperformedusingthe -gflag:
npminstall-glodash
Whenthishappens,npmwon'tinstallthepackageunderthelocalfolder,butinstead,itwilluseagloballocation.
Where,exactly?
The npmroot-gcommandwilltellyouwherethatexactlocationisonyourmachine.
OnmacOSorLinuxthislocationcouldbe /usr/local/lib/node_modules.OnWindowsitcouldbe C:\Users\YOU\AppData\Roaming\npm\node_modules
Ifyouuse nvmtomanageNode.jsversions,however,thatlocationwoulddiffer.
Iforexampleuse nvmandmypackageslocationwasshownas/Users/flavio/.nvm/versions/node/v8.9.0/lib/node_modules.
Wheredoesnpminstallthepackages
49
HowtouseorexecuteapackageinstalledusingnpmHowtoincludeanduseinyourcodeapackageinstalledinyournode_modulesfolder
Whenyouinstallusing npmapackageintoyour node_modulesfolder,oralsoglobally,howdoyouuseitinyourNodecode?
Sayyouinstall lodash,thepopularJavaScriptutilitylibrary,using
npminstalllodash
Thisisgoingtoinstallthepackageinthelocal node_modulesfolder.
Touseitinyourcode,youjustneedtoimportitintoyourprogramusing require:
const_=require('lodash)
Whatifyourpackageisanexecutable?
Inthiscase,itwillputtheexecutablefileunderthe node_modules/.bin/folder.
Oneeasywaytodemonstratethisiscowsay.
Thecowsaypackageprovidesacommandlineprogramthatcanbeexecutedtomakeacowsaysomething(andotheranimalsaswellࢤ ).
Whenyouinstallthepackageusing npminstallcowsay,itwillinstallitselfandafewdependenciesinthenode_modulesfolder:
Howtouseorexecuteapackageinstalledusingnpm
50
Thereisahidden.binfolder,whichcontainssymboliclinkstothecowsaybinaries:
Howdoyouexecutethose?
Youcanofcoursetype ./node_modules/.bin/cowsaytorunit,anditworks,butnpx,includedintherecentversionsofnpm(since5.2),isamuchbetteroption.Youjustrun:
npxcowsay
andnpxwillfindthepackagelocation.
Howtouseorexecuteapackageinstalledusingnpm
51
Howtouseorexecuteapackageinstalledusingnpm
52
Thepackage.jsonfileThepackage.jsonfileisakeyelementinlotsofappcodebasesbasedontheNode.jsecosystem.
IfyouworkwithJavaScript,oryou'veeverinteractedwithaJavaScriptproject,Node.jsorafrontendproject,yousurelymetthe package.jsonfile.
What'sthatfor?Whatshouldyouknowaboutit,andwhataresomeofthecoolthingsyoucandowithit?
The package.jsonfileiskindofamanifestforyourproject.Itcandoalotofthings,completelyunrelated.It'sacentralrepositoryofconfigurationfortools,forexample.It'salsowhere npmand yarnstorethenamesandversionsofthepackageitinstalled.
ThefilestructurePropertiesbreakdown
name
author
contributors
bugs
homepage
version
license
keywords
description
repository
main
private
scripts
dependencies
devDependencies
engines
browserslist
Command-specificpropertiesPackageversions
ThefilestructureHere'sanexamplepackage.jsonfile:
Thepackage.jsonfile
53
{
}
It'sempty!Therearenofixedrequirementsofwhatshouldbeina package.jsonfile,foranapplication.TheonlyrequirementisthatitrespectstheJSONformat,otherwiseitcannotbereadbyprogramsthattrytoaccessitspropertiesprogrammatically.
Ifyou'rebuildingaNode.jspackagethatyouwanttodistributeover npmthingschangeradically,andyoumusthaveasetofpropertiesthatwillhelpotherpeopleuseit.We'llseemoreaboutthislateron.
Thisisanotherpackage.json:
{
"name":"test-project"
}
Itdefinesa nameproperty,whichtellsthenameoftheapp,orpackage,that'scontainedinthesamefolderwherethisfilelives.
Here'samuchmorecomplexexample,whichIextractedthisfromasampleVue.jsapplication:
{
"name":"test-project",
"version":"1.0.0",
"description":"AVue.jsproject",
"main":"src/main.js",
"private":true,
"scripts":{
"dev":"webpack-dev-server--inline--progress--configbuild/webpack.dev.conf.js",
"start":"npmrundev",
"unit":"jest--configtest/unit/jest.conf.js--coverage",
"test":"npmrununit",
"lint":"eslint--ext.js,.vuesrctest/unit",
"build":"nodebuild/build.js"
},
"dependencies":{
"vue":"^2.5.2"
},
"devDependencies":{
"autoprefixer":"^7.1.2",
"babel-core":"^6.22.1",
"babel-eslint":"^8.2.1",
"babel-helper-vue-jsx-merge-props":"^2.0.3",
"babel-jest":"^21.0.2",
"babel-loader":"^7.1.1",
"babel-plugin-dynamic-import-node":"^1.2.0",
"babel-plugin-syntax-jsx":"^6.18.0",
Thepackage.jsonfile
54
"babel-plugin-transform-es2015-modules-commonjs":"^6.26.0",
"babel-plugin-transform-runtime":"^6.22.0",
"babel-plugin-transform-vue-jsx":"^3.5.0",
"babel-preset-env":"^1.3.2",
"babel-preset-stage-2":"^6.22.0",
"chalk":"^2.0.1",
"copy-webpack-plugin":"^4.0.1",
"css-loader":"^0.28.0",
"eslint":"^4.15.0",
"eslint-config-airbnb-base":"^11.3.0",
"eslint-friendly-formatter":"^3.0.0",
"eslint-import-resolver-webpack":"^0.8.3",
"eslint-loader":"^1.7.1",
"eslint-plugin-import":"^2.7.0",
"eslint-plugin-vue":"^4.0.0",
"extract-text-webpack-plugin":"^3.0.0",
"file-loader":"^1.1.4",
"friendly-errors-webpack-plugin":"^1.6.1",
"html-webpack-plugin":"^2.30.1",
"jest":"^22.0.4",
"jest-serializer-vue":"^0.3.0",
"node-notifier":"^5.1.2",
"optimize-css-assets-webpack-plugin":"^3.2.0",
"ora":"^1.2.0",
"portfinder":"^1.0.13",
"postcss-import":"^11.0.0",
"postcss-loader":"^2.0.8",
"postcss-url":"^7.2.1",
"rimraf":"^2.6.0",
"semver":"^5.3.0",
"shelljs":"^0.7.6",
"uglifyjs-webpack-plugin":"^1.1.1",
"url-loader":"^0.5.8",
"vue-jest":"^1.0.2",
"vue-loader":"^13.3.0",
"vue-style-loader":"^3.0.1",
"vue-template-compiler":"^2.5.2",
"webpack":"^3.6.0",
"webpack-bundle-analyzer":"^2.9.0",
"webpack-dev-server":"^2.9.1",
"webpack-merge":"^4.1.0"
},
"engines":{
"node":">=6.0.0",
"npm":">=3.0.0"
},
"browserslist":[
">1%",
"last2versions",
"notie<=8"
]
}
therearelotsofthingsgoingonhere:
Thepackage.jsonfile
55
namesetstheapplication/packagenameversionindicatesthecurrentversiondescriptionisabriefdescriptionoftheapp/packagemainsettheentrypointfortheapplicationprivateifsetto truepreventstheapp/packagetobeaccidentallypublishedon npmscriptsdefinesasetofnodescriptsyoucanrundependenciessetsalistof npmpackagesinstalledasdependenciesdevDependenciessetsalistof npmpackagesinstalledasdevelopmentdependenciesenginessetswhichversionsofNodethispackage/appworksonbrowserslistisusedtotellwhichbrowsers(andtheirversions)youwanttosupport
Allthosepropertiesareusedbyeither npmorothertoolsthatwecanuse.
PropertiesbreakdownThissectiondescribesthepropertiesyoucanuseindetail.Ireferto"package"butthesamethingappliestolocalapplicationswhichyoudonotuseaspackages.
Mostofthosepropertiesareonlyusedonthehttps://www.npmjs.com/,otherbyscriptsthatinteractwithyourcode,like npmorothers.
name
Setsthepackagename.
Example:
"name":"test-project"
Thenamemustbelessthan214characters,mustnothavespaces,itcanonlycontainlowercaseletters,hyphens( -)orunderscores( _).
Thisisbecausewhenapackageispublishedon npm,itgetsitsownURLbasedonthisproperty.
IfyoupublishedthispackagepubliclyonGitHub,agoodvalueforthispropertyistheGitHubrepositoryname.
author
Liststhepackageauthorname
Example:
Thepackage.jsonfile
56
{
"author":"FlavioCopes<[email protected]>(https://flaviocopes.com)"
}
Canalsobeusedwiththisformat:
{
"author":{
"name":"FlavioCopes",
"email":"[email protected]",
"url":"https://flaviocopes.com"
}
}
contributors
Aswellastheauthor,theprojectcanhaveoneormorecontributors.Thispropertyisanarraythatliststhem.
Example:
{
"contributors":[
"FlavioCopes<[email protected]>(https://flaviocopes.com)"
]
}
Canalsobeusedwiththisformat:
{
"contributors":[
{
"name":"FlavioCopes",
"email":"[email protected]",
"url":"https://flaviocopes.com"
}
]
}
bugs
Linkstothepackageissuetracker,mostlikelyaGitHubissuespage
Example:
{
"bugs":"https://github.com/flaviocopes/package/issues"
Thepackage.jsonfile
57
}
homepage
Setsthepackagehomepage
Example:
{
"homepage":"https://flaviocopes.com/package"
}
version
Indicatesthecurrentversionofthepackage.
Example:
"version":"1.0.0"
Thispropertyfollowsthesemanticversioning(semver)notationforversions,whichmeanstheversionisalwaysexpressedwith3numbers: x.x.x.
Thefirstnumberisthemajorversion,thesecondtheminorversionandthethirdisthepatchversion.
Thereisameaninginthesenumbers:areleasethatonlyfixesbugsisapatchrelease,areleasethatintroducesbackward-compatiblechangesisaminorrelease,amajorreleasecanhavebreakingchanges.
license
Indicatesthelicenseofthepackage.
Example:
"license":"MIT"
keywords
Thispropertycontainsanarrayofkeywordsthatassociatewithwhatyourpackagedoes.
Example:
Thepackage.jsonfile
58
"keywords":[
"email",
"machinelearning",
"ai"
]
Thishelpspeoplefindyourpackagewhennavigatingsimilarpackages,orwhenbrowsingthehttps://www.npmjs.com/website.
description
Thispropertycontainsabriefdescriptionofthepackage
Example:
"description":"Apackagetoworkwithstrings"
Thisisespeciallyusefulifyoudecidetopublishyourpackageto npmsothatpeoplecanfindoutwhatthepackageisabout.
repository
Thispropertyspecifieswherethispackagerepositoryislocated.
Example:
"repository":"github:flaviocopes/testing",
Noticethe githubprefix.Thereareotherpopularservicesbakedin:
"repository":"gitlab:flaviocopes/testing",
"repository":"bitbucket:flaviocopes/testing",
Youcanexplicitlysettheversioncontrolsystem:
"repository":{
"type":"git",
"url":"https://github.com/flaviocopes/testing.git"
}
Youcanusedifferentversioncontrolsystems:
Thepackage.jsonfile
59
"repository":{
"type":"svn",
"url":"..."
}
main
Setstheentrypointforthepackage.
Whenyouimportthispackageinanapplication,that'swheretheapplicationwillsearchforthemoduleexports.
Example:
"main":"src/main.js"
private
ifsetto truepreventstheapp/packagetobeaccidentallypublishedon npm
Example:
"private":true
scripts
Definesasetofnodescriptsyoucanrun
Example:
"scripts":{
"dev":"webpack-dev-server--inline--progress--configbuild/webpack.dev.conf.js",
"start":"npmrundev",
"unit":"jest--configtest/unit/jest.conf.js--coverage",
"test":"npmrununit",
"lint":"eslint--ext.js,.vuesrctest/unit",
"build":"nodebuild/build.js"
}
Thesescriptsarecommandlineapplications.Youcanrunthembycalling npmrunXXXXoryarnXXXX,where XXXXisthecommandname.Example: npmrundev.
Youcanuseanynameyouwantforacommand,andscriptscandoliterallyanythingyouwant.
Thepackage.jsonfile
60
dependencies
Setsalistof npmpackagesinstalledasdependencies.
Whenyouinstallapackageusingnpmoryarn:
npminstall<PACKAGENAME>
yarnadd<PACKAGENAME>
thatpackageisautomaticallyinsertedinthislist.
Example:
"dependencies":{
"vue":"^2.5.2"
}
devDependencies
Setsalistof npmpackagesinstalledasdevelopmentdependencies.
Theydifferfrom dependenciesbecausetheyaremeanttobeinstalledonlyonadevelopmentmachine,notneededtorunthecodeinproduction.
Whenyouinstallapackageusingnpmoryarn:
npminstall--dev<PACKAGENAME>
yarnadd--dev<PACKAGENAME>
thatpackageisautomaticallyinsertedinthislist.
Example:
"devDependencies":{
"autoprefixer":"^7.1.2",
"babel-core":"^6.22.1"
}
engines
SetswhichversionsofNode.jsandothercommandsthispackage/appworkon
Example:
"engines":{
Thepackage.jsonfile
61
"node":">=6.0.0",
"npm":">=3.0.0",
"yarn":"^0.13.0"
}
browserslist
Isusedtotellwhichbrowsers(andtheirversions)youwanttosupport.It'sreferencedbyBabel,Autoprefixer,andothertools,toonlyaddthepolyfillsandfallbacksneededtothebrowsersyoutarget.
Example:
"browserslist":[
">1%",
"last2versions",
"notie<=8"
]
Thisconfigurationmeansyouwanttosupportthelast2majorversionsofallbrowserswithatleast1%ofusage(fromtheCanIUse.comstats),exceptIE8andlower.
(seemore)
Command-specificproperties
The package.jsonfilecanalsohostcommand-specificconfiguration,forexampleforBabel,ESLint,andmore.
Eachhasaspecificproperty,like eslintConfig, babelandothers.Thosearecommand-specific,andyoucanfindhowtousethoseintherespectivecommand/projectdocumentation.
PackageversionsYouhaveseeninthedescriptionaboveversionnumberslikethese: ~3.0.0or 0.13.0.Whatdotheymean,andwhichotherversionspecifierscanyouuse?
Thatsymbolspecifieswhichupdatesyoupackageaccepts,fromthatdependency.
Giventhatusingsemver(semanticversioning)allversionshave3digits,thefirstbeingthemajorrelease,thesecondtheminorreleaseandthethirdisthepatchrelease,youhavetheserules:
~:ifyouwrite ~0.13.0,youwanttoonlyupdatepatchreleases: 0.13.1isok,but
Thepackage.jsonfile
62
0.14.0isnot. :ifyouwrite 0.13.0,youwanttoupdatepatchandminorreleases: 0.13.1, 0.14.0andsoon.*:ifyouwrite *,thatmeansyouacceptallupdates,includingmajorversionupgrades.>:youacceptanyversionhigherthantheoneyouspecify>=:youacceptanyversionequaltoorhigherthantheoneyouspecify<=:youacceptanyversionequalorlowertotheoneyouspecify<:youacceptanyversionlowertotheoneyouspecify
Thereareotherrules,too:
nosymbol:youacceptonlythatspecificversionyouspecifylatest:youwanttousethelatestversionavailable
andyoucancombinemostoftheaboveinranges,likethis: 1.0.0||>=1.1.0<1.2.0,toeitheruse1.0.0oronereleasefrom1.1.0up,butlowerthan1.2.0.
Thepackage.jsonfile
63
Thepackage-lock.jsonfileThepackage-lock.jsonfileisautomaticallygeneratedwheninstallingnodepackages.Learnwhatit'sabout
Inversion5,npmintroducedthe package-lock.jsonfile.
What'sthat?Youprobablyknowaboutthe package.jsonfile,whichismuchmorecommonandhasbeenaroundformuchlonger.
Thegoalofthefileistokeeptrackoftheexactversionofeverypackagethatisinstalledsothataproductis100%reproducibleinthesamewayevenifpackagesareupdatedbytheirmaintainers.
Thissolvesaveryspecificproblemthat package.jsonleftunsolved.Inpackage.jsonyoucansetwhichversionsyouwanttoupgradeto(patchorminor),usingthesemvernotation,forexample:
ifyouwrite ~0.13.0,youwanttoonlyupdatepatchreleases: 0.13.1isok,but 0.14.0isnot.ifyouwrite 0.13.0,youwanttoupdatepatchandminorreleases: 0.13.1, 0.14.0andsoon.ifyouwrite 0.13.0,thatistheexactversionthatwillbeused,always
Youdon'tcommittoGityournode_modulesfolder,whichisgenerallyhuge,andwhenyoutrytoreplicatetheprojectonanothermachinebyusingthe npminstallcommand,ifyouspecifiedthe ~syntaxandapatchreleaseofapackagehasbeenreleased,thatoneisgoingtobeinstalled.Samefor andminorreleases.
Ifyouspecifyexactversions,like 0.13.0intheexample,youarenotaffectedbythisproblem.
Itcouldbeyou,oranotherpersontryingtoinitializetheprojectontheothersideoftheworldbyrunning npminstall.
Soyouroriginalprojectandthenewlyinitializedprojectareactuallydifferent.Evenifapatchorminorreleaseshouldnotintroducebreakingchanges,weallknowbugscan(andso,theywill)slidein.
The package-lock.jsonsetsyourcurrentlyinstalledversionofeachpackageinstone,andnpmwillusethoseexactversionswhenrunning npminstall.
Thisconceptisnotnew,andotherprogramminglanguagespackagemanagers(likeComposerinPHP)useasimilarsystemforyears.
Thepackage-lock.jsonfile
64
The package-lock.jsonfileneedstobecommittedtoyourGitrepository,soitcanbefetchedbyotherpeople,iftheprojectispublicoryouhavecollaborators,orifyouuseGitasasourcefordeployments.
Thedependenciesversionswillbeupdatedinthe package-lock.jsonfilewhenyourun npmupdate.
AnexampleThisisanexamplestructureofa package-lock.jsonfilewegetwhenwerun npminstallcowsayinanemptyfolder:
{
"requires":true,
"lockfileVersion":1,
"dependencies":{
"ansi-regex":{
"version":"3.0.0",
"resolved":"https://registry.npmjs.org/ansi-regex/-/ansi-regex-3.
0.0.tgz",
"integrity":"sha1-7QMXwyIGT3lGbAKWa922Bas32Zg="
},
"cowsay":{
"version":"1.3.1",
"resolved":"https://registry.npmjs.org/cowsay/-/cowsay-1.3.1.tgz"
,
"integrity":"sha512-3PVFe6FePVtPj1HTeLin9v8WyLl+VmM1l1H/5P+BTTDkM
Ajufp+0F9eLjzRnOHzVAYeIYFF5po5NjRrgefnRMQ==",
"requires":{
"get-stdin":"^5.0.1",
"optimist":"~0.6.1",
"string-width":"~2.1.1",
"strip-eof":"^1.0.0"
}
},
"get-stdin":{
"version":"5.0.1",
"resolved":"https://registry.npmjs.org/get-stdin/-/get-stdin-5.0.
1.tgz",
"integrity":"sha1-Ei4WFZHiH/TFJTAwVpPyDmOTo5g="
},
"is-fullwidth-code-point":{
"version":"2.0.0",
"resolved":"https://registry.npmjs.org/is-fullwidth-code-point/-/
is-fullwidth-code-point-2.0.0.tgz",
"integrity":"sha1-o7MKXE8ZkYMWeqq5O+764937ZU8="
},
"minimist":{
"version":"0.0.10",
"resolved":"https://registry.npmjs.org/minimist/-/minimist-0.0.10
.tgz",
Thepackage-lock.jsonfile
65
"integrity":"sha1-3j+YVD2/lggr5IrRoMfNqDYwHc8="
},
"optimist":{
"version":"0.6.1",
"resolved":"https://registry.npmjs.org/optimist/-/optimist-0.6.1.tgz",
"integrity":"sha1-2j6nRob6IaGaERwybpDrFaAZZoY=",
"requires":{
"minimist":"~0.0.1",
"wordwrap":"~0.0.2"
}
},
"string-width":{
"version":"2.1.1",
"resolved":"https://registry.npmjs.org/string-width/-/string-width-2.1.1.tgz",
"integrity":"sha512-nOqH59deCq9SRHlxq1Aw85Jnt4w6KvLKqWVik6oA9ZklXLNIOlqg4F2yrT1MVaT
jAqvVwdfeZ7w7aCvJD7ugkw==",
"requires":{
"is-fullwidth-code-point":"^2.0.0",
"strip-ansi":"^4.0.0"
}
},
"strip-ansi":{
"version":"4.0.0",
"resolved":"https://registry.npmjs.org/strip-ansi/-/strip-ansi-4.0.0.tgz",
"integrity":"sha1-qEeQIusaw2iocTibY1JixQXuNo8=",
"requires":{
"ansi-regex":"^3.0.0"
}
},
"strip-eof":{
"version":"1.0.0",
"resolved":"https://registry.npmjs.org/strip-eof/-/strip-eof-1.0.0.tgz",
"integrity":"sha1-u0P/VZim6wXYm1n80SnJgzE2Br8="
},
"wordwrap":{
"version":"0.0.3",
"resolved":"https://registry.npmjs.org/wordwrap/-/wordwrap-0.0.3.tgz",
"integrity":"sha1-o9XabNXAvAAI03I0u68b7WMFkQc="
}
}
}
Weinstalled cowsay,whichdependson
get-stdin
optimist
string-width
strip-eof
Inturn,thosepackagesrequireotherpackages,aswecanseefromthe requirespropertythatsomehave:
Thepackage-lock.jsonfile
66
ansi-regex
is-fullwidth-code-point
minimist
wordwrap
strip-eof
Theyareaddedinalphabeticalorderintothefile,andeachonehasa versionfield,aresolvedfieldthatpointstothepackagelocation,andan integritystringthatwecanusetoverifythepackage.
Thepackage-lock.jsonfile
67
FindtheinstalledversionofannpmpackageHowtofindoutwhichversionofaparticularpackageyouhaveinstalledinyourapp
Toseethelatestversionofallthenpmpackageinstalled,includingtheirdependencies:
npmlist
Example:
❯npmlist/Users/flavio/dev/node/cowsay
Youcanalsojustopenthe package-lock.jsonfile,butthisinvolvessomevisualscanning.
npmlist-gisthesame,butforgloballyinstalledpackages.
Togetonlyyourtop-levelpackages(basically,theonesyoutoldnpmtoinstallandyoulistedinthe package.json),run npmlist--depth=0:
❯npmlist--depth=0/Users/flavio/dev/node/cowsay
Youcangettheversionofaspecificpackagebyspecifyingthename:
❯npmlistcowsay/Users/flavio/dev/node/cowsay
Thisalsoworksfordependenciesofpackagesyouinstalled:
Findtheinstalledversionofannpmpackage
68
❯npmlistminimist/Users/flavio/dev/node/cowsay
Ifyouwanttoseewhat'sthelatestavailableversionofthepackageonthenpmrepository,runnpmview[package_name]version:
❯npmviewcowsayversion
1.3.1
Findtheinstalledversionofannpmpackage
69
HowtoinstallanolderversionofannpmpackageLearnhowtoinstallanolderversionofannpmpackage,somethingthatmightbeusefultosolveacompatibilityproblem
Youcaninstallanoldversionofannpmpackageusingthe @syntax:
npminstall<package>@<version>
Example:
npminstallcowsay
installsversion1.3.1(atthetimeofwriting).
Installversion1.2.0with:
Thesamecanbedonewithglobalpackages:
Youmightalsobeinterestedinlistingallthepreviousversionofapackage.Youcandoitwithnpmview<package>versions:
❯npmviewcowsayversions
['1.0.0',
'1.0.1',
'1.0.2',
'1.0.3',
'1.1.0',
'1.1.1',
'1.1.2',
'1.1.3',
'1.1.4',
'1.1.5',
'1.1.6',
'1.1.7',
'1.1.8',
'1.1.9',
'1.2.0',
'1.2.1',
Howtoinstallanolderversionofannpmpackage
70
'1.3.0',
'1.3.1']
Howtoinstallanolderversionofannpmpackage
71
HowtoupdatealltheNodedependenciestotheirlatestversionHowdoyouupdateallthenpmdependenciesstoreinthepackage.jsonfile,totheirlatestversionavailable?
Whenyouinstallapackageusing npminstall<packagename>,thelatestavailableversionofthepackageisdownloadedandputinthe node_modulesfolder,andacorrespondingentryisaddedtothe package.jsonand package-lock.jsonfilesthatarepresentinyourcurrentfolder.
npmcalculatesthedependenciesandinstallsthelatestavailableversionofthoseaswell.
Let'ssayyouinstall cowsay,acoolcommandlinetoolthatletsyoumakeacowsaythings.
Whenyou npminstallcowsay,thisentryisaddedtothe package.jsonfile:
{
"dependencies":{
"cowsay":"^1.3.1"
}
}
andthisisanextractof package-lock.json,whereIremovedthenesteddependenciesforclarity:
{
"requires":true,
"lockfileVersion":1,
"dependencies":{
"cowsay":{
"version":"1.3.1",
"resolved":"https://registry.npmjs.org/cowsay/-/cowsay-1.3.1.tgz",
"integrity":"sha512-3PVFe6FePVtPj1HTeLin9v8WyLl+VmM1l1H/5P+BTTDkMAjufp+0F9eLjzRnOHz
VAYeIYFF5po5NjRrgefnRMQ==",
"requires":{
"get-stdin":"^5.0.1",
"optimist":"~0.6.1",
"string-width":"~2.1.1",
"strip-eof":"^1.0.0"
}
}
}
}
HowtoupdatealltheNodedependenciestotheirlatestversion
72
Nowthose2filestellusthatweinstalledversion 1.3.1ofcowsay,andourruleforupdatesis 1.3.1,whichforthenpmversioningrulesmeansthatnpmcanupdatetopatchandminorreleases: 0.13.1, 0.14.0andsoon.
Ifthereisanewminororpatchreleaseandwetype npmupdate,theinstalledversionisupdated,andthe package-lock.jsonfilediligentlyfilledwiththenewversion.
package.jsonremainsunchanged.
Todiscovernewreleasesofthepackages,yourun npmoutdated.
Here'sthelistofafewoutdatedpackagesinonerepositoryIdidn'tupdateforquiteawhile:
Someofthoseupdatesaremajorreleases.Running npmupdatewon'tupdatetheversionofthose.Majorreleasesareneverupdatedinthiswaybecausethey(bydefinition)introducebreakingchanges,and npmwanttosaveyoutrouble.
Toupdatetoanewmajorversionallthepackages,installthe npm-check-updatespackageglobally:
npminstall-gnpm-check-updates
thenrunit:
ncu-u
HowtoupdatealltheNodedependenciestotheirlatestversion
73
thiswillupgradealltheversionhintsinthe package.jsonfile,to dependenciesanddevDependencies,sonpmcaninstallthenewmajorversion.
Youarenowreadytoruntheupdate:
npmupdate
Ifyoujustdownloadedtheprojectwithoutthe node_modulesdependenciesandyouwanttoinstalltheshinynewversionsfirst,justrun
npminstall
HowtoupdatealltheNodedependenciestotheirlatestversion
74
SemanticversioningrulesSemanticVersioningisaconventionusedtoprovideameaningtoversions
Ifthere'sonegreatthinginNode.jspackages,isthatallagreedonusingSemanticVersioningfortheirversionnumbering.
TheSemanticVersioningconceptissimple:allversionshave3digits: x.y.z.
thefirstdigitisthemajorversiontheseconddigitistheminorversionthethirddigitisthepatchversion
Whenyoumakeanewrelease,youdon'tjustupanumberasyouplease,butyouhaverules:
youupthemajorversionwhenyoumakeincompatibleAPIchangesyouuptheminorversionwhenyouaddfunctionalityinabackward-compatiblemanneryouupthepatchversionwhenyoumakebackward-compatiblebugfixes
Theconventionisadoptedallacrossprogramminglanguages,anditisveryimportantthatevery npmpackageadherestoit,becausethewholesystemdependsonthat.
Whyisthatsoimportant?
Because npmsetsomeruleswecanuseinthe package.jsonfiletochoosewhichversionsitcanupdateourpackagesto,whenwerun npmupdate.
Therulesusethosesymbols:
~
>
>=
<
<=
=
-
||
Let'sseethoserulesindetail:
:ifyouwrite 0.13.0whenrunning npmupdateitcanupdatetopatchandminorreleases: 0.13.1, 0.14.0andsoon.~:ifyouwrite ~0.13.0,whenrunning npmupdateitcanupdatetopatchreleases:0.13.1isok,but 0.14.0isnot.
Semanticversioningrules
75
>:youacceptanyversionhigherthantheoneyouspecify>=:youacceptanyversionequaltoorhigherthantheoneyouspecify<=:youacceptanyversionequalorlowertotheoneyouspecify<:youacceptanyversionlowertotheoneyouspecify=:youacceptthatexactversion-:youacceptarangeofversions.Example: 2.1.0-2.6.2||:youcombinesets.Example: <2.1||>2.6
Youcancombinesomeofthosenotations,forexampleuse 1.0.0||>=1.1.0<1.2.0toeitheruse1.0.0oronereleasefrom1.1.0up,butlowerthan1.2.0.
Thereareotherrules,too:
nosymbol:youacceptonlythatspecificversionyouspecify( 1.2.1)latest:youwanttousethelatestversionavailable
Semanticversioningrules
76
UninstallingnpmpackagesHowtouninstallannpmNodepackage,locallyorglobally
Touninstallapackageyouhavepreviouslyinstalledlocally(using npminstall<package-name>inthe node_modulesfolder,run
npmuninstall<package-name>
fromtheprojectrootfolder(thefolderthatcontainsthenode_modulesfolder).
Usingthe -Sflag,or --save,thisoperationwillalsoremovethereferenceinthepackage.jsonfile.
Ifthepackagewasadevelopmentdependency,listedinthedevDependenciesofthepackage.jsonfile,youmustusethe -D/ --save-devflagtoremoveitfromthefile:
npmuninstall-S<package-name>
npmuninstall-D<package-name>
Ifthepackageisinstalledglobally,youneedtoaddthe -g/ --globalflag:
npmuninstall-g<package-name>
forexample:
npmuninstall-gwebpack
andyoucanrunthiscommandfromanywhereyouwantonyoursystembecausethefolderwhereyoucurrentlyaredoesnotmatter.
Uninstallingnpmpackages
77
GlobalorlocalpackagesWhenisapackagebestinstalledglobally?Why?
Themaindifferencebetweenlocalandglobalpackagesisthis:
localpackagesareinstalledinthedirectorywhereyourun npminstall<package-name>,andtheyareputinthe node_modulesfolderunderthisdirectoryglobalpackagesareallputinasingleplaceinyoursystem(exactlywheredependsonyoursetup),regardlessofwhereyourun npminstall-g<package-name>
Inyourcode,theyarebothrequiredinthesameway:
require('package-name')
sowhenshouldyouinstallinonewayoranother?
Ingeneral,allpackagesshouldbeinstalledlocally.
Thismakessureyoucanhavedozensofapplicationsinyourcomputer,allrunningadifferentversionofeachpackageifneeded.
Updatingaglobalpackagewouldmakeallyourprojectsusethenewrelease,andasyoucanimaginethismightcausenightmaresintermsofmaintenance,assomepackagesmightbreakcompatibilitywithfurtherdependencies,andsoon.
Allprojectshavetheirownlocalversionofapackage,evenifthismightappearlikeawasteofresources,it'sminimalcomparedtothepossiblenegativeconsequences.
Apackageshouldbeinstalledgloballywhenitprovidesanexecutablecommandthatyourunfromtheshell(CLI),andit'sreusedacrossprojects.
Youcanalsoinstallexecutablecommandslocallyandrunthemusingnpx,butsomepackagesarejustbetterinstalledglobally.
Greatexamplesofpopularglobalpackageswhichyoumightknoware
npm
create-react-app
vue-cli
grunt-cli
mocha
react-native-cli
gatsby-cli
Globalorlocalpackages
78
forever
nodemon
Youprobablyhavesomepackagesinstalledgloballyalreadyonyoursystem.Youcanseethembyrunning
npmlist-g--depth0
onyourcommandline.
Globalorlocalpackages
79
npmdependenciesanddevDependenciesWhenisapackageadependency,andwhenisitadevdependency?
Whenyouinstallannpmpackageusing npminstall<package-name>,youareinstallingitasadependency.
Thepackageisautomaticallylistedinthepackage.jsonfile,underthe dependencieslist(asofnpm5:beforeyouhadtomanuallyspecify --save).
Whenyouaddthe -Dflag,or --save-dev,youareinstallingitasadevelopmentdependency,whichaddsittothe devDependencieslist.
Developmentdependenciesareintendedasdevelopment-onlypackages,thatareunneededinproduction.Forexampletestingpackages,webpackorBabel.
Whenyougoinproduction,ifyoutype npminstallandthefoldercontainsa package.jsonfile,theyareinstalled,asnpmassumesthisisadevelopmentdeploy.
Youneedtosetthe --productionflag( npminstall--production)toavoidinstallingthosedevelopmentdependencies.
npmdependenciesanddevDependencies
80
npxnpxisaverycoolwaytorunNodecode,andprovidesmanyusefulfeatures
Inthispost,Iwanttointroduceaverypowerfulcommandthat'sbeenavailableinnpmstartingversion5.2,releasedinJuly2017:npx.
Ifyoudon'twanttoinstallnpm,youcaninstallnpxasastandalonepackage
npxletsyouruncodebuiltwithNodeandpublishedthroughthenpmregistry.
EasilyrunlocalcommandsNodedevelopersusedtopublishmostoftheexecutablecommandsasglobalpackages,inorderforthemtobeinthepathandexecutableimmediately.
Thiswasapainbecauseyoucouldnotreallyinstalldifferentversionsofthesamecommand.
Running npxcommandnameautomaticallyfindsthecorrectreferenceofthecommandinsidethenode_modulesfolderofaproject,withoutneedingtoknowtheexactpath,andwithoutrequiringthepackagetobeinstalledgloballyandintheuser'spath.
Installation-lesscommandexecutionThereisanothergreatfeatureof npm,whichisallowingtoruncommandswithoutfirstinstallingthem.
Thisisprettyuseful,mostlybecause:
1)youdon'tneedtoinstallanything2)youcanrundifferentversionsofthesamecommand,usingthesyntax@version
Atypicaldemonstrationofusing npxisthroughthe cowsaycommand. cowsaywillprintacowsayingwhatyouwroteinthecommand.Forexample:
cowsay"Hello"willprint
_______
<Hello>
-------
\^__^
\(oo)\_______
(__)\)\/\
||----w|
npx
81
||||
Now,thisifyouhavethe cowsaycommandgloballyinstalledfromnpmpreviously,otherwiseyou'llgetanerrorwhenyoutrytorunthecommand.
npxallowsyoutorunthatnpmcommandwithouthavingitinstalledlocally:
npxcowsay"Hello"
willdothejob.
Now,thisisafunnyuselesscommand.Otherscenariosinclude:
runningthe vueCLItooltocreatenewapplicationsandrunthem: npxvuecreatemy-vue-app
creatinganewReactappusing create-react-app: npxcreate-react-appmy-react-app
andmanymore.
Oncedownloaded,thedownloadedcodewillbewiped.
RunsomecodeusingadifferentNodeversionUsethe @tospecifytheversion,andcombinethatwiththe nodenpmpackage:
npxnode@6-v#v6.14.3
npxnode@8-v#v8.11.3
Thishelpstoavoidtoolslike nvmortheotherNodeversionmanagementtools.
RunarbitrarycodesnippetsdirectlyfromaURLnpxdoesnotlimityoutothepackagespublishedonthenpmregistry.
YoucanruncodethatsitsinaGitHubgist,forexample:
npxhttps://gist.github.com/zkat/4bc19503fe9e9309e2bfaa2c58074d32
Ofcourse,youneedtobecarefulwhenrunningcodethatyoudonotcontrol,aswithgreatpowercomesgreatresponsibility.
npx
82
npx
83
TheeventloopTheEventLoopisoneofthemostimportantaspectstounderstandaboutJavaScript.Thispostexplainsitinsimpleterms
IntroductionBlockingtheeventloopThecallstackAsimpleeventloopexplanationQueuingfunctionexecutionTheMessageQueueES6JobQueue
IntroductionTheEventLoopisoneofthemostimportantaspectstounderstandaboutJavaScript.
I'veprogrammedforyearswithJavaScript,yetI'veneverfullyunderstoodhowthingsworkunderthehoods.It'scompletelyfinetonotknowthisconceptindetail,butasusual,it'shelpfultoknowhowitworks,andalsoyoumightjustbealittlecuriousatthispoint.
ThispostaimstoexplaintheinnerdetailsofhowJavaScriptworkswithasinglethread,andhowithandlesasynchronousfunctions.
YourJavaScriptcoderunssinglethreaded.Thereisjustonethinghappeningatatime.
Thisisalimitationthat'sactuallyveryhelpful,asitsimplifiesalothowyouprogramwithoutworryingaboutconcurrencyissues.
Youjustneedtopayattentiontohowyouwriteyourcodeandavoidanythingthatcouldblockthethread,likesynchronousnetworkcallsorinfiniteloops.
Ingeneral,inmostbrowsersthereisaneventloopforeverybrowsertab,tomakeeveryprocessisolatedandavoidawebpagewithinfiniteloopsorheavyprocessingtoblockyourentirebrowser.
Theenvironmentmanagesmultipleconcurrenteventloops,tohandleAPIcallsforexample.WebWorkersrunintheirowneventloopaswell.
Youmainlyneedtobeconcernedthatyourcodewillrunonasingleeventloop,andwritecodewiththisthinginmindtoavoidblockingit.
Theeventloop
84
BlockingtheeventloopAnyJavaScriptcodethattakestoolongtoreturnbackcontroltotheeventloopwillblocktheexecutionofanyJavaScriptcodeinthepage,evenblocktheUIthread,andtheusercannotclickaround,scrollthepage,andsoon.
AlmostalltheI/OprimitivesinJavaScriptarenon-blocking.Networkrequests,Node.jsfilesystemoperations,andsoon.Beingblockingistheexception,andthisiswhyJavaScriptisbasedsomuchoncallbacks,andmorerecentlyonpromisesandasync/await.
ThecallstackThecallstackisaLIFOqueue(LastIn,FirstOut).
Theeventloopcontinuouslychecksthecallstacktoseeifthere'sanyfunctionthatneedstorun.
Whiledoingso,itaddsanyfunctioncallitfindstothecallstackandexecuteseachoneinorder.
Youknowtheerrorstacktraceyoumightbefamiliarwith,inthedebuggerorinthebrowserconsole?Thebrowserlooksupthefunctionnamesinthecallstacktoinformyouwhichfunctionoriginatesthecurrentcall:
Theeventloop
85
AsimpleeventloopexplanationLet'spickanexample:
constbar=()=>console.log('bar')
constbaz=()=>console.log('baz')
constfoo=()=>{
console.log('foo')
bar()
baz()
}
foo()
Theeventloop
86
Thiscodeprints
foo
bar
baz
asexpected.
Whenthiscoderuns,first foo()iscalled.Inside foo()wefirstcall bar(),thenwecallbaz().
Atthispointthecallstacklookslikethis:
Theeventlooponeveryiterationlooksifthere'ssomethinginthecallstack,andexecutesit:
Theeventloop
87
untilthecallstackisempty.
QueuingfunctionexecutionTheaboveexamplelooksnormal,there'snothingspecialaboutit:JavaScriptfindsthingstoexecute,runstheminorder.
Theeventloop
88
Let'sseehowtodeferafunctionuntilthestackisclear.
Theusecaseof setTimeout(()=>{}),0)istocallafunction,butexecuteitonceeveryotherfunctioninthecodehasexecuted.
Takethisexample:
constbar=()=>console.log('bar')
constbaz=()=>console.log('baz')
constfoo=()=>{
console.log('foo')
setTimeout(bar,0)
baz()
}
foo()
Thiscodeprints,maybesurprisingly:
foo
baz
bar
Whenthiscoderuns,firstfoo()iscalled.Insidefoo()wefirstcallsetTimeout,passing barasanargument,andweinstructittorunimmediatelyasfastasitcan,passing0asthetimer.Thenwecallbaz().
Atthispointthecallstacklookslikethis:
Theeventloop
89
Hereistheexecutionorderforallthefunctionsinourprogram:
Theeventloop
90
Whyisthishappening?
TheMessageQueueWhensetTimeout()iscalled,theBrowserorNode.jsstartthetimer.Oncethetimerexpires,inthiscaseimmediatelyasweput0asthetimeout,thecallbackfunctionisputintheMessageQueue.
Theeventloop
91
TheMessageQueueisalsowhereuser-initiatedeventslikeclickorkeyboardevents,orfetchresponsesarequeuedbeforeyourcodehastheopportunitytoreacttothem.OralsoDOMeventslike onLoad.
Theloopgivesprioritytothecallstack,anditfirstprocesseseverythingitfindsinthecallstack,andoncethere'snothinginthere,itgoestopickupthingsintheeventqueue.
Wedon'thavetowaitforfunctionslike setTimeout,fetchorotherthingstodotheirownwork,becausetheyareprovidedbythebrowser,andtheyliveontheirownthreads.Forexample,ifyousetthe setTimeouttimeoutto2seconds,youdon'thavetowait2seconds-thewaithappenselsewhere.
ES6JobQueueECMAScript2015introducedtheconceptoftheJobQueue,whichisusedbyPromises(alsointroducedinES6/ES2015).It'sawaytoexecutetheresultofanasyncfunctionassoonaspossible,ratherthanbeingputattheendofthecallstack.
Promisesthatresolvebeforethecurrentfunctionendswillbeexecutedrightafterthecurrentfunction.
Ifindnicetheanalogyofarollercoasterrideatanamusementpark:themessagequeueputsyoubackinqueuewithafteralltheotherpeopleinthequeue,whilethejobqueueisthefastpassticketthatletsyoutakeanotherriderightafteryoufinishedthepreviousone.
Example:
constbar=()=>console.log('bar')
constbaz=()=>console.log('baz')
constfoo=()=>{
console.log('foo')
setTimeout(bar,0)
newPromise((resolve,reject)=>
resolve('shouldberightafterbaz,beforebar')
).then(resolve=>console.log(resolve))
baz()
}
foo()
Thisprints
foo
Theeventloop
92
baz
shouldberightafterbaz,beforebar
bar
That'sabigdifferencebetweenPromises(andAsync/await,whichisbuiltonpromises)andplainoldasynchronousfunctionsthrough setTimeout()orotherplatformAPIs.
Theeventloop
93
nextTickTheNode.jsprocess.nextTickfunctioninteractswiththeeventloopinaspecialway
AsyoutrytounderstandtheNode.jseventloop,oneimportantpartofitisprocess.nextTick().
Everytimetheeventlooptakesafulltrip,wecallitatick.
Whenwepassafunctionto process.nextTick(),weinstructtheenginetoinvokethisfunctionattheendofthecurrentoperation,beforethenexteventlooptickstarts:
process.nextTick(()=>{
//dosomething
})
Theeventloopisbusyprocessingthecurrentfunctioncode.
Whenthisoperationends,theJSenginerunsallthefunctionspassedto nextTickcallsduringthatoperation.
It'sthewaywecantelltheJSenginetoprocessafunctionasynchronously(afterthecurrentfunction),butassoonaspossible,notqueueit.
Calling setTimeout(()=>{},0)willexecutethefunctioninthenexttick,muchlaterthanwhenusing nextTick().
Use nextTick()whenyouwanttomakesurethatinthenexteventloopiterationthatcodeisalreadyexecuted.
nextTick
94
setImmediateTheNode.jssetImmediatefunctioninteractswiththeeventloopinaspecialway
Whenyouwanttoexecutesomepieceofcodeasynchronously,butassoonaspossible,oneoptionistousethe setImmediate()functionprovidedbyNode.js:
setImmediate(()=>{
//runsomething
})
AnyfunctionpassedasthesetImmediate()argumentisacallbackthat'sexecutedinthenextiterationoftheeventloop.
Howis setImmediate()differentfrom setTimeout(()=>{},0)(passinga0mstimeout),andfrom process.nextTick()?
Afunctionpassedto process.nextTick()isgoingtobeexecutedonthecurrentiterationoftheeventloop,afterthecurrentoperationends.ThismeansitwillalwaysexecutebeforesetTimeoutand setImmediate.
A setTimeout()callbackwitha0msdelayisverysimilarto setImmediate().Theexecutionorderwilldependonvariousfactors,buttheywillbebothruninthenextiterationoftheeventloop.
setImmediate
95
TimersWhenwritingJavaScriptcode,youmightwanttodelaytheexecutionofafunction.LearnhowtousesetTimeoutandsetIntervaltoschedulefunctionsinthefuture
setTimeout()
ZerodelaysetInterval()
RecursivesetTimeout
setTimeout()
WhenwritingJavaScriptcode,youmightwanttodelaytheexecutionofafunction.
Thisisthejobof setTimeout.Youspecifyacallbackfunctiontoexecutelater,andavalueexpressinghowlateryouwantittorun,inmilliseconds:
setTimeout(()=>{
//runsafter2seconds
},2000)
setTimeout(()=>{
//runsafter50milliseconds
},50)
Thissyntaxdefinesanewfunction.Youcancallwhateverotherfunctionyouwantinthere,oryoucanpassanexistingfunctionname,andasetofparameters:
constmyFunction=(firstParam,secondParam)=>{
//dosomething
}
Timers
96
//runsafter2seconds
setTimeout(myFunction,2000,firstParam,secondParam)
setTimeoutreturnsthetimerid.Thisisgenerallynotused,butyoucanstorethisid,andclearitifyouwanttodeletethisscheduledfunctionexecution:
constid=setTimeout(()=>{
//shouldrunafter2seconds
},2000)
//Ichangedmymind
clearTimeout(id)
Zerodelay
Ifyouspecifythetimeoutdelayto 0,thecallbackfunctionwillbeexecutedassoonaspossible,butafterthecurrentfunctionexecution:
setTimeout(()=>{
console.log('after')
},0)
console.log('before')
willprint beforeafter.
ThisisespeciallyusefultoavoidblockingtheCPUonintensivetasksandletotherfunctionsbeexecutedwhileperformingaheavycalculation,byqueuingfunctionsinthescheduler.
Somebrowsers(IEandEdge)implementa setImmediate()methodthatdoesthissameexactfunctionality,butit'snotstandardandunavailableonotherbrowsers.Butit'sastandardfunctioninNode.js.
setInterval()
setIntervalisafunctionsimilarto setTimeout,withadifference:insteadofrunningthecallbackfunctiononce,itwillrunitforever,atthespecifictimeintervalyouspecify(inmilliseconds):
setInterval(()=>{
//runsevery2seconds
},2000)
Timers
97
Thefunctionaboverunsevery2secondsunlessyoutellittostop,using clearInterval,passingittheintervalidthat setIntervalreturned:
constid=setInterval(()=>{
//runsevery2seconds
},2000)
clearInterval(id)
It'scommontocall clearIntervalinsidethesetIntervalcallbackfunction,toletitauto-determineifitshouldrunagainorstop.ForexamplethiscoderunssomethingunlessApp.somethingIWaithasthevalue arrived:
constinterval=setInterval(()=>{
if(App.somethingIWait==='arrived'){
clearInterval(interval)
return
}
//otherwisedothings
},100)
RecursivesetTimeoutsetIntervalstartsafunctioneverynmilliseconds,withoutanyconsiderationaboutwhenafunctionfinisheditsexecution.
Ifafunctiontakesalwaysthesameamountoftime,it'sallfine:
Maybethefunctiontakesdifferentexecutiontimes,dependingonnetworkconditionsforexample:
Andmaybeonelongexecutionoverlapsthenextone:
Timers
98
Toavoidthis,youcanschedulearecursivesetTimeouttobecalledwhenthecallbackfunctionfinishes:
constmyFunction=()=>{
//dosomething
setTimeout(myFunction,1000)
}
setTimeout(
myFunction()
},1000)
toachievethisscenario:
setTimeoutand setIntervalareavailableinNode.js,throughtheTimersmodule.
Node.jsalsoprovides setImmediate(),whichisequivalenttousing setTimeout(()=>{},0),mostlyusedtoworkwiththeNode.jsEventLoop.
Timers
99
CallbacksJavaScriptissynchronousbydefault,andissinglethreaded.Thismeansthatcodecannotcreatenewthreadsandruninparallel.Findoutwhatasynchronouscodemeansandhowitlookslike
AsynchronicityinProgrammingLanguagesJavaScriptCallbacksHandlingerrorsincallbacksTheproblemwithcallbacksAlternativestocallbacks
AsynchronicityinProgrammingLanguagesComputersareasynchronousbydesign.
Asynchronousmeansthatthingscanhappenindependentlyofthemainprogramflow.
Callbacks
100
Inthecurrentconsumercomputers,everyprogramrunsforaspecifictimeslot,andthenitstopsitsexecutiontoletanotherprogramcontinueitsexecution.Thisthingrunsinacyclesofastthat'simpossibletonotice,andwethinkourcomputersrunmanyprogramssimultaneously,butthisisanillusion(exceptonmultiprocessormachines).
Programsinternallyuseinterrupts,asignalthat'semittedtotheprocessortogaintheattentionofthesystem.
Iwon'tgointotheinternalsofthis,butjustkeepinmindthatit'snormalforprogramstobeasynchronous,andhalttheirexecutionuntiltheyneedattention,andthecomputercanexecuteotherthingsinthemeantime.Whenaprogramiswaitingforaresponsefromthenetwork,itcannothalttheprocessoruntiltherequestfinishes.
Normally,programminglanguagesaresynchronous,andsomeprovideawaytomanageasynchronicity,inthelanguageorthroughlibraries.C,Java,C#,PHP,Go,Ruby,Swift,Python,theyareallsynchronousbydefault.Someofthemhandleasyncbyusingthreads,spawninganewprocess.
JavaScriptJavaScriptissynchronousbydefaultandissinglethreaded.Thismeansthatcodecannotcreatenewthreadsandruninparallel.
Linesofcodeareexecutedinseries,oneafteranother,forexample:
consta=1
constb=2
constc=a*b
console.log(c)
doSomething()
ButJavaScriptwasborninsidethebrowser,itsmainjob,inthebeginning,wastorespondtouseractions,like onClick, onMouseOver, onChange, onSubmitandsoon.Howcoulditdothiswithasynchronousprogrammingmodel?
Theanswerwasinitsenvironment.ThebrowserprovidesawaytodoitbyprovidingasetofAPIsthatcanhandlethiskindoffunctionality.
Morerecently,Node.jsintroducedanon-blockingI/Oenvironmenttoextendthisconcepttofileaccess,networkcallsandsoon.
Callbacks
Callbacks
101
Youcan'tknowwhenauserisgoingtoclickabutton,sowhatyoudois,youdefineaneventhandlerfortheclickevent.Thiseventhandleracceptsafunction,whichwillbecalledwhentheeventistriggered:
document.getElementById('button').addEventListener('click',()=>{
//itemclicked
})
Thisistheso-calledcallback.
Acallbackisasimplefunctionthat'spassedasavaluetoanotherfunction,andwillonlybeexecutedwhentheeventhappens.WecandothisbecauseJavaScripthasfirst-classfunctions,whichcanbeassignedtovariablesandpassedaroundtootherfunctions(calledhigher-orderfunctions)
It'scommontowrapallyourclientcodeina loadeventlisteneronthe windowobject,whichrunsthecallbackfunctiononlywhenthepageisready:
window.addEventListener('load',()=>{
//windowloaded
//dowhatyouwant
})
Callbacksareusedeverywhere,notjustinDOMevents.
Onecommonexampleisbyusingtimers:
setTimeout(()=>{
//runsafter2seconds
},2000)
XHRrequestsalsoacceptacallback,inthisexamplebyassigningafunctiontoapropertythatwillbecalledwhenaparticulareventoccurs(inthiscase,thestateoftherequestchanges):
constxhr=newXMLHttpRequest()
xhr.onreadystatechange=()=>{
if(xhr.readyState===4){
xhr.status===200?console.log(xhr.responseText):console.error('error')
}
}
xhr.open('GET','https://yoursite.com')
xhr.send()
Handlingerrorsincallbacks
Callbacks
102
Howdoyouhandleerrorswithcallbacks?OneverycommonstrategyistousewhatNode.jsadopted:thefirstparameterinanycallbackfunctionistheerrorobject:error-firstcallbacks
Ifthereisnoerror,theobjectis null.Ifthereisanerror,itcontainssomedescriptionoftheerrorandotherinformation.
fs.readFile('/file.json',(err,data)=>{
if(err!==null){
//handleerror
console.log(err)
return
}
//noerrors,processdata
console.log(data)
})
TheproblemwithcallbacksCallbacksaregreatforsimplecases!
Howevereverycallbackaddsalevelofnesting,andwhenyouhavelotsofcallbacks,thecodestartstobecomplicatedveryquickly:
window.addEventListener('load',()=>{
document.getElementById('button').addEventListener('click',()=>{
setTimeout(()=>{
items.forEach(item=>{
//yourcodehere
})
},2000)
})
})
Thisisjustasimple4-levelscode,butI'veseenmuchmorelevelsofnestingandit'snotfun.
Howdowesolvethis?
AlternativestocallbacksStartingwithES6,JavaScriptintroducedseveralfeaturesthathelpuswithasynchronouscodethatdonotinvolveusingcallbacks:
Promises(ES6)Async/Await(ES8)
Callbacks
103
Callbacks
104
PromisesPromisesareonewaytodealwithasynchronouscodeinJavaScript,withoutwritingtoomanycallbacksinyourcode.
IntroductiontopromisesHowpromiseswork,inbriefWhichJSAPIusepromises?
CreatingapromiseConsumingapromiseChainingpromises
ExampleofchainingpromisesHandlingerrors
CascadingerrorsOrchestratingpromises
Promise.all()
Promise.race()
CommonerrorsUncaughtTypeError:undefinedisnotapromise
IntroductiontopromisesApromiseiscommonlydefinedasaproxyforavaluethatwilleventuallybecomeavailable.
Promisesareonewaytodealwithasynchronouscode,withoutwritingtoomanycallbacksinyourcode.
Althoughbeingaroundsinceyears,theyhavebeenstandardizedandintroducedinES2015,andnowtheyhavebeensupersededinES2017byasyncfunctions.
AsyncfunctionsusethepromisesAPIastheirbuildingblock,sounderstandingthemisfundamentalevenifinnewercodeyou'lllikelyuseasyncfunctionsinsteadofpromises.
Howpromiseswork,inbrief
Onceapromisehasbeencalled,itwillstartinpendingstate.Thismeansthatthecallerfunctioncontinuestheexecution,whileitwaitsforthepromisetodoitsownprocessing,andgivethecallerfunctionsomefeedback.
Promises
105
Atthispoint,thecallerfunctionwaitsforittoeitherreturnthepromiseinaresolvedstate,orinarejectedstate,butasyouknowJavaScriptisasynchronous,sothefunctioncontinuesitsexecutionwhilethepromisedoesitwork.
WhichJSAPIusepromises?
Inadditiontoyourowncodeandlibrariescode,promisesareusedbystandardmodernWebAPIssuchas:
theBatteryAPItheFetchAPIServiceWorkers
It'sunlikelythatinmodernJavaScriptyou'llfindyourselfnotusingpromises,solet'sstartdivingrightintothem.
CreatingapromiseThePromiseAPIexposesaPromiseconstructor,whichyouinitializeusing newPromise():
letdone=true
constisItDoneYet=newPromise(
(resolve,reject)=>{
if(done){
constworkDone='HereisthethingIbuilt'
resolve(workDone)
}else{
constwhy='Stillworkingonsomethingelse'
reject(why)
}
}
)
Asyoucanseethepromisechecksthe doneglobalconstant,andifthat'strue,wereturnaresolvedpromise,otherwisearejectedpromise.
Using resolveand rejectwecancommunicatebackavalue,intheabovecasewejustreturnastring,butitcouldbeanobjectaswell.
Consumingapromise
Promises
106
Inthelastsection,weintroducedhowapromiseiscreated.
Nowlet'sseehowthepromisecanbeconsumedorused.
constisItDoneYet=newPromise(
//...
)
constcheckIfItsDone=()=>{
isItDoneYet
.then((ok)=>{
console.log(ok)
})
.catch((err)=>{
console.error(err)
})
}
Running checkIfItsDone()willexecutethe isItDoneYet()promiseandwillwaitforittoresolve,usingthe thencallback,andifthereisanerror,itwillhandleitinthe catchcallback.
ChainingpromisesApromisecanbereturnedtoanotherpromise,creatingachainofpromises.
AgreatexampleofchainingpromisesisgivenbytheFetchAPI,alayerontopoftheXMLHttpRequestAPI,whichwecanusetogetaresourceandqueueachainofpromisestoexecutewhentheresourceisfetched.
TheFetchAPIisapromise-basedmechanism,andcalling fetch()isequivalenttodefiningourownpromiseusing newPromise().
Exampleofchainingpromises
conststatus=(response)=>{
if(response.status>=200&&response.status<300){
returnPromise.resolve(response)
}
returnPromise.reject(newError(response.statusText))
}
constjson=(response)=>response.json()
fetch('/todos.json')
.then(status)
Promises
107
.then(json)
.then((data)=>{console.log('RequestsucceededwithJSONresponse',data)})
.catch((error)=>{console.log('Requestfailed',error)})
Inthisexample,wecall fetch()togetalistofTODOitemsfromthe todos.jsonfilefoundinthedomainroot,andwecreateachainofpromises.
Running fetch()returnsaresponse,whichhasmanyproperties,andwithinthosewereference:
status,anumericvaluerepresentingtheHTTPstatuscodestatusText,astatusmessage,whichis OKiftherequestsucceeded
responsealsohasa json()method,whichreturnsapromisethatwillresolvewiththecontentofthebodyprocessedandtransformedintoJSON.
Sogiventhosepremises,thisiswhathappens:thefirstpromiseinthechainisafunctionthatwedefined,called status(),thatcheckstheresponsestatusandifit'snotasuccessresponse(between200and299),itrejectsthepromise.
Thisoperationwillcausethepromisechaintoskipallthechainedpromiseslistedandwillskipdirectlytothe catch()statementatthebottom,loggingthe Requestfailedtextalongwiththeerrormessage.
Ifthatsucceedsinstead,itcallsthejson()functionwedefined.Sincethepreviouspromise,whensuccessful,returnedthe responseobject,wegetitasaninputtothesecondpromise.
Inthiscase,wereturnthedataJSONprocessed,sothethirdpromisereceivestheJSONdirectly:
.then((data)=>{
console.log('RequestsucceededwithJSONresponse',data)
})
andwesimplylogittotheconsole.
HandlingerrorsIntheexample,intheprevioussection,wehada catchthatwasappendedtothechainofpromises.
Whenanythinginthechainofpromisesfailsandraisesanerrororrejectsthepromise,thecontrolgoestothenearest catch()statementdownthechain.
Promises
108
newPromise((resolve,reject)=>{
thrownewError('Error')
})
.catch((err)=>{console.error(err)})
//or
newPromise((resolve,reject)=>{
reject('Error')
})
.catch((err)=>{console.error(err)})
Cascadingerrors
Ifinsidethe catch()youraiseanerror,youcanappendasecond catch()tohandleit,andsoon.
newPromise((resolve,reject)=>{
thrownewError('Error')
})
.catch((err)=>{thrownewError('Error')})
.catch((err)=>{console.error(err)})
Orchestratingpromises
Promise.all()
Ifyouneedtosynchronizedifferentpromises, Promise.all()helpsyoudefinealistofpromises,andexecutesomethingwhentheyareallresolved.
Example:
constf1=fetch('/something.json')
constf2=fetch('/something2.json')
Promise.all([f1,f2]).then((res)=>{
console.log('Arrayofresults',res)
})
.catch((err)=>{
console.error(err)
})
TheES2015destructuringassignmentsyntaxallowsyoutoalsodo
Promise.all([f1,f2]).then(([res1,res2])=>{
Promises
109
console.log('Results',res1,res2)
})
Youarenotlimitedtousing fetchofcourse,anypromiseisgoodtogo.
Promise.race()
Promise.race()runswhenthefirstofthepromisesyoupasstoitresolves,anditrunstheattachedcallbackjustonce,withtheresultofthefirstpromiseresolved.
Example:
constfirst=newPromise((resolve,reject)=>{
setTimeout(resolve,500,'first')
})
constsecond=newPromise((resolve,reject)=>{
setTimeout(resolve,100,'second')
})
Promise.race([first,second]).then((result)=>{
console.log(result)//second
})
Commonerrors
UncaughtTypeError:undefinedisnotapromise
Ifyougetthe UncaughtTypeError:undefinedisnotapromiseerrorintheconsole,makesureyouuse newPromise()insteadofjust Promise()
Promises
110
async/awaitDiscoverthemodernapproachtoasynchronousfunctionsinJavaScript.JavaScriptevolvedinaveryshorttimefromcallbackstoPromises,andsinceES2017asynchronousJavaScriptisevensimplerwiththeasync/awaitsyntax
IntroductionWhywereasync/awaitintroduced?HowitworksAquickexamplePromiseallthethingsThecodeismuchsimplertoreadMultipleasyncfunctionsinseriesEasierdebugging
IntroductionJavaScriptevolvedinaveryshorttimefromcallbackstopromises(ES2015),andsinceES2017asynchronousJavaScriptisevensimplerwiththeasync/awaitsyntax.
Asyncfunctionsareacombinationofpromisesandgenerators,andbasically,theyareahigherlevelabstractionoverpromises.Letmerepeat:async/awaitisbuiltonpromises.
Whywereasync/awaitintroduced?Theyreducetheboilerplatearoundpromises,andthe"don'tbreakthechain"limitationofchainingpromises.
WhenPromiseswereintroducedinES2015,theyweremeanttosolveaproblemwithasynchronouscode,andtheydid,butoverthe2yearsthatseparatedES2015andES2017,itwasclearthatpromisescouldnotbethefinalsolution.
Promiseswereintroducedtosolvethefamouscallbackhellproblem,buttheyintroducedcomplexityontheirown,andsyntaxcomplexity.
Theyweregoodprimitivesaroundwhichabettersyntaxcouldbeexposedtothedevelopers,sowhenthetimewasrightwegotasyncfunctions.
Theymakethecodelooklikeit'ssynchronous,butit'sasynchronousandnon-blockingbehindthescenes.
async/await
111
HowitworksAnasyncfunctionreturnsapromise,likeinthisexample:
constdoSomethingAsync=()=>{
returnnewPromise((resolve)=>{
setTimeout(()=>resolve('Ididsomething'),3000)
})
}
Whenyouwanttocallthisfunctionyouprepend await,andthecallingcodewillstopuntilthepromiseisresolvedorrejected.Onecaveat:theclientfunctionmustbedefinedasasync.Here'sanexample:
constdoSomething=async()=>{
console.log(awaitdoSomethingAsync())
}
AquickexampleThisisasimpleexampleofasync/awaitusedtorunafunctionasynchronously:
constdoSomethingAsync=()=>{
returnnewPromise((resolve)=>{
setTimeout(()=>resolve('Ididsomething'),3000)
})
}
constdoSomething=async()=>{
console.log(awaitdoSomethingAsync())
}
console.log('Before')
doSomething()
console.log('After')
Theabovecodewillprintthefollowingtothebrowserconsole:
Before
After
Ididsomething//after3s
Promiseallthethings
async/await
112
Prependingthe asynckeywordtoanyfunctionmeansthatthefunctionwillreturnapromise.
Evenifit'snotdoingsoexplicitly,itwillinternallymakeitreturnapromise.
Thisiswhythiscodeisvalid:
constaFunction=async()=>{
return'test'
}
aFunction().then(alert)//Thiswillalert'test'
andit'sthesameas:
constaFunction=async()=>{
returnPromise.resolve('test')
}
aFunction().then(alert)//Thiswillalert'test'
ThecodeismuchsimplertoreadAsyoucanseeintheexampleabove,ourcodelooksverysimple.Compareittocodeusingplainpromises,withchainingandcallbackfunctions.
Andthisisaverysimpleexample,themajorbenefitswillarisewhenthecodeismuchmorecomplex.
Forexamplehere'showyouwouldgetaJSONresource,andparseit,usingpromises:
constgetFirstUserData=()=>{
returnfetch('/users.json')//getuserslist
.then(response=>response.json())//parseJSON
.then(users=>users[0])//pickfirstuser
.then(user=>fetch(`/users/${user.name}`))//getuserdata
.then(userResponse=>response.json())//parseJSON
}
getFirstUserData()
Andhereisthesamefunctionalityprovidedusingawait/async:
constgetFirstUserData=async()=>{
constresponse=awaitfetch('/users.json')//getuserslist
constusers=awaitresponse.json()//parseJSON
constuser=users[0]//pickfirstuser
constuserResponse=awaitfetch(`/users/${user.name}`)//getuserdata
async/await
113
constuserData=awaituser.json()//parseJSON
returnuserData
}
getFirstUserData()
MultipleasyncfunctionsinseriesAsyncfunctionscanbechainedveryeasily,andthesyntaxismuchmorereadablethanwithplainpromises:
constpromiseToDoSomething=()=>{
returnnewPromise(resolve=>{
setTimeout(()=>resolve('Ididsomething'),10000)
})
}
constwatchOverSomeoneDoingSomething=async()=>{
constsomething=awaitpromiseToDoSomething()
returnsomething+'andIwatched'
}
constwatchOverSomeoneWatchingSomeoneDoingSomething=async()=>{
constsomething=awaitwatchOverSomeoneDoingSomething()
returnsomething+'andIwatchedaswell'
}
watchOverSomeoneWatchingSomeoneDoingSomething().then((res)=>{
console.log(res)
})
Willprint:
IdidsomethingandIwatchedandIwatchedaswell
EasierdebuggingDebuggingpromisesishardbecausethedebuggerwillnotstepoverasynchronouscode.
Async/awaitmakesthisveryeasybecausetothecompilerit'sjustlikesynchronouscode.
async/await
114
TheNodeEventEmitterHowtoworkwithcustomeventsinNode
IfyouworkedwithJavaScriptinthebrowser,youknowhowmuchoftheinteractionoftheuserishandledthroughevents:mouseclicks,keyboardbuttonpresses,reactingtomousemovements,andsoon.
Onthebackendside,Nodeoffersustheoptiontobuildasimilarsystemusingthe eventsmodule.
Thismodule,inparticular,offersthe EventEmitterclass,whichwe'llusetohandleourevents.
Youinitializethatusing
consteventEmitter=require('events').EventEmitter()
Thisobjectexposes,amongmanyothers,the onand emitmethods.
emitisusedtotriggeraneventonisusedtoaddacallbackfunctionthat'sgoingtobeexecutedwhentheeventistriggered
Forexample,let'screatea startevent,andasamatterofprovidingasample,wereacttothatbyjustloggingtotheconsole:
eventEmitter.on('start',()=>{
console.log('started')
})
Whenwerun
eventEmitter.emit('start')
theeventhandlerfunctionistriggered,andwegettheconsolelog.
Youcanpassargumentstotheeventhandlerbypassingthemasadditionalargumentstoemit():
eventEmitter.on('start',(number)=>{
console.log(`started${number}`)
})
eventEmitter.emit('start',23)
TheNodeEventEmitter
115
Multiplearguments:
eventEmitter.on('start',(start,end)=>{
console.log(`startedfrom${start}to${end}`)
})
eventEmitter.emit('start',1,100)
TheEventEmitterobjectalsoexposesseveralothermethodstointeractwithevents,like
once():addaone-timelistenerremoveListener()/ off():removeaneventlistenerfromaneventremoveAllListeners():removealllistenersforanevent
Youcanreadalltheirdetailsontheeventsmodulepageathttps://nodejs.org/api/events.html
TheNodeEventEmitter
116
HTTPAdetaileddescriptionofhowtheHTTPprotocol,andtheWeb,work
HTTP(HyperTextTransferProtocol)isoneoftheapplicationprotocolsofTCP/IP,thesuiteofprotocolsthatpowerstheInternet.
Letmefixthat:it'snotoneoftheprotocols,it'sthemostsuccessfulandpopularone,byallmeans.
HTTPiswhatmakestheWorldWideWebwork,givingbrowsersalanguagetocommunicatetoremoteserversthathostwebpages.
HTTPwasfirststandardizedin1991,asaresultoftheworkthatTimBerners-LeedidatCERN,theEuropeanCenterofNuclearResearch,since1989.
Thegoalwastoallowresearcherstoeasilyexchangeandinterlinktheirpapers.Itwasmeantasawayforthescientificcommunitytoworkbetter.
BackthentheinternetmainapplicationsbasicallyconsistedinFTP(theFileTransferProtocol),EmailandUsenet(newsgroups,todayalmostabandoned).
In1993Mosaic,thefirstgraphicalwebbrowser,wasreleased,andthingsskyrocketedfromthere.
TheWebbecamethekillerappoftheInternet.
OvertimetheWebandtheecosystemaroundithavedramaticallyevolved,butthebasicsstillremain.Oneexampleofevolution:HTTPnowpowers,inadditiontowebpages,RESTAPIs,onecommonwaytoprogrammaticallyaccessaserviceovertheInternet.
HTTPgotaminorrevisionin1997withHTTP/1.1,andin2015itssuccessor,HTTP/2,wasstandardizedandit'snowbeingimplementedbythemajorWebServersusedacrosstheglobe.
TheHTTPprotocolisconsideredinsecure,justlikeanyotherprotocol(SMTP,FTP..)notservedoveranencryptedconnection.ThisiswhythereisabigpushnowadaystowardsusingHTTPS,whichisHTTPservedoverTLS.
Thatsaid,thebuildingblocksofHTTP/2andHTTPShavetheirrootsinHTTP,andinthisarticleI'llintroducehowHTTPworks.
HTMLdocuments
HTTP
117
HTTPisthewaywebbrowserslikeChrome,Firefox,Edgeandmanyothers(alsocalledclientsfromhereon)communicatewithwebservers.
ThenameHyperTextTransferProtocolderivesfromtheneedoftransferringnotjustfiles,likeinFTP-the"FileTransferProtocol",buthypertexts,whichwouldbewrittenusingHTML,andthenrepresentedgraphicallybythebrowserwithanicepresentationandinteractivelinks.
Linkswerethedrivingforcethatdroveadoption,alongwiththeeaseofcreationofnewwebpages.
HTTPiswhattransferthosehypertextfiles(andaswe'llseealsoimagesandotherfiletypes)overthenetwork.
HyperlinksInsideawebbrowser,adocumentcanpointtoanotherdocumentusinglinks.
Alinkiscomposedbyafirstpartthatdeterminestheprotocolandtheserveraddress,eitherthroughadomainnameoranIP.
ThispartisnotuniquetoHTTP,ofcourse.
Thenthere'sthedocumentpart.Anythingappendedtotheaddresspartrepresentsthedocumentpath.
Forexample,thisdocumentaddressis https://flaviocopes.com/http/:
httpsistheprotocol.flaviocopes.comisthedomainnamethatpointstomyserver/http/isthedocumentURLrelativetotheserverrootpath.
Thepathcanbenested: https://flaviocopes.com/page/privacy/andinthiscasethedocumentURLis /page/privacy.
Thewebserverisresponsibleforinterpretingtherequestand,onceanalyzed,servingthecorrectresponse.
ArequestWhat'sinarequest?
ThefirstthingistheURL,whichwe'vealreadyseenbefore.
Whenweenteranaddressandpressenterinourbrowser,underthehoodstheserversendstothecorrectIPaddressarequestlikethis:
HTTP
118
GET/a-page
where/a-pageistheURLyourequested.
ThesecondthingistheHTTPmethod(alsocalledverb).
HTTPintheearlydaysdefined3ofthem:
GET
POST
HEAD
andHTTP/1.1introduced
PUT
DELETE
OPTIONS
TRACE
We'llseethemindetailinaminute.
ThethirdthingthatcomposesarequestisasetofHTTPheaders.
Headersareasetof key:valuepairsthatareusedtocommunicatetotheserver-specificinformationthatispredefined,sotheservercanknowwhatwemean.
IdescribedthemindetailintheHTTPrequestheaderslist.
Givethatlistaquicklook.Allofthoseheadersareoptional,except Host.
HTTPmethods
GET
GETisthemostusedmethodhere.It'stheonethat'susedwhenyoutypeanURLinthebrowseraddressbar,orwhenyouclickalink.
Itaskstheservertosendtherequestedresourceasaresponse.
HEAD
HEADisjustlikeGET,buttellstheservertonotsendtheresponsebodyback.Justtheheaders.
HTTP
119
POST
TheclientusesthePOSTmethodtosenddatatotheserver.It'stypicallyusedinforms,forexample,butalsowheninteractingwithaRESTAPI.
PUT
ThePUTmethodisintendedtocreatearesourceatthatspecificURL,withtheparameterspassedintherequestbody.MainlyusedinRESTAPIs
DELETE
TheDELETEmethodiscalledagainstanURLtorequestdeletionofthatresource.MainlyusedinRESTAPIs
OPTIONS
WhenaserverreceivesanOPTIONSrequestitshouldsendbackthelistofHTTPmethodsallowedforthatspecificURL.
TRACE
Returnsbacktotheclienttherequestthathasbeenreceived.Usedfordebuggingordiagnosticpurposes.
HTTPClient/ServercommunicationHTTP,asmostoftheprotocolsthatbelongtotheTCP/IPsuite,isastatelessprotocol.
Servershavenoideawhat'sthecurrentstateoftheclient.Alltheycareaboutisthattheygetrequestandtheyneedtofulfillthem.
Anypriorrequestismeaninglessinthiscontext,andthismakesitpossibleforawebservertobeveryfast,asthere'slesstoprocess,andalsoitgivesitbandwidthtohandlealotofconcurrentrequests.
HTTPisalsoverylean,andcommunicationisveryfastintermsofoverhead.ThiscontrastswiththeprotocolsthatwerethemostusedatthetimeHTTPwasintroduced:TCPandPOP/SMTP,themailprotocols,whichinvolvelotsofhandshakingandconfirmationsonthereceivingends.
HTTP
120
Graphicalbrowsersabstractallthiscommunication,butwe'llillustrateithereforlearningpurposes.
Amessageiscomposedbyafirstline,whichstartswiththeHTTPmethod,thencontainstheresourcerelativepath,andtheprotocolversion:
GET/a-pageHTTP/1.1
Afterthat,weneedtoaddtheHTTPrequestheaders.Asmentionedabove,therearemanyheaders,buttheonlymandatoryoneis Host:
GET/a-pageHTTP/1.1
Host:flaviocopes.com
Howcanyoutestthis?Usingtelnet.Thisisacommand-linetoolthatletsusconnecttoanyserverandsenditcommands.
Openyourterminal,andtype telnetflaviocopes.com80
Thiswillopenaterminal,thattellsyou
Trying178.128.202.129...
Connectedtoflaviocopes.com.
Escapecharacteris'^]'.
YouareconnectedtotheNetlifywebserverthatpowersmyblog.Youcannowtype:
GET/axios/HTTP/1.1
Host:flaviocopes.com
andpressenteronanemptylinetofiretherequest.
Theresponsewillbe:
HTTP/1.1301MovedPermanently
Cache-Control:public,max-age=0,must-revalidate
Content-Length:46
Content-Type:text/plain
Date:Sun,29Jul201814:07:07GMT
Location:https://flaviocopes.com/axios/
Age:0
Connection:keep-alive
Server:Netlify
Redirectingtohttps://flaviocopes.com/axios/
HTTP
121
See,thisisanHTTPresponsewegotbackfromtheserver.It'sa301MovedPermanentlyrequest.SeetheHTTPstatuscodeslisttoknowmoreaboutthestatuscodes.
Itbasicallytellsustheresourcehaspermanentlymovedtoanotherlocation.
Why?Becauseweconnectedtoport80,whichisthedefaultforHTTP,butonmyserverIsetupanautomaticredirectiontoHTTPS.
Thenewlocationisspecifiedinthe LocationHTTPresponseheader.
Thereareotherheaders,alldescribedintheHTTPresponseheaderslist.
Inboththerequestandtheresponse,anemptylineseparatestherequestheaderfromtherequestbody.Therequestbodyinthiscasecontainsthestring
Redirectingtohttps://flaviocopes.com/axios/
whichis46byteslong,asspecifiedinthe Content-Lengthheader.Itisshowninthebrowserwhenyouopenthepage,whileitautomaticallyredirectsyoutothecorrectlocation.
Inthiscasewe'reusingtelnet,thelow-leveltoolthatwecanusetoconnecttoanyserver,sowecan'thaveanykindofautomaticredirect.
Let'sdothisprocessagain,nowconnectingtoport443,whichisthedefaultportoftheHTTPSprotocol.Wecan'tusetelnetbecauseoftheSSLhandshakethatmusthappen.
Let'skeepthingssimpleanduse curl,anothercommand-linetool.WecannotdirectlytypetheHTTPrequest,butwe'llseetheresponse:
curl-ihttps://flaviocopes.com/axios/
thisiswhatwe'llgetinreturn:
HTTP/1.1200OK
Cache-Control:public,max-age=0,must-revalidate
Content-Type:text/html;charset=UTF-8
Date:Sun,29Jul201814:20:45GMT
Etag:"de3153d6eacef2299964de09db154b32-ssl"
Strict-Transport-Security:max-age=31536000
Age:152
Content-Length:9797
Connection:keep-alive
Server:Netlify
<!DOCTYPEhtml>
<htmlprefix="og:http://ogp.me/ns#"lang="en">
<head>
<metacharset="utf-8">
HTTP
122
<metahttp-equiv="X-UA-Compatible"content="IE=edge">
<title>HTTPrequestsusingAxios</title>
....
Icuttheresponse,butyoucanseethattheHTMLofthepageisbeingreturnednow.
OtherresourcesAnHTTPserverwillnotjusttransferHTMLfiles,buttypicallyitwillalsoserveotherfiles:CSS,JS,SVG,PNG,JPG,lotsofdifferentfiletypes.
Thisdependsontheconfiguration.
HTTPisperfectlycapableoftransferringthosefilesaswell,andtheclientwillknowaboutthefiletype,thusinterpretthemintherightway.
Thisishowthewebworks:whenanHTMLpageisretrievedbythebrowser,it'sinterpretedandanyotherresourceitneedstodisplayproperty(CSS,JavaScript,images..)isretrievedthroughadditionalHTTPrequeststothesameserver.
HTTP
123
HowHTTPRequestsworkWhathappenswhenyoutypeanURLinthebrowser,fromstarttofinish
TheHTTPprotocolIanalyzeURLrequestsonlyThingsrelatetomacOS/LinuxDNSLookupphase
gethostbynameTCPrequesthandshakingSendingtherequest
TherequestlineTherequestheaderTherequestbody
TheresponseParsetheHTML
ThisarticledescribeshowbrowsersperformpagerequestsusingtheHTTP/1.1protocol
Ifyoueverdidaninterview,youmighthavebeenasked:"whathappenswhenyoutypesomethingintotheGooglesearchboxandpressenter".
It'soneofthemostpopularquestionsyougetasked.Peoplejustwanttoseeifyoucanexplainsomeratherbasicconceptsandifyouhaveanycluehowtheinternetactuallyworks.
Inthispost,I'llanalyzewhathappenswhenyoutypeanURLintheaddressbarofyourbrowserandpressenter.
It'saveryinterestingtopictodissectinablogpost,asittouchesmanytechnologiesIcandiveintoinseparateposts.
Thisistechthatisveryrarelychanged,andpowersonethemostcomplexandwideecosystemseverbuiltbyhumans.
TheHTTPprotocolFirst,ImentionHTTPSinparticularbecausethingsaredifferentfromanHTTPSconnection.
IanalyzeURLrequestsonly
HowHTTPRequestswork
124
ModernbrowsershavethecapabilityofknowingifthethingyouwroteintheaddressbarisanactualURLorasearchterm,andtheywillusethedefaultsearchengineifit'snotavalidURL.
IassumeyoutypeanactualURL.
WhenyouentertheURLandpressenter,thebrowserfirstbuildsthefullURL.
Ifyoujustenteredadomain,like flaviocopes.com,thebrowserbydefaultwillprependHTTP://toit,defaultingtotheHTTPprotocol.
ThingsrelatetomacOS/LinuxJustFYI.Windowsmightdosomethingsslightlydifferently.
DNSLookupphaseThebrowserstartstheDNSlookuptogettheserverIPaddress.
Thedomainnameisahandyshortcutforushumans,buttheinternetisorganizedinsuchawaythatcomputerscanlookuptheexactlocationofaserverthroughitsIPaddress,whichisasetofnumberslike 222.324.3.1(IPv4).
First,itcheckstheDNSlocalcache,toseeifthedomainhasalreadybeenresolvedrecently.
ChromehasahandyDNScachevisualizeryoucanseeatchrome://net-internals/#dns
Ifnothingisfoundthere,thebrowserusestheDNSresolver,usingthe gethostbynamePOSIXsystemcalltoretrievethehostinformation.
gethostbyname
gethostbynamefirstlooksinthelocalhostsfile,whichonmacOSorLinuxislocatedin/etc/hosts,toseeifthesystemprovidestheinformationlocally.
Ifthisdoesnotgiveanyinformationaboutthedomain,thesystemmakesarequesttotheDNSserver.
TheaddressoftheDNSserverisstoredinthesystempreferences.
Thoseare2popularDNSservers:
8.8.8.8:theGooglepublicDNSserver1.1.1.1:theCloudFlareDNSserver
MostpeopleusetheDNSserverprovidedbytheirinternetprovider.
HowHTTPRequestswork
125
ThebrowserperformstheDNSrequestusingtheUDPprotocol.
TCPandUDParetwoofthefoundationalprotocolsofcomputernetworking.Theysitatthesameconceptuallevel,butTCPisconnection-oriented,whileUDPisaconnectionlessprotocol,morelightweight,usedtosendmessageswithlittleoverhead.
HowtheUDPrequestisperformedisnotinthescopeofthistutorial
TheDNSservermighthavethedomainIPinthecache.Itnot,itwillasktherootDNSserver.That'sasystem(composedof13actualservers,distributedacrosstheplanet)thatdrivestheentireinternet.
TheDNSserverdoesnotknowtheaddressofeachandeverydomainnameontheplanet.
Whatitknowsiswherethetop-levelDNSresolversare.
Atop-leveldomainisthedomainextension: .com, .it, .pizzaandsoon.
OncetherootDNSserverreceivestherequest,itforwardstherequesttothattop-leveldomain(TLD)DNSserver.
Sayyouarelookingfor flaviocopes.com.TherootdomainDNSserverreturnstheIPofthe.comTLDserver.
NowourDNSresolverwillcachetheIPofthatTLDserver,soitdoesnothavetoasktherootDNSserveragainforit.
TheTLDDNSserverwillhavetheIPaddressesoftheauthoritativeNameServersforthedomainwearelookingfor.
How?Whenyoubuyadomain,thedomainregistrarsendstheappropriateTDLthenameservers.Whenyouupdatethenameservers(forexample,whenyouchangethehostingprovider),thisinformationwillbeautomaticallyupdatedbyyourdomainregistrar.
ThosearetheDNSserversofthehostingprovider.Theyareusuallymorethan1,toserveasbackup.
Forexample:
ns1.dreamhost.com
ns2.dreamhost.com
ns3.dreamhost.com
TheDNSresolverstartswiththefirst,andtriestoasktheIPofthedomain(withthesubdomain,too)youarelookingfor.
ThatistheultimatesourceoftruthfortheIPaddress.
HowHTTPRequestswork
126
NowthatwehavetheIPaddress,wecangooninourjourney.
TCPrequesthandshakingWiththeserverIPaddressavailable,nowthebrowsercaninitiateaTCPconnectiontothat.
ATCPconnectionrequiresabitofhandshakingbeforeitcanbefullyinitializedandyoucanstartsendingdata.
Oncetheconnectionisestablished,wecansendtherequest
SendingtherequestTherequestisaplaintextdocumentstructuredinaprecisewaydeterminedbythecommunicationprotocol.
It'scomposedof3parts:
therequestlinetherequestheadertherequestbody
Therequestline
Therequestlinesets,onasingleline:
theHTTPmethodtheresourcelocationtheprotocolversion
Example:
GET/HTTP/1.1
Therequestheader
Therequestheaderisasetof field:valuepairsthatsetcertainvalues.
Thereare2mandatoryfields,oneofwhichis Host,andtheotheris Connection,whilealltheotherfieldsareoptional:
Host:flaviocopes.com
Connection:close
HowHTTPRequestswork
127
Hostindicatesthedomainnamewhichwewanttotarget,while Connectionisalwayssettocloseunlesstheconnectionmustbekeptopen.
Someofthemostusedheaderfieldsare:
Origin
Accept
Accept-Encoding
Cookie
Cache-Control
Dnt
butmanymoreexist.
Theheaderpartisterminatedbyablankline.
Therequestbody
Therequestbodyisoptional,notusedinGETrequestsbutverymuchusedinPOSTrequestsandsometimesinotherverbstoo,anditcancontaindatainJSONformat.
Sincewe'renowanalyzingaGETrequest,thebodyisblankandwe'llnotlookmoreintoit.
TheresponseOncetherequestissent,theserverprocessesitandsendsbackaresponse.
Theresponsestartswiththestatuscodeandthestatusmessage.Iftherequestissuccessfulandreturnsa200,itwillstartwith:
200OK
Therequestmightreturnadifferentstatuscodeandmessage,likeoneofthese:
404NotFound
403Forbidden
301MovedPermanently
500InternalServerError
304NotModified
401Unauthorized
TheresponsethencontainsalistofHTTPheadersandtheresponsebody(which,sincewe'remakingtherequestinthebrowser,isgoingtobeHTML)
HowHTTPRequestswork
128
ParsetheHTMLThebrowsernowhasreceivedtheHTMLandstartstoparseit,andwillrepeattheexactsameprocesswedidnotforalltheresourcesrequiredbythepage:
CSSfilesimagesthefaviconJavaScriptfiles...
Howbrowsersrenderthepagethenisoutofthescope,butit'simportanttounderstandthattheprocessIdescribedisnotjustfortheHTMLpages,butforanyitemthat'sservedoverHTTP.
HowHTTPRequestswork
129
BuildanHTTPserverHowtobuildanHTTPserverwithNode.js
HereistheHTTPwebserverweusedastheNodeHelloWorldapplicationintheNode.jsintroduction
consthttp=require('http')
constport=3000
constserver=http.createServer((req,res)=>{
res.statusCode=200
res.setHeader('Content-Type','text/plain')
res.end('HelloWorld\n')
})
server.listen(port,()=>{
console.log(`Serverrunningathttp://${hostname}:${port}/`)
})
Let'sanalyzeitbriefly.Weincludethe httpmodule.
WeusethemoduletocreateanHTTPserver.
Theserverissettolistenonthespecifiedport, 3000.Whentheserverisready,the listencallbackfunctioniscalled.
Thecallbackfunctionwepassistheonethat'sgoingtobeexecuteduponeveryrequestthatcomesin.Wheneveranewrequestisreceived,the requesteventiscalled,providingtwoobjects:arequest(an http.IncomingMessageobject)andaresponse(an http.ServerResponseobject).
requestprovidestherequestdetails.Throughit,weaccesstherequestheadersandrequestdata.
responseisusedtopopulatethedatawe'regoingtoreturntotheclient.
Inthiscasewith
res.statusCode=200
wesetthestatusCodepropertyto200,toindicateasuccessfulresponse.
WealsosettheContent-Typeheader:
BuildanHTTPserver
130
res.setHeader('Content-Type','text/plain')
andweendclosetheresponse,addingthecontentasanargumentto end():
res.end('HelloWorld\n')
BuildanHTTPserver
131
MakingHTTPrequestsHowtoperformHTTPrequestswithNode.jsusingGET,POST,PUTandDELETE
IusethetermHTTP,butHTTPSiswhatshouldbeusedeverywhere,thereforetheseexamplesuseHTTPSinsteadofHTTP.
PerformaGETRequest
consthttps=require('https')
constoptions={
hostname:'flaviocopes.com',
port:443,
path:'/todos',
method:'GET'
}
constreq=https.request(options,(res)=>{
console.log(`statusCode:${res.statusCode}`)
res.on('data',(d)=>{
process.stdout.write(d)
})
})
req.on('error',(error)=>{
console.error(error)
})
req.end()
PerformaPOSTRequest
consthttps=require('https')
constdata=JSON.stringify({
todo:'Buythemilk'
})
constoptions={
hostname:'flaviocopes.com',
port:443,
path:'/todos',
method:'POST',
headers:{
MakingHTTPrequests
132
'Content-Type':'application/json',
'Content-Length':data.length
}
}
constreq=https.request(options,(res)=>{
console.log(`statusCode:${res.statusCode}`)
res.on('data',(d)=>{
process.stdout.write(d)
})
})
req.on('error',(error)=>{
console.error(error)
})
req.write(data)
req.end()
PUTandDELETEPUTandDELETErequestsusethesamePOSTrequestformat,andjustchangetheoptions.methodvalue.
MakingHTTPrequests
133
AxiosAxiosisaveryconvenientJavaScriptlibrarytoperformHTTPrequestsinNode.js
IntroductionInstallationTheAxiosAPIGETrequestsAddparameterstoGETrequestsPOSTRequests
IntroductionAxiosisaverypopularJavaScriptlibraryyoucanusetoperformHTTPrequests,thatworksinbothBrowserandNode.jsplatforms.
Itsupportsallmodernbrowsers,includingsupportforIE8andhigher.
Itispromise-based,andthisletsuswriteasync/awaitcodetoperformXHRrequestsveryeasily.
UsingAxioshasquiteafewadvantagesoverthenativeFetchAPI:
Axios
134
supportsolderbrowsers(Fetchneedsapolyfill)hasawaytoabortarequesthasawaytosetaresponsetimeouthasbuilt-inCSRFprotectionsupportsuploadprogressperformsautomaticJSONdatatransformationworksinNode.js
InstallationAxioscanbeinstalledusingnpm:
npminstallaxios
oryarn:
yarnaddaxios
orsimplyincludeitinyourpageusingunpkg.com:
<scriptsrc="https://unpkg.com/axios/dist/axios.min.js"></script>
TheAxiosAPIYoucanstartanHTTPrequestfromthe axiosobject:
axios({
url:'https://dog.ceo/api/breeds/list/all',
method:'get',
data:{
foo:'bar'
}
})
butforconvenience,youwillgenerallyuse
axios.get()
axios.post()
(likeinjQueryyouwoulduse $.get()and $.post()insteadof $.ajax())
AxiosoffersmethodsforalltheHTTPverbs,whicharelesspopularbutstillused:
Axios
135
axios.delete()
axios.put()
axios.patch()
axios.options()
andamethodtogettheHTTPheadersofarequest,discardingthebody:
axios.head()
GETrequestsOneconvenientwaytouseAxiosistousethemodern(ES2017)async/awaitsyntax.
ThisNode.jsexamplequeriestheDogAPItoretrievealistofallthedogsbreeds,usingaxios.get(),anditcountsthem:
constaxios=require('axios')
constgetBreeds=async()=>{
try{
returnawaitaxios.get('https://dog.ceo/api/breeds/list/all')
}catch(error){
console.error(error)
}
}
constcountBreeds=async()=>{
constbreeds=awaitgetBreeds()
if(breeds.data.message){
console.log(`Got${Object.entries(breeds.data.message).length}breeds`)
}
}
countBreeds()
Ifyoudon'twanttouseasync/awaityoucanusethePromisessyntax:
constaxios=require('axios')
constgetBreeds=()=>{
try{
returnaxios.get('https://dog.ceo/api/breeds/list/all')
}catch(error){
console.error(error)
}
}
constcountBreeds=async()=>{
constbreeds=getBreeds()
Axios
136
.then(response=>{
if(response.data.message){
console.log(
`Got${Object.entries(response.data.message).length}breeds`
)
}
})
.catch(error=>{
console.log(error)
})
}
countBreeds()
AddparameterstoGETrequestsAGETresponsecancontainparametersintheURL,likethis: https://site.com/?foo=bar.
WithAxiosyoucanperformthisbysimplyusingthatURL:
axios.get('https://site.com/?foo=bar')
oryoucanusea paramspropertyintheoptions:
axios.get('https://site.com/',{
params:{
foo:'bar'
}
})
POSTRequestsPerformingaPOSTrequestisjustlikedoingaGETrequest,butinsteadof axios.get,youuse axios.post:
axios.post('https://site.com/')
AnobjectcontainingthePOSTparametersisthesecondargument:
axios.post('https://site.com/',{
foo:'bar'
})
Axios
137
Axios
138
WebsocketsWebSocketsareanalternativetoHTTPcommunicationinWebApplications.Theyofferalonglived,bidirectionalcommunicationchannelbetweenclientandserver.
WebSocketsareanalternativetoHTTPcommunicationinWebApplications.
Theyofferalonglived,bidirectionalcommunicationchannelbetweenclientandserver.
Onceestablished,thechanneliskeptopen,offeringaveryfastconnectionwithlowlatencyandoverhead.
BrowsersupportforWebSocketsWebSocketsaresupportedbyallmodernbrowsers.
Websockets
139
HowWebSocketsdifferfromHTTPHTTPisaverydifferentprotocol,andalsoadifferentwayofcommunicate.
HTTPisarequest/responseprotocol:theserverreturnssomedatawhentheclientrequestsit.
WithWebSockets:
theservercansendamessagetotheclientwithouttheclientexplicitlyrequestingsomethingtheclientandtheservercantalktoeachothersimultaneouslyverylittledataoverheadneedstobeexchangedtosendmessages.Thismeansalowlatencycommunication.
WebSocketsaregreatforreal-timeandlong-livedcommunications.
HTTPisgreatforoccasionaldataexchangeandinteractionsinitiatedbytheclient.
HTTPismuchsimplertoimplement,whileWebSocketsrequireabitmoreoverhead.
SecuredWebSocketsAlwaysusethesecure,encryptedprotocolforWebSockets, wss://.
ws://referstotheunsafeWebSocketsversion(the http://ofWebSockets),andshouldbeavoidedforobviousreasons.
CreateanewWebSocketsconnection
consturl='wss://myserver.com/something'
constconnection=newWebSocket(url)
connectionisaWebSocketobject.
Whentheconnectionissuccessfullyestablished,the openeventisfired.
Listenforitbyassigningacallbackfunctiontothe onopenpropertyofthe connectionobject:
connection.onopen=()=>{
//...
}
Ifthere'sanyerror,the onerrorfunctioncallbackisfired:
Websockets
140
connection.onerror=error=>{
console.log(`WebSocketerror:${error}`)
}
SendingdatatotheserverusingWebSocketsOncetheconnectionisopen,youcansenddatatotheserver.
Youcandosoconvenientlyinsidethe onopencallbackfunction:
connection.onopen=()=>{
connection.send('hey')
}
ReceivingdatafromtheserverusingWebSocketsListenwithacallbackfunctionon onmessage,whichiscalledwhenthe messageeventisreceived:
connection.onmessage=e=>{
console.log(e.data)
}
ImplementaWebSocketsserverinNode.jswsisapopularWebSocketslibraryforNode.js.
We'lluseittobuildaWebSocketsserver.Itcanalsobeusedtoimplementaclient,anduseWebSocketstocommunicatebetweentwobackendservices.
Easilyinstallitusing
yarninit
yarnaddws
Thecodeyouneedtowriteisverylittle:
constWebSocket=require('ws')
constwss=newWebSocket.Server({port:8080})
Websockets
141
wss.on('connection',ws=>{
ws.on('message',message=>{
console.log(`Receivedmessage=>${message}`)
})
ws.send('ho!')
})
Thiscodecreatesanewserveronport8080(thedefaultportforWebSockets),andaddsacallbackfunctionwhenaconnectionisestablished,sending ho!totheclient,andloggingthemessagesitreceives.
SeealiveexampleonGlitchHereisaliveexampleofaWebSocketsserver:https://glitch.com/edit/#!/flavio-websockets-server-example
HereisaWebSocketsclientthatinteractswiththeserver:https://glitch.com/edit/#!/flavio-websockets-client-example
Websockets
142
HTTPS,secureconnectionsTheHTTPSprotocolisanextensionofHTTP,theHyperTextTransferProtocol,thatprovidesecurecommunication
HTTPininsecurebydesign.
Whenyouopenyourbrowserandaskawebservertosendyouawebpage,yourdataperforms2trips:1fromthebrowsertothewebserver,and1fromthewebservertothebrowser.
Then,dependingonthecontentofthewebpage,youmighthavemoreconnectionsrequiredtogettheCSSfiles,theJavaScriptfiles,images,andsoon.
Duringanyofthoseconnections,anynetworkyourdataisgoingtocrosscanbeinspectedandmanipulated.
Theconsequencescanbeserious:youmighthaveallyournetworkactivitymonitoredandlogged,bya3rdpartyouarenotevenawareitexist,somenetworksmightinjectads,andyoumightbesubjecttoaman-in-the-middleattack,asecuritythreatwheretheattackercanmanipulateyourdataandevenimpersonateyourcomputeroverthenetwork.It'sveryeasyforsomeonetojustlistentoHTTPpacketsbeingtransmittedoverapublicandunencryptedWi-Finetwork.
HTTPSaimstosolvetheproblemattheroot:theentirecommunicationbetweenyourbrowserandthewebserverisencrypted.
Privacyandsecurityareamajorconcernintoday'sinternet.Afewyearsago,youcouldgetawaywithjustusinganencryptedconnectioninlogin-protectedpages,orduringane-commercecheckout.AlsobecauseofSSLcertificatespricingandcomplications,mostwebsitesjustusedHTTP.
TodayHTTPSisarequirementonanysite.Morethan50%ofthewholeWebusesitnow.GoogleChromerecentlystartedmarkingHTTPsitesasinsecure,justtogiveyouavalidreasontohaveHTTPSmandatory(andforced)onallyourwebsites.
WhenusingHTTPthedefaultserverportis80,andonHTTPSit's443.Itdoesnotneedtobeexplicitlyaddediftheserverusesthedefaultport,ofcourse.
HTTPSisalsosometimescalledHTTPoverSSL,orHTTPoverTLS.
Thedifferencebetweenthetwoissimple:TLSisthesuccessorofSSL.
WhenusingHTTPS,theonlythingthatisnotencryptedisthewebserverdomain,andtheserverport.
HTTPS,secureconnections
143
Everyotherinformation,includingtheresourcepath,headers,cookiesandqueryparametersareallencrypted.
Iwon'tgointhedetailsofanalyzinghowtheTLSprotocolworksunderthehoods,butyoumightthinkit'saddingagoodamountofoverhead,andyouwouldberight.
Anycomputationthat'saddedtotheprocessingofnetworkresourcescausesoverheadbothontheclient,theserver,andtothetransmittedpacketssize.
HoweverHTTPSenablestheuseofthenewestprotocolHTTP/2,whichhasahugeadvantageoverHTTP/1.1:itwayfaster.
Why?Therearemanyreasons,oneisheadercompression,oneisresourcemultiplexing.Oneisserverpush:theservercanpushmoreresourceswhenoneresourceisrequested.So,ifthebrowserrequestsapage,itwillalsoreceivealltheresourcesneeded(images,CSS,JS).
Detailsaside,HTTP/2isahugeimprovementoverHTTP/1.1anditrequiresHTTPS.ThismeansthatHTTPS,despitehavingtheencryptionoverhead,happenstobewayfasterthanHTTP,ifthingsareproperlyconfiguredwithamodernsetup.
HTTPS,secureconnections
144
FiledescriptorsHowtointeractwithfiledescriptorsusingNode
Beforeyou'reabletointeractwithafilethatsitsinyourfilesystem,youmustgetafiledescriptor.
Afiledescriptoriswhat'sreturnedbyopeningthefileusingthe open()methodofferedbythefsmodule:
constfs=require('fs')
fs.open('/Users/flavio/test.txt','r',(err,fd)=>{
//fdisourfiledescriptor
})
Noticethe rweusedasthesecondparametertothe fs.open()call.
Thatflagmeansweopenthefileforreading.
Otherflagsyou'llcommonlyuseare
r+openthefileforreadingandwritingw+openthefileforreadingandwriting,positioningthestreamatthebeginningofthefile.Thefileiscreatedifnotexistingaopenthefileforwriting,positioningthestreamattheendofthefile.Thefileiscreatedifnotexistinga+openthefileforreadingandwriting,positioningthestreamattheendofthefile.Thefileiscreatedifnotexisting
Youcanalsoopenthefilebyusingthe fs.openSyncmethod,whichinsteadofprovidingthefiledescriptorobjectinacallback,itreturnsit:
constfs=require('fs')
try{
constfd=fs.openSync('/Users/flavio/test.txt','r')
}catch(err){
console.error(err)
}
Onceyougetthefiledescriptor,inwhateverwayyouchoose,youcanperformalltheoperationsthatrequireit,likecalling fs.open()andmanyotheroperationsthatinteractwiththefilesystem.
Filedescriptors
145
Filedescriptors
146
FilestatsHowtogetthedetailsofafileusingNode
EveryfilecomeswithasetofdetailsthatwecaninspectusingNode.
Inparticular,usingthe stat()methodprovidedbythe fsmodule.
Youcallitpassingafilepath,andonceNodegetsthefiledetailsitwillcallthecallbackfunctionyoupass,with2parameters:anerrormessage,andthefilestats:
constfs=require('fs')
fs.stat('/Users/flavio/test.txt',(err,stats)=>{
if(err){
console.error(err)
return
}
//wehaveaccesstothefilestatsin`stats`
})
Nodeprovidesalsoasyncmethod,whichblocksthethreaduntilthefilestatsareready:
constfs=require('fs')
try{
conststats=fs.stat('/Users/flavio/test.txt')
}catch(err){
console.error(err)
}
Thefileinformationisincludedinthestatsvariable.Whatkindofinformationcanweextractusingthestats?
Alot,including:
ifthefileisadirectoryorafile,using stats.isFile()and stats.isDirectory()ifthefileisasymboliclinkusing stats.isSymbolicLink()thefilesizeinbytesusing stats.size.
Thereareotheradvancedmethods,butthebulkofwhatyou'lluseinyourday-to-dayprogrammingisthis.
constfs=require('fs')
fs.stat('/Users/flavio/test.txt',(err,stats)=>{
if(err){
console.error(err)
return
}
Filestats
147
stats.isFile()//true
stats.isDirectory()//false
stats.isSymbolicLink()//false
stats.size//1024000//=1MB
})
Filestats
148
FilepathsHowtointeractwithfilepathsandmanipulatetheminNode
GettinginformationoutofapathWorkingwithpaths
Everyfileinthesystemhasapath.
OnLinuxandmacOS,apathmightlooklike:
/users/flavio/file.txt
whileWindowscomputersaredifferent,andhaveastructuresuchas:
C:\users\flavio\file.txt
Youneedtopayattentionwhenusingpathsinyourapplications,asthisdifferencemustbetakenintoaccount.
Youincludethismoduleinyourfilesusing
constpath=require('path')
andyoucanstartusingitsmethods.
GettinginformationoutofapathGivenapath,youcanextractinformationoutofitusingthosemethods:
dirname:gettheparentfolderofafilebasename:getthefilenamepartextname:getthefileextension
Example:
constnotes='/users/flavio/notes.txt'
path.dirname(notes)///users/flavio
path.basename(notes)//notes.txt
path.extname(notes)//.txt
Youcangetthefilenamewithouttheextensionbyspecifyingasecondargumenttobasename:
Filepaths
149
path.basename(notes,path.extname(notes))//notes
WorkingwithpathsYoucanjointwoormorepartsofapathbyusing path.join():
constname='flavio'
path.join('/','users',name,'notes.txt')//'/users/flavio/notes.txt'
Youcangettheabsolutepathcalculationofarelativepathusing path.resolve():
path.resolve('flavio.txt')//'/Users/flavio/flavio.txt'ifrunfrommyhomefolder
InthiscaseNodewillsimplyappend /flavio.txttothecurrentworkingdirectory.Ifyouspecifyasecondparameterfolder, resolvewillusethefirstasabaseforthesecond:
path.resolve('tmp','flavio.txt')//'/Users/flavio/tmp/flavio.txt'ifrunfrommyhomefold
er
Ifthefirstparameterstartswithaslash,thatmeansit'sanabsolutepath:
path.resolve('/etc','flavio.txt')//'/etc/flavio.txt'
path.normalize()isanotherusefulfunction,thatwilltryandcalculatetheactualpath,whenitcontainsrelativespecifierslike .or ..,ordoubleslashes:
path.normalize('/users/flavio/..//test.txt')///users/test.txt
Bothresolveandnormalizewillnotcheckifthepathexists.Theyjustcalculateapathbasedontheinformationtheygot.
Filepaths
150
ReadingfilesHowtoreadfilesusingNode
ThesimplestwaytoreadafileinNodeistousethe fs.readFile()method,passingitthefilepathandacallbackfunctionthatwillbecalledwiththefiledata(andtheerror):
constfs=require('fs')
fs.readFile('/Users/flavio/test.txt',(err,data)=>{
if(err){
console.error(err)
return
}
console.log(data)
})
Alternatively,youcanusethesynchronousversion fs.readFileSync():
constfs=require('fs')
try{
constdata=fs.readFileSync('/Users/flavio/test.txt','utf8')
console.log(data)
}catch(err){
console.error(err)
}
Thedefaultencodingisutf8,butyoucanspecifyacustomencodingusingaasecondparameter.
Both fs.readFile()and fs.readFileSync()readthefullcontentofthefileinmemorybeforereturningthedata.
Thismeansthatbigfilesaregoingtohaveamajorimpactonyourmemoryconsumptionandspeedofexecutionoftheprogram.
Inthiscase,abetteroptionistoreadthefilecontentusingstreams.
Readingfiles
151
WritingfilesHowtowritefilesusingNode
TheeasiestwaytowritetofilesinNode.jsistousethe fs.writeFile()API.
Example:
constfs=require('fs')
constcontent='Somecontent!'
fs.writeFile('/Users/flavio/test.txt',content,(err)=>{
if(err){
console.error(err)
return
}
//filewrittensuccessfully
})
Alternatively,youcanusethesynchronousversion fs.writeFileSync():
constfs=require('fs')
constcontent='Somecontent!'
try{
constdata=fs.writeFileSync('/Users/flavio/test.txt',content)
//filewrittensuccessfully
}catch(err){
console.error(err)
}
Bydefault,thisAPIwillreplacethecontentsofthefileifitdoesalreadyexist.
Youcanmodifythedefaultbyspecifyingaflag:
fs.writeFile('/Users/flavio/test.txt',content,{flag:'a+'},(err)=>{})
Theflagsyou'lllikelyuseare
r+openthefileforreadingandwritingw+openthefileforreadingandwriting,positioningthestreamatthebeginningofthefile.Thefileiscreatedifnotexistingaopenthefileforwriting,positioningthestreamattheendofthefile.Thefileiscreatedifnotexisting
Writingfiles
152
a+openthefileforreadingandwriting,positioningthestreamattheendofthefile.Thefileiscreatedifnotexisting
(youcanfindmoreflagsathttps://nodejs.org/api/fs.html#fs_file_system_flags)
AppendtoafileAhandymethodtoappendcontenttotheendofafileis fs.appendFile()(anditsfs.appendFileSync()counterpart):
constcontent='Somecontent!'
fs.appendFile('file.log',content,(err)=>{
if(err){
console.error(err)
return
}
//done!
})
UsingstreamsAllthosemethodswritethefullcontenttothefilebeforereturningthecontrolbacktoyourprogram(intheasyncversion,thismeansexecutingthecallback)
Inthiscase,abetteroptionistowritethefilecontentusingstreams.
Writingfiles
153
WorkingwithfoldersHowtointeractwithfoldersusingNode
TheNode.js fscoremoduleprovidesmanyhandymethodsyoucanusetoworkwithfolders.
CheckifafolderexistsUse fs.access()tocheckifthefolderexistsandNodecanaccessitwithitspermissions.
CreateanewfolderUse fs.mkdir()or fs.mkdirSync()tocreateanewfolder.
constfs=require('fs')
constfolderName='/Users/flavio/test'
try{
if(!fs.existsSync(dir)){
fs.mkdirSync(dir)
}
}catch(err){
console.error(err)
}
ReadthecontentofadirectoryUse fs.readdir()or fs.readdirSynctoreadthecontentsofadirectory.
Thispieceofcodereadsthecontentofafolder,bothfilesandsubfolders,andreturnstheirrelativepath:
constfs=require('fs')
constpath=require('path')
constfolderPath='/Users/flavio'
fs.readdirSync(folderPath)
Youcangetthefullpath:
Workingwithfolders
154
fs.readdirSync(folderPath).map(fileName=>{
returnpath.join(folderPath,fileName)
}
Youcanalsofiltertheresultstoonlyreturnthefiles,andexcludethefolders:
constisFile=fileName=>{
returnfs.lstatSync(fileName).isFile()
}
fs.readdirSync(folderPath).map(fileName=>{
returnpath.join(folderPath,fileName)).filter(isFile)
}
RenameafolderUse fs.rename()or fs.renameSync()torenamefolder.Thefirstparameteristhecurrentpath,thesecondthenewpath:
constfs=require('fs')
fs.rename('/Users/flavio','/Users/roger',(err)=>{
if(err){
console.error(err)
return
}
//done
})
fs.renameSync()isthesynchronousversion:
constfs=require('fs')
try{
fs.renameSync('/Users/flavio','/Users/roger')
}catch(err){
console.error(err)
}
RemoveafolderUse fs.rmdir()or fs.rmdirSync()toremoveafolder.
Removingafolderthathascontentcanbemorecomplicatedthanyouneed.
Workingwithfolders
155
InthiscaseIrecommendinstallingthe fs-extramodule,whichisverypopularandwellmaintained,andit'sadrop-inreplacementofthe fsmodule,providingmorefeaturesontopofit.
Inthiscasethe remove()methodiswhatyouwant.
Installitusing
npminstallfs-extra
anduseitlikethis:
constfs=require('fs-extra')
constfolder='/Users/flavio'
fs.remove(folder,err=>{
console.error(err)
})
Itcanalsobeusedwithpromises:
fs.remove(folder).then(()=>{
//done
}).catch(err=>{
console.error(err)
})
orwithasync/await:
asyncfunctionremoveFolder(folder){
try{
awaitfs.remove(folder)
//done
}catch(err){
console.error(err)
}
}
constfolder='/Users/flavio'
removeFolder(folder)
Workingwithfolders
156
ThefsmoduleThefsmoduleofNode.jsprovidesusefulfunctionstointeractwiththefilesystem
The fsmoduleprovidesalotofveryusefulfunctionalitytoaccessandinteractwiththefilesystem.
Thereisnoneedtoinstallit.BeingpartoftheNodecore,itcanbeusedbysimplyrequiringit:
constfs=require('fs')
Onceyoudoso,youhaveaccesstoallitsmethods,whichinclude:
fs.access():checkifthefileexistsandNodecanaccessitwithitspermissionsfs.appendFile():appenddatatoafile.Ifthefiledoesnotexist,it'screatedfs.chmod():changethepermissionsofafilespecifiedbythefilenamepassed.Related:fs.lchmod(), fs.fchmod()fs.chown():changetheownerandgroupofafilespecifiedbythefilenamepassed.Related: fs.fchown(), fs.lchown()fs.close():closeafiledescriptorfs.copyFile():copiesafilefs.createReadStream():createareadablefilestreamfs.createWriteStream():createawritablefilestreamfs.link():createanewhardlinktoafilefs.mkdir():createanewfolderfs.mkdtemp():createatemporarydirectoryfs.open():setthefilemodefs.readdir():readthecontentsofadirectoryfs.readFile():readthecontentofafile.Related: fs.read()fs.readlink():readthevalueofasymboliclinkfs.realpath():resolverelativefilepathpointers( ., ..)tothefullpathfs.rename():renameafileorfolderfs.rmdir():removeafolderfs.stat():returnsthestatusofthefileidentifiedbythefilenamepassed.Related:fs.fstat(), fs.lstat()fs.symlink():createanewsymboliclinktoafilefs.truncate():truncatetothespecifiedlengththefileidentifiedbythefilenamepassed.Related: fs.ftruncate()fs.unlink():removeafileorasymboliclink
Thefsmodule
157
fs.unwatchFile():stopwatchingforchangesonafilefs.utimes():changethetimestampofthefileidentifiedbythefilenamepassed.Related:fs.futimes()
fs.watchFile():startwatchingforchangesonafile.Related: fs.watch()fs.writeFile():writedatatoafile.Related: fs.write()
Onepeculiarthingaboutthe fsmoduleisthatallthemethodsareasynchronousbydefault,buttheycanalsoworksynchronouslybyappending Sync.
Forexample:
fs.rename()
fs.renameSync()
fs.write()
fs.writeSync()
Thismakesahugedifferenceinyourapplicationflow.
Node10includesexperimentalsupportforapromisebasedAPI
Forexamplelet'sexaminethe fs.rename()method.TheasynchronousAPIisusedwithacallback:
constfs=require('fs')
fs.rename('before.json','after.json',(err)=>{
if(err){
returnconsole.error(err)
}
//done
})
AsynchronousAPIcanbeusedlikethis,withatry/catchblocktohandleerrors:
constfs=require('fs')
try{
fs.renameSync('before.json','after.json')
//done
}catch(err){
console.error(err)
}
Thekeydifferencehereisthattheexecutionofyourscriptwillblockinthesecondexample,untilthefileoperationsucceeded.
Thefsmodule
158
Thefsmodule
159
ThepathmoduleThepathmoduleofNode.jsprovidesusefulfunctionstointeractwithfilepaths
The pathmoduleprovidesalotofveryusefulfunctionalitytoaccessandinteractwiththefilesystem.
Thereisnoneedtoinstallit.BeingpartoftheNodecore,itcanbeusedbysimplyrequiringit:
constpath=require('path')
Thismoduleprovides path.sepwhichprovidesthepathsegmentseparator( \onWindows,and /onLinux/macOS),and path.delimiterwhichprovidesthepathdelimiter( ;onWindows,and :onLinux/macOS).
Thesearethe pathmethods:
path.basename()
path.dirname()
path.extname()
path.isAbsolute()
path.join()
path.normalize()
path.parse()
path.relative()
path.resolve()
path.basename()
Returnthelastportionofapath.Asecondparametercanfilteroutthefileextension:
require('path').basename('/test/something')//something
require('path').basename('/test/something.txt')//something.txt
require('path').basename('/test/something.txt','.txt')//something
path.dirname()
Returnthedirectorypartofapath:
require('path').dirname('/test/something')///test
require('path').dirname('/test/something/file.txt')///test/something
Thepathmodule
160
path.extname()
Returntheextensionpartofapath
require('path').dirname('/test/something')//''
require('path').dirname('/test/something/file.txt')//'.txt'
path.isAbsolute()
Returnstrueifit'sanabsolutepath
require('path').isAbsolute('/test/something')//true
require('path').isAbsolute('./test/something')//false
path.join()
Joinstwoormorepartsofapath:
constname='flavio'
require('path').join('/','users',name,'notes.txt')//'/users/flavio/notes.txt'
path.normalize()
Triestocalculatetheactualpathwhenitcontainsrelativespecifierslike .or ..,ordoubleslashes:
require('path').normalize('/users/flavio/..//test.txt')///users/test.txt
path.parse()
Parsesapathtoanobjectwiththesegmentsthatcomposeit:
root:therootdir:thefolderpathstartingfromtherootbase:thefilename+extensionname:thefilenameext:thefileextension
Example:
require('path').parse('/users/test.txt')
Thepathmodule
161
resultsin
{
root:'/',
dir:'/users',
base:'test.txt',
ext:'.txt',
name:'test'
}
path.relative()
Accepts2pathsasarguments.Returnsthetherelativepathfromthefirstpathtothesecond,basedonthecurrentworkingdirectory.
Example:
require('path').relative('/Users/flavio','/Users/flavio/test.txt')//'test.txt'
require('path').relative('/Users/flavio','/Users/flavio/something/test.txt')//'something
/test.txt'
path.resolve()
Youcangettheabsolutepathcalculationofarelativepathusing path.resolve():
path.resolve('flavio.txt')//'/Users/flavio/flavio.txt'ifrunfrommyhomefolder
Byspecifyingasecondparameter, resolvewillusethefirstasabaseforthesecond:
path.resolve('tmp','flavio.txt')//'/Users/flavio/tmp/flavio.txt'ifrunfrommyhomefold
er
Ifthefirstparameterstartswithaslash,thatmeansit'sanabsolutepath:
path.resolve('/etc','flavio.txt')//'/etc/flavio.txt'
Thepathmodule
162
TheosmoduleTheosmoduleofNode.jsprovidesusefulfunctionstointeractwithunderlyingsystem
Thismoduleprovidesmanyfunctionsthatyoucanusetoretrieveinformationfromtheunderlyingoperatingsystemandthecomputertheprogramrunson,andinteractwithit.
constos=require('os')
Thereareafewusefulpropertiesthattellussomekeythingsrelatedtohandlingfiles:
os.EOLgivesthelinedelimitersequence.It's \nonLinuxandmacOS,and \r\nonWindows.
WhenIsayLinuxandmacOSImeanPOSIXplatforms.ForsimplicityIexcludeotherlesspopularoperatingsystemsNodecanrunon.
os.constants.signalstellsusalltheconstantsrelatedtohandlingprocesssignals,likeSIGHUP,SIGKILLandsoon.
os.constants.errnosetstheconstantsforerrorreporting,likeEADDRINUSE,EOVERFLOWandmore.
Youcanreadthemallonhttps://nodejs.org/api/os.html#os_signal_constants.
Let'snowseethemainmethodsthat osprovides:
os.arch()
os.cpus()
os.endianness()
os.freemem()
os.homedir()
os.hostname()
os.loadavg()
os.networkInterfaces()
os.platform()
os.release()
os.tmpdir()
os.totalmem()
os.type()
os.uptime()
os.userInfo()
Theosmodule
163
os.arch()
Returnthestringthatidentifiestheunderlyingarchitecture,like arm, x64, arm64.
os.cpus()
ReturninformationontheCPUsavailableonyoursystem.
Example:
[{model:'Intel(R)Core(TM)[email protected]',
speed:2400,
times:
{user:281685380,
nice:0,
sys:187986530,
idle:685833750,
irq:0}},
{model:'Intel(R)Core(TM)[email protected]',
speed:2400,
times:
{user:282348700,
nice:0,
sys:161800480,
idle:703509470,
irq:0}}]
os.endianness()
Return BEor LEdependingifNodewascompiledwithBigEndianorLittleEndian.
os.freemem()
Returnthenumberofbytesthatrepresentthefreememoryinthesystem.
os.homedir()
Returnthepathtothehomedirectoryofthecurrentuser.
Example:
'/Users/flavio'
Theosmodule
164
os.hostname()
Returnthehostname.
os.loadavg()
Returnthecalculationmadebytheoperatingsystemontheloadaverage.
ItonlyreturnsameaningfulvalueonLinuxandmacOS.
Example:
[3.68798828125,4.00244140625,11.1181640625]
os.networkInterfaces()
Returnsthedetailsofthenetworkinterfacesavailableonyoursystem.
Example:
{lo0:
[{address:'127.0.0.1',
netmask:'255.0.0.0',
family:'IPv4',
mac:'fe:82:00:00:00:00',
internal:true},
{address:'::1',
netmask:'ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff',
family:'IPv6',
mac:'fe:82:00:00:00:00',
scopeid:0,
internal:true},
{address:'fe80::1',
netmask:'ffff:ffff:ffff:ffff::',
family:'IPv6',
mac:'fe:82:00:00:00:00',
scopeid:1,
internal:true}],
en1:
[{address:'fe82::9b:8282:d7e6:496e',
netmask:'ffff:ffff:ffff:ffff::',
family:'IPv6',
mac:'06:00:00:02:0e:00',
scopeid:5,
internal:false},
{address:'192.168.1.38',
netmask:'255.255.255.0',
family:'IPv4',
mac:'06:00:00:02:0e:00',
Theosmodule
165
internal:false}],
utun0:
[{address:'fe80::2513:72bc:f405:61d0',
netmask:'ffff:ffff:ffff:ffff::',
family:'IPv6',
mac:'fe:80:00:20:00:00',
scopeid:8,
internal:false}]}
os.platform()
ReturntheplatformthatNodewascompiledfor:
darwin
freebsd
linux
openbsd
win32
...more
os.release()
Returnsastringthatidentifiestheoperatingsystemreleasenumber
os.tmpdir()
Returnsthepathtotheassignedtempfolder.
os.totalmem()
Returnsthenumberofbytesthatrepresentthetotalmemoryavailableinthesystem.
os.type()
Identifiestheoperatingsystem:
Linux
DarwinonmacOSWindows_NTonWindows
os.uptime()
Theosmodule
166
Returnsthenumberofsecondsthecomputerhasbeenrunningsinceitwaslastrebooted.
os.userInfo()
Theosmodule
167
TheeventsmoduleTheeventsmoduleofNode.jsprovidestheEventEmitterclass
The eventsmoduleprovidesustheEventEmitterclass,whichiskeytoworkingwitheventsinNode.
Ipublishedafullarticleonthat,sohereIwilljustdescribetheAPIwithoutfurtherexamplesonhowtouseit.
constEventEmitter=require('events')
constdoor=newEventEmitter()
Theeventlistenereatsitsowndogfoodandusestheseevents:
newListenerwhenalistenerisaddedremoveListenerwhenalistenerisremoved
Here'sadetaileddescriptionofthemostusefulmethods:
emitter.addListener()
emitter.emit()
emitter.eventNames()
emitter.getMaxListeners()
emitter.listenerCount()
emitter.listeners()
emitter.off()
emitter.on()
emitter.once()
emitter.prependListener()
emitter.prependOnceListener()
emitter.removeAllListeners()
emitter.removeListener()
emitter.setMaxListeners()
emitter.addListener()
Aliasfor emitter.on().
emitter.emit()
Theeventsmodule
168
Emitsanevent.Itsynchronouslycallseveryeventlistenerintheordertheywereregistered.
emitter.eventNames()
ReturnanarrayofstringsthatrepresenttheeventsregisteredonthecurrentEventListener:
door.eventNames()
emitter.getMaxListeners()
GetthemaximumamountoflistenersonecanaddtoanEventListenerobject,whichdefaultsto10butcanbeincreasedorloweredbyusing setMaxListeners()
door.getMaxListeners()
emitter.listenerCount()
Getthecountoflistenersoftheeventpassedasparameter:
door.listenerCount('open')
emitter.listeners()
Getsanarrayoflistenersoftheeventpassedasparameter:
door.listeners('open')
emitter.off()
Aliasfor emitter.removeListener()addedinNode10
emitter.on()
Addsacallbackfunctionthat'scalledwhenaneventisemitted.
Usage:
door.on('open',()=>{
Theeventsmodule
169
console.log('Doorwasopened')
})
emitter.once()
Addsacallbackfunctionthat'scalledwhenaneventisemittedforthefirsttimeafterregisteringthis.Thiscallbackisonlygoingtobecalledonce,neveragain.
constEventEmitter=require('events')
constee=newEventEmitter()
ee.once('my-event',()=>{
//callcallbackfunctiononce
})
emitter.prependListener()
Whenyouaddalistenerusing onor addListener,it'saddedlastinthequeueoflisteners,andcalledlast.Using prependListenerit'sadded,andcalled,beforeotherlisteners.
emitter.prependOnceListener()
Whenyouaddalistenerusing once,it'saddedlastinthequeueoflisteners,andcalledlast.Using prependOnceListenerit'sadded,andcalled,beforeotherlisteners.
emitter.removeAllListeners()
Removesalllistenersofaneventemitterobjectlisteningtoaspecificevent:
door.removeAllListeners('open')
emitter.removeListener()
Removeaspecificlistener.Youcandothisbysavingthecallbackfunctiontoavariable,whenadded,soyoucanreferenceitlater:
constdoSomething=()=>{}
door.on('open',doSomething)
door.removeListener('open',doSomething)
Theeventsmodule
170
emitter.setMaxListeners()
SetsthemaximumamountoflistenersonecanaddtoanEventListenerobject,whichdefaultsto10butcanbeincreasedorlowered.
door.setMaxListeners(50)
Theeventsmodule
171
ThehttpmoduleThehttpmoduleofNode.jsprovidesusefulfunctionsandclassestobuildanHTTPserver
TheHTTPcoremoduleisakeymoduletoNodenetworking.
Propertieshttp.METHODS
http.STATUS_CODES
http.globalAgent
Methodshttp.createServer()
http.request()
http.get()
Classeshttp.Agent
http.ClientRequest
http.Server
http.ServerResponse
http.IncomingMessage
Itcanbeincludedusing
consthttp=require('http')
Themoduleprovidessomepropertiesandmethods,andsomeclasses.
Properties
http.METHODS
ThispropertylistsalltheHTTPmethodssupported:
>require('http').METHODS
['ACL',
'BIND',
'CHECKOUT',
'CONNECT',
'COPY',
'DELETE',
'GET',
Thehttpmodule
172
'HEAD',
'LINK',
'LOCK',
'M-SEARCH',
'MERGE',
'MKACTIVITY',
'MKCALENDAR',
'MKCOL',
'MOVE',
'NOTIFY',
'OPTIONS',
'PATCH',
'POST',
'PROPFIND',
'PROPPATCH',
'PURGE',
'PUT',
'REBIND',
'REPORT',
'SEARCH',
'SUBSCRIBE',
'TRACE',
'UNBIND',
'UNLINK',
'UNLOCK',
'UNSUBSCRIBE']
http.STATUS_CODES
ThispropertylistsalltheHTTPstatuscodesandtheirdescription:
>require('http').STATUS_CODES
{'100':'Continue',
'101':'SwitchingProtocols',
'102':'Processing',
'200':'OK',
'201':'Created',
'202':'Accepted',
'203':'Non-AuthoritativeInformation',
'204':'NoContent',
'205':'ResetContent',
'206':'PartialContent',
'207':'Multi-Status',
'208':'AlreadyReported',
'226':'IMUsed',
'300':'MultipleChoices',
'301':'MovedPermanently',
'302':'Found',
'303':'SeeOther',
'304':'NotModified',
'305':'UseProxy',
'307':'TemporaryRedirect',
'308':'PermanentRedirect',
'400':'BadRequest',
Thehttpmodule
173
'401':'Unauthorized',
'402':'PaymentRequired',
'403':'Forbidden',
'404':'NotFound',
'405':'MethodNotAllowed',
'406':'NotAcceptable',
'407':'ProxyAuthenticationRequired',
'408':'RequestTimeout',
'409':'Conflict',
'410':'Gone',
'411':'LengthRequired',
'412':'PreconditionFailed',
'413':'PayloadTooLarge',
'414':'URITooLong',
'415':'UnsupportedMediaType',
'416':'RangeNotSatisfiable',
'417':'ExpectationFailed',
'418':'I\'mateapot',
'421':'MisdirectedRequest',
'422':'UnprocessableEntity',
'423':'Locked',
'424':'FailedDependency',
'425':'UnorderedCollection',
'426':'UpgradeRequired',
'428':'PreconditionRequired',
'429':'TooManyRequests',
'431':'RequestHeaderFieldsTooLarge',
'451':'UnavailableForLegalReasons',
'500':'InternalServerError',
'501':'NotImplemented',
'502':'BadGateway',
'503':'ServiceUnavailable',
'504':'GatewayTimeout',
'505':'HTTPVersionNotSupported',
'506':'VariantAlsoNegotiates',
'507':'InsufficientStorage',
'508':'LoopDetected',
'509':'BandwidthLimitExceeded',
'510':'NotExtended',
'511':'NetworkAuthenticationRequired'}
http.globalAgent
PointstotheglobalinstanceoftheAgentobject,whichisaninstanceofthe http.Agentclass.
It'susedtomanageconnectionspersistanceandreuseforHTTPclients,andit'sakeycomponentofNodeHTTPnetworking.
Moreinthe http.Agentclassdescriptionlateron.
Methods
Thehttpmodule
174
http.createServer()
Returnanewinstanceofthe http.Serverclass.
Usage:
constserver=http.createServer((req,res)=>{
//handleeverysinglerequestwiththiscallback
})
http.request()
MakesanHTTPrequesttoaserver,creatinganinstanceofthe http.ClientRequestclass.
http.get()
Similarto http.request(),butautomaticallysetstheHTTPmethodtoGET,andcallsreq.end()automatically.
ClassesTheHTTPmoduleprovides5classes:
http.Agent
http.ClientRequest
http.Server
http.ServerResponse
http.IncomingMessage
http.Agent
Nodecreatesaglobalinstanceofthe http.AgentclasstomanageconnectionspersistanceandreuseforHTTPclients,akeycomponentofNodeHTTPnetworking.
Thisobjectmakessurethateveryrequestmadetoaserverisqueuedandasinglesocketisreused.
Italsomaintainsapoolofsockets.Thisiskeyforperformancereasons.
http.ClientRequest
An http.ClientRequestobjectiscreatedwhen http.request()or http.get()iscalled.
Thehttpmodule
175
Whenaresponseisreceived,the responseeventiscalledwiththeresponse,withanhttp.IncomingMessageinstanceasargument.
Thereturneddataofaresponsecanbereadin2ways:
youcancallthe response.read()methodinthe responseeventhandleryoucansetupaneventlistenerforthe dataevent,soyoucanlistenforthedatastreamedinto.
http.Server
Thisclassiscommonlyinstantiatedandreturnedwhencreatinganewserverusinghttp.createServer().
Onceyouhaveaserverobject,youhaveaccesstoitsmethods:
close()stopstheserverfromacceptingnewconnectionslisten()startstheHTTPserverandlistensforconnections
http.ServerResponse
Createdbyan http.Serverandpassedasthesecondparametertothe requesteventitfires.
Commonlyknownandusedincodeas res:
constserver=http.createServer((req,res)=>{
//resisanhttp.ServerResponseobject
})
Themethodyou'llalwayscallinthehandleris end(),whichclosestheresponse,themessageiscompleteandtheservercansendittotheclient.Itmustbecalledoneachresponse.
ThesemethodsareusedtointeractwithHTTPheaders:
getHeaderNames()getthelistofthenamesoftheHTTPheadersalreadysetgetHeaders()getacopyoftheHTTPheadersalreadysetsetHeader('headername',value)setsanHTTPheadervaluegetHeader('headername')getsanHTTPheaderalreadysetremoveHeader('headername')removesanHTTPheaderalreadysethasHeader('headername')returntrueiftheresponsehasthatheadersetheadersSent()returntrueiftheheadershavealreadybeensenttotheclient
Thehttpmodule
176
Afterprocessingtheheadersyoucansendthemtotheclientbycalling response.writeHead(),whichacceptsthestatusCodeasthefirstparameter,theoptionalstatusmessage,andtheheadersobject.
Tosenddatatotheclientintheresponsebody,youuse write().ItwillsendbuffereddatatotheHTTPresponsestream.
Iftheheaderswerenotsentyetusing response.writeHead(),itwillsendtheheadersfirst,withthestatuscodeandmessagethat'ssetintherequest,whichyoucaneditbysettingthestatusCodeand statusMessagepropertiesvalues:
response.statusCode=500
response.statusMessage='InternalServerError'
http.IncomingMessage
An http.IncomingMessageobjectiscreatedby:
http.Serverwhenlisteningtothe requesteventhttp.ClientRequestwhenlisteningtothe responseevent
Itcanbeusedtoaccesstheresponse:
statususingits statusCodeand statusMessagemethodsheadersusingits headersmethodor rawHeadersHTTPmethodusingits methodmethodHTTPversionusingthe httpVersionmethodURLusingthe urlmethodunderlyingsocketusingthe socketmethod
Thedataisaccessedusingstreams,since http.IncomingMessageimplementstheReadableStreaminterface.
Thehttpmodule
177
StreamsLearnwhatstreamsarefor,whyaretheysoimportant,andhowtousethem.
WhatarestreamsWhystreamsAnexampleofastreampipe()Streams-poweredNodeAPIsDifferenttypesofstreamsHowtocreateareadablestreamHowtocreateawritablestreamHowtogetdatafromareadablestreamHowtosenddatatoawritablestreamSignalingawritablestreamthatyouendedwritingConclusion
WhatarestreamsStreamsareoneofthefundamentalconceptsthatpowerNode.jsapplications.
Theyareawaytohandlereading/writingfiles,networkcommunications,oranykindofend-to-endinformationexchangeinanefficientway.
StreamsarenotaconceptuniquetoNode.js.TheywereintroducedintheUnixoperatingsystemdecadesago,andprogramscaninteractwitheachotherpassingstreamsthroughthepipeoperator( |).
Forexample,inthetraditionalway,whenyoutelltheprogramtoreadafile,thefileisreadintomemory,fromstarttofinish,andthenyouprocessit.
Usingstreamsyoureaditpiecebypiece,processingitscontentwithoutkeepingitallinmemory.
TheNode.js streammoduleprovidesthefoundationuponwhichallstreamingAPIsarebuild.
WhystreamsStreamsbasicallyprovidetwomajoradvantagesusingotherdatahandlingmethods:
Memoryefficiency:youdon'tneedtoloadlargeamountsofdatainmemorybeforeyou
Streams
178
areabletoprocessitTimeefficiency:ittakeswaylesstimetostartprocessingdataassoonasyouhaveit,ratherthanwaitingtillthewholedatapayloadisavailabletostart
AnexampleofastreamAtypicalexampleistheoneofreadingfilesfromadisk.
UsingtheNode fsmoduleyoucanreadafile,andserveitoverHTTPwhenanewconnectionisestablishedtoyourhttpserver:
consthttp=require('http')
constfs=require('fs')
constserver=http.createServer(function(req,res){
fs.readFile(__dirname+'/data.txt',(err,data)=>{
res.end(data)
})
})
server.listen(3000)
readFile()readsthefullcontentsofthefile,andinvokesthecallbackfunctionwhenit'sdone.
res.end(data)inthecallbackwillreturnthefilecontentstotheHTTPclient.
Ifthefileisbig,theoperationwilltakequiteabitoftime.Hereisthesamethingwrittenusingstreams:
consthttp=require('http')
constfs=require('fs')
constserver=http.createServer((req,res)=>{
conststream=fs.createReadStream(__dirname+'/data.txt')
stream.pipe(res)
})
server.listen(3000)
Insteadofwaitinguntilthefileisfullyread,westartstreamingittotheHTTPclientassoonaswehaveachunkofdatareadytobesent.
pipe()Theaboveexampleusestheline stream.pipe(res):the pipe()methodiscalledonthefilestream.
Streams
179
Whatdoesthiscodedo?Ittakesthesource,andpipesitintoadestination.
Youcallitonthesourcestream,sointhiscase,thefilestreamispipedtotheHTTPresponse.
Thereturnvalueofthe pipe()methodisthedestinationstream,whichisaveryconvenientthingthatletsuschainmultiple pipe()calls,likethis:
src.pipe(dest1).pipe(dest2)
Thisconstructisthesameasdoing
src.pipe(dest1)
dest1.pipe(dest2)
Streams-poweredNodeAPIsDuetotheiradvantages,manyNode.jscoremodulesprovidenativestreamhandlingcapabilities,mostnotably:
process.stdinreturnsastreamconnectedtostdinprocess.stdoutreturnsastreamconnectedtostdoutprocess.stderrreturnsastreamconnectedtostderrfs.createReadStream()createsareadablestreamtoafilefs.createWriteStream()createsawritablestreamtoafilenet.connect()initiatesastream-basedconnectionhttp.request()returnsaninstanceofthehttp.ClientRequestclass,whichisawritablestreamzlib.createGzip()compressdatausinggzip(acompressionalgorithm)intoastreamzlib.createGunzip()decompressagzipstream.zlib.createDeflate()compressdatausingdeflate(acompressionalgorithm)intoastreamzlib.createInflate()decompressadeflatestream
DifferenttypesofstreamsTherearefourclassesofstreams:
Readable:astreamyoucanpipefrom,butnotpipeinto(youcanreceivedata,butnotsenddatatoit).Whenyoupushdataintoareadablestream,itisbuffered,untilaconsumerstartstoreadthedata.Writable:astreamyoucanpipeinto,butnotpipefrom(youcansenddata,butnot
Streams
180
receivefromit)Duplex:astreamyoucanbothpipeintoandpipefrom,basicallyacombinationofaReadableandWritablestreamTransform:aTransformstreamissimilartoaDuplex,buttheoutputisatransformofitsinput
HowtocreateareadablestreamWegettheReadablestreamfromthe streammodule,andweinitializeit
constStream=require('stream')
constreadableStream=newStream.Readable()
Nowthatthestreamisinitialized,wecansenddatatoit:
readableStream.push('hi!')
readableStream.push('ho!')
HowtocreateawritablestreamTocreateawritablestreamweextendthebase Writableobject,andweimplementits_write()method.
Firstcreateastreamobject:
constStream=require('stream')
constwritableStream=newStream.Writable()
thenimplement _write:
writableStream._write=(chunk,encoding,next)=>{
console.log(chunk.toString())
next()
}
Youcannowpipeareadablestreamin:
process.stdin.pipe(writableStream)
Howtogetdatafromareadablestream
Streams
181
Howdowereaddatafromareadablestream?Usingawritablestream:
constStream=require('stream')
constreadableStream=newStream.Readable()
constwritableStream=newStream.Writable()
writableStream._write=(chunk,encoding,next)=>{
console.log(chunk.toString())
next()
}
readableStream.pipe(writableStream)
readableStream.push('hi!')
readableStream.push('ho!')
Youcanalsoconsumeareadablestreamdirectly,usingthe readableevent:
readableStream.on('readable',()=>{
console.log(readableStream.read())
})
HowtosenddatatoawritablestreamUsingthestream write()method:
writableStream.write('hey!\n')
SignalingawritablestreamthatyouendedwritingUsethe end()method:
constStream=require('stream')
constreadableStream=newStream.Readable()
constwritableStream=newStream.Writable()
writableStream._write=(chunk,encoding,next)=>{
console.log(chunk.toString())
next()
}
readableStream.pipe(writableStream)
Streams
182
readableStream.push('hi!')
readableStream.push('ho!')
writableStream.end()
ConclusionThisisanintroductiontostreams.Therearemuchmorecomplicatedaspectstoanalyze,andIhopetocoverthemsoon.
Streams
183
WorkingwithMySQLMySQLisoneofthemostpopularrelationaldatabasesintheworld.FindouthowtomakeitworkwithNode.js
MySQLisoneofthemostpopularrelationaldatabasesintheworld.
TheNodeecosystemofcoursehasseveraldifferentpackagesthatallowyoutointerfacewithMySQL,storedata,retrievedata,andsoon.
We'lluse mysqljs/mysql,apackagethathasover12.000GitHubstarsandhasbeenaroundforyears.
InstallingtheNodemysqlpackageYouinstallitusing
npminstallmysql
InitializingtheconnectiontothedatabaseYoufirstincludethepackage:
constmysql=require('mysql')
andyoucreateaconnection:
constoptions={
user:'the_mysql_user_name',
password:'the_mysql_user_password',
database:'the_mysql_database_name'
}
constconnection=mysql.createConnection(options)
Youinitiateanewconnectionbycalling:
connection.connect(err=>{
if(err){
console.error('AnerroroccurredwhileconnectingtotheDB')
throwerr
}
})
WorkingwithMySQL
184
TheconnectionoptionsIntheaboveexample,the optionsobjectcontained3options:
constoptions={
user:'the_mysql_user_name',
password:'the_mysql_user_password',
database:'the_mysql_database_name'
}
Therearemanymoreyoucanuse,including:
host,thedatabasehostname,defaultsto localhostport,theMySQLserverportnumber,defaultsto3306socketPath,usedtospecifyaunixsocketinsteadofhostandportdebug,bydefaultdisabled,canbeusedfordebuggingtrace,bydefaultenabled,printsstacktraceswhenerrorsoccurssl,usedtosetupanSSLconnectiontotheserver(outofthescopeofthistutorial)
PerformaSELECTqueryNowyouarereadytoperformanSQLqueryonthedatabase.Thequeryonceexecutedwillinvokeacallbackfunctionwhichcontainsaneventualerror,theresultsandthefields.
connection.query('SELECT*FROMtodos',(error,todos,fields)=>{
if(error){
console.error('Anerroroccurredwhileexecutingthequery')
throwerror
}
console.log(todos)
})
Youcanpassinvalueswhichwillbeautomaticallyescaped:
constid=223
connection.query('SELECT*FROMtodosWHEREid=?',[id],(error,todos,fields)=>{
if(error){
console.error('Anerroroccurredwhileexecutingthequery')
throwerror
}
console.log(todos)
})
WorkingwithMySQL
185
Topassmultiplevalues,justputmoreelementsinthearrayyoupassasthesecondparameter:
constid=223
constauthor='Flavio'
connection.query('SELECT*FROMtodosWHEREid=?ANDauthor=?',[id,author],(error,
todos,fields)=>{
if(error){
console.error('Anerroroccurredwhileexecutingthequery')
throwerror
}
console.log(todos)
})
PerformanINSERTqueryYoucanpassanobject
consttodo={
thing:'Buythemilk'
author:'Flavio'
}
connection.query('INSERTINTOtodosSET?',todo,(error,results,fields)=>{
if(error){
console.error('Anerroroccurredwhileexecutingthequery')
throwerror
}
})
Ifthetablehasaprimarykeywith auto_increment,thevalueofthatwillbereturnedintheresults.insertIdvalue:
consttodo={
thing:'Buythemilk'
author:'Flavio'
}
connection.query('INSERTINTOtodosSET?',todo,(error,results,fields)=>{
if(error){
console.error('Anerroroccurredwhileexecutingthequery')
throwerror
}}
constid=results.resultId
console.log(id)
)
Closetheconnection
WorkingwithMySQL
186
Whenyouneedtoterminatetheconnectiontothedatabaseyoucancallthe end()method:
connection.end()
Thismakessureanypendingquerygetssent,andtheconnectionisgracefullyterminated.
WorkingwithMySQL
187
DifferencebetweendevelopmentandproductionLearnhowtosetupdifferentconfigurationsforproductionanddevelopmentenvironments
Youcanhavedifferentconfigurationsforproductionanddevelopmentenvironments.
Nodeassumesit'salwaysrunninginadevelopmentenvironment.YoucansignalNode.jsthatyouarerunninginproductionbysettingthe NODE_ENV=productionenvironmentvariable.
Thisisusuallydonebyexecutingthecommand
exportNODE_ENV=production
intheshell,butit'sbettertoputitinyourshellconfigurationfile(e.g. .bash_profilewiththeBashshell)becauseotherwisethesettingdoesnotpersistincaseofasystemrestart.
Youcanalsoapplytheenvironmentvariablebyprependingittoyourapplicationinitializationcommand:
NODE_ENV=productionnodeapp.js
Thisenvironmentvariableisaconventionthatiswidelyusedinexternallibrariesaswell.
Settingtheenvironmentto productiongenerallyensuresthat
loggingiskepttoaminimum,essentiallevelmorecachinglevelstakeplacetooptimizeperformance
ForexamplePug,thetemplatinglibraryusedbyExpress,compilesindebugmodeifNODE_ENVisnotsetto production.Expressviewsarecompiledineveryrequestindevelopmentmode,whileinproductiontheyarecached.Therearemanymoreexamples.
Expressprovidesconfigurationhooksspecifictotheenvironment,whichareautomaticallycalledbasedontheNODE_ENVvariablevalue:
app.configure('development',()=>{
//...
})
app.configure('production',()=>{
//...
})
app.configure('production','staging',()=>{
//...
Differencebetweendevelopmentandproduction
188
})
Forexampleyoucanusethistosetdifferenterrorhandlersfordifferentmode:
app.configure('development',()=>{
app.use(express.errorHandler({dumpExceptions:true,showStack:true}));
})
app.configure('production',()=>{
app.use(express.errorHandler())
})
Differencebetweendevelopmentandproduction
189