You Don't Know JS: Async & Performance

1. Introduction2. Preface3. Foreword4. TableofContent5. Chapter01:Asynchrony:Now&Later6. Chapter02:Callbacks7. Chapter03:Promises8. Chapter04:Generators9. Chapter05:ProgramPerformance10. Chapter06:Benchmarking&Tuning11. AppendixA:àsynquence`Library12. AppendixB:AdvancedAsyncPatterns13. AppendixC:Acknowledgments

TableofContents

Purchasedigital/printcopyfromO'Reilly

TableofContents

Foreword(byJakeArchibald)PrefaceChapter1:Asynchrony:Now&LaterChapter2:CallbacksChapter3:PromisesChapter4:GeneratorsChapter5:ProgramPerformanceChapter6:Benchmarking&TuningAppendixA:Library:asynquenceAppendixB:AdvancedAsyncPatternsAppendixC:ThankYou's!

YouDon'tKnowJS:Async&Performance

http://shop.oreilly.com/product/0636920033752.do

http://jakearchibald.com

I'msureyounoticed,but"JS"inthebookseriestitleisnotanabbreviationforwordsusedtocurseaboutJavaScript,thoughcursingatthelanguage'squirksissomethingwecanprobablyallidentifywith!

Fromtheearliestdaysoftheweb,JavaScripthasbeenafoundationaltechnologythatdrivesinteractiveexperiencearoundthecontentweconsume.Whileflickeringmousetrailsandannoyingpop-uppromptsmaybewhereJavaScriptstarted,nearly2decadeslater,thetechnologyandcapabilityofJavaScripthasgrownmanyordersofmagnitude,andfewdoubtitsimportanceattheheartoftheworld'smostwidelyavailablesoftwareplatform:theweb.

Butasalanguage,ithasperpetuallybeenatargetforagreatdealofcriticism,owingpartlytoitsheritagebutevenmoretoitsdesignphilosophy.Eventhenameevokes,asBrendanEichonceputit,"dumbkidbrother"statusnexttoitsmorematureolderbrother"Java".Butthenameismerelyanaccidentofpoliticsandmarketing.Thetwolanguagesarevastlydifferentinmanyimportantways."JavaScript"isasrelatedto"Java"as"Carnival"isto"Car".

BecauseJavaScriptborrowsconceptsandsyntaxidiomsfromseverallanguages,includingproudC-styleproceduralrootsaswellassubtle,lessobviousScheme/Lisp-stylefunctionalroots,itisexceedinglyapproachabletoabroadaudienceofdevelopers,eventhosewithjustlittletonoprogrammingexperience.The"HelloWorld"ofJavaScriptissosimplethatthelanguageisinvitingandeasytogetcomfortablewithinearlyexposure.

WhileJavaScriptisperhapsoneoftheeasiestlanguagestogetupandrunningwith,itseccentricitiesmakesolidmasteryofthelanguageavastlylesscommonoccurrencethaninmanyotherlanguages.Whereittakesaprettyin-depthknowledgeofalanguagelikeCorC++towriteafull-scaleprogram,full-scaleproductionJavaScriptcan,andoftendoes,barelyscratchthesurfaceofwhatthelanguagecando.

Sophisticatedconceptswhicharedeeplyrootedintothelanguagetendinsteadtosurfacethemselvesinseeminglysimplisticways,suchaspassingaroundfunctionsascallbacks,whichencouragestheJavaScriptdevelopertojustusethelanguageas-isandnotworrytoomuchaboutwhat'sgoingonunderthehood.

Itissimultaneouslyasimple,easy-to-uselanguagethathasbroadappeal,andacomplexandnuancedcollectionoflanguagemechanicswhichwithoutcarefulstudywilleludetrueunderstandingevenforthemostseasonedofJavaScriptdevelopers.

ThereinliestheparadoxofJavaScript,theAchilles'Heelofthelanguage,thechallengewearepresentlyaddressing.BecauseJavaScriptcanbeusedwithoutunderstanding,theunderstandingofthelanguageisoftenneverattained.

IfateverypointthatyouencounterasurpriseorfrustrationinJavaScript,yourresponseistoaddittotheblacklist,assomeareaccustomedtodoing,yousoonwillberelegatedtoahollowshelloftherichnessofJavaScript.

Whilethissubsethasbeenfamouslydubbed"TheGoodParts",Iwouldimploreyou,dearreader,toinsteadconsideritthe"TheEasyParts","TheSafeParts",oreven"TheIncompleteParts".

ThisYouDon'tKnowJavaScriptbookseriesoffersacontrarychallenge:learnanddeeplyunderstandallofJavaScript,evenandespecially"TheToughParts".

Here,weaddressheadonthetendencyofJSdeveloperstolearn"justenough"togetby,withouteverforcingthemselvestolearnexactlyhowandwhythelanguagebehavesthewayitdoes.Furthermore,weeschewthecommonadvicetoretreatwhentheroadgetsrough.

Iamnotcontent,norshouldyoube,atstoppingoncesomethingjustworks,andnotreallyknowingwhy.Igentlychallenge

YouDon'tKnowJS

Preface

Mission

youtojourneydownthatbumpy"roadlesstraveled"andembraceallthatJavaScriptisandcando.Withthatknowledge,notechnique,noframework,nopopularbuzzwordacronymoftheweek,willbebeyondyourunderstanding.

Thesebookseachtakeonspecificcorepartsofthelanguagewhicharemostcommonlymisunderstoodorunder-understood,anddiveverydeepandexhaustivelyintothem.Youshouldcomeawayfromreadingwithafirmconfidenceinyourunderstanding,notjustofthetheoretical,butthepractical"whatyouneedtoknow"bits.

TheJavaScriptyouknowrightnowisprobablypartshandeddowntoyoubyotherswho'vebeenburnedbyincompleteunderstanding.ThatJavaScriptisbutashadowofthetruelanguage.Youdon'treallyknowJavaScript,yet,butifyoudigintothisseries,youwill.Readon,myfriends.JavaScriptawaitsyou.

JavaScriptisawesome.It'seasytolearnpartially,andmuchhardertolearncompletely(orevensufficiently).Whendevelopersencounterconfusion,theyusuallyblamethelanguageinsteadoftheirlackofunderstanding.Thesebooksaimtofixthat,inspiringastrongappreciationforthelanguageyoucannow,andshould,deeplyknow.

Note:Manyoftheexamplesinthisbookassumemodern(andfuture-reaching)JavaScriptengineenvironments,suchasES6.Somecodemaynotworkasdescribedifruninolder(pre-ES6)engines.

Summary

Overtheyears,myemployerhastrustedmeenoughtoconductinterviews.Ifwe'relookingforsomeonewithskillsinJavaScript,myfirstlineofquestioning…actuallythat'snottrue,Ifirstcheckifthecandidateneedsthebathroomand/oradrink,becausecomfortisimportant,butonceI'mpastthebitaboutthecandidate'sfluidin/out-take,IsetaboutdeterminingifthecandidateknowsJavaScript,orjustjQuery.

Notthatthere'sanythingwrongwithjQuery.ItletsyoudoalotwithoutreallyknowingJavaScript,andthat'safeaturenotabug.ButifthejobcallsforadvancedskillsinJavaScriptperformanceandmaintainability,youneedsomeonewhoknowshowlibrariessuchasjQueryareputtogether.YouneedtobeabletoharnessthecoreofJavaScriptthesamewaytheydo.

IfIwanttogetapictureofsomeone'scoreJavaScriptskill,I'mmostinterestedinwhattheymakeofclosures(you'vereadthatbookofthisseriesalready,right?)andhowtogetthemostoutofasynchronicity,whichbringsustothisbook.

Forstarters,you'llbetakenthroughcallbacks,thebreadandbutterofasynchronousprogramming.Ofcourse,breadandbutterdoesnotmakeforaparticularlysatisfyingmeal,butthenextcourseisfulloftastytastypromises!

Ifyoudon'tknowpromises,nowisthetimetolearn.PromisesarenowtheofficialwaytoprovideasyncreturnvaluesinbothJavaScriptandtheDOM.AllfutureasyncDOMAPIswillusethem,manyalreadydo,sobeprepared!Atthetimeofwriting,Promiseshaveshippedinmostmajorbrowsers,withIEshippingsoon.Onceyou'vefinishedthat,Ihopeyouleftroomforthenextcourse,Generators.

GeneratorssnucktheirwayintostableversionsofChromeandFirefoxwithouttoomuchpompandceremony,because,frankly,they'remorecomplicatedthantheyareinteresting.Or,that'swhatIthoughtuntilIsawthemcombinedwithpromises.There,theybecomeanimportanttoolinreadabilityandmaintenance.

Fordessert,well,Iwon'tspoilthesurprise,butpreparetogazeintothefutureofJavaScript!Featuresthatgiveyoumoreandmorecontroloverconcurrencyandasynchronicity.

Well,Iwon'tblockyourenjoymentofthebookanylonger,onwiththeshow!Ifyou'vealreadyreadpartofthebookbeforereadingthisForeword,giveyourself10asynchronouspoints!Youdeservethem!

JakeArchibaldjakearchibald.com,@jaffathecakeDeveloperAdvocateatGoogleChrome

http://jakearchibald.com

http://twitter.com/jaffathecake

ForewordPrefaceChapter1:Asynchrony:Now&Later

AProgramInChunksEventLoopParallelThreadingConcurrencyJobsStatementOrdering

Chapter2:CallbacksContinuationsSequentialBrainTrustIssuesTryingToSaveCallbacks

Chapter3:PromisesWhatisaPromise?ThenableDuck-TypingPromiseTrustChainFlowErrorHandlingPromisePatternsPromiseAPIRecapPromiseLimitations

Chapter4:GeneratorsBreakingRun-to-completionGenerator'ingValuesIteratingGeneratorsAsynchronouslyGenerators+PromisesGeneratorDelegationGeneratorConcurrencyThunksPre-ES6Generators

Chapter5:ProgramPerformanceWebWorkersParallelJSSIMDasm.js

Chapter6:Benchmarking&TuningBenchmarkingContextIsKingjsPerf.comWritingGoodTestsMicroperformanceTailCallOptimization(TCO)

AppendixA:asynquenceLibraryAppendixB:AdvancedAsyncPatternsAppendixC:Acknowledgments


TableofContents

OneofthemostimportantandyetoftenmisunderstoodpartsofprogramminginalanguagelikeJavaScriptishowtoexpressandmanipulateprogrambehaviorspreadoutoveraperiodoftime.

Thisisnotjustaboutwhathappensfromthebeginningofaforlooptotheendofaforloop,whichofcoursetakessometime(microsecondstomilliseconds)tocomplete.It'saboutwhathappenswhenpartofyourprogramrunsnow,andanotherpartofyourprogramrunslater--there'sagapbetweennowandlaterwhereyourprogramisn'tactivelyexecuting.

Practicallyallnontrivialprogramseverwritten(especiallyinJS)haveinsomewayoranotherhadtomanagethisgap,whetherthatbeinwaitingforuserinput,requestingdatafromadatabaseorfilesystem,sendingdataacrossthenetworkandwaitingforaresponse,orperformingarepeatedtaskatafixedintervaloftime(likeanimation).Inallthesevariousways,yourprogramhastomanagestateacrossthegapintime.AstheyfamouslysayinLondon(ofthechasmbetweenthesubwaydoorandtheplatform):"mindthegap."

Infact,therelationshipbetweenthenowandlaterpartsofyourprogramisattheheartofasynchronousprogramming.

AsynchronousprogramminghasbeenaroundsincethebeginningofJS,forsure.ButmostJSdevelopershaveneverreallycarefullyconsideredexactlyhowandwhyitcropsupintheirprograms,orexploredvariousotherwaystohandleit.Thegoodenoughapproachhasalwaysbeenthehumblecallbackfunction.Manytothisdaywillinsistthatcallbacksaremorethansufficient.

ButasJScontinuestogrowinbothscopeandcomplexity,tomeettheever-wideningdemandsofafirst-classprogramminglanguagethatrunsinbrowsersandserversandeveryconceivabledeviceinbetween,thepainsbywhichwemanageasynchronyarebecomingincreasinglycrippling,andtheycryoutforapproachesthatarebothmorecapableandmorereason-able.

Whilethisallmayseemratherabstractrightnow,Iassureyouwe'lltackleitmorecompletelyandconcretelyaswegoonthroughthisbook.We'llexploreavarietyofemergingtechniquesforasyncJavaScriptprogrammingoverthenextseveralchapters.

Butbeforewecangetthere,we'regoingtohavetounderstandmuchmoredeeplywhatasynchronyisandhowitoperatesinJS.

YoumaywriteyourJSprograminone.jsfile,butyourprogramisalmostcertainlycomprisedofseveralchunks,onlyoneofwhichisgoingtoexecutenow,andtherestofwhichwillexecutelater.Themostcommonunitofchunkisthefunction.

TheproblemmostdevelopersnewtoJSseemtohaveisthatlaterdoesn'thappenstrictlyandimmediatelyafternow.Inotherwords,tasksthatcannotcompletenoware,bydefinition,goingtocompleteasynchronously,andthuswewillnothaveblockingbehaviorasyoumightintuitivelyexpectorwant.

Consider:

//ajax(..)issomearbitraryAjaxfunctiongivenbyalibrary

vardata=ajax("http://some.url.1");

console.log(data);

//Oops!`data`generallywon'thavetheAjaxresults

You'reprobablyawarethatstandardAjaxrequestsdon'tcompletesynchronously,whichmeanstheajax(..)functiondoes


Chapter1:Asynchrony:Now&Later

APrograminChunks

notyethaveanyvaluetoreturnbacktobeassignedtodatavariable.Ifajax(..)couldblockuntiltheresponsecameback,thenthedata=..assignmentwouldworkfine.

Butthat'snothowwedoAjax.WemakeanasynchronousAjaxrequestnow,andwewon'tgettheresultsbackuntillater.

Thesimplest(butdefinitelynotonly,ornecessarilyevenbest!)wayof"waiting"fromnowuntillateristouseafunction,commonlycalledacallbackfunction:


ajax("http://some.url.1",functionmyCallbackFunction(data){

console.log(data);//Yay,Igotsmesome`data`!

});

Warning:Youmayhaveheardthatit'spossibletomakesynchronousAjaxrequests.Whilethat'stechnicallytrue,youshouldnever,everdoit,underanycircumstances,becauseitlocksthebrowserUI(buttons,menus,scrolling,etc.)andpreventsanyuserinteractionwhatsoever.Thisisaterribleidea,andshouldalwaysbeavoided.

Beforeyouprotestindisagreement,no,yourdesiretoavoidthemessofcallbacksisnotjustificationforblocking,synchronousAjax.

Forexample,considerthiscode:

functionnow(){

return21;

}

functionlater(){

answer=answer*2;

console.log("Meaningoflife:",answer);

}

varanswer=now();

setTimeout(later,1000);//Meaningoflife:42

Therearetwochunkstothisprogram:thestuffthatwillrunnow,andthestuffthatwillrunlater.Itshouldbefairlyobviouswhatthosetwochunksare,butlet'sbesuperexplicit:

Now:

functionnow(){

return21;

}

functionlater(){..}

varanswer=now();

setTimeout(later,1000);

Later:

answer=answer*2;


Thenowchunkrunsrightaway,assoonasyouexecuteyourprogram.ButsetTimeout(..)alsosetsupanevent(atimeout)tohappenlater,sothecontentsofthelater()functionwillbeexecutedatalatertime(1,000millisecondsfromnow).

Anytimeyouwrapaportionofcodeintoafunctionandspecifythatitshouldbeexecutedinresponsetosomeevent(timer,mouseclick,Ajaxresponse,etc.),youarecreatingalaterchunkofyourcode,andthusintroducingasynchronytoyourprogram.

Thereisnospecificationorsetofrequirementsaroundhowtheconsole.*methodswork--theyarenotofficiallypartofJavaScript,butareinsteadaddedtoJSbythehostingenvironment(seetheTypes&Grammartitleofthisbookseries).

So,differentbrowsersandJSenvironmentsdoastheyplease,whichcansometimesleadtoconfusingbehavior.

Inparticular,therearesomebrowsersandsomeconditionsthatconsole.log(..)doesnotactuallyimmediatelyoutputwhatit'sgiven.ThemainreasonthismayhappenisbecauseI/Oisaveryslowandblockingpartofmanyprograms(notjustJS).So,itmayperformbetter(fromthepage/UIperspective)forabrowsertohandleconsoleI/Oasynchronouslyinthebackground,withoutyouperhapsevenknowingthatoccurred.

Anotterriblycommon,butpossible,scenariowherethiscouldbeobservable(notfromcodeitselfbutfromtheoutside):

vara={

index:1

};

//later

console.log(a);//??

//evenlater

a.index++;

We'dnormallyexpecttoseetheaobjectbesnapshottedattheexactmomentoftheconsole.log(..)statement,printingsomethinglike{index:1},suchthatinthenextstatmentwhena.index++happens,it'smodifyingsomethingdifferentthan,orjuststrictlyafter,theoutputofa.

Mostofthetime,theprecedingcodewillprobablyproduceanobjectrepresentationinyourdevelopertools'consolethat'swhatyou'dexpect.Butit'spossiblethissamecodecouldruninasituationwherethebrowserfeltitneededtodefertheconsoleI/Otothebackground,inwhichcaseit'spossiblethatbythetimetheobjectisrepresentedinthebrowserconsole,thea.index++hasalreadyhappened,anditshows{index:2}.

It'samovingtargetunderwhatconditionsexactlyconsoleI/Owillbedeferred,orevenwhetheritwillbeobservable.JustbeawareofthispossibleasynchronicityinI/Oincaseyoueverrunintoissuesindebuggingwhereobjectshavebeenmodifiedafteraconsole.log(..)statementandyetyouseetheunexpectedmodificationsshowup.

Note:Ifyourunintothisrarescenario,thebestoptionistousebreakpointsinyourJSdebuggerinsteadofrelyingonconsoleoutput.Thenextbestoptionwouldbetoforcea"snapshot"oftheobjectinquestionbyserializingittoastring,likewithJSON.stringify(..).

Let'smakea(perhapsshocking)claim:despiteyourclearlybeingabletowriteasynchronousJScode(likethetimeoutwejustlookedat),upuntilrecently(ES6),JavaScriptitselfhasactuallyneverhadanydirectnotionofasynchronybuiltintoit.

What!?Thatseemslikeacrazyclaim,right?Infact,it'squitetrue.TheJSengineitselfhasneverdoneanythingmorethanexecuteasinglechunkofyourprogramatanygivenmoment,whenaskedto.

"Askedto."Bywhom?That'stheimportantpart!

TheJSenginedoesn'truninisolation.Itrunsinsideahostingenvironment,whichisformostdevelopersthetypicalwebbrowser.Overthelastseveralyears(butbynomeansexlusively),JShasexpandedbeyondthebrowserintootherenvironments,suchasservers,viathingslikeNode.js.Infact,JavaScriptgetsembeddedintoallkindsofdevicesthese

AsyncConsole

EventLoop

days,fromrobotstolightbulbs.

Buttheonecommon"thread"(that'sanot-so-subtleasynchronousjoke,forwhatit'sworth)ofalltheseenvironmentsisthattheyhaveamechanisminthemthathandlesexecutingmultiplechunksofyourprogramovertime,ateachmomentinvokingtheJSengine,calledthe"eventloop."

Inotherwords,theJSenginehashadnoinnatesenseoftime,buthasinsteadbeenanon-demandexecutionenvironmentforanyarbitrarysnippetofJS.It'sthesurroundingenvironmentthathasalwaysscheduled"events"(JScodeexecutions).

So,forexample,whenyourJSprogrammakesanAjaxrequesttofetchsomedatafromaserver,yousetupthe"response"codeinafunction(commonlycalleda"callback"),andtheJSenginetellsthehostingenvironment,"Hey,I'mgoingtosuspendexecutionfornow,butwheneveryoufinishwiththatnetworkrequest,andyouhavesomedata,pleasecallthisfunctionback."

Thebrowseristhensetuptolistenfortheresponsefromthenetwork,andwhenithassomethingtogiveyou,itschedulesthecallbackfunctiontobeexecutedbyinsertingitintotheeventloop.

Sowhatistheeventloop?

Let'sconceptualizeitfirstthroughsomefake-ishcode:

//èventLoopìsanarraythatactsasaqueue(first-in,first-out)

vareventLoop=[];

varevent;

//keepgoing"forever"

while(true){

//performa"tick"

if(eventLoop.length>0){

//getthenexteventinthequeue

event=eventLoop.shift();

//now,executethenextevent

try{

event();

}

catch(err){

reportError(err);

}

}

}

Thisis,ofcourse,vastlysimplifiedpseudocodetoillustratetheconcepts.Butitshouldbeenoughtohelpgetabetterunderstanding.

Asyoucansee,there'sacontinuouslyrunninglooprepresentedbythewhileloop,andeachiterationofthisloopiscalleda"tick."Foreachtick,ifaneventiswaitingonthequeue,it'stakenoffandexecuted.Theseeventsareyourfunctioncallbacks.

It'simportanttonotethatsetTimeout(..)doesn'tputyourcallbackontheeventloopqueue.Whatitdoesissetupatimer;whenthetimerexpires,theenvironmentplacesyourcallbackintotheeventloop,suchthatsomefuturetickwillpickitupandexecuteit.

Whatiftherearealready20itemsintheeventloopatthatmoment?Yourcallbackwaits.Itgetsinlinebehindtheothers--there'snotnormallyapathforpreemptingthequeueandskippingaheadinline.ThisexplainswhysetTimeout(..)timersmaynotfirewithperfecttemporalaccuracy.You'reguaranteed(roughlyspeaking)thatyourcallbackwon'tfirebeforethetimeintervalyouspecify,butitcanhappenatorafterthattime,dependingonthestateoftheeventqueue.

So,inotherwords,yourprogramisgenerallybrokenupintolotsofsmallchunks,whichhappenoneaftertheotherintheeventloopqueue.Andtechnically,othereventsnotrelateddirectlytoyourprogramcanbeinterleavedwithinthequeueaswell.

Note:Wementioned"upuntilrecently"inrelationtoES6changingthenatureofwheretheeventloopqueueismanaged.

It'smostlyaformaltechnicality,butES6nowspecifieshowtheeventloopworks,whichmeanstechnicallyit'swithinthepurviewoftheJSengine,ratherthanjustthehostingenvironment.OnemainreasonforthischangeistheintroductionofES6Promises,whichwe'lldiscussinChapter3,becausetheyrequiretheabilitytohavedirect,fine-grainedcontroloverschedulingoperationsontheeventloopqueue(seethediscussionofsetTimeout(..0)inthe"Cooperation"section).

It'sverycommontoconflatetheterms"async"and"parallel,"buttheyareactuallyquitedifferent.Remember,asyncisaboutthegapbetweennowandlater.Butparallelisaboutthingsbeingabletooccursimultaneously.

Themostcommontoolsforparallelcomputingareprocessesandthreads.Processesandthreadsexecuteindependentlyandmayexecutesimultaneously:onseparateprocessors,orevenseparatecomputers,butmultiplethreadscansharethememoryofasingleprocess.

Aneventloop,bycontrast,breaksitsworkintotasksandexecutestheminserial,disallowingparallelaccessandchangestosharedmemory.Parallelismand"serialism"cancoexistintheformofcooperatingeventloopsinseparatethreads.

Theinterleavingofparallelthreadsofexecutionandtheinterleavingofasynchronouseventsoccuratverydifferentlevelsofgranularity.

Forexample:

functionlater(){

answer=answer*2;


}

Whiletheentirecontentsoflater()wouldberegardedasasingleeventloopqueueentry,whenthinkingaboutathreadthiscodewouldrunon,there'sactuallyperhapsadozendifferentlow-leveloperations.Forexample,answer=answer*2requiresfirstloadingthecurrentvalueofanswer,thenputting2somewhere,thenperformingthemultiplication,thentakingtheresultandstoringitbackintoanswer.

Inasingle-threadedenvironment,itreallydoesn'tmatterthattheitemsinthethreadqueuearelow-leveloperations,becausenothingcaninterruptthethread.Butifyouhaveaparallelsystem,wheretwodifferentthreadsareoperatinginthesameprogram,youcouldverylikelyhaveunpredictablebehavior.

Consider:

vara=20;

functionfoo(){

a=a+1;

}

functionbar(){

a=a*2;

}


ajax("http://some.url.1",foo);

ajax("http://some.url.2",bar);

InJavaScript'ssingle-threadedbehavior,iffoo()runsbeforebar(),theresultisthatahas42,butifbar()runsbeforefoo()theresultinawillbe41.

IfJSeventssharingthesamedataexecutedinparallel,though,theproblemswouldbemuchmoresubtle.Considerthesetwolistsofpseudocodetasksasthethreadsthatcouldrespectivelyrunthecodeinfoo()andbar(),andconsiderwhathappensiftheyarerunningatexactlythesametime:

ParallelThreading

Thread1(XandYaretemporarymemorylocations):

foo():

a.loadvalueofàìn`X`

b.store`1ìn`Y`

c.add`Xànd`Y`,storeresultin`X`

d.storevalueof`Xìnà`

Thread2(XandYaretemporarymemorylocations):

bar():

a.loadvalueofàìn`X`

b.store`2ìn`Y`

c.multiply`Xànd`Y`,storeresultin`X`

d.storevalueof`Xìnà`

Now,let'ssaythatthetwothreadsarerunningtrulyinparallel.Youcanprobablyspottheproblem,right?TheyusesharedmemorylocationsXandYfortheirtemporarysteps.

What'stheendresultinaifthestepshappenlikethis?

1a(loadvalueofàìn`X`==>`20`)


1b(store`1ìn`Y`==>`1`)

2b(store`2ìn`Y`==>`2`)

1c(add`Xànd`Y`,storeresultin`X`==>`22`)

1d(storevalueof`Xìnà`==>`22`)

2c(multiply`Xànd`Y`,storeresultin`X`==>`44`)


Theresultinawillbe44.Butwhataboutthisordering?



2b(store`2ìn`Y`==>`2`)

1b(store`1ìn`Y`==>`1`)

2c(multiply`Xànd`Y`,storeresultin`X`==>`20`)

1c(add`Xànd`Y`,storeresultin`X`==>`21`)



Theresultinawillbe21.

So,threadedprogrammingisverytricky,becauseifyoudon'ttakespecialstepstopreventthiskindofinterruption/interleavingfromhappening,youcangetverysurprising,nondeterministicbehaviorthatfrequentlyleadstoheadaches.

JavaScriptneversharesdataaccrossthreads,whichmeansthatlevelofnondeterminismisn'taconcern.Butthatdoesn'tmeanJSisalwaysdeterministic.Rememberearlier,wheretherelativeorderingoffoo()andbar()producestwodifferentresults(41or42)?

Note:Itmaynotbeobviousyet,butnotallnondeterminismisbad.Sometimesit'sirrelevant,andsometimesit'sintentional.We'llseemoreexamplesofthatthroughoutthisandthenextfewchapters.

BecauseofJavaScript'ssingle-threading,thecodeinsideoffoo()(andbar())isatomic,whichmeansthatoncefoo()startsrunning,theentiretyofitscodewillfinishbeforeanyofthecodeinbar()canrun,orviceversa.Thisiscalled"run-to-completion"behavior.

Run-to-Completion

Infact,therun-to-completionsemanticsaremoreobviouswhenfoo()andbar()havemorecodeinthem,suchas:

vara=1;

varb=2;

functionfoo(){

a++;

b=b*a;

a=b+3;

}

functionbar(){

b--;

a=8+b;

b=a*2;

}




Becausefoo()can'tbeinterruptedbybar(),andbar()can'tbeinterruptedbyfoo(),thisprogramonlyhastwopossibleoutcomesdependingonwhichstartsrunningfirst--ifthreadingwerepresent,andtheindividualstatementsinfoo()andbar()couldbeinterleaved,thenumberofpossibleoutcomeswouldbegreatlyincreased!

Chunk1issynchronous(happensnow),butchunks2and3areasynchronous(happenlater),whichmeanstheirexecutionwillbeseparatedbyagapoftime.

Chunk1:

vara=1;

varb=2;

Chunk2(foo()):

a++;

b=b*a;

a=b+3;

Chunk3(bar()):

b--;

a=8+b;

b=a*2;

Chunks2and3mayhappenineither-firstorder,sotherearetwopossibleoutcomesforthisprogram,asillustratedhere:

Outcome1:

vara=1;

varb=2;

//foo()

a++;

b=b*a;

a=b+3;

//bar()

b--;

a=8+b;

b=a*2;

a;//11

b;//22

Outcome2:

vara=1;

varb=2;

//bar()

b--;

a=8+b;

b=a*2;

//foo()

a++;

b=b*a;

a=b+3;

a;//183

b;//180

Twooutcomesfromthesamecodemeanswestillhavenondeterminism!Butit'satthefunction(event)orderinglevel,ratherthanatthestatementorderinglevel(or,infact,theexpressionoperationorderinglevel)asitiswiththreads.Inotherwords,it'smoredeterministicthanthreadswouldhavebeen.

AsappliedtoJavaScript'sbehavior,thisfunction-orderingnondeterminismisthecommonterm"racecondition,"asfoo()andbar()areracingagainsteachothertoseewhichrunsfirst.Specifically,it'sa"racecondition"becauseyoucannotpredictreliablyhowaandbwillturnout.

Note:IftherewasafunctioninJSthatsomehowdidnothaverun-to-completionbehavior,wecouldhavemanymorepossibleoutcomes,right?ItturnsoutES6introducesjustsuchathing(seeChapter4"Generators"),butdon'tworryrightnow,we'llcomebacktothat!

Let'simagineasitethatdisplaysalistofstatusupdates(likeasocialnetworknewsfeed)thatprogressivelyloadsastheuserscrollsdownthelist.Tomakesuchafeatureworkcorrectly,(atleast)twoseparate"processes"willneedtobeexecutingsimultaneously(i.e.,duringthesamewindowoftime,butnotnecessarilyatthesameinstant).

Note:We'reusing"process"inquotesherebecausetheyaren'ttrueoperatingsystem–levelprocessesinthecomputersciencesense.They'revirtualprocesses,ortasks,thatrepresentalogicallyconnected,sequentialseriesofoperations.We'llsimplyprefer"process"over"task"becauseterminology-wise,itwillmatchthedefinitionsoftheconceptswe'reexploring.

Thefirst"process"willrespondtoonscrollevents(makingAjaxrequestsfornewcontent)astheyfirewhentheuserhasscrolledthepagefurtherdown.Thesecond"process"willreceiveAjaxresponsesback(torendercontentontothepage).

Obviously,ifauserscrollsfastenough,youmayseetwoormoreonscrolleventsfiredduringthetimeittakestogetthefirstresponsebackandprocess,andthusyou'regoingtohaveonscrolleventsandAjaxresponseeventsfiringrapidly,interleavedwitheachother.

Concurrencyiswhentwoormore"processes"areexecutingsimultaneouslyoverthesameperiod,regardlessofwhethertheirindividualconstituentoperationshappeninparallel(atthesameinstantonseparateprocessorsorcores)ornot.Youcanthinkofconcurrencythenas"process"-level(ortask-level)parallelism,asopposedtooperation-levelparallelism(separate-processorthreads).

Note:Concurrencyalsointroducesanoptionalnotionofthese"processes"interactingwitheachother.We'llcomebacktothatlater.

Foragivenwindowoftime(afewsecondsworthofauserscrolling),let'svisualizeeachindependent"process"asaseries

Concurrency

ofevents/operations:

"Process"1(onscrollevents):

onscroll,request1

onscroll,request2

onscroll,request3

onscroll,request4

onscroll,request5

onscroll,request6

onscroll,request7

"Process"2(Ajaxresponseevents):

response1

response2

response3

response4

response5

response6

response7

It'squitepossiblethatanonscrolleventandanAjaxresponseeventcouldbereadytobeprocessedatexactlythesamemoment.Forexamplelet'svisualizetheseeventsinatimeline:

onscroll,request1

onscroll,request2response1


response3

onscroll,request4

onscroll,request5


onscroll,request7

response6

response5

response7

But,goingbacktoournotionoftheeventloopfromearilerinthechapter,JSisonlygoingtobeabletohandleoneeventatatime,soeitheronscroll,request2isgoingtohappenfirstorresponse1isgoingtohappenfirst,buttheycannothappenatliterallythesamemoment.Justlikekidsataschoolcafeteria,nomatterwhatcrowdtheyformoutsidethedoors,they'llhavetomergeintoasinglelinetogettheirlunch!

Let'svisualizetheinterleavingofalltheseeventsontotheeventloopqueue.

EventLoopQueue:

onscroll,request1<---Process1starts

onscroll,request2

response1<---Process2starts

onscroll,request3

response2

response3

onscroll,request4

onscroll,request5

onscroll,request6

response4

onscroll,request7<---Process1finishes

response6

response5

response7<---Process2finishes

"Process1"and"Process2"runconcurrently(task-levelparallel),buttheirindividualeventsrunsequentiallyontheeventloopqueue.

Bytheway,noticehowresponse6andresponse5camebackoutofexpectedorder?

Thesingle-threadedeventloopisoneexpressionofconcurrency(therearecertainlyothers,whichwe'llcomebacktolater).

Astwoormore"processes"areinterleavingtheirsteps/eventsconcurrentlywithinthesameprogram,theydon'tnecessarilyneedtointeractwitheachotherifthetasksareunrelated.Iftheydon'tinteract,nondeterminismisperfectlyacceptable.

Forexample:

varres={};

functionfoo(results){

res.foo=results;

}

functionbar(results){

res.bar=results;

}




foo()andbar()aretwoconcurrent"processes,"andit'snondeterminatewhichordertheywillbefiredin.Butwe'veconstructedtheprogramsoitdoesn'tmatterwhatordertheyfirein,becausetheyactindependentlyandassuchdon'tneedtointeract.

Thisisnota"racecondition"bug,asthecodewillalwaysworkcorrectly,regardlessoftheordering.

Morecommonly,concurrent"processes"willbynecessityinteract,indirectlythroughscopeand/ortheDOM.Whensuchinteractionwilloccur,youneedtocoordinatetheseinteractionstoprevent"raceconditions,"asdescribedearlier.

Here'sasimpleexampleoftwoconcurrent"processes"thatinteractbecauseofimpliedordering,whichisonlysometimesbroken:

varres=[];

functionresponse(data){

res.push(data);

}


ajax("http://some.url.1",response);


Theconcurrent"processes"arethetworesponse()callsthatwillbemadetohandletheAjaxresponses.Theycanhappenineither-firstorder.

Let'sassumetheexpectedbehavioristhatres[0]hastheresultsofthe"http://some.url.1"call,andres[1]hastheresultsofthe"http://some.url.2"call.Sometimesthatwillbethecase,butsometimesthey'llbeflipped,dependingonwhichcallfinishesfirst.There'saprettygoodlikelihoodthatthisnondeterminismisa"racecondition"bug.

Note:Beextremelywaryofassumptionsyoumighttendtomakeinthesesituations.Forexample,it'snotuncommonforadevelopertoobservethat"http://some.url.2"is"always"muchslowertorespondthan"http://some.url.1",perhapsbyvirtueofwhattasksthey'redoing(e.g.,oneperformingadatabasetaskandtheotherjustfetchingastaticfile),sothe

Noninteracting

Interaction

observedorderingseemstoalwaysbeasexpected.Evenifbothrequestsgotothesameserver,anditintentionallyrespondsinacertainorder,there'snorealguaranteeofwhatordertheresponseswillarrivebackinthebrowser.

So,toaddresssucharacecondition,youcancoordinateorderinginteraction:

varres=[];


if(data.url=="http://some.url.1"){

res[0]=data;

}

elseif(data.url=="http://some.url.2"){

res[1]=data;

}

}




RegardlessofwhichAjaxresponsecomesbackfirst,weinspectthedata.url(assumingoneisreturnedfromtheserver,ofcourse!)tofigureoutwhichpositiontheresponsedatashouldoccupyintheresarray.res[0]willalwaysholdthe"http://some.url.1"resultsandres[1]willalwaysholdthe"http://some.url.2"results.Throughsimplecoordination,weeliminatedthe"racecondition"nondeterminism.

ThesamereasoningfromthisscenariowouldapplyifmultipleconcurrentfunctioncallswereinteractingwitheachotherthroughthesharedDOM,likeoneupdatingthecontentsofa<div>andtheotherupdatingthestyleorattributesofthe<div>(e.g.,tomaketheDOMelementvisibleonceithascontent).Youprobablywouldn'twanttoshowtheDOMelementbeforeithadcontent,sothecoordinationmustensureproperorderinginteraction.

Someconcurrencyscenariosarealwaysbroken(notjustsometimes)withoutcoordinatedinteraction.Consider:

vara,b;

functionfoo(x){

a=x*2;

baz();

}

functionbar(y){

b=y*2;

baz();

}

functionbaz(){

console.log(a+b);

}




Inthisexample,whetherfoo()orbar()firesfirst,itwillalwayscausebaz()toruntooearly(eitheraorbwillstillbeundefined),butthesecondinvocationofbaz()willwork,asbothaandbwillbeavailable.

Therearedifferentwaystoaddresssuchacondition.Here'sonesimpleway:

vara,b;

functionfoo(x){

a=x*2;

if(a&&b){

baz();

}

}

functionbar(y){

b=y*2;

if(a&&b){

baz();

}

}

functionbaz(){

console.log(a+b);

}




Theif(a&&b)conditionalaroundthebaz()callistraditionallycalleda"gate,"becausewe'renotsurewhatorderaandbwillarrive,butwewaitforbothofthemtogettherebeforeweproceedtoopenthegate(callbaz()).

Anotherconcurrencyinteractionconditionyoumayrunintoissometimescalleda"race,"butmorecorrectlycalleda"latch."It'scharacterizedby"onlythefirstonewins"behavior.Here,nondeterminismisacceptable,inthatyouareexplicitlysayingit'sOKforthe"race"tothefinishlinetohaveonlyonewinner.

Considerthisbrokencode:

vara;

functionfoo(x){

a=x*2;

baz();

}

functionbar(x){

a=x/2;

baz();

}

functionbaz(){

console.log(a);

}




Whicheverone(foo()orbar())fireslastwillnotonlyoverwritetheassignedavaluefromtheother,butitwillalsoduplicatethecalltobaz()(likelyundesired).

So,wecancoordinatetheinteractionwithasimplelatch,toletonlythefirstonethrough:

vara;

functionfoo(x){

if(!a){

a=x*2;

baz();

}

}

functionbar(x){

if(!a){

a=x/2;

baz();

}

}

functionbaz(){

console.log(a);

}




Theif(!a)conditionalallowsonlythefirstoffoo()orbar()through,andthesecond(andindeedanysubsequent)callswouldjustbeignored.There'sjustnovirtueincominginsecondplace!

Note:Inallthesescenarios,we'vebeenusingglobalvariablesforsimplisticillustrationpurposes,butthere'snothingaboutourreasoningherethatrequiresit.Aslongasthefunctionsinquestioncanaccessthevariables(viascope),they'llworkasintended.Relyingonlexicallyscopedvariables(seetheScope&Closurestitleofthisbookseries),andinfactglobalvariablesasintheseexamples,isoneobviousdownsidetotheseformsofconcurrencycoordination.Aswegothroughthenextfewchapters,we'llseeotherwaysofcoordinationthataremuchcleanerinthatrespect.

Anotherexpressionofconcurrencycoordinationiscalled"cooperativeconcurrency."Here,thefocusisn'tsomuchoninteractingviavaluesharinginscopes(thoughthat'sobviouslystillallowed!).Thegoalistotakealong-running"process"andbreakitupintostepsorbatchessothatotherconcurrent"processes"haveachancetointerleavetheiroperationsintotheeventloopqueue.

Forexample,consideranAjaxresponsehandlerthatneedstorunthroughalonglistofresultstotransformthevalues.We'lluseArray#map(..)tokeepthecodeshorter:

varres=[];

//`response(..)`receivesarrayofresultsfromtheAjaxcall


//addontoexisting`resàrray

res=res.concat(

//makeanewtransformedarraywithall`data`valuesdoubled

data.map(function(val){

returnval*2;

})

);

}




If"http://some.url.1"getsitsresultsbackfirst,theentirelistwillbemappedintoresallatonce.Ifit'safewthousandorlessrecords,thisisnotgenerallyabigdeal.Butifit'ssay10millionrecords,thatcantakeawhiletorun(severalsecondsonapowerfullaptop,muchlongeronamobiledevice,etc.).

Whilesucha"process"isrunning,nothingelseinthepagecanhappen,includingnootherresponse(..)calls,noUIupdates,notevenusereventslikescrolling,typing,buttonclicking,andthelike.That'sprettypainful.

So,tomakeamorecooperativelyconcurrentsystem,onethat'sfriendlieranddoesn'thogtheeventloopqueue,youcanprocesstheseresultsinasynchronousbatches,aftereachone"yielding"backtotheeventlooptoletotherwaitingeventshappen.

Here'saverysimpleapproach:

varres=[];

//`response(..)`receivesarrayofresultsfromtheAjaxcall


//let'sjustdo1000atatime

varchunk=data.splice(0,1000);

//addontoexisting`resàrray

res=res.concat(

//makeanewtransformedarraywithall`chunk`valuesdoubled

chunk.map(function(val){

returnval*2;

})

);

Cooperation

//anythinglefttoprocess?

if(data.length>0){

//asyncschedulenextbatch

setTimeout(function(){

response(data);

},0);

}

}




Weprocessthedatasetinmaximum-sizedchunksof1,000items.Bydoingso,weensureashort-running"process,"evenifthatmeansmanymoresubsequent"processes,"astheinterleavingontotheeventloopqueuewillgiveusamuchmoreresponsive(performant)site/app.

Ofcourse,we'renotinteraction-coordinatingtheorderingofanyofthese"processes,"sotheorderofresultsinreswon'tbepredictable.Iforderingwasrequired,you'dneedtouseinteractiontechniqueslikethosewediscussedearlier,oroneswewillcoverinlaterchaptersofthisbook.

WeusethesetTimeout(..0)(hack)forasyncscheduling,whichbasicallyjustmeans"stickthisfunctionattheendofthecurrenteventloopqueue."

Note:setTimeout(..0)isnottechnicallyinsertinganitemdirectlyontotheeventloopqueue.Thetimerwillinserttheeventatitsnextopportunity.Forexample,twosubsequentsetTimeout(..0)callswouldnotbestrictlyguaranteedtobeprocessedincallorder,soitispossibletoseevariousconditionsliketimerdriftwheretheorderingofsucheventsisn'tpredictable.InNode.js,asimilarapproachisprocess.nextTick(..).Despitehowconvenient(andusuallymoreperformant)itwouldbe,there'snotasingledirectway(atleastyet)acrossallenvironmentstoensureasynceventordering.Wecoverthistopicinmoredetailinthenextsection.

AsofES6,there'sanewconceptlayeredontopoftheeventloopqueue,calledthe"Jobqueue."Themostlikelyexposureyou'llhavetoitiswiththeasynchronousbehaviorofPromises(seeChapter3).

Unfortunately,atthemomentit'samechanismwithoutanexposedAPI,andthusdemonstratingitisabitmoreconvoluted.Sowe'regoingtohavetojustdescribeitconceptually,suchthatwhenwediscussasyncbehaviorwithPromisesinChapter3,you'llunderstandhowthoseactionsarebeingscheduledandprocessed.

So,thebestwaytothinkaboutthisthatI'vefoundisthatthe"Jobqueue"isaqueuehangingofftheendofeverytickintheeventloopqueue.Certainasync-impliedactionsthatmayoccurduringatickoftheeventloopwillnotcauseawholeneweventtobeaddedtotheeventloopqueue,butwillinsteadaddanitem(akaJob)totheendofthecurrenttick'sJobqueue.

It'skindalikesaying,"oh,here'sthisotherthingIneedtodolater,butmakesureithappensrightawaybeforeanythingelsecanhappen."

Or,touseametaphor:theeventloopqueueislikeanamusementparkride,whereonceyoufinishtheride,youhavetogotothebackofthelinetorideagain.ButtheJobqueueislikefinishingtheride,butthencuttinginlineandgettingrightbackon.

AJobcanalsocausemoreJobstobeaddedtotheendofthesamequeue.So,it'stheoreticallypossiblethataJob"loop"(aJobthatkeepsaddinganotherJob,etc.)couldspinindefinitely,thusstarvingtheprogramoftheabilitytomoveontothenexteventlooptick.Thiswouldconceptuallybealmostthesameasjustexpressingalong-runningorinfiniteloop(likewhile(true)..)inyourcode.

JobsarekindoflikethespiritofthesetTimeout(..0)hack,butimplementedinsuchawayastohaveamuchmorewell-definedandguaranteedordering:later,butassoonaspossible.

Jobs

Let'simagineanAPIforschedulingJobs(directly,withouthacks),andcallitschedule(..).Consider:

console.log("A");


console.log("B");

},0);

//theoretical"JobAPI"

schedule(function(){

console.log("C");

schedule(function(){

console.log("D");

});

});

YoumightexpectthistoprintoutABCD,butinsteaditwouldprintoutACDB,becausetheJobshappenattheendofthecurrenteventlooptick,andthetimerfirestoscheduleforthenexteventlooptick(ifavailable!).

InChapter3,we'llseethattheasynchronousbehaviorofPromisesisbasedonJobs,soit'simportanttokeepclearhowthatrelatestoeventloopbehavior.

TheorderinwhichweexpressstatementsinourcodeisnotnecessarilythesameorderastheJSenginewillexecutethem.Thatmayseemlikequiteastrangeassertiontomake,sowe'lljustbrieflyexploreit.

Butbeforewedo,weshouldbecrystalclearonsomething:therules/grammarofthelanguage(seetheTypes&Grammartitleofthisbookseries)dictateaverypredictableandreliablebehaviorforstatementorderingfromtheprogrampointofview.Sowhatwe'reabouttodiscussarenotthingsyoushouldeverbeabletoobserveinyourJSprogram.

Warning:Ifyouareeverabletoobservecompilerstatementreorderinglikewe'reabouttoillustrate,that'dbeaclearviolationofthespecification,anditwouldunquestionablybeduetoabugintheJSengineinquestion--onewhichshouldpromptlybereportedandfixed!Butit'svastlymorecommonthatyoususpectsomethingcrazyishappeningintheJSengine,wheninfactit'sjustabug(probablya"racecondition"!)inyourowncode--solooktherefirst,andagainandagain.TheJSdebugger,usingbreakpointsandsteppingthroughcodelinebyline,willbeyourmostpowerfultoolforsniffingoutsuchbugsinyourcode.

Consider:

vara,b;

a=10;

b=30;

a=a+1;

b=b+1;

console.log(a+b);//42

Thiscodehasnoexpressedasynchronytoit(otherthantherareconsoleasyncI/Odiscussedearlier!),sothemostlikelyassumptionisthatitwouldprocesslinebylineintop-downfashion.

Butit'spossiblethattheJSengine,aftercompilingthiscode(yes,JSiscompiled--seetheScope&Closurestitleofthisbookseries!)mightfindopportunitiestorunyourcodefasterbyrearranging(safely)theorderofthesestatements.Essentially,aslongasyoucan'tobservethereordering,anything'sfairgame.

Forexample,theenginemightfindit'sfastertoactuallyexecutethecodelikethis:

StatementOrdering

vara,b;

a=10;

a++;

b=30;

b++;


Orthis:

vara,b;

a=11;

b=31;


Oreven:

//becauseàànd`bàren'tusedanymore,wecan

//inlineanddon'tevenneedthem!

console.log(42);//42

Inallthesecases,theJSengineisperformingsafeoptimizationsduringitscompilation,astheendobservableresultwillbethesame.

Buthere'sascenariowherethesespecificoptimizationswouldbeunsafeandthuscouldn'tbeallowed(ofcourse,nottosaythatit'snotoptimizedatall):

vara,b;

a=10;

b=30;

//weneedàànd`bìntheirpreincrementedstate!

console.log(a*b);//300

a=a+1;

b=b+1;


Otherexampleswherethecompilerreorderingcouldcreateobservablesideeffects(andthusmustbedisallowed)wouldincludethingslikeanyfunctioncallwithsideeffects(evenandespeciallygetterfunctions),orES6Proxyobjects(seetheES6&Beyondtitleofthisbookseries).

Consider:

functionfoo(){

console.log(b);

return1;

}

vara,b,c;

//ES5.1getterliteralsyntax

c={

getbar(){

console.log(a);

return1;

}

};

a=10;

b=30;

a+=foo();//30

b+=c.bar;//11


Ifitweren'tfortheconsole.log(..)statementsinthissnippet(justusedasaconvenientformofobservablesideeffectfortheillustration),theJSenginewouldlikelyhavebeenfree,ifitwantedto(whoknowsifitwould!?),toreorderthecodeto:

//...

a=10+foo();

b=30+c.bar;

//...

WhileJSsemanticsthankfullyprotectusfromtheobservablenightmaresthatcompilerstatementreorderingwouldseemtobeindangerof,it'sstillimportanttounderstandjusthowtenuousalinkthereisbetweenthewaysourcecodeisauthored(intop-downfashion)andthewayitrunsaftercompilation.

Compilerstatementreorderingisalmostamicro-metaphorforconcurrencyandinteraction.Asageneralconcept,suchawarenesscanhelpyouunderstandasyncJScodeflowissuesbetter.

AJavaScriptprogramis(practically)alwaysbrokenupintotwoormorechunks,wherethefirstchunkrunsnowandthenextchunkrunslater,inresponsetoanevent.Eventhoughtheprogramisexecutedchunk-by-chunk,allofthemsharethesameaccesstotheprogramscopeandstate,soeachmodificationtostateismadeontopofthepreviousstate.

Wheneverthereareeventstorun,theeventlooprunsuntilthequeueisempty.Eachiterationoftheeventloopisa"tick."Userinteraction,IO,andtimersenqueueeventsontheeventqueue.

Atanygivenmoment,onlyoneeventcanbeprocessedfromthequeueatatime.Whileaneventisexecuting,itcandirectlyorindirectlycauseoneormoresubsequentevents.

Concurrencyiswhentwoormorechainsofeventsinterleaveovertime,suchthatfromahigh-levelperspective,theyappeartoberunningsimultaneously(eventhoughatanygivenmomentonlyoneeventisbeingprocessed).

It'softennecessarytodosomeformofinteractioncoordinationbetweentheseconcurrent"processes"(asdistinctfromoperatingsystemprocesses),forinstancetoensureorderingortoprevent"raceconditions."These"processes"canalsocooperatebybreakingthemselvesintosmallerchunksandtoallowother"process"interleaving.

Review

InChapter1,weexploredtheterminologyandconceptsaroundasynchronousprogramminginJavaScript.Ourfocusisonunderstandingthesingle-threaded(one-at-a-time)eventloopqueuethatdrivesall"events"(asyncfunctioninvocations).Wealsoexploredvariouswaysthatconcurrencypatternsexplaintherelationships(ifany!)betweensimultaneouslyrunningchainsofevents,or"processes"(tasks,functioncalls,etc.).

AllourexamplesinChapter1usedthefunctionastheindividual,indivisibleunitofoperations,wherebyinsidethefunction,statementsruninpredictableorder(abovethecompilerlevel!),butatthefunction-orderinglevel,events(akaasyncfunctioninvocations)canhappeninavarietyoforders.

Inallthesecases,thefunctionisactingasa"callback,"becauseitservesasthetargetfortheeventloopto"callbackinto"theprogram,wheneverthatiteminthequeueisprocessed.

Asyounodoubthaveobserved,callbacksarebyfarthemostcommonwaythatasynchronyinJSprogramsisexpressedandmanaged.Indeed,thecallbackisthemostfundamentalasyncpatterninthelanguage.

CountlessJSprograms,evenverysophisticatedandcomplexones,havebeenwrittenuponnootherasyncfoundationthanthecallback(withofcoursetheconcurrencyinteractionpatternsweexploredinChapter1).ThecallbackfunctionistheasyncworkhorseforJavaScript,anditdoesitsjobrespectably.

Except...callbacksarenotwithouttheirshortcomings.Manydevelopersareexcitedbythepromise(punintended!)ofbetterasyncpatterns.Butit'simpossibletoeffectivelyuseanyabstractionifyoudon'tunderstandwhatit'sabstracting,andwhy.

Inthischapter,wewillexploreacoupleofthoseindepth,asmotivationforwhymoresophisticatedasyncpatterns(exploredinsubsequentchaptersofthisbook)arenecessaryanddesired.

Let'sgobacktotheasynccallbackexamplewestartedwithinChapter1,butletmeslightlymodifyittoillustrateapoint:

//A

ajax("..",function(..){

//C

});

//B

//Aand//Brepresentthefirsthalfoftheprogram(akathenow),and//Cmarksthesecondhalfoftheprogram(akathelater).Thefirsthalfexecutesrightaway,andthenthere'sa"pause"ofindeterminatelength.Atsomefuturemoment,iftheAjaxcallcompletes,thentheprogramwillpickupwhereitleftoff,andcontinuewiththesecondhalf.

Inotherwords,thecallbackfunctionwrapsorencapsulatesthecontinuationoftheprogram.

Let'smakethecodeevensimpler:

//A


//C

},1000);

//B


Chapter2:Callbacks

Continuations

Stopforamomentandaskyourselfhowyou'ddescribe(tosomeoneelselessinformedabouthowJSworks)thewaythatprogrambehaves.Goahead,tryitoutloud.It'sagoodexercisethatwillhelpmynextpointsmakemoresense.

Mostreadersjustnowprobablythoughtorsaidsomethingtotheeffectof:"DoA,thensetupatimeouttowait1,000milliseconds,thenoncethatfires,doC."Howclosewasyourrendition?

Youmighthavecaughtyourselfandself-editedto:"DoA,setupthetimeoutfor1,000milliseconds,thendoB,thenafterthetimeoutfires,doC."That'smoreaccuratethanthefirstversion.Canyouspotthedifference?

Eventhoughthesecondversionismoreaccurate,bothversionsaredeficientinexplainingthiscodeinawaythatmatchesourbrainstothecode,andthecodetotheJSengine.Thedisconnectisbothsubtleandmonumental,andisattheveryheartofunderstandingtheshortcomingsofcallbacksasasyncexpressionandmanagement.

Assoonasweintroduceasinglecontinuation(orseveraldozenasmanyprogramsdo!)intheformofacallbackfunction,wehaveallowedadivergencetoformbetweenhowourbrainsworkandthewaythecodewilloperate.Anytimethesetwodiverge(andthisisbyfarnottheonlyplacethathappens,asI'msureyouknow!),werunintotheinevitablefactthatourcodebecomeshardertounderstand,reasonabout,debug,andmaintain.

I'mprettysuremostofyoureadershaveheardsomeonesay(evenmadetheclaimyourself),"I'mamultitasker."Theeffectsoftryingtoactasamultitaskerrangefromhumorous(e.g.,thesillypatting-head-rubbing-stomachkids'game)tomundane(chewinggumwhilewalking)todownrightdangerous(textingwhiledriving).

Butarewemultitaskers?Canwereallydotwoconscious,intentionalactionsatonceandthink/reasonaboutbothofthematexactlythesamemoment?Doesourhighestlevelofbrainfunctionalityhaveparallelmultithreadinggoingon?

Theanswermaysurpriseyou:probablynot.

That'sjustnotreallyhowourbrainsappeartobesetup.We'remuchmoresingletaskersthanmanyofus(especiallyA-typepersonalities!)wouldliketoadmit.Wecanreallyonlythinkaboutonethingatanygiveninstant.

I'mnottalkingaboutallourinvoluntary,subconscious,automaticbrainfunctions,suchasheartbeating,breathing,andeyelidblinking.Thoseareallvitaltaskstooursustainedlife,butwedon'tintentionallyallocateanybrainpowertothem.Thankfully,whileweobsessaboutcheckingsocialnetworkfeedsforthe15thtimeinthreeminutes,ourbraincarriesoninthebackground(threads!)withallthoseimportanttasks.

We'reinsteadtalkingaboutwhatevertaskisattheforefrontofourmindsatthemoment.Forme,it'swritingthetextinthisbookrightnow.AmIdoinganyotherhigherlevelbrainfunctionatexactlythissamemoment?Nope,notreally.Igetdistractedquicklyandeasily--afewdozentimesintheselastcoupleofparagraphs!

Whenwefakemultitasking,suchastryingtotypesomethingatthesametimewe'retalkingtoafriendorfamilymemberonthephone,whatwe'reactuallymostlikelydoingisactingasfastcontextswitchers.Inotherwords,weswitchbackandforthbetweentwoormoretasksinrapidsuccession,simultaneouslyprogressingoneachtaskintiny,fastlittlechunks.Wedoitsofastthattotheoutsideworlditappearsasifwe'redoingthesethingsinparallel.

Doesthatsoundsuspiciouslylikeasynceventedconcurrency(likethesortthathappensinJS)toyou?!Ifnot,gobackandreadChapter1again!

Infact,onewayofsimplifying(i.e.,abusing)themassivelycomplexworldofneurologyintosomethingIcanremotelyhopetodiscusshereisthatourbrainsworkkindaliketheeventloopqueue.

Ifyouthinkabouteverysingleletter(orword)Itypeasasingleasyncevent,injustthissentencealonethereareseveraldozenopportunitiesformybraintobeinterruptedbysomeotherevent,suchasfrommysenses,orevenjustmyrandomthoughts.

Idon'tgetinterruptedandpulledtoanother"process"ateveryopportunitythatIcouldbe(thankfully--orthisbookwould

SequentialBrain

neverbewritten!).ButithappensoftenenoughthatIfeelmyownbrainisnearlyconstantlyswitchingtovariousdifferentcontexts(aka"processes").Andthat'sanawfullotlikehowtheJSenginewouldprobablyfeel.

OK,soourbrainscanbethoughtofasoperatinginsingle-threadedeventloopqueuelikeways,ascantheJSengine.Thatsoundslikegoodmatch.

Butweneedtobemorenuancedthanthatinouranalysis.There'sabig,observabledifferencebetweenhowweplanvarioustasks,andhowourbrainsactuallyoperatethosetasks.

Again,backtothewritingofthistextasmymetaphor.Myroughmentaloutlineplanhereistokeepwritingandwriting,goingsequentiallythroughasetofpointsIhaveorderedinmythoughts.Idon'tplantohaveanyinterruptionsornonlinearactivityinthiswriting.Butyet,mybrainisneverthelessswitchingaroundallthetime.

Eventhoughatanoperationallevelourbrainsareasyncevented,weseemtoplanouttasksinasequential,synchronousway."Ineedtogotothestore,thenbuysomemilk,thendropoffmydrycleaning."

You'llnoticethatthishigherlevelthinking(planning)doesn'tseemveryasynceventedinitsformulation.Infact,it'skindofrareforustodeliberatelythinksolelyintermsofevents.Instead,weplanthingsoutcarefully,sequentially(AthenBthenC),andweassumetoanextentasortoftemporalblockingthatforcesBtowaitonA,andCtowaitonB.

Whenadeveloperwritescode,theyareplanningoutasetofactionstooccur.Ifthey'reanygoodatbeingadeveloper,they'recarefullyplanningitout."Ineedtosetztothevalueofx,andthenxtothevalueofy,"andsoforth.

Whenwewriteoutsynchronouscode,statementbystatement,itworksalotlikeourerrandsto-dolist:

//swap`xànd`y`(viatempvariable`z`)

z=x;

x=y;

y=z;

Thesethreeassignmentstatementsaresynchronous,sox=ywaitsforz=xtofinish,andy=zinturnwaitsforx=ytofinish.Anotherwayofsayingitisthatthesethreestatementsaretemporallyboundtoexecuteinacertainorder,onerightaftertheother.Thankfully,wedon'tneedtobebotheredwithanyasynceventeddetailshere.Ifwedid,thecodegetsalotmorecomplex,quickly!

Soifsynchronousbrainplanningmapswelltosynchronouscodestatements,howwelldoourbrainsdoatplanningoutasynchronouscode?

Itturnsoutthathowweexpressasynchrony(withcallbacks)inourcodedoesn'tmapverywellatalltothatsynchronousbrainplanningbehavior.

Canyouactuallyimaginehavingalineofthinkingthatplansoutyourto-doerrandslikethis?

"Ineedtogotothestore,butonthewayI'msureI'llgetaphonecall,so'Hi,Mom',andwhileshestartstalking,I'llbelookingupthestoreaddressonGPS,butthat'lltakeasecondtoload,soI'llturndowntheradiosoIcanhearMombetter,thenI'llrealizeIforgottoputonajacketandit'scoldoutside,butnomatter,keepdrivingandtalkingtoMom,andthentheseatbeltdingremindsmetobuckleup,so'Yes,Mom,Iamwearingmyseatbelt,Ialwaysdo!'.Ah,finallytheGPSgotthedirections,now..."

Asridiculousasthatsoundsasaformulationforhowweplanourdayoutandthinkaboutwhattodoandinwhatorder,nonethelessit'sexactlyhowourbrainsoperateatafunctionallevel.Remember,that'snotmultitasking,it'sjustfastcontextswitching.

Thereasonit'sdifficultforusasdeveloperstowriteasynceventedcode,especiallywhenallwehaveisthecallbacktodoit,isthatstreamofconsciousnessthinking/planningisunnaturalformostofus.

DoingVersusPlanning

Wethinkinstep-by-stepterms,butthetools(callbacks)availabletousincodearenotexpressedinastep-by-stepfashiononcewemovefromsynchronoustoasynchronous.

Andthatiswhyit'ssohardtoaccuratelyauthorandreasonaboutasyncJScodewithcallbacks:becauseit'snothowourbrainplanningworks.

Note:Theonlythingworsethannotknowingwhysomecodebreaksisnotknowingwhyitworkedinthefirstplace!It'stheclassic"houseofcards"mentality:"itworks,butnotsurewhy,sonobodytouchit!"Youmayhaveheard,"Hellisotherpeople"(Sartre),andtheprogrammermemetwist,"Hellisotherpeople'scode."Ibelievetruly:"Hellisnotunderstandingmyowncode."Andcallbacksareonemainculprit.

Consider:

listen("click",functionhandler(evt){

setTimeout(functionrequest(){

ajax("http://some.url.1",functionresponse(text){

if(text=="hello"){

handler();

}

elseif(text=="world"){

request();

}

});

},500);

});

There'sagoodchancecodelikethatisrecognizabletoyou.We'vegotachainofthreefunctionsnestedtogether,eachonerepresentingastepinanasynchronousseries(task,"process").

Thiskindofcodeisoftencalled"callbackhell,"andsometimesalsoreferredtoasthe"pyramidofdoom"(foritssideways-facingtriangularshapeduetothenestedindentation).

But"callbackhell"actuallyhasalmostnothingtodowiththenesting/indentation.It'safardeeperproblemthanthat.We'llseehowandwhyaswecontinuethroughtherestofthischapter.

First,we'rewaitingforthe"click"event,thenwe'rewaitingforthetimertofire,thenwe'rewaitingfortheAjaxresponsetocomeback,atwhichpointitmightdoitallagain.

Atfirstglance,thiscodemayseemtomapitsasynchronynaturallytosequentialbrainplanning.

First(now),we:

listen("..",functionhandler(..){

//..

});

Thenlater,we:

setTimeout(functionrequest(..){

//..

},500);

Thenstilllater,we:

ajax("..",functionresponse(..){

//..

});

Nested/ChainedCallbacks

Andfinally(mostlater),we:

if(..){

//..

}

else..

Butthere'sseveralproblemswithreasoningaboutthiscodelinearlyinsuchafashion.

First,it'sanaccidentoftheexamplethatourstepsareonsubsequentlines(1,2,3,and4...).InrealasyncJSprograms,there'softenalotmorenoiseclutteringthingsup,noisethatwehavetodeftlymaneuverpastinourbrainsaswejumpfromonefunctiontothenext.Understandingtheasyncflowinsuchcallback-ladencodeisnotimpossible,butit'scertainlynotnaturaloreasy,evenwithlotsofpractice.

Butalso,there'ssomethingdeeperwrong,whichisn'tevidentjustinthatcodeexample.Letmemakeupanotherscenario(pseudocode-ish)toillustrateit:

doA(function(){

doB();

doC(function(){

doD();

})

doE();

});

doF();

Whiletheexperiencedamongyouwillcorrectlyidentifythetrueorderofoperationshere,I'mbettingitismorethanalittleconfusingatfirstglance,andtakessomeconcertedmentalcyclestoarriveat.Theoperationswillhappeninthisorder:

doA()

doF()

doB()

doC()

doE()

doD()

Didyougetthatrighttheveryfirsttimeyouglancedatthecode?

OK,someofyouarethinkingIwasunfairinmyfunctionnaming,tointentionallyleadyouastray.IswearIwasjustnamingintop-downappearanceorder.Butletmetryagain:

doA(function(){

doC();

doD(function(){

doF();

})

doE();

});

doB();

Now,I'venamedthemalphabeticallyinorderofactualexecution.ButIstillbet,evenwithexperiencenowinthisscenario,tracingthroughtheA->B->C->D->E->Forderdoesn'tcomenaturaltomanyifanyofyoureaders.Certainlyyoureyesdoanawfullotofjumpingupanddownthecodesnippet,right?

Butevenifthatallcomesnaturaltoyou,there'sstillonemorehazardthatcouldwreakhavoc.Canyouspotwhatitis?

WhatifdoA(..)ordoD(..)aren'tactuallyasync,thewayweobviouslyassumedthemtobe?Uhoh,nowtheorderisdifferent.Ifthey'rebothsync(andmaybeonlysometimes,dependingontheconditionsoftheprogramatthetime),theorderisnowA->C->D->F->E->B.

ThatsoundyoujustheardfaintlyinthebackgroundisthesighsofthousandsofJSdeveloperswhojusthadaface-in-handsmoment.

Isnestingtheproblem?Isthatwhatmakesitsohardtotracetheasyncflow?That'spartofit,certainly.

Butletmerewritethepreviousnestedevent/timeout/Ajaxexamplewithoutusingnesting:

listen("click",handler);

functionhandler(){

setTimeout(request,500);

}

functionrequest(){


}

functionresponse(text){

if(text=="hello"){

handler();

}

elseif(text=="world"){

request();

}

}

Thisformulationofthecodeisnothardlyasrecognizableashavingthenesting/indentationwoesofitspreviousform,andyetit'severybitassusceptibleto"callbackhell."Why?

Aswegotolinearly(sequentially)reasonaboutthiscode,wehavetoskipfromonefunction,tothenext,tothenext,andbounceallaroundthecodebaseto"see"thesequenceflow.Andremember,thisissimplifiedcodeinsortofbest-casefashion.WeallknowthatrealasyncJSprogramcodebasesareoftenfantasticallymorejumbled,whichmakessuchreasoningordersofmagnitudemoredifficult.

Anotherthingtonotice:togetsteps2,3,and4linkedtogethersotheyhappeninsuccession,theonlyaffordancecallbacksalonegivesusistohardcodestep2intostep1,step3intostep2,step4intostep3,andsoon.Thehardcodingisn'tnecessarilyabadthing,ifitreallyisafixedconditionthatstep2shouldalwaysleadtostep3.

Butthehardcodingdefinitelymakesthecodeabitmorebrittle,asitdoesn'taccountforanythinggoingwrongthatmightcauseadeviationintheprogressionofsteps.Forexample,ifstep2fails,step3nevergetsreached,nordoesstep2retry,ormovetoanalternateerrorhandlingflow,andsoon.

Alloftheseissuesarethingsyoucanmanuallyhardcodeintoeachstep,butthatcodeisoftenveryrepetitiveandnotreusableinotherstepsorinotherasyncflowsinyourprogram.

Eventhoughourbrainsmightplanoutaseriesoftasksinasequentialtypeofway(this,thenthis,thenthis),theeventednatureofourbrainoperationmakesrecovery/retry/forkingofflowcontrolalmosteffortless.Ifyou'reoutrunningerrands,andyourealizeyouleftashoppinglistathome,itdoesn'tendthedaybecauseyoudidn'tplanthataheadoftime.Yourbrainroutesaroundthishiccupeasily:yougohome,getthelist,thenheadrightbackouttothestore.

Butthebrittlenatureofmanuallyhardcodedcallbacks(evenwithhardcodederrorhandling)isoftenfarlessgraceful.Onceyouendupspecifying(akapre-planning)allthevariouseventualities/paths,thecodebecomessoconvolutedthatit'shardtoevermaintainorupdateit.

Thatiswhat"callbackhell"isallabout!Thenesting/indentationarebasicallyasideshow,aredherring.

Andasifallthat'snotenough,wehaven'teventouchedwhathappenswhentwoormorechainsofthesecallbackcontinuationsarehappeningsimultaneously,orwhenthethirdstepbranchesoutinto"parallel"callbackswithgatesorlatches,or...OMG,mybrainhurts,howaboutyours!?

Areyoucatchingthenotionherethatoursequential,blockingbrainplanningbehaviorsjustdon'tmapwellontocallback-orientedasynccode?That'sthefirstmajordeficiencytoarticulateaboutcallbacks:theyexpressasynchronyincodeinwaysourbrainshavetofightjusttokeepinsyncwith(punintended!).

Themismatchbetweensequentialbrainplanningandcallback-drivenasyncJScodeisonlypartoftheproblemwithcallbacks.There'ssomethingmuchdeepertobeconcernedabout.

Let'sonceagainrevisitthenotionofacallbackfunctionasthecontinuation(akathesecondhalf)ofourprogram:

//A

ajax("..",function(..){

//C

});

//B

//Aand//Bhappennow,underthedirectcontrolofthemainJSprogram.But//Cgetsdeferredtohappenlater,andunderthecontrolofanotherparty--inthiscase,theajax(..)function.Inabasicsense,thatsortofhand-offofcontroldoesn'tregularlycauselotsofproblemsforprograms.

Butdon'tbefooledbyitsinfrequencythatthiscontrolswitchisn'tabigdeal.Infact,it'soneoftheworst(andyetmostsubtle)problemsaboutcallback-drivendesign.Itrevolvesaroundtheideathatsometimesajax(..)(i.e.,the"party"youhandyourcallbackcontinuationto)isnotafunctionthatyouwrote,orthatyoudirectlycontrol.Manytimesit'sautilityprovidedbysomethirdparty.

Wecallthis"inversionofcontrol,"whenyoutakepartofyourprogramandgiveovercontrolofitsexecutiontoanotherthirdparty.There'sanunspoken"contract"thatexistsbetweenyourcodeandthethird-partyutility--asetofthingsyouexpecttobemaintained.

Itmightnotbeterriblyobviouswhythisissuchabigdeal.Letmeconstructanexaggeratedscenariotoillustratethehazardsoftrustatplay.

Imagineyou'readevelopertaskedwithbuildingoutanecommercecheckoutsystemforasitethatsellsexpensiveTVs.Youalreadyhaveallthevariouspagesofthecheckoutsystembuiltoutjustfine.Onthelastpage,whentheuserclicks"confirm"tobuytheTV,youneedtocallathird-partyfunction(providedsaybysomeanalyticstrackingcompany)sothatthesalecanbetracked.

Younoticethatthey'veprovidedwhatlookslikeanasynctrackingutility,probablyforthesakeofperformancebestpractices,whichmeansyouneedtopassinacallbackfunction.Inthiscontinuationthatyoupassin,youwillhavethefinalcodethatchargesthecustomer'screditcardanddisplaysthethankyoupage.

Thiscodemightlooklike:

analytics.trackPurchase(purchaseData,function(){

chargeCreditCard();

displayThankyouPage();

});

Easyenough,right?Youwritethecode,testit,everythingworks,andyoudeploytoproduction.Everyone'shappy!

TrustIssues

TaleofFiveCallbacks

Sixmonthsgobyandnoissues.You'vealmostforgottenyouevenwrotethatcode.Onemorning,you'reatacoffeeshopbeforework,casuallyenjoyingyourlatte,whenyougetapanickedcallfromyourbossinsistingyoudropthecoffeeandrushintoworkrightaway.

Whenyouarrive,youfindoutthatahigh-profilecustomerhashadhiscreditcardchargedfivetimesforthesameTV,andhe'sunderstandablyupset.Customerservicehasalreadyissuedanapologyandprocessedarefund.Butyourbossdemandstoknowhowthiscouldpossiblyhavehappened."Don'twehavetestsforstufflikethis!?"

Youdon'tevenrememberthecodeyouwrote.Butyoudigbackinandstarttryingtofindoutwhatcouldhavegoneawry.

Afterdiggingthroughsomelogs,youcometotheconclusionthattheonlyexplanationisthattheanalyticsutilitysomehow,forsomereason,calledyourcallbackfivetimesinsteadofonce.Nothingintheirdocumentationmentionsanythingaboutthis.

Frustrated,youcontactcustomersupport,whoofcourseisasastonishedasyouare.Theyagreetoescalateittotheirdevelopers,andpromisetogetbacktoyou.Thenextday,youreceivealengthyemailexplainingwhattheyfound,whichyoupromptlyforwardtoyourboss.

Apparently,thedevelopersattheanalyticscompanyhadbeenworkingonsomeexperimentalcodethat,undercertainconditions,wouldretrytheprovidedcallbackoncepersecond,forfiveseconds,beforefailingwithatimeout.Theyhadneverintendedtopushthatintoproduction,butsomehowtheydid,andthey'retotallyembarrassedandapologetic.Theygointoplentyofdetailabouthowthey'veidentifiedthebreakdownandwhatthey'lldotoensureitneverhappensagain.Yadda,yadda.

What'snext?

Youtalkitoverwithyourboss,buthe'snotfeelingparticularlycomfortablewiththestateofthings.Heinsists,andyoureluctantlyagree,thatyoucan'ttrustthemanymore(that'swhatbityou),andthatyou'llneedtofigureouthowtoprotectthecheckoutcodefromsuchavulnerabilityagain.

Aftersometinkering,youimplementsomesimpleadhoccodelikethefollowing,whichtheteamseemshappywith:

vartracked=false;

analytics.trackPurchase(purchaseData,function(){

if(!tracked){

tracked=true;

chargeCreditCard();

displayThankyouPage();

}

});

Note:ThisshouldlookfamiliartoyoufromChapter1,becausewe'reessentiallycreatingalatchtohandleiftherehappentobemultipleconcurrentinvocationsofourcallback.

ButthenoneofyourQAengineersasks,"whathappensiftheynevercallthecallback?"Oops.Neitherofyouhadthoughtaboutthat.

Youbegintochasedowntherabbithole,andthinkofallthepossiblethingsthatcouldgowrongwiththemcallingyourcallback.Here'sroughlythelistyoucomeupwithofwaystheanalyticsutilitycouldmisbehave:

Callthecallbacktooearly(beforeit'sbeentracked)Callthecallbacktoolate(ornever)Callthecallbacktoofewortoomanytimes(liketheproblemyouencountered!)Failtopassalonganynecessaryenvironment/parameterstoyourcallbackSwallowanyerrors/exceptionsthatmayhappen...

Thatshouldfeellikeatroublinglist,becauseitis.You'reprobablyslowlystartingtorealizethatyou'regoingtohaveto

inventanawfullotofadhoclogicineachandeverysinglecallbackthat'spassedtoautilityyou'renotpositiveyoucantrust.

Nowyourealizeabitmorecompletelyjusthowhellish"callbackhell"is.

SomeofyoumaybeskepticalatthispointwhetherthisisasbigadealasI'mmakingitouttobe.Perhapsyoudon'tinteractwithtrulythird-partyutilitiesmuchifatall.PerhapsyouuseversionedAPIsorself-hostsuchlibraries,sothatitsbehaviorcan'tbechangedoutfromunderneathyou.

So,contemplatethis:canyouevenreallytrustutilitiesthatyoudotheoreticallycontrol(inyourowncodebase)?

Thinkofitthisway:mostofusagreethatatleasttosomeextentweshouldbuildourowninternalfunctionswithsomedefensivechecksontheinputparameters,toreduce/preventunexpectedissues.

Overlytrustingofinput:

functionaddNumbers(x,y){

//+isoverloadedwithcoerciontoalsobe

//stringconcatenation,sothisoperation

//isn'tstrictlysafedependingonwhat's

//passedin.

returnx+y;

}

addNumbers(21,21);//42

addNumbers(21,"21");//"2121"

Defensiveagainstuntrustedinput:


//ensurenumericalinput

if(typeofx!="number"||typeofy!="number"){

throwError("Badparameters");

}

//ifwegethere,+willsafelydonumericaddition

returnx+y;

}


addNumbers(21,"21");//Error:"Badparameters"

Orperhapsstillsafebutfriendlier:


//ensurenumericalinput

x=Number(x);

y=Number(y);

//+willsafelydonumericaddition

returnx+y;

}


addNumbers(21,"21");//42

Howeveryougoaboutit,thesesortsofchecks/normalizationsarefairlycommononfunctioninputs,evenwithcodewetheoreticallyentirelytrust.Inacrudesortofway,it'sliketheprogrammingequivalentofthegeopoliticalprincipleof"TrustButVerify."

So,doesn'titstandtoreasonthatweshoulddothesamethingaboutcompositionofasyncfunctioncallbacks,notjustwith

NotJustOthers'Code

trulyexternalcodebutevenwithcodeweknowisgenerally"underourowncontrol"?Ofcourseweshould.

Butcallbacksdon'treallyofferanythingtoassistus.Wehavetoconstructallthatmachineryourselves,anditoftenendsupbeingalotofboilerplate/overheadthatwerepeatforeverysingleasynccallback.

Themosttroublesomeproblemwithcallbacksisinversionofcontrolleadingtoacompletebreakdownalongallthosetrustlines.

Ifyouhavecodethatusescallbacks,especiallybutnotexclusivelywiththird-partyutilities,andyou'renotalreadyapplyingsomesortofmitigationlogicforalltheseinversionofcontroltrustissues,yourcodehasbugsinitrightnoweventhoughtheymaynothavebittenyouyet.Latentbugsarestillbugs.

Hellindeed.

Thereareseveralvariationsofcallbackdesignthathaveattemptedtoaddresssome(notall!)ofthetrustissueswe'vejustlookedat.It'savaliant,butdoomed,efforttosavethecallbackpatternfromimplodingonitself.

Forexample,regardingmoregracefulerrorhandling,someAPIdesignsprovideforsplitcallbacks(oneforthesuccessnotification,onefortheerrornotification):

functionsuccess(data){

console.log(data);

}

functionfailure(err){

console.error(err);

}

ajax("http://some.url.1",success,failure);

InAPIsofthisdesign,oftenthefailure()errorhandlerisoptional,andifnotprovideditwillbeassumedyouwanttheerrorsswallowed.Ugh.

Note:Thissplit-callbackdesigniswhattheES6PromiseAPIuses.We'llcoverES6Promisesinmuchmoredetailinthenextchapter.

Anothercommoncallbackpatterniscalled"error-firststyle"(sometimescalled"Nodestyle,"asit'salsotheconventionusedacrossnearlyallNode.jsAPIs),wherethefirstargumentofasinglecallbackisreservedforanerrorobject(ifany).Ifsuccess,thisargumentwillbeempty/falsy(andanysubsequentargumentswillbethesuccessdata),butifanerrorresultisbeingsignaled,thefirstargumentisset/truthy(andusuallynothingelseispassed):

functionresponse(err,data){

//error?

if(err){

console.error(err);

}

//otherwise,assumesuccess

else{

console.log(data);

}

}


Inbothofthesecases,severalthingsshouldbeobserved.

First,ithasnotreallyresolvedthemajorityoftrustissueslikeitmayappear.There'snothingabouteithercallbackthatpreventsorfiltersunwantedrepeatedinvocations.Moreover,thingsareworsenow,becauseyoumaygetbothsuccessand

TryingtoSaveCallbacks

errorsignals,orneither,andyoustillhavetocodearoundeitherofthoseconditions.

Also,don'tmissthefactthatwhileit'sastandardpatternyoucanemploy,it'sdefinitelymoreverboseandboilerplate-ishwithoutmuchreuse,soyou'regoingtogetwearyoftypingallthatoutforeverysinglecallbackinyourapplication.

Whataboutthetrustissueofneverbeingcalled?Ifthisisaconcern(anditprobablyshouldbe!),youlikelywillneedtosetupatimeoutthatcancelstheevent.Youcouldmakeautility(proof-of-conceptonlyshown)tohelpyouwiththat:

functiontimeoutify(fn,delay){

varintv=setTimeout(function(){

intv=null;

fn(newError("Timeout!"));

},delay)

;

returnfunction(){

//timeouthasn'thappenedyet?

if(intv){

clearTimeout(intv);

fn.apply(this,arguments);

}

};

}

Here'showyouuseit:

//using"error-firststyle"callbackdesign

functionfoo(err,data){

if(err){

console.error(err);

}

else{

console.log(data);

}

}

ajax("http://some.url.1",timeoutify(foo,500));

Anothertrustissueisbeingcalled"tooearly."Inapplication-specificterms,thismayactuallyinvolvebeingcalledbeforesomecriticaltaskiscomplete.Butmoregenerally,theproblemisevidentinutilitiesthatcaneitherinvokethecallbackyouprovidenow(synchronously),orlater(asynchronously).

Thisnondeterminismaroundthesync-or-asyncbehaviorisalmostalwaysgoingtoleadtoverydifficulttotrackdownbugs.Insomecircles,thefictionalinsanity-inducingmonsternamedZalgoisusedtodescribethesync/asyncnightmares."Don'treleaseZalgo!"isacommoncry,anditleadstoverysoundadvice:alwaysinvokecallbacksasynchronously,evenifthat's"rightaway"onthenextturnoftheeventloop,sothatallcallbacksarepredictablyasync.

Note:FormoreinformationonZalgo,seeOrenGolan's"Don'tReleaseZalgo!"(https://github.com/oren/oren.github.io/blob/master/posts/zalgo.md)andIsaacZ.Schlueter's"DesigningAPIsforAsynchrony"(http://blog.izs.me/post/59142742143/designing-apis-for-asynchrony).

Consider:

functionresult(data){

console.log(a);

}

vara=0;

ajax("..pre-cached-url..",result);

a++;

Willthiscodeprint0(synccallbackinvocation)or1(asynccallbackinvocation)?Depends...ontheconditions.

https://github.com/oren/oren.github.io/blob/master/posts/zalgo.md

http://blog.izs.me/post/59142742143/designing-apis-for-asynchrony

YoucanseejusthowquicklytheunpredictabilityofZalgocanthreatenanyJSprogram.Sothesilly-sounding"neverreleaseZalgo"isactuallyincrediblycommonandsolidadvice.Alwaysbeasyncing.

Whatifyoudon'tknowwhethertheAPIinquestionwillalwaysexecuteasync?Youcouldinventautilitylikethisasyncify(..)proof-of-concept:

functionasyncify(fn){

varorig_fn=fn,

intv=setTimeout(function(){

intv=null;

if(fn)fn();

},0)

;

fn=null;

returnfunction(){

//firingtooquickly,beforeìntv`timerhasfiredto

//indicateasyncturnhaspassed?

if(intv){

fn=orig_fn.bind.apply(

orig_fn,

//addthewrapper's`this`tothe`bind(..)`

//callparameters,aswellascurryingany

//passedinparameters

[this].concat([].slice.call(arguments))

);

}

//alreadyasync

else{

//invokeoriginalfunction

orig_fn.apply(this,arguments);

}

};

}

Youuseasyncify(..)likethis:

functionresult(data){

console.log(a);

}

vara=0;

ajax("..pre-cached-url..",asyncify(result));

a++;

WhethertheAjaxrequestisinthecacheandresolvestotrytocallthecallbackrightaway,ormustbefetchedoverthewireandthuscompletelaterasynchronously,thiscodewillalwaysoutput1insteadof0--result(..)cannothelpbutbeinvokedasynchronously,whichmeansthea++hasachancetorunbeforeresult(..)does.

Yay,anothertrustissued"solved"!Butit'sinefficient,andyetagainmorebloatedboilerplatetoweighyourprojectdown.

That'sjustthestory,overandoveragain,withcallbacks.Theycandoprettymuchanythingyouwant,butyouhavetobewillingtoworkhardtogetit,andoftentimesthiseffortismuchmorethanyoucanorshouldspendonsuchcodereasoning.

Youmightfindyourselfwishingforbuilt-inAPIsorotherlanguagemechanicstoaddresstheseissues.FinallyES6hasarrivedonthescenewithsomegreatanswers,sokeepreading!

CallbacksarethefundamentalunitofasynchronyinJS.Butthey'renotenoughfortheevolvinglandscapeofasyncprogrammingasJSmatures.

First,ourbrainsplanthingsoutinsequential,blocking,single-threadedsemanticways,butcallbacksexpress

Review

asynchronousflowinarathernonlinear,nonsequentialway,whichmakesreasoningproperlyaboutsuchcodemuchharder.Badtoreasonaboutcodeisbadcodethatleadstobadbugs.

Weneedawaytoexpressasynchronyinamoresynchronous,sequential,blockingmanner,justlikeourbrainsdo.

Second,andmoreimportantly,callbackssufferfrominversionofcontrolinthattheyimplicitlygivecontrolovertoanotherparty(oftenathird-partyutilitynotinyourcontrol!)toinvokethecontinuationofyourprogram.Thiscontroltransferleadsustoatroublinglistoftrustissues,suchaswhetherthecallbackiscalledmoretimesthanweexpect.

Inventingadhoclogictosolvethesetrustissuesispossible,butit'smoredifficultthanitshouldbe,anditproducesclunkierandhardertomaintaincode,aswellascodethatislikelyinsufficientlyprotectedfromthesehazardsuntilyougetvisiblybittenbythebugs.

Weneedageneralizedsolutiontoallofthetrustissues,onethatcanbereusedforasmanycallbacksaswecreatewithoutalltheextraboilerplateoverhead.

Weneedsomethingbetterthancallbacks.They'veserveduswelltothispoint,butthefutureofJavaScriptdemandsmoresophisticatedandcapableasyncpatterns.Thesubsequentchaptersinthisbookwilldiveintothoseemergingevolutions.

InChapter2,weidentifiedtwomajorcategoriesofdeficiencieswithusingcallbackstoexpressprogramasynchronyandmanageconcurrency:lackofsequentialityandlackoftrustability.Nowthatweunderstandtheproblemsmoreintimately,it'stimeweturnourattentiontopatternsthatcanaddressthem.

Theissuewewanttoaddressfirstistheinversionofcontrol,thetrustthatissofragilelyheldandsoeasilylost.

Recallthatwewrapupthecontinuationofourprograminacallbackfunction,andhandthatcallbackovertoanotherparty(potentiallyevenexternalcode)andjustcrossourfingersthatitwilldotherightthingwiththeinvocationofthecallback.

Wedothisbecausewewanttosay,"here'swhathappenslater,afterthecurrentstepfinishes."

Butwhatifwecoulduninvertthatinversionofcontrol?Whatifinsteadofhandingthecontinuationofourprogramtoanotherparty,wecouldexpectittoreturnusacapabilitytoknowwhenitstaskfinishes,andthenourcodecoulddecidewhattodonext?

ThisparadigmiscalledPromises.

PromisesarestartingtotaketheJSworldbystorm,asdevelopersandspecificationwritersalikedesperatelyseektountangletheinsanityofcallbackhellintheircode/design.Infact,mostnewasyncAPIsbeingaddedtoJS/DOMplatformarebeingbuiltonPromises.Soit'sprobablyagoodideatodiginandlearnthem,don'tyouthink!?

Note:Theword"immediately"willbeusedfrequentlyinthischapter,generallytorefertosomePromiseresolutionaction.However,inessentiallyallcases,"immediately"meansintermsoftheJobqueuebehavior(seeChapter1),notinthestrictlysynchronousnowsense.

Whendevelopersdecidetolearnanewtechnologyorpattern,usuallytheirfirststepis"Showmethecode!"It'squitenaturalforustojustjumpinfeetfirstandlearnaswego.

ButitturnsoutthatsomeabstractionsgetlostontheAPIsalone.Promisesareoneofthosetoolswhereitcanbepainfullyobviousfromhowsomeoneusesitwhethertheyunderstandwhatit'sforandaboutversusjustlearningandusingtheAPI.

SobeforeIshowthePromisecode,IwanttofullyexplainwhataPromisereallyisconceptually.IhopethiswillthenguideyoubetterasyouexploreintegratingPromisetheoryintoyourownasyncflow.

Withthatinmind,let'slookattwodifferentanalogiesforwhataPromiseis.

Imaginethisscenario:Iwalkuptothecounteratafast-foodrestaurant,andplaceanorderforacheeseburger.Ihandthecashier$1.47.Byplacingmyorderandpayingforit,I'vemadearequestforavalueback(thecheeseburger).I'vestartedatransaction.

Butoften,thechesseburgerisnotimmediatelyavailableforme.Thecashierhandsmesomethinginplaceofmycheeseburger:areceiptwithanordernumberonit.ThisordernumberisanIOU("Ioweyou")promisethatensuresthateventually,Ishouldreceivemycheeseburger.

SoIholdontomyreceiptandordernumber.Iknowitrepresentsmyfuturecheeseburger,soIdon'tneedtoworryaboutitanymore--asidefrombeinghungry!


Chapter3:Promises

WhatIsaPromise?

FutureValue

WhileIwait,Icandootherthings,likesendatextmessagetoafriendthatsays,"Hey,canyoucomejoinmeforlunch?I'mgoingtoeatacheeseburger."

Iamreasoningaboutmyfuturecheeseburgeralready,eventhoughIdon'thaveitinmyhandsyet.Mybrainisabletodothisbecauseit'streatingtheordernumberasaplaceholderforthecheeseburger.Theplaceholderessentiallymakesthevaluetimeindependent.It'safuturevalue.

Eventually,Ihear,"Order113!"andIgleefullywalkbackuptothecounterwithreceiptinhand.Ihandmyreceipttothecashier,andItakemycheeseburgerinreturn.

Inotherwords,oncemyfuturevaluewasready,Iexchangedmyvalue-promiseforthevalueitself.

Butthere'sanotherpossibleoutcome.Theycallmyordernumber,butwhenIgotoretrievemycheeseburger,thecashierregretfullyinformsme,"I'msorry,butweappeartobealloutofcheeseburgers."Settingasidethecustomerfrustrationofthisscenarioforamoment,wecanseeanimportantcharacteristicoffuturevalues:theycaneitherindicateasuccessorfailure.

EverytimeIorderacheeseburger,IknowthatI'lleithergetacheeseburgereventually,orI'llgetthesadnewsofthecheeseburgershortage,andI'llhavetofigureoutsomethingelsetoeatforlunch.

Note:Incode,thingsarenotquiteassimple,becausemetaphoricallytheordernumbermayneverbecalled,inwhichcasewe'releftindefinitelyinanunresolvedstate.We'llcomebacktodealingwiththatcaselater.

Thisallmightsoundtoomentallyabstracttoapplytoyourcode.Solet'sbemoreconcrete.

However,beforewecanintroducehowPromisesworkinthisfashion,we'regoingtoderiveincodethatwealreadyunderstand--callbacks!--howtohandlethesefuturevalues.

Whenyouwritecodetoreasonaboutavalue,suchasperformingmathonanumber,whetheryourealizeitornot,you'vebeenassumingsomethingveryfundamentalaboutthatvalue,whichisthatit'saconcretenowvaluealready:

varx,y=2;

console.log(x+y);//NaN<--because`xìsn'tsetyet

Thex+yoperationassumesbothxandyarealreadyset.Intermswe'llexpoundonshortly,weassumethexandyvaluesarealreadyresolved.

Itwouldbenonsensetoexpectthatthe+operatorbyitselfwouldsomehowbemagicallycapableofdetectingandwaitingarounduntilbothxandyareresolved(akaready),onlythentodotheoperation.Thatwouldcausechaosintheprogramifdifferentstatementsfinishednowandothersfinishedlater,right?

Howcouldyoupossiblyreasonabouttherelationshipsbetweentwostatementsifeitherone(orboth)ofthemmightnotbefinishedyet?Ifstatement2reliesonstatement1beingfinished,therearejusttwooutcomes:eitherstatement1finishedrightnowandeverythingproceedsfine,orstatement1didn'tfinishyet,andthusstatement2isgoingtofail.

IfthissortofthingsoundsfamiliarfromChapter1,good!

Let'sgobacktoourx+ymathoperation.Imagineiftherewasawaytosay,"Addxandy,butifeitherofthemisn'treadyyet,justwaituntiltheyare.Addthemassoonasyoucan."

Yourbrainmighthavejustjumpedtocallbacks.OK,so...

functionadd(getX,getY,cb){

varx,y;

getX(function(xVal){

ValuesNowandLater

x=xVal;

//bothareready?

if(y!=undefined){

cb(x+y);//sendalongsum

}

});

getY(function(yVal){

y=yVal;

//bothareready?

if(x!=undefined){

cb(x+y);//sendalongsum

}

});

}

//`fetchX()ànd`fetchY()àresyncorasync

//functions

add(fetchX,fetchY,function(sum){

console.log(sum);//thatwaseasy,huh?

});

Takejustamomenttoletthebeauty(orlackthereof)ofthatsnippetsinkin(whistlespatiently).

Whiletheuglinessisundeniable,there'ssomethingveryimportanttonoticeaboutthisasyncpattern.

Inthatsnippet,wetreatedxandyasfuturevalues,andweexpressanoperationadd(..)that(fromtheoutside)doesnotcarewhetherxoryorbothareavailablerightawayornot.Inotherwords,itnormalizesthenowandlater,suchthatwecanrelyonapredictableoutcomeoftheadd(..)operation.

Byusinganadd(..)thatistemporallyconsistent--itbehavesthesameacrossnowandlatertimes--theasynccodeismucheasiertoreasonabout.

Toputitmoreplainly:toconsistentlyhandlebothnowandlater,wemakebothofthemlater:alloperationsbecomeasync.

Ofcourse,thisroughcallbacks-basedapproachleavesmuchtobedesired.It'sjustafirsttinysteptowardrealizingthebenefitsofreasoningaboutfuturevalueswithoutworryingaboutthetimeaspectofwhenit'savailableornot.

We'lldefinitelygointoalotmoredetailaboutPromiseslaterinthechapter--sodon'tworryifsomeofthisisconfusing--butlet'sjustbrieflyglimpseathowwecanexpressthex+yexampleviaPromises:

functionadd(xPromise,yPromise){

//`Promise.all([..])`takesanarrayofpromises,

//andreturnsanewpromisethatwaitsonthem

//alltofinish

returnPromise.all([xPromise,yPromise])

//whenthatpromiseisresolved,let'stakethe

//received`Xànd`Y`valuesandaddthemtogether.

.then(function(values){

//`valuesìsanarrayofthemessagesfromthe

//previouslyresolvedpromises

returnvalues[0]+values[1];

});

}

//`fetchX()ànd`fetchY()`returnpromisesfor

//theirrespectivevalues,whichmaybeready

//*now*or*later*.

add(fetchX(),fetchY())

//wegetapromisebackforthesumofthose

//twonumbers.

//nowwechain-call`then(..)`towaitforthe

//resolutionofthatreturnedpromise.

.then(function(sum){

console.log(sum);//thatwaseasier!

});

PromiseValue

TherearetwolayersofPromisesinthissnippet.

fetchX()andfetchY()arecalleddirectly,andthevaluestheyreturn(promises!)arepassedintoadd(..).Theunderlyingvaluesthosepromisesrepresentmaybereadynoworlater,buteachpromisenormalizesthebehaviortobethesameregardless.WereasonaboutXandYvaluesinatime-independentway.Theyarefuturevalues.

Thesecondlayeristhepromisethatadd(..)creates(viaPromise.all([..]))andreturns,whichwewaitonbycallingthen(..).Whentheadd(..)operationcompletes,oursumfuturevalueisreadyandwecanprintitout.Wehideinsideofadd(..)thelogicforwaitingontheXandYfuturevalues.

Note:Insideadd(..),thePromise.all([..])callcreatesapromise(whichiswaitingonpromiseXandpromiseYtoresolve).Thechainedcallto.then(..)createsanotherpromise,whichthereturnvalues[0]+values[1]lineimmediatelyresolves(withtheresultoftheaddition).Thus,thethen(..)callwechainofftheendoftheadd(..)call--attheendofthesnippet--isactuallyoperatingonthatsecondpromisereturned,ratherthanthefirstonecreatedbyPromise.all([..]).Also,thoughwearenotchainingofftheendofthatsecondthen(..),ittoohascreatedanotherpromise,hadwechosentoobserve/useit.ThisPromisechainingstuffwillbeexplainedinmuchgreaterdetaillaterinthischapter.

Justlikewithcheeseburgerorders,it'spossiblethattheresolutionofaPromiseisrejectioninsteadoffulfillment.UnlikeafulfilledPromise,wherethevalueisalwaysprogrammatic,arejectionvalue--commonlycalleda"rejectionreason"--caneitherbesetdirectlybytheprogramlogic,oritcanresultimplicitlyfromaruntimeexception.

WithPromises,thethen(..)callcanactuallytaketwofunctions,thefirstforfulfillment(asshownearlier),andthesecondforrejection:

add(fetchX(),fetchY())

.then(

//fullfillmenthandler

function(sum){

console.log(sum);

},

//rejectionhandler

function(err){

console.error(err);//bummer!

}

);

IfsomethingwentwronggettingXorY,orsomethingsomehowfailedduringtheaddition,thepromisethatadd(..)returnsisrejected,andthesecondcallbackerrorhandlerpassedtothen(..)willreceivetherejectionvaluefromthepromise.

BecausePromisesencapsulatethetime-dependentstate--waitingonthefulfillmentorrejectionoftheunderlyingvalue--fromtheoutside,thePromiseitselfistime-independent,andthusPromisescanbecomposed(combined)inpredictablewaysregardlessofthetimingoroutcomeunderneath.

Moreover,onceaPromiseisresolved,itstaysthatwayforever--itbecomesanimmutablevalueatthatpoint--andcanthenbeobservedasmanytimesasnecessary.

Note:BecauseaPromiseisexternallyimmutableonceresolved,it'snowsafetopassthatvaluearoundtoanypartyandknowthatitcannotbemodifiedaccidentallyormaliciously.ThisisespeciallytrueinrelationtomultiplepartiesobservingtheresolutionofaPromise.Itisnotpossibleforonepartytoaffectanotherparty'sabilitytoobservePromiseresolution.Immutabilitymaysoundlikeanacademictopic,butit'sactuallyoneofthemostfundamentalandimportantaspectsofPromisedesign,andshouldn'tbecasuallypassedover.

That'soneofthemostpowerfulandimportantconceptstounderstandaboutPromises.Withafairamountofwork,youcouldadhoccreatethesameeffectswithnothingbutuglycallbackcomposition,butthat'snotreallyaneffectivestrategy,especiallybecauseyouhavetodoitoverandoveragain.

Promisesareaneasilyrepeatablemechanismforencapsulatingandcomposingfuturevalues.

Aswejustsaw,anindividualPromisebehavesasafuturevalue.Butthere'sanotherwaytothinkoftheresolutionofaPromise:asaflow-controlmechanism--atemporalthis-then-that--fortwoormorestepsinanasynchronoustask.

Let'simaginecallingafunctionfoo(..)toperformsometask.Wedon'tknowaboutanyofitsdetails,nordowecare.Itmaycompletethetaskrightaway,oritmaytakeawhile.

Wejustsimplyneedtoknowwhenfoo(..)finishessothatwecanmoveontoournexttask.Inotherwords,we'dlikeawaytobenotifiedoffoo(..)'scompletionsothatwecancontinue.

IntypicalJavaScriptfashion,ifyouneedtolistenforanotification,you'dlikelythinkofthatintermsofevents.Sowecouldreframeourneedfornotificationasaneedtolistenforacompletion(orcontinuation)eventemittedbyfoo(..).

Note:Whetheryoucallita"completionevent"ora"continuationevent"dependsonyourperspective.Isthefocusmoreonwhathappenswithfoo(..),orwhathappensafterfoo(..)finishes?Bothperspectivesareaccurateanduseful.Theeventnotificationtellsusthatfoo(..)hascompleted,butalsothatit'sOKtocontinuewiththenextstep.Indeed,thecallbackyoupasstobecalledfortheeventnotificationisitselfwhatwe'vepreviouslycalledacontinuation.Becausecompletioneventisabitmorefocusedonthefoo(..),whichmorehasourattentionatpresent,weslightlyfavorcompletioneventfortherestofthistext.

Withcallbacks,the"notification"wouldbeourcallbackinvokedbythetask(foo(..)).ButwithPromises,weturntherelationshiparound,andexpectthatwecanlistenforaneventfromfoo(..),andwhennotified,proceedaccordingly.

First,considersomepseudocode:

foo(x){

//startdoingsomethingthatcouldtakeawhile

}

foo(42)

on(foo"completion"){

//nowwecandothenextstep!

}

on(foo"error"){

//oops,somethingwentwrongin`foo(..)`

}

Wecallfoo(..)andthenwesetuptwoeventlisteners,onefor"completion"andonefor"error"--thetwopossiblefinaloutcomesofthefoo(..)call.Inessence,foo(..)doesn'tevenappeartobeawarethatthecallingcodehassubscribedtotheseevents,whichmakesforaveryniceseparationofconcerns.

Unfortunately,suchcodewouldrequiresome"magic"oftheJSenvironmentthatdoesn'texist(andwouldlikelybeabitimpractical).Here'sthemorenaturalwaywecouldexpressthatinJS:

functionfoo(x){


//makea`listenerèventnotification

//capabilitytoreturn

returnlistener;

}

varevt=foo(42);

evt.on("completion",function(){

//nowwecandothenextstep!

});

evt.on("failure",function(err){


});

CompletionEvent

foo(..)expresslycreatesaneventsubscriptioncapabilitytoreturnback,andthecallingcodereceivesandregistersthetwoeventhandlersagainstit.

Theinversionfromnormalcallback-orientedcodeshouldbeobvious,andit'sintentional.Insteadofpassingthecallbackstofoo(..),itreturnsaneventcapabilitywecallevt,whichreceivesthecallbacks.

ButifyourecallfromChapter2,callbacksthemselvesrepresentaninversionofcontrol.Soinvertingthecallbackpatternisactuallyaninversionofinversion,oranuninversionofcontrol--restoringcontrolbacktothecallingcodewherewewantedittobeinthefirstplace.

Oneimportantbenefitisthatmultipleseparatepartsofthecodecanbegiventheeventlisteningcapability,andtheycanallindependentlybenotifiedofwhenfoo(..)completestoperformsubsequentstepsafteritscompletion:

varevt=foo(42);

//let`bar(..)`listento`foo(..)`'scompletion

bar(evt);

//also,let`baz(..)`listento`foo(..)`'scompletion

baz(evt);

Uninversionofcontrolenablesanicerseparationofconcerns,wherebar(..)andbaz(..)don'tneedtobeinvolvedinhowfoo(..)iscalled.Similarly,foo(..)doesn'tneedtoknoworcarethatbar(..)andbaz(..)existorarewaitingtobenotifiedwhenfoo(..)completes.

Essentially,thisevtobjectisaneutralthird-partynegotiationbetweentheseparateconcerns.

Asyoumayhaveguessedbynow,theevteventlisteningcapabilityisananalogyforaPromise.

InaPromise-basedapproach,theprevioussnippetwouldhavefoo(..)creatingandreturningaPromiseinstance,andthatpromisewouldthenbepassedtobar(..)andbaz(..).

Note:ThePromiseresolution"events"welistenforaren'tstrictlyevents(thoughtheycertainlybehavelikeeventsforthesepurposes),andthey'renottypicallycalled"completion"or"error".Instead,weusethen(..)toregistera"then"event.Orperhapsmoreprecisely,then(..)registers"fulfillment"and/or"rejection"event(s),thoughwedon'tseethosetermsusedexplicitlyinthecode.

Consider:

functionfoo(x){


//constructandreturnapromise

returnnewPromise(function(resolve,reject){

//eventually,call`resolve(..)òr`reject(..)`,

//whicharetheresolutioncallbacksfor

//thepromise.

});

}

varp=foo(42);

bar(p);

baz(p);

Note:ThepatternshownwithnewPromise(function(..){..})isgenerallycalledthe"revealingconstructor".Thefunctionpassedinisexecutedimmediately(notasyncdeferred,ascallbackstothen(..)are),andit'sprovidedtwo

Promise"Events"

http://domenic.me/2014/02/13/the-revealing-constructor-pattern/

parameters,whichinthiscasewe'venamedresolveandreject.Thesearetheresolutionfunctionsforthepromise.resolve(..)generallysignalsfulfillment,andreject(..)signalsrejection.

Youcanprobablyguesswhattheinternalsofbar(..)andbaz(..)mightlooklike:

functionbar(fooPromise){

//listenfor`foo(..)`tocomplete

fooPromise.then(

function(){

//`foo(..)`hasnowfinished,so

//do`bar(..)`'stask

},

function(){


}

);

}

//dittofor`baz(..)`

Promiseresolutiondoesn'tnecessarilyneedtoinvolvesendingalongamessage,asitdidwhenwewereexaminingPromisesasfuturevalues.Itcanjustbeaflow-controlsignal,asusedintheprevioussnippet.

Anotherwaytoapproachthisis:

functionbar(){

//`foo(..)`hasdefinitelyfinished,so

//do`bar(..)`'stask

}

functionoopsBar(){

//oops,somethingwentwrongin`foo(..)`,

//so`bar(..)`didn'trun

}

//dittofor`baz()àndòopsBaz()`

varp=foo(42);

p.then(bar,oopsBar);

p.then(baz,oopsBaz);

Note:Ifyou'veseenPromise-basedcodingbefore,youmightbetemptedtobelievethatthelasttwolinesofthatcodecouldbewrittenasp.then(..).then(..),usingchaining,ratherthanp.then(..);p.then(..).Thatwouldhaveanentirelydifferentbehavior,sobecareful!Thedifferencemightnotbeclearrightnow,butit'sactuallyadifferentasyncpatternthanwe'veseenthusfar:splitting/forking.Don'tworry!We'llcomebacktothispointlaterinthischapter.

Insteadofpassingtheppromisetobar(..)andbaz(..),weusethepromisetocontrolwhenbar(..)andbaz(..)willgetexecuted,ifever.Theprimarydifferenceisintheerrorhandling.

Inthefirstsnippet'sapproach,bar(..)iscalledregardlessofwhetherfoo(..)succeedsorfails,andithandlesitsownfallbacklogicifit'snotifiedthatfoo(..)failed.Thesameistrueforbaz(..),obviously.

Inthesecondsnippet,bar(..)onlygetscallediffoo(..)succeeds,andotherwiseoopsBar(..)getscalled.Dittoforbaz(..).

Neitherapproachiscorrectperse.Therewillbecaseswhereoneispreferredovertheother.

Ineithercase,thepromisepthatcomesbackfromfoo(..)isusedtocontrolwhathappensnext.

Moreover,thefactthatbothsnippetsendupcallingthen(..)twiceagainstthesamepromisepillustratesthepointmadeearlier,whichisthatPromises(onceresolved)retaintheirsameresolution(fulfillmentorrejection)forever,andcansubsequentlybeobservedasmanytimesasnecessary.

Wheneverpisresolved,thenextstepwillalwaysbethesame,bothnowandlater.

InPromises-land,animportantdetailishowtoknowforsureifsomevalueisagenuinePromiseornot.Ormoredirectly,isitavaluethatwillbehavelikeaPromise?

GiventhatPromisesareconstructedbythenewPromise(..)syntax,youmightthinkthatpinstanceofPromisewouldbeanacceptablecheck.Butunfortunately,thereareanumberofreasonsthat'snottotallysufficient.

Mainly,youcanreceiveaPromisevaluefromanotherbrowserwindow(iframe,etc.),whichwouldhaveitsownPromisedifferentfromtheoneinthecurrentwindow/frame,andthatcheckwouldfailtoidentifythePromiseinstance.

Moreover,alibraryorframeworkmaychoosetovenditsownPromisesandnotusethenativeES6Promiseimplementationtodoso.Infact,youmayverywellbeusingPromiseswithlibrariesinolderbrowsersthathavenoPromiseatall.

WhenwediscussPromiseresolutionprocesseslaterinthischapter,itwillbecomemoreobviouswhyanon-genuine-but-Promise-likevaluewouldstillbeveryimportanttobeabletorecognizeandassimilate.Butfornow,justtakemywordforitthatit'sacriticalpieceofthepuzzle.

Assuch,itwasdecidedthatthewaytorecognizeaPromise(orsomethingthatbehaveslikeaPromise)wouldbetodefinesomethingcalleda"thenable"asanyobjectorfunctionwhichhasathen(..)methodonit.ItisassumedthatanysuchvalueisaPromise-conformingthenable.

Thegeneraltermfor"typechecks"thatmakeassumptionsaboutavalue's"type"basedonitsshape(whatpropertiesarepresent)iscalled"ducktyping"--"Ifitlookslikeaduck,andquackslikeaduck,itmustbeaduck"(seetheTypes&Grammartitleofthisbookseries).Sotheducktypingcheckforathenablewouldroughlybe:

if(

p!==null&&

(

typeofp==="object"||

typeofp==="function"

)&&

typeofp.then==="function"

){

//assumeit'sathenable!

}

else{

//notathenable

}

Yuck!Settingasidethefactthatthislogicisabituglytoimplementinvariousplaces,there'ssomethingdeeperandmoretroublinggoingon.

IfyoutrytofulfillaPromisewithanyobject/functionvaluethathappenstohaveathen(..)functiononit,butyouweren'tintendingittobetreatedasaPromise/thenable,you'reoutofluck,becauseitwillautomaticallyberecognizedasthenableandtreatedwithspecialrules(seelaterinthechapter).

Thisiseventrueifyoudidn'trealizethevaluehasathen(..)onit.Forexample:

varo={then:function(){}};

//make`v`be`[[Prototype]]`-linkedtoò`

varv=Object.create(o);

v.someStuff="cool";

v.otherStuff="notsocool";

v.hasOwnProperty("then");//false

ThenableDuckTyping

vdoesn'tlooklikeaPromiseorthenableatall.It'sjustaplainobjectwithsomepropertiesonit.You'reprobablyjustintendingtosendthatvaluearoundlikeanyotherobject.

Butunknowntoyou,visalso[[Prototype]]-linked(seethethis&ObjectPrototypestitleofthisbookseries)toanotherobjecto,whichhappenstohaveathen(..)onit.Sothethenableducktypingcheckswillthinkandassumevisathenable.Uhoh.

Itdoesn'tevenneedtobesomethingasdirectlyintentionalasthat:

Object.prototype.then=function(){};

Array.prototype.then=function(){};

varv1={hello:"world"};

varv2=["Hello","World"];

Bothv1andv2willbeassumedtobethenables.Youcan'tcontrolorpredictifanyothercodeaccidentallyormaliciouslyaddsthen(..)toObject.prototype,Array.prototype,oranyoftheothernativeprototypes.Andifwhat'sspecifiedisafunctionthatdoesn'tcalleitherofitsparametersascallbacks,thenanyPromiseresolvedwithsuchavaluewilljustsilentlyhangforever!Crazy.

Soundimplausibleorunlikely?Perhaps.

Butkeepinmindthattherewereseveralwell-knownnon-PromiselibrariespreexistinginthecommunitypriortoES6thathappenedtoalreadyhaveamethodonthemcalledthen(..).Someofthoselibrarieschosetorenametheirownmethodstoavoidcollision(thatsucks!).Othershavesimplybeenrelegatedtotheunfortunatestatusof"incompatiblewithPromise-basedcoding"inrewardfortheirinabilitytochangetogetoutoftheway.

Thestandardsdecisiontohijackthepreviouslynonreserved--andcompletelygeneral-purposesounding--thenpropertynamemeansthatnovalue(oranyofitsdelegates),eitherpast,present,orfuture,canhaveathen(..)functionpresent,eitheronpurposeorbyaccident,orthatvaluewillbeconfusedforathenableinPromisessystems,whichwillprobablycreatebugsthatarereallyhardtotrackdown.

Warning:IdonotlikehowweendedupwithducktypingofthenablesforPromiserecognition.Therewereotheroptions,suchas"branding"oreven"anti-branding";whatwegotseemslikeaworst-casecompromise.Butit'snotalldoomandgloom.Thenableducktypingcanbehelpful,aswe'llseelater.JustbewarethatthenableducktypingcanbehazardousifitincorrectlyidentifiessomethingasaPromisethatisn't.

We'venowseentwostronganalogiesthatexplaindifferentaspectsofwhatPromisescandoforourasynccode.Butifwestopthere,we'vemissedperhapsthesinglemostimportantcharacteristicthatthePromisepatternestablishes:trust.

Whereasthefuturevaluesandcompletioneventsanalogiesplayoutexplicitlyinthecodepatternswe'veexplored,itwon'tbeentirelyobviouswhyorhowPromisesaredesignedtosolvealloftheinversionofcontroltrustissueswelaidoutinthe"TrustIssues"sectionofChapter2.Butwithalittledigging,wecanuncoversomeimportantguaranteesthatrestoretheconfidenceinasynccodingthatChapter2toredown!

Let'sstartbyreviewingthetrustissueswithcallbacks-onlycoding.Whenyoupassacallbacktoautilityfoo(..),itmight:

CallthecallbacktooearlyCallthecallbacktoolate(ornever)CallthecallbacktoofewortoomanytimesFailtopassalonganynecessaryenvironment/parametersswallowanyerrors/exceptionsthatmayhappen

ThecharacteristicsofPromisesareintentionallydesignedtoprovideuseful,repeatableanswerstoalltheseconcerns.

PromiseTrust

Primarily,thisisaconcernofwhethercodecanintroduceZalgo-likeeffects(seeChapter2),wheresometimesataskfinishessynchronouslyandsometimesasynchronously,whichcanleadtoraceconditions.

Promisesbydefinitioncannotbesusceptibletothisconcern,becauseevenanimmediatelyfulfilledPromise(likenewPromise(function(resolve){resolve(42);}))cannotbeobservedsynchronously.

Thatis,whenyoucallthen(..)onaPromise,evenifthatPromisewasalreadyresolved,thecallbackyouprovidetothen(..)willalwaysbecalledasynchronously(formoreonthis,referbackto"Jobs"inChapter1).

NomoreneedtoinsertyourownsetTimeout(..,0)hacks.PromisespreventZalgoautomatically.

Similartothepreviouspoint,aPromise'sthen(..)registeredobservationcallbacksareautomaticallyscheduledwheneitherresolve(..)orreject(..)arecalledbythePromisecreationcapability.Thosescheduledcallbackswillpredictablybefiredatthenextasynchronousmoment(see"Jobs"inChapter1).

It'snotpossibleforsynchronousobservation,soit'snotpossibleforasynchronouschainoftaskstoruninsuchawaytoineffect"delay"anothercallbackfromhappeningasexpected.Thatis,whenaPromiseisresolved,allthen(..)registeredcallbacksonitwillbecalled,inorder,immediatelyatthenextasynchronousopportunity(again,see"Jobs"inChapter1),andnothingthathappensinsideofoneofthosecallbackscanaffect/delaythecallingoftheothercallbacks.

Forexample:

p.then(function(){

p.then(function(){

console.log("C");

});

console.log("A");

});

p.then(function(){

console.log("B");

});

//ABC

Here,"C"cannotinterruptandprecede"B",byvirtueofhowPromisesaredefinedtooperate.

It'simportanttonote,though,thattherearelotsofnuancesofschedulingwheretherelativeorderingbetweencallbackschainedofftwoseparatePromisesisnotreliablypredictable.

Iftwopromisesp1andp2arebothalreadyresolved,itshouldbetruethatp1.then(..);p2.then(..)wouldendupcallingthecallback(s)forp1beforetheonesforp2.Buttherearesubtlecaseswherethatmightnotbetrue,suchasthefollowing:

varp3=newPromise(function(resolve,reject){

resolve("B");

});


resolve(p3);

});


resolve("A");

});

p1.then(function(v){

console.log(v);

});

CallingTooEarly

CallingTooLate

PromiseSchedulingQuirks


console.log(v);

});

//AB<--notBAasyoumightexpect

We'llcoverthismorelater,butasyoucansee,p1isresolvednotwithanimmediatevalue,butwithanotherpromisep3whichisitselfresolvedwiththevalue"B".Thespecifiedbehavioristounwrapp3intop1,butasynchronously,sop1'scallback(s)arebehindp2'scallback(s)intheasynchronusJobqueue(seeChapter1).

Toavoidsuchnuancednightmares,youshouldneverrelyonanythingabouttheordering/schedulingofcallbacksacrossPromises.Infact,agoodpracticeisnottocodeinsuchawaywheretheorderingofmultiplecallbacksmattersatall.Avoidthatifyoucan.

Thisisaverycommonconcern.It'saddressableinseveralwayswithPromises.

First,nothing(notevenaJSerror)canpreventaPromisefromnotifyingyouofitsresolution(ifit'sresolved).IfyouregisterbothfulfillmentandrejectioncallbacksforaPromise,andthePromisegetsresolved,oneofthetwocallbackswillalwaysbecalled.

Ofcourse,ifyourcallbacksthemselveshaveJSerrors,youmaynotseetheoutcomeyouexpect,butthecallbackwillinfacthavebeencalled.We'llcoverlaterhowtobenotifiedofanerrorinyourcallback,becauseeventhosedon'tgetswallowed.

ButwhatifthePromiseitselfnevergetsresolvedeitherway?EventhatisaconditionthatPromisesprovideananswerfor,usingahigherlevelabstractioncalleda"race":

//autilityfortimingoutaPromise

functiontimeoutPromise(delay){



reject("Timeout!");

},delay);

});

}

//setupatimeoutfor`foo()`

Promise.race([

foo(),//attempt`foo()`

timeoutPromise(3000)//giveit3seconds

])

.then(

function(){

//`foo(..)`fulfilledintime!

},

function(err){

//either`foo()`rejected,oritjust

//didn'tfinishintime,soinspect

//èrr`toknowwhich

}

);

TherearemoredetailstoconsiderwiththisPromisetimeoutpattern,butwe'llcomebacktoitlater.

Importantly,wecanensureasignalastotheoutcomeoffoo(),topreventitfromhangingourprogramindefinitely.

Bydefinition,oneistheappropriatenumberoftimesforthecallbacktobecalled.The"toofew"casewouldbezerocalls,whichisthesameasthe"never"casewejustexamined.

NeverCallingtheCallback

CallingTooFeworTooManyTimes

The"toomany"caseiseasytoexplain.Promisesaredefinedsothattheycanonlyberesolvedonce.IfforsomereasonthePromisecreationcodetriestocallresolve(..)orreject(..)multipletimes,ortriestocallboth,thePromisewillacceptonlythefirstresolution,andwillsilentlyignoreanysubsequentattempts.

BecauseaPromisecanonlyberesolvedonce,anythen(..)registeredcallbackswillonlyeverbecalledonce(each).

Ofcourse,ifyouregisterthesamecallbackmorethanonce,(e.g.,p.then(f);p.then(f);),it'llbecalledasmanytimesasitwasregistered.Theguaranteethataresponsefunctioniscalledonlyoncedoesnotpreventyoufromshootingyourselfinthefoot.

Promisescanhave,atmost,oneresolutionvalue(fulfillmentorrejection).

Ifyoudon'texplicitlyresolvewithavalueeitherway,thevalueisundefined,asistypicalinJS.Butwhateverthevalue,itwillalwaysbepassedtoallregistered(andappropriate:fulfillmentorrejection)callbacks,eithernoworinthefuture.

Somethingtobeawareof:Ifyoucallresolve(..)orreject(..)withmultipleparameters,allsubsequentparametersbeyondthefirstwillbesilentlyignored.Althoughthatmightseemaviolationoftheguaranteewejustdescribed,it'snotexactly,becauseitconstitutesaninvalidusageofthePromisemechanism.OtherinvalidusagesoftheAPI(suchascallingresolve(..)multipletimes)aresimilarlyprotected,sothePromisebehaviorhereisconsistent(ifnotatinybitfrustrating).

Ifyouwanttopassalongmultiplevalues,youmustwraptheminanothersinglevaluethatyoupass,suchasanarrayoranobject.

Asforenvironment,functionsinJSalwaysretaintheirclosureofthescopeinwhichthey'redefined(seetheScope&Closurestitleofthisseries),sotheyofcoursewouldcontinuetohaveaccesstowhateversurroundingstateyouprovide.Ofcourse,thesameistrueofcallbacks-onlydesign,sothisisn'taspecificaugmentationofbenefitfromPromises--butit'saguaranteewecanrelyonnonetheless.

Inthebasesense,thisisarestatementofthepreviouspoint.IfyourejectaPromisewithareason(akaerrormessage),thatvalueispassedtotherejectioncallback(s).

Butthere'ssomethingmuchbiggeratplayhere.IfatanypointinthecreationofaPromise,orintheobservationofitsresolution,aJSexceptionerroroccurs,suchasaTypeErrororReferenceError,thatexceptionwillbecaught,anditwillforcethePromiseinquestiontobecomerejected.

Forexample:

varp=newPromise(function(resolve,reject){

foo.bar();//`fooìsnotdefined,soerror!

resolve(42);//nevergetshere:(

});

p.then(

functionfulfilled(){

//nevergetshere:(

},

functionrejected(err){

//èrr`willbea`TypeErrorèxceptionobject

//fromthe`foo.bar()`line.

}

);

TheJSexceptionthatoccursfromfoo.bar()becomesaPromiserejectionthatyoucancatchandrespondto.

Thisisanimportantdetail,becauseiteffectivelysolvesanotherpotentialZalgomoment,whichisthaterrorscouldcreateasynchronousreactionwhereasnonerrorswouldbeasynchronous.PromisesturnevenJSexceptionsintoasynchronous

FailingtoPassAlongAnyParameters/Environment

SwallowingAnyErrors/Exceptions

behavior,therebyreducingtheraceconditionchancesgreatly.

ButwhathappensifaPromiseisfulfilled,butthere'saJSexceptionerrorduringtheobservation(inathen(..)registeredcallback)?Eventhosearen'tlost,butyoumayfindhowthey'rehandledabitsurprising,untilyoudiginalittledeeper:


resolve(42);

});

p.then(

functionfulfilled(msg){

foo.bar();

console.log(msg);//nevergetshere:(

},


//nevergetshereeither:(

}

);

Wait,thatmakesitseemliketheexceptionfromfoo.bar()reallydidgetswallowed.Neverfear,itdidn't.Butsomethingdeeperiswrong,whichisthatwe'vefailedtolistenforit.Thep.then(..)callitselfreturnsanotherpromise,andit'sthatpromisethatwillberejectedwiththeTypeErrorexception.

Whycouldn'titjustcalltheerrorhandlerwehavedefinedthere?Seemslikealogicalbehavioronthesurface.ButitwouldviolatethefundamentalprinciplethatPromisesareimmutableonceresolved.pwasalreadyfulfilledtothevalue42,soitcan'tlaterbechangedtoarejectionjustbecausethere'sanerrorinobservingp'sresolution.

Besidestheprincipleviolation,suchbehaviorcouldwreakhavoc,ifsaythereweremultiplethen(..)registeredcallbacksonthepromisep,becausesomewouldgetcalledandotherswouldn't,anditwouldbeveryopaqueastowhy.

There'sonelastdetailtoexaminetoestablishtrustbasedonthePromisepattern.

You'venodoubtnoticedthatPromisesdon'tgetridofcallbacksatall.Theyjustchangewherethecallbackispassedto.Insteadofpassingacallbacktofoo(..),wegetsomething(ostensiblyagenuinePromise)backfromfoo(..),andwepassthecallbacktothatsomethinginstead.

Butwhywouldthisbeanymoretrustablethanjustcallbacksalone?HowcanwebesurethesomethingwegetbackisinfactatrustablePromise?Isn'titbasicallyalljustahouseofcardswherewecantrustonlybecausewealreadytrusted?

Oneofthemostimportant,butoftenoverlooked,detailsofPromisesisthattheyhaveasolutiontothisissueaswell.IncludedwiththenativeES6PromiseimplementationisPromise.resolve(..).

Ifyoupassanimmediate,non-Promise,non-thenablevaluetoPromise.resolve(..),yougetapromisethat'sfulfilledwiththatvalue.Inotherwords,thesetwopromisesp1andp2willbehavebasicallyidentically:


resolve(42);

});

varp2=Promise.resolve(42);

ButifyoupassagenuinePromisetoPromise.resolve(..),youjustgetthesamepromiseback:


varp2=Promise.resolve(p1);

p1===p2;//true

TrustablePromise?

Evenmoreimportantly,ifyoupassanon-PromisethenablevaluetoPromise.resolve(..),itwillattempttounwrapthatvalue,andtheunwrappingwillkeepgoinguntilaconcretefinalnon-Promise-likevalueisextracted.

Recallourpreviousdiscussionofthenables?

Consider:

varp={

then:function(cb){

cb(42);

}

};

//thisworksOK,butonlybygoodfortune

p

.then(

functionfulfilled(val){

console.log(val);//42

},


//nevergetshere

}

);

Thispisathenable,butit'snotagenuinePromise.Luckily,it'sreasonable,asmostwillbe.Butwhatifyougotbackinsteadsomethingthatlookedlike:

varp={

then:function(cb,errcb){

cb(42);

errcb("evillaugh");

}

};

p

.then(



},


//oops,shouldn'thaverun

console.log(err);//evillaugh

}

);

Thispisathenablebutit'snotsowellbehavedofapromise.Isitmalicious?OrisitjustignorantofhowPromisesshouldwork?Itdoesn'treallymatter,tobehonest.Ineithercase,it'snottrustableasis.

Nonetheless,wecanpasseitheroftheseversionsofptoPromise.resolve(..),andwe'llgetthenormalized,saferesultwe'dexpect:

Promise.resolve(p)

.then(



},


//nevergetshere

}

);

Promise.resolve(..)willacceptanythenable,andwillunwrapittoitsnon-thenablevalue.ButyougetbackfromPromise.resolve(..)areal,genuinePromiseinitsplace,onethatyoucantrust.IfwhatyoupassedinisalreadyagenuinePromise,youjustgetitrightback,sothere'snodownsideatalltofilteringthroughPromise.resolve(..)togaintrust.

Solet'ssaywe'recallingafoo(..)utilityandwe'renotsurewecantrustitsreturnvaluetobeawell-behavingPromise,butweknowit'satleastathenable.Promise.resolve(..)willgiveusatrustablePromisewrappertochainoffof:

//don'tjustdothis:

foo(42)

.then(function(v){

console.log(v);

});

//instead,dothis:

Promise.resolve(foo(42))

.then(function(v){

console.log(v);

});

Note:AnotherbeneficialsideeffectofwrappingPromise.resolve(..)aroundanyfunction'sreturnvalue(thenableornot)isthatit'saneasywaytonormalizethatfunctioncallintoawell-behavingasynctask.Iffoo(42)returnsanimmediatevaluesometimes,oraPromiseothertimes,Promise.resolve(foo(42))makessureit'salwaysaPromiseresult.AndavoidingZalgomakesformuchbettercode.

Hopefullythepreviousdiscussionnowfully"resolves"(punintended)inyourmindwhythePromiseistrustable,andmoreimportantly,whythattrustissocriticalinbuildingrobust,maintainablesoftware.

CanyouwriteasynccodeinJSwithouttrust?Ofcourseyoucan.WeJSdevelopershavebeencodingasyncwithnothingbutcallbacksfornearlytwodecades.

Butonceyoustartquestioningjusthowmuchyoucantrustthemechanismsyoubuildupontoactuallybepredictableandreliable,youstarttorealizecallbackshaveaprettyshakytrustfoundation.

Promisesareapatternthataugmentscallbackswithtrustablesemantics,sothatthebehaviorismorereason-ableandmorereliable.Byuninvertingtheinversionofcontrolofcallbacks,weplacethecontrolwithatrustablesystem(Promises)thatwasdesignedspecificallytobringsanitytoourasync.

We'vehintedatthisacoupleoftimesalready,butPromisesarenotjustamechanismforasingle-stepthis-then-thatsortofoperation.That'sthebuildingblock,ofcourse,butitturnsoutwecanstringmultiplePromisestogethertorepresentasequenceofasyncsteps.

ThekeytomakingthisworkisbuiltontwobehaviorsintrinsictoPromises:

Everytimeyoucallthen(..)onaPromise,itcreatesandreturnsanewPromise,whichwecanchainwith.Whatevervalueyoureturnfromthethen(..)call'sfulfillmentcallback(thefirstparameter)isautomaticallysetasthefulfillmentofthechainedPromise(fromthefirstpoint).

Let'sfirstillustratewhatthatmeans,andthenwe'llderivehowthathelpsuscreateasyncsequencesofflowcontrol.Considerthefollowing:

varp=Promise.resolve(21);

varp2=p.then(function(v){

console.log(v);//21

//fulfill`p2`withvalue`42`

returnv*2;

});

//chainoff`p2`


TrustBuilt

ChainFlow

console.log(v);//42

});

Byreturningv*2(i.e.,42),wefulfillthep2promisethatthefirstthen(..)callcreatedandreturned.Whenp2'sthen(..)callruns,it'sreceivingthefulfillmentfromthereturnv*2statement.Ofcourse,p2.then(..)createsyetanotherpromise,whichwecouldhavestoredinap3variable.

Butit'salittleannoyingtohavetocreateanintermediatevariablep2(orp3,etc.).Thankfully,wecaneasilyjustchainthesetogether:


p

.then(function(v){

console.log(v);//21

//fulfillthechainedpromisewithvalue`42`

returnv*2;

})

//here'sthechainedpromise

.then(function(v){

console.log(v);//42

});

Sonowthefirstthen(..)isthefirststepinanasyncsequence,andthesecondthen(..)isthesecondstep.Thiscouldkeepgoingforaslongasyouneededittoextend.Justkeepchainingoffapreviousthen(..)witheachautomaticallycreatedPromise.

Butthere'ssomethingmissinghere.Whatifwewantstep2towaitforstep1todosomethingasynchronous?We'reusinganimmediatereturnstatement,whichimmediatelyfulfillsthechainedpromise.

ThekeytomakingaPromisesequencetrulyasynccapableateverystepistorecallhowPromise.resolve(..)operateswhenwhatyoupasstoitisaPromiseorthenableinsteadofafinalvalue.Promise.resolve(..)directlyreturnsareceivedgenuinePromise,oritunwrapsthevalueofareceivedthenable--andkeepsgoingrecursivelywhileitkeepsunwrappingthenables.

ThesamesortofunwrappinghappensifyoureturnathenableorPromisefromthefulfillment(orrejection)handler.Consider:


p.then(function(v){

console.log(v);//21

//createapromiseandreturnit


//fulfillwithvalue`42`

resolve(v*2);

});

})

.then(function(v){

console.log(v);//42

});

Eventhoughwewrapped42upinapromisethatwereturned,itstillgotunwrappedandendedupastheresolutionofthechainedpromise,suchthatthesecondthen(..)stillreceived42.Ifweintroduceasynchronytothatwrappingpromise,everythingstillnicelyworksthesame:


p.then(function(v){

console.log(v);//21

//createapromisetoreturn


//introduceasynchrony!


//fulfillwithvalue`42`

resolve(v*2);

},100);

});

})

.then(function(v){

//runsafterthe100msdelayinthepreviousstep

console.log(v);//42

});

That'sincrediblypowerful!Nowwecanconstructasequenceofhowevermanyasyncstepswewant,andeachstepcandelaythenextstep(ornot!),asnecessary.

Ofcourse,thevaluepassingfromsteptostepintheseexamplesisoptional.Ifyoudon'treturnanexplicitvalue,animplicitundefinedisassumed,andthepromisesstillchaintogetherthesameway.EachPromiseresolutionisthusjustasignaltoproceedtothenextstep.

Tofurtherthechainillustration,let'sgeneralizeadelay-Promisecreation(withoutresolutionmessages)intoautilitywecanreuseformultiplesteps:

functiondelay(time){


setTimeout(resolve,time);

});

}

delay(100)//step1

.then(functionSTEP2(){

console.log("step2(after100ms)");

returndelay(200);

})


console.log("step3(afteranother200ms)");

})


console.log("step4(nextJob)");

returndelay(50);

})


console.log("step5(afteranother50ms)");

})

...

Callingdelay(200)createsapromisethatwillfulfillin200ms,andthenwereturnthatfromthefirstthen(..)fulfillmentcallback,whichcausesthesecondthen(..)'spromisetowaitonthat200mspromise.

Note:Asdescribed,technicallytherearetwopromisesinthatinterchange:the200ms-delaypromiseandthechainedpromisethatthesecondthen(..)chainsfrom.Butyoumayfinditeasiertomentallycombinethesetwopromisestogether,becausethePromisemechanismautomaticallymergestheirstatesforyou.Inthatrespect,youcouldthinkofreturndelay(200)ascreatingapromisethatreplacestheearlier-returnedchainedpromise.

Tobehonest,though,sequencesofdelayswithnomessagepassingisn'taterriblyusefulexampleofPromiseflowcontrol.Let'slookatascenariothat'salittlemorepractical.

Insteadoftimers,let'sconsidermakingAjaxrequests:

//assumeanàjax({url},{callback})ùtility

//Promise-awareajax

functionrequest(url){


//theàjax(..)`callbackshouldbeour

//promise's`resolve(..)`function

ajax(url,resolve);

});

}

Wefirstdefinearequest(..)utilitythatconstructsapromisetorepresentthecompletionoftheajax(..)call:

request("http://some.url.1/")

.then(function(response1){

returnrequest("http://some.url.2/?v="+response1);

})


console.log(response2);

});

Note:DeveloperscommonlyencountersituationsinwhichtheywanttodoPromise-awareasyncflowcontrolwithutilitiesthatarenotthemselvesPromise-enabled(likeajax(..)here,whichexpectsacallback).AlthoughthenativeES6Promisemechanismdoesn'tautomaticallysolvethispatternforus,practicallyallPromiselibrariesdo.Theyusuallycallthisprocess"lifting"or"promisifying"orsomevariationthereof.We'llcomebacktothistechniquelater.

UsingthePromise-returningrequest(..),wecreatethefirststepinourchainimplicitlybycallingitwiththefirstURL,andchainoffthatreturnedpromisewiththefirstthen(..).

Onceresponse1comesback,weusethatvaluetoconstructasecondURL,andmakeasecondrequest(..)call.Thatsecondrequest(..)promiseisreturnedsothatthethirdstepinourasyncflowcontrolwaitsforthatAjaxcalltocomplete.Finally,weprintresponse2onceitreturns.

ThePromisechainweconstructisnotonlyaflowcontrolthatexpressesamultistepasyncsequence,butitalsoactsasamessagechanneltopropagatemessagesfromsteptostep.

WhatifsomethingwentwronginoneofthestepsofthePromisechain?Anerror/exceptionisonaper-Promisebasis,whichmeansit'spossibletocatchsuchanerroratanypointinthechain,andthatcatchingactstosortof"reset"thechainbacktonormaloperationatthatpoint:

//step1:


//step2:


foo.bar();//undefined,error!

//nevergetshere

returnrequest("http://some.url.2/?v="+response1);

})

//step3:

.then(

functionfulfilled(response2){

//nevergetshere

},

//rejectionhandlertocatchtheerror


console.log(err);//`TypeError`from`foo.bar()èrror

return42;

}

)

//step4:

.then(function(msg){

console.log(msg);//42

});

Whentheerroroccursinstep2,therejectionhandlerinstep3catchesit.Thereturnvalue(42inthissnippet),ifany,fromthatrejectionhandlerfulfillsthepromiseforthenextstep(4),suchthatthechainisnowbackinafulfillmentstate.

Note:Aswediscussedearlier,whenreturningapromisefromafulfillmenthandler,it'sunwrappedandcandelaythenextstep.That'salsotrueforreturningpromisesfromrejectionhandlers,suchthatifthereturn42instep3insteadreturneda

promise,thatpromisecoulddelaystep4.Athrownexceptioninsideeitherthefulfillmentorrejectionhandlerofathen(..)callcausesthenext(chained)promisetobeimmediatelyrejectedwiththatexception.

Ifyoucallthen(..)onapromise,andyouonlypassafulfillmenthandlertoit,anassumedrejectionhandlerissubstituted:


reject("Oops");

});

varp2=p.then(


//nevergetshere

}

//assumedrejectionhandler,ifomittedor

//anyothernon-functionvaluepassed

//function(err){

//throwerr;

//}

);

Asyoucansee,theassumedrejectionhandlersimplyrethrowstheerror,whichendsupforcingp2(thechainedpromise)torejectwiththesameerrorreason.Inessence,thisallowstheerrortocontinuepropagatingalongaPromisechainuntilanexplicitlydefinedrejectionhandlerisencountered.

Note:We'llcovermoredetailsoferrorhandlingwithPromisesalittlelater,becausethereareothernuanceddetailstobeconcernedabout.

Ifapropervalidfunctionisnotpassedasthefulfillmenthandlerparametertothen(..),there'salsoadefaulthandlersubstituted:


p.then(

//assumedfulfillmenthandler,ifomittedor

//anyothernon-functionvaluepassed

//function(v){

//returnv;

//}

null,


//nevergetshere

}

);

Asyoucansee,thedefaultfulfillmenthandlersimplypasseswhatevervalueitreceivesalongtothenextstep(Promise).

Note:Thethen(null,function(err){..})pattern--onlyhandlingrejections(ifany)butlettingfulfillmentspassthrough--hasashortcutintheAPI:catch(function(err){..}).We'llcovercatch(..)morefullyinthenextsection.

Let'sreviewbrieflytheintrinsicbehaviorsofPromisesthatenablechainingflowcontrol:

Athen(..)callagainstonePromiseautomaticallyproducesanewPromisetoreturnfromthecall.Insidethefulfillment/rejectionhandlers,ifyoureturnavalueoranexceptionisthrown,thenewreturned(chainable)Promiseisresolvedaccordingly.IfthefulfillmentorrejectionhandlerreturnsaPromise,itisunwrapped,sothatwhateveritsresolutioniswillbecometheresolutionofthechainedPromisereturnedfromthecurrentthen(..).

Whilechainingflowcontrolishelpful,it'sprobablymostaccuratetothinkofitasasidebenefitofhowPromisescompose(combine)together,ratherthanthemainintent.Aswe'vediscussedindetailseveraltimesalready,Promisesnormalizeasynchronyandencapsulatetime-dependentvaluestate,andthatiswhatletsuschainthemtogetherinthisusefulway.

Certainly,thesequentialexpressivenessofthechain(this-then-this-then-this...)isabigimprovementoverthetangledmessofcallbacksasweidentifiedinChapter2.Butthere'sstillafairamountofboilerplate(then(..)andfunction(){..})to

wadethrough.Inthenextchapter,we'llseeasignificantlynicerpatternforsequentialflowcontrolexpressivity,withgenerators.

There'ssomeslightconfusionaroundtheterms"resolve,""fulfill,"and"reject"thatweneedtoclearup,beforeyougettoomuchdeeperintolearningaboutPromises.Let'sfirstconsiderthePromise(..)constructor:

varp=newPromise(function(X,Y){

//X()forfulfillment

//Y()forrejection

});

Asyoucansee,twocallbacks(herelabeledXandY)areprovided.ThefirstisusuallyusedtomarkthePromiseasfulfilled,andthesecondalwaysmarksthePromiseasrejected.Butwhat'sthe"usually"about,andwhatdoesthatimplyaboutaccuratelynamingthoseparameters?

Ultimately,it'sjustyourusercodeandtheidentifiernamesaren'tinterpretedbytheenginetomeananything,soitdoesn'ttechnicallymatter;foo(..)andbar(..)areequallyfunctional.Butthewordsyouusecanaffectnotonlyhowyouarethinkingaboutthecode,buthowotherdevelopersonyourteamwillthinkaboutit.Thinkingwronglyaboutcarefullyorchestratedasynccodeisalmostsurelygoingtobeworsethanthespaghetti-callbackalternatives.

Soitactuallydoeskindofmatterwhatyoucallthem.

Thesecondparameteriseasytodecide.Almostallliteratureusesreject(..)asitsname,andbecausethat'sexactly(andonly!)whatitdoes,that'saverygoodchoiceforthename.I'dstronglyrecommendyoualwaysusereject(..).

Butthere'salittlemoreambiguityaroundthefirstparameter,whichinPromiseliteratureisoftenlabeledresolve(..).Thatwordisobviouslyrelatedto"resolution,"whichiswhat'susedacrosstheliterature(includingthisbook)todescribesettingafinalvalue/statetoaPromise.We'vealreadyused"resolvethePromise"severaltimestomeaneitherfulfillingorrejectingthePromise.

ButifthisparameterseemstobeusedtospecificallyfulfillthePromise,whyshouldn'twecallitfulfill(..)insteadofresolve(..)tobemoreaccurate?Toanswerthatquestion,let'salsotakealookattwoofthePromiseAPImethods:

varfulfilledPr=Promise.resolve(42);

varrejectedPr=Promise.reject("Oops");

Promise.resolve(..)createsaPromisethat'sresolvedtothevaluegiventoit.Inthisexample,42isanormal,non-Promise,non-thenablevalue,sothefulfilledpromisefulfilledPriscreatedforthevalue42.Promise.reject("Oops")createstherejectedpromiserejectedPrforthereason"Oops".

Let'snowillustratewhytheword"resolve"(suchasinPromise.resolve(..))isunambiguousandindeedmoreaccurate,ifusedexplicitlyinacontextthatcouldresultineitherfulfillmentorrejection:

varrejectedTh={

then:function(resolved,rejected){

rejected("Oops");

}

};

varrejectedPr=Promise.resolve(rejectedTh);

Aswediscussedearlierinthischapter,Promise.resolve(..)willreturnareceivedgenuinePromisedirectly,orunwrapareceivedthenable.Ifthatthenableunwrappingrevealsarejectedstate,thePromisereturnedfromPromise.resolve(..)isinfactinthatsamerejectedstate.

Terminology:Resolve,Fulfill,andReject

SoPromise.resolve(..)isagood,accuratenamefortheAPImethod,becauseitcanactuallyresultineitherfulfillmentorrejection.

ThefirstcallbackparameterofthePromise(..)constructorwillunwrapeitherathenable(identicallytoPromise.resolve(..))oragenuinePromise:

varrejectedPr=newPromise(function(resolve,reject){

//resolvethispromisewitharejectedpromise

resolve(Promise.reject("Oops"));

});

rejectedPr.then(


//nevergetshere

},


console.log(err);//"Oops"

}

);

Itshouldbeclearnowthatresolve(..)istheappropriatenameforthefirstcallbackparameterofthePromise(..)constructor.

Warning:Thepreviouslymentionedreject(..)doesnotdotheunwrappingthatresolve(..)does.IfyoupassaPromise/thenablevaluetoreject(..),thatuntouchedvaluewillbesetastherejectionreason.AsubsequentrejectionhandlerwouldreceivetheactualPromise/thenableyoupassedtoreject(..),notitsunderlyingimmediatevalue.

Butnowlet'sturnourattentiontothecallbacksprovidedtothen(..).Whatshouldtheybecalled(bothinliteratureandincode)?Iwouldsuggestfulfilled(..)andrejected(..):


console.log(msg);

}


console.error(err);

}

p.then(

fulfilled,

rejected

);

Inthecaseofthefirstparametertothen(..),it'sunambiguouslyalwaysthefulfillmentcase,sothere'snoneedforthedualityof"resolve"terminology.Asasidenote,theES6specificationusesonFulfilled(..)andonRejected(..)tolabelthesetwocallbacks,sotheyareaccurateterms.

We'vealreadyseenseveralexamplesofhowPromiserejection--eitherintentionalthroughcallingreject(..)oraccidentalthroughJSexceptions--allowssanererrorhandlinginasynchronousprogramming.Let'scirclebackthoughandbeexplicitaboutsomeofthedetailsthatweglossedover.

Themostnaturalformoferrorhandlingformostdevelopersisthesynchronoustry..catchconstruct.Unfortunately,it'ssynchronous-only,soitfailstohelpinasynccodepatterns:

functionfoo(){


baz.bar();

},100);

}

ErrorHandling

try{

foo();

//laterthrowsglobalerrorfrom`baz.bar()`

}

catch(err){

//nevergetshere

}

try..catchwouldcertainlybenicetohave,butitdoesn'tworkacrossasyncoperations.Thatis,unlessthere'ssomeadditionalenvironmentalsupport,whichwe'llcomebacktowithgeneratorsinChapter4.

Incallbacks,somestandardshaveemergedforpatternederrorhandling,mostnotablythe"error-firstcallback"style:

functionfoo(cb){


try{

varx=baz.bar();

cb(null,x);//success!

}

catch(err){

cb(err);

}

},100);

}

foo(function(err,val){

if(err){

console.error(err);//bummer:(

}

else{

console.log(val);

}

});

Note:Thetry..catchhereworksonlyfromtheperspectivethatthebaz.bar()callwilleithersucceedorfailimmediately,synchronously.Ifbaz.bar()wasitselfitsownasynccompletingfunction,anyasyncerrorsinsideitwouldnotbecatchable.

Thecallbackwepasstofoo(..)expectstoreceiveasignalofanerrorbythereservedfirstparametererr.Ifpresent,errorisassumed.Ifnot,successisassumed.

Thissortoferrorhandlingistechnicallyasynccapable,butitdoesn'tcomposewellatall.Multiplelevelsoferror-firstcallbackswoventogetherwiththeseubiquitousifstatementchecksinevitablywillleadyoutotheperilsofcallbackhell(seeChapter2).

SowecomebacktoerrorhandlinginPromises,withtherejectionhandlerpassedtothen(..).Promisesdon'tusethepopular"error-firstcallback"designstyle,butinsteaduse"splitcallbacks"style;there'sonecallbackforfulfillmentandoneforrejection:

varp=Promise.reject("Oops");

p.then(


//nevergetshere

},


console.log(err);//"Oops"

}

);

Whilethispatternoferrorhandlingmakesfinesenseonthesurface,thenuancesofPromiseerrorhandlingareoftenafairbitmoredifficulttofullygrasp.

Consider:


p.then(


//numbersdon'thavestringfunctions,

//sowillthrowanerror

console.log(msg.toLowerCase());

},


//nevergetshere

}

);

Ifthemsg.toLowerCase()legitimatelythrowsanerror(itdoes!),whydoesn'tourerrorhandlergetnotified?Asweexplainedearlier,it'sbecausethaterrorhandlerisfortheppromise,whichhasalreadybeenfulfilledwithvalue42.Theppromiseisimmutable,sotheonlypromisethatcanbenotifiedoftheerroristheonereturnedfromp.then(..),whichinthiscasewedon'tcapture.

ThatshouldpaintaclearpictureofwhyerrorhandlingwithPromisesiserror-prone(punintended).It'sfartooeasytohaveerrorsswallowed,asthisisveryrarelywhatyou'dintend.

Warning:IfyouusethePromiseAPIinaninvalidwayandanerroroccursthatpreventsproperPromiseconstruction,theresultwillbeanimmediatelythrownexception,notarejectedPromise.SomeexamplesofincorrectusagethatfailPromiseconstruction:newPromise(null),Promise.all(),Promise.race(42),andsoon.Youcan'tgetarejectedPromiseifyoudon'tusethePromiseAPIvalidlyenoughtoactuallyconstructaPromiseinthefirstplace!

JeffAtwoodnotedyearsago:programminglanguagesareoftensetupinsuchawaythatbydefault,developersfallintothe"pitofdespair"(http://blog.codinghorror.com/falling-into-the-pit-of-success/)--whereaccidentsarepunished--andthatyouhavetotryhardertogetitright.Heimploredustoinsteadcreatea"pitofsuccess,"wherebydefaultyoufallintoexpected(successful)action,andthuswouldhavetotryhardtofail.

Promiseerrorhandlingisunquestionably"pitofdespair"design.Bydefault,itassumesthatyouwantanyerrortobeswallowedbythePromisestate,andifyouforgettoobservethatstate,theerrorsilentlylanguishes/diesinobscurity--usuallydespair.

Toavoidlosinganerrortothesilenceofaforgotten/discardedPromise,somedevelopershaveclaimedthata"bestpractice"forPromisechainsistoalwaysendyourchainwithafinalcatch(..),like:


p.then(





}

)

.catch(handleErrors);

Becausewedidn'tpassarejectionhandlertothethen(..),thedefaulthandlerwassubstituted,whichsimplypropagatestheerrortothenextpromiseinthechain.Assuch,botherrorsthatcomeintop,anderrorsthatcomeafterpinitsresolution(likethemsg.toLowerCase()one)willfilterdowntothefinalhandleErrors(..).

Problemsolved,right?Notsofast!

WhathappensifhandleErrors(..)itselfalsohasanerrorinit?Whocatchesthat?There'sstillyetanotherunattendedpromise:theonecatch(..)returns,whichwedon'tcaptureanddon'tregisterarejectionhandlerfor.

Youcan'tjuststickanothercatch(..)ontheendofthatchain,becauseittoocouldfail.ThelaststepinanyPromisechain,whateveritis,alwayshasthepossibility,evendecreasinglyso,ofdanglingwithanuncaughterrorstuckinsideanunobservedPromise.

PitofDespair

http://blog.codinghorror.com/falling-into-the-pit-of-success/

Soundlikeanimpossibleconundrumyet?

It'snotexactlyaneasyproblemtosolvecompletely.Thereareotherwaystoapproachitwhichmanywouldsayarebetter.

SomePromiselibrarieshaveaddedmethodsforregisteringsomethinglikea"globalunhandledrejection"handler,whichwouldbecalledinsteadofagloballythrownerror.Buttheirsolutionforhowtoidentifyanerroras"uncaught"istohaveanarbitrary-lengthtimer,say3seconds,runningfromtimeofrejection.IfaPromiseisrejectedbutnoerrorhandlerisregisteredbeforethetimerfires,thenit'sassumedthatyouwon'teverberegisteringahandler,soit's"uncaught."

Inpractice,thishasworkedwellformanylibraries,asmostusagepatternsdon'ttypicallycallforsignificantdelaybetweenPromiserejectionandobservationofthatrejection.Butthispatternistroublesomebecause3secondsissoarbitrary(evenifempirical),andalsobecausethereareindeedsomecaseswhereyouwantaPromisetoholdontoitsrejectednessforsomeindefiniteperiodoftime,andyoudon'treallywanttohaveyour"uncaught"handlercalledforallthosefalsepositives(not-yet-handled"uncaughterrors").

AnothermorecommonsuggestionisthatPromisesshouldhaveadone(..)addedtothem,whichessentiallymarksthePromisechainas"done."done(..)doesn'tcreateandreturnaPromise,sothecallbackspassedtodone(..)areobviouslynotwireduptoreportproblemstoachainedPromisethatdoesn'texist.

Sowhathappensinstead?It'streatedasyoumightusuallyexpectinuncaughterrorconditions:anyexceptioninsideadone(..)rejectionhandlerwouldbethrownasaglobaluncaughterror(inthedeveloperconsole,basically):


p.then(





}

)

.done(null,handleErrors);

//if`handleErrors(..)`causeditsownexception,itwould

//bethrowngloballyhere

Thismightsoundmoreattractivethanthenever-endingchainorthearbitrarytimeouts.Butthebiggestproblemisthatit'snotpartoftheES6standard,sonomatterhowgooditsounds,atbestit'salotlongerwayofffrombeingareliableandubiquitoussolution.

Arewejuststuck,then?Notentirely.

Browsershaveauniquecapabilitythatourcodedoesnothave:theycantrackandknowforsurewhenanyobjectgetsthrownawayandgarbagecollected.So,browserscantrackPromiseobjects,andwhenevertheygetgarbagecollected,iftheyhavearejectioninthem,thebrowserknowsforsurethiswasalegitimate"uncaughterror,"andcanthusconfidentlyknowitshouldreportittothedeveloperconsole.

Note:Atthetimeofthiswriting,bothChromeandFirefoxhaveearlyattemptsatthatsortof"uncaughtrejection"capability,thoughsupportisincompleteatbest.

However,ifaPromisedoesn'tgetgarbagecollected--it'sexceedinglyeasyforthattoaccidentallyhappenthroughlotsofdifferentcodingpatterns--thebrowser'sgarbagecollectionsniffingwon'thelpyouknowanddiagnosethatyouhaveasilentlyrejectedPromiselayingaround.

Isthereanyotheralternative?Yes.

UncaughtHandling

PitofSuccess

Thefollowingisjusttheoretical,howPromisescouldbesomedaychangedtobehave.Ibelieveitwouldbefarsuperiortowhatwecurrentlyhave.AndIthinkthischangewouldbepossibleevenpost-ES6becauseIdon'tthinkitwouldbreakwebcompatibilitywithES6Promises.Moreover,itcanbepolyfilled/prollyfilledin,ifyou'recareful.Let'stakealook:

Promisescoulddefaulttoreporting(tothedeveloperconsole)anyrejection,onthenextJoboreventlooptick,ifatthatexactmomentnoerrorhandlerhasbeenregisteredforthePromise.ForthecaseswhereyouwantarejectedPromisetoholdontoitsrejectedstateforanindefiniteamountoftimebeforeobserving,youcouldcalldefer(),whichsuppressesautomaticerrorreportingonthatPromise.

IfaPromiseisrejected,itdefaultstonoisilyreportingthatfacttothedeveloperconsole(insteadofdefaultingtosilence).Youcanoptoutofthatreportingeitherimplicitly(byregisteringanerrorhandlerbeforerejection),orexplicitly(withdefer()).Ineithercase,youcontrolthefalsepositives.

Consider:

varp=Promise.reject("Oops").defer();

//`foo(..)ìsPromise-aware

foo(42)

.then(


returnp;

},


//handle`foo(..)èrror

}

);

...

Whenwecreatep,weknowwe'regoingtowaitawhiletouse/observeitsrejection,sowecalldefer()--thusnoglobalreporting.defer()simplyreturnsthesamepromise,forchainingpurposes.

Thepromisereturnedfromfoo(..)getsanerrorhandlerattachedrightaway,soit'simplicitlyoptedoutandnoglobalreportingforitoccurseither.

Butthepromisereturnedfromthethen(..)callhasnodefer()orerrorhandlerattached,soifitrejects(frominsideeitherresolutionhandler),thenitwillbereportedtothedeveloperconsoleasanuncaughterror.

Thisdesignisapitofsuccess.Bydefault,allerrorsareeitherhandledorreported--whatalmostalldevelopersinalmostallcaseswouldexpect.Youeitherhavetoregisterahandleroryouhavetointentionallyoptout,andindicateyouintendtodefererrorhandlinguntillater;you'reoptingfortheextraresponsibilityinjustthatspecificcase.

Theonlyrealdangerinthisapproachisifyoudefer()aPromisebutthenfailtoactuallyeverobserve/handleitsrejection.

Butyouhadtointentionallycalldefer()tooptintothatpitofdespair--thedefaultwasthepitofsuccess--sothere'snotmuchelsewecoulddotosaveyoufromyourownmistakes.

Ithinkthere'sstillhopeforPromiseerrorhandling(post-ES6).Ihopethepowersthatbewillrethinkthesituationandconsiderthisalternative.Inthemeantime,youcanimplementthisyourself(achallengingexerciseforthereader!),oruseasmarterPromiselibrarythatdoessoforyou!

Note:Thisexactmodelforerrorhandling/reportingisimplementedinmyasynquencePromiseabstractionlibrary,whichwillbediscussedinAppendixAofthisbook.

We'vealreadyimplicitlyseenthesequencepatternwithPromisechains(this-then-this-then-thatflowcontrol)buttherearelotsofvariationsonasynchronouspatternsthatwecanbuildasabstractionsontopofPromises.Thesepatternsservetosimplifytheexpressionofasyncflowcontrol--whichhelpsmakeourcodemorereason-ableandmoremaintainable--

PromisePatterns

eveninthemostcomplexpartsofourprograms.

TwosuchpatternsarecodifieddirectlyintothenativeES6Promiseimplementation,sowegetthemforfree,touseasbuildingblocksforotherpatterns.

Inanasyncsequence(Promisechain),onlyoneasynctaskisbeingcoordinatedatanygivenmoment--step2strictlyfollowsstep1,andstep3strictlyfollowsstep2.Butwhataboutdoingtwoormorestepsconcurrently(aka"inparallel")?

Inclassicprogrammingterminology,a"gate"isamechanismthatwaitsontwoormoreparallel/concurrenttaskstocompletebeforecontinuing.Itdoesn'tmatterwhatordertheyfinishin,justthatallofthemhavetocompleteforthegatetoopenandlettheflowcontrolthrough.

InthePromiseAPI,wecallthispatternall([..]).

SayyouwantedtomaketwoAjaxrequestsatthesametime,andwaitforbothtofinish,regardlessoftheirorder,beforemakingathirdAjaxrequest.Consider:

//`request(..)ìsaPromise-awareAjaxutility,

//likewedefinedearlierinthechapter

varp1=request("http://some.url.1/");


Promise.all([p1,p2])

.then(function(msgs){

//both`p1ànd`p2`fulfillandpassin

//theirmessageshere

returnrequest(

"http://some.url.3/?v="+msgs.join(",")

);

})


console.log(msg);

});

Promise.all([..])expectsasingleargument,anarray,consistinggenerallyofPromiseinstances.ThepromisereturnedfromthePromise.all([..])callwillreceiveafulfillmentmessage(msgsinthissnippet)thatisanarrayofallthefulfillmentmessagesfromthepassedinpromises,inthesameorderasspecified(regardlessoffulfillmentorder).

Note:Technically,thearrayofvaluespassedintoPromise.all([..])canincludePromises,thenables,orevenimmediatevalues.EachvalueinthelistisessentiallypassedthroughPromise.resolve(..)tomakesureit'sagenuinePromisetobewaitedon,soanimmediatevaluewilljustbenormalizedintoaPromiseforthatvalue.Ifthearrayisempty,themainPromiseisimmediatelyfulfilled.

ThemainpromisereturnedfromPromise.all([..])willonlybefulfilledifandwhenallitsconstituentpromisesarefulfilled.Ifanyoneofthosepromisesinsteadisrejected,themainPromise.all([..])promiseisimmediatelyrejected,discardingallresultsfromanyotherpromises.

Remembertoalwaysattacharejection/errorhandlertoeverypromise,evenandespeciallytheonethatcomesbackfromPromise.all([..]).

WhilePromise.all([..])coordinatesmultiplePromisesconcurrentlyandassumesallareneededforfulfillment,sometimesyouonlywanttorespondtothe"firstPromisetocrossthefinishline,"lettingtheotherPromisesfallaway.

Thispatternisclassicallycalleda"latch,"butinPromisesit'scalleda"race."

Warning:Whilethemetaphorof"onlythefirstacrossthefinishlinewins"fitsthebehaviorwell,unfortunately"race"iskind

Promise.all([..])

Promise.race([..])

ofaloadedterm,because"raceconditions"aregenerallytakenasbugsinprograms(seeChapter1).Don'tconfusePromise.race([..])with"racecondition."

Promise.race([..])alsoexpectsasinglearrayargument,containingoneormorePromises,thenables,orimmediatevalues.Itdoesn'tmakemuchpracticalsensetohavearacewithimmediatevalues,becausethefirstonelistedwillobviouslywin--likeafootracewhereonerunnerstartsatthefinishline!

SimilartoPromise.all([..]),Promise.race([..])willfulfillifandwhenanyPromiseresolutionisafulfillment,anditwillrejectifandwhenanyPromiseresolutionisarejection.

Warning:A"race"requiresatleastone"runner,"soifyoupassanemptyarray,insteadofimmediatelyresolving,themainrace([..])Promisewillneverresolve.Thisisafootgun!ES6shouldhavespecifiedthatiteitherfulfills,rejects,orjustthrowssomesortofsynchronouserror.Unfortunately,becauseofprecedenceinPromiselibrariespredatingES6Promise,theyhadtoleavethisgotchainthere,sobecarefulnevertosendinanemptyarray.

Let'srevisitourpreviousconcurrentAjaxexample,butinthecontextofaracebetweenp1andp2:

//`request(..)ìsaPromise-awareAjaxutility,

//likewedefinedearlierinthechapter



Promise.race([p1,p2])


//either`p1òr`p2`willwintherace

returnrequest(

"http://some.url.3/?v="+msg

);

})


console.log(msg);

});

Becauseonlyonepromisewins,thefulfillmentvalueisasinglemessage,notanarrayasitwasforPromise.all([..]).

Wesawthisexampleearlier,illustratinghowPromise.race([..])canbeusedtoexpressthe"promisetimeout"pattern:

//`foo()ìsaPromise-awarefunction

//`timeoutPromise(..)`,definedealier,returns

//aPromisethatrejectsafteraspecifieddelay

//setupatimeoutfor`foo()`

Promise.race([



])

.then(

function(){

//`foo(..)`fulfilledintime!

},

function(err){

//either`foo()`rejected,oritjust

//didn'tfinishintime,soinspect

//èrr`toknowwhich

}

);

Thistimeoutpatternworkswellinmostcases.Buttherearesomenuancestoconsider,andfranklytheyapplytobothPromise.race([..])andPromise.all([..])equally.

TimeoutRace

"Finally"

Thekeyquestiontoaskis,"Whathappenstothepromisesthatgetdiscarded/ignored?"We'renotaskingthatquestionfromtheperformanceperspective--theywouldtypicallyendupgarbagecollectioneligible--butfromthebehavioralperspective(sideeffects,etc.).Promisescannotbecanceled--andshouldn'tbeasthatwoulddestroytheexternalimmutabilitytrustdiscussedinthe"PromiseUncancelable"sectionlaterinthischapter--sotheycanonlybesilentlyignored.

Butwhatiffoo()inthepreviousexampleisreservingsomesortofresourceforusage,butthetimeoutfiresfirstandcausesthatpromisetobeignored?Isthereanythinginthispatternthatproactivelyfreesthereservedresourceafterthetimeout,orotherwisecancelsanysideeffectsitmayhavehad?Whatifallyouwantedwastologthefactthatfoo()timedout?

SomedevelopershaveproposedthatPromisesneedafinally(..)callbackregistration,whichisalwayscalledwhenaPromiseresolves,andallowsyoutospecifyanycleanupthatmaybenecessary.Thisdoesn'texistinthespecificationatthemoment,butitmaycomeinES7+.We'llhavetowaitandsee.

Itmightlooklike:


p.then(something)

.finally(cleanup)

.then(another)

.finally(cleanup);

Note:InvariousPromiselibraries,finally(..)stillcreatesandreturnsanewPromise(tokeepthechaingoing).Ifthecleanup(..)functionweretoreturnaPromise,itwouldbelinkedintothechain,whichmeansyoucouldstillhavetheunhandledrejectionissueswediscussedearlier.

Inthemeantime,wecouldmakeastatichelperutilitythatletsusobserve(withoutinterfering)theresolutionofaPromise:

//polyfill-safeguardcheck

if(!Promise.observe){

Promise.observe=function(pr,cb){

//side-observe`pr`'sresolution

pr.then(


//schedulecallbackasync(asJob)

Promise.resolve(msg).then(cb);

},


//schedulecallbackasync(asJob)

Promise.resolve(err).then(cb);

}

);

//returnoriginalpromise

returnpr;

};

}

Here'showwe'duseitinthetimeoutexamplefrombefore:

Promise.race([

Promise.observe(


functioncleanup(msg){

//cleanupafter`foo()`,evenifit

//didn'tfinishbeforethetimeout

}

),


])

ThisPromise.observe(..)helperisjustanillustrationofhowyoucouldobservethecompletionsofPromiseswithoutinterferingwiththem.OtherPromiselibrarieshavetheirownsolutions.Regardlessofhowyoudoit,you'lllikelyhaveplaceswhereyouwanttomakesureyourPromisesaren'tjustsilentlyignoredbyaccident.

WhilenativeES6Promisescomewithbuilt-inPromise.all([..])andPromise.race([..]),thereareseveralothercommonlyusedpatternswithvariationsonthosesemantics:

none([..])islikeall([..]),butfulfillmentsandrejectionsaretransposed.AllPromisesneedtoberejected--rejectionsbecomethefulfillmentvaluesandviceversa.any([..])islikeall([..]),butitignoresanyrejections,soonlyoneneedstofulfillinsteadofallofthem.first([..])isalikearacewithany([..]),whichisthatitignoresanyrejectionsandfulfillsassoonasthefirstPromisefulfills.last([..])islikefirst([..]),butonlythelatestfulfillmentwins.

SomePromiseabstractionlibrariesprovidethese,butyoucouldalsodefinethemyourselfusingthemechanicsofPromises,race([..])andall([..]).

Forexample,here'showwecoulddefinefirst([..]):


if(!Promise.first){

Promise.first=function(prs){


//loopthroughallpromises

prs.forEach(function(pr){

//normalizethevalue

Promise.resolve(pr)

//whicheveronefulfillsfirstwins,and

//getstoresolvethemainpromise

.then(resolve);

});

});

};

}

Note:Thisimplementationoffirst(..)doesnotrejectifallitspromisesreject;itsimplyhangs,muchlikeaPromise.race([])does.Ifdesired,youcouldaddadditionallogictotrackeachpromiserejectionandifallreject,callreject()onthemainpromise.We'llleavethatasanexerciseforthereader.

SometimesyouwanttoiterateoveralistofPromisesandperformsometaskagainstallofthem,muchlikeyoucandowithsynchronousarrays(e.g.,forEach(..),map(..),some(..),andevery(..)).IfthetasktoperformagainsteachPromiseisfundamentallysynchronous,theseworkfine,justasweusedforEach(..)intheprevioussnippet.

Butifthetasksarefundamentallyasynchronous,orcan/shouldotherwisebeperformedconcurrently,youcanuseasyncversionsoftheseutilitiesasprovidedbymanylibraries.

Forexample,let'sconsideranasynchronousmap(..)utilitythattakesanarrayofvalues(couldbePromisesoranythingelse),plusafunction(task)toperformagainsteach.map(..)itselfreturnsapromisewhosefulfillmentvalueisanarraythatholds(inthesamemappingorder)theasyncfulfillmentvaluefromeachtask:

if(!Promise.map){

Promise.map=function(vals,cb){

//newpromisethatwaitsforallmappedpromises

returnPromise.all(

//note:regulararray`map(..)`,turns

//thearrayofvaluesintoanarrayof

//promises

vals.map(function(val){

Variationsonall([..])andrace([..])

ConcurrentIterations

//replace`val`withanewpromisethat

//resolvesafter`valìsasyncmapped

returnnewPromise(function(resolve){

cb(val,resolve);

});

})

);

};

}

Note:Inthisimplementationofmap(..),youcan'tsignalasyncrejection,butifasynchronousexception/erroroccursinsideofthemappingcallback(cb(..)),themainPromise.map(..)returnedpromisewouldreject.

Let'sillustrateusingmap(..)withalistofPromises(insteadofsimplevalues):



varp3=Promise.reject("Oops");

//doublevaluesinlistevenifthey'rein

//Promises

Promise.map([p1,p2,p3],function(pr,done){

//makesuretheitemitselfisaPromise

Promise.resolve(pr)

.then(

//extractvalueas`v`

function(v){

//mapfulfillment`v`tonewvalue

done(v*2);

},

//or,maptopromiserejectionmessage

done

);

})

.then(function(vals){

console.log(vals);//[42,84,"Oops"]

});

Let'sreviewtheES6PromiseAPIthatwe'vealreadyseenunfoldinbitsandpiecesthroughoutthischapter.

Note:ThefollowingAPIisnativeonlyasofES6,buttherearespecification-compliantpolyfills(notjustextendedPromiselibraries)whichcandefinePromiseandallitsassociatedbehaviorsothatyoucanusenativePromiseseveninpre-ES6browsers.Onesuchpolyfillis"NativePromiseOnly"(http://github.com/getify/native-promise-only),whichIwrote!

TherevealingconstructorPromise(..)mustbeusedwithnew,andmustbeprovidedafunctioncallbackthatissynchronously/immediatelycalled.Thisfunctionispassedtwofunctioncallbacksthatactasresolutioncapabilitiesforthepromise.Wecommonlylabeltheseresolve(..)andreject(..):


//`resolve(..)`toresolve/fulfillthepromise

//`reject(..)`torejectthepromise

});

reject(..)simplyrejectsthepromise,butresolve(..)caneitherfulfillthepromiseorrejectit,dependingonwhatit'spassed.Ifresolve(..)ispassedanimmediate,non-Promise,non-thenablevalue,thenthepromiseisfulfilledwiththatvalue.

Butifresolve(..)ispassedagenuinePromiseorthenablevalue,thatvalueisunwrappedrecursively,andwhateveritsfinalresolution/stateiswillbeadoptedbythepromise.

PromiseAPIRecap

newPromise(..)Constructor

http://github.com/getify/native-promise-only

Ashortcutforcreatinganalready-rejectedPromiseisPromise.reject(..),sothesetwopromisesareequivalent:


reject("Oops");

});


Promise.resolve(..)isusuallyusedtocreateanalready-fulfilledPromiseinasimilarwaytoPromise.reject(..).However,Promise.resolve(..)alsounwrapsthenablevalues(asdiscusssedseveraltimesalready).Inthatcase,thePromisereturnedadoptsthefinalresolutionofthethenableyoupassedin,whichcouldeitherbefulfillmentorrejection:

varfulfilledTh={

then:function(cb){cb(42);}

};

varrejectedTh={

then:function(cb,errCb){

errCb("Oops");

}

};

varp1=Promise.resolve(fulfilledTh);

varp2=Promise.resolve(rejectedTh);

//`p1`willbeafulfilledpromise

//`p2`willbearejectedpromise

Andremember,Promise.resolve(..)doesn'tdoanythingifwhatyoupassisalreadyagenuinePromise;itjustreturnsthevaluedirectly.Sothere'snooverheadtocallingPromise.resolve(..)onvaluesthatyoudon'tknowthenatureof,ifonehappenstoalreadybeagenuinePromise.

EachPromiseinstance(notthePromiseAPInamespace)hasthen(..)andcatch(..)methods,whichallowregisteringoffulfillmentandrejectionhandlersforthePromise.OncethePromiseisresolved,oneortheotherofthesehandlerswillbecalled,butnotboth,anditwillalwaysbecalledasynchronously(see"Jobs"inChapter1).

then(..)takesoneortwoparameters,thefirstforthefulfillmentcallback,andthesecondfortherejectioncallback.Ifeitherisomittedorisotherwisepassedasanon-functionvalue,adefaultcallbackissubstitutedrespectively.Thedefaultfulfillmentcallbacksimplypassesthemessagealong,whilethedefaultrejectioncallbacksimplyrethrows(propagates)theerrorreasonitreceives.

catch(..)takesonlytherejectioncallbackasaparameter,andautomaticallysubstitutesthedefaultfulfillmentcallback,asjustdiscussed.Inotherwords,it'sequivalenttothen(null,..):

p.then(fulfilled);

p.then(fulfilled,rejected);

p.catch(rejected);//or`p.then(null,rejected)`

then(..)andcatch(..)alsocreateandreturnanewpromise,whichcanbeusedtoexpressPromisechainflowcontrol.Ifthefulfillmentorrejectioncallbackshaveanexceptionthrown,thereturnedpromiseisrejected.Ifeithercallbackreturnsanimmediate,non-Promise,non-thenablevalue,thatvalueissetasthefulfillmentforthereturnedpromise.Ifthefulfillmenthandlerspecificallyreturnsapromiseorthenablevalue,thatvalueisunwrappedandbecomestheresolutionofthereturnedpromise.

Promise.resolve(..)andPromise.reject(..)

then(..)andcatch(..)

ThestatichelpersPromise.all([..])andPromise.race([..])ontheES6PromiseAPIbothcreateaPromiseastheirreturnvalue.Theresolutionofthatpromiseiscontrolledentirelybythearrayofpromisesthatyoupassin.

ForPromise.all([..]),allthepromisesyoupassinmustfulfillforthereturnedpromisetofulfill.Ifanypromiseisrejected,themainreturnedpromiseisimmediatelyrejected,too(discardingtheresultsofanyoftheotherpromises).Forfulfillment,youreceiveanarrayofallthepassedinpromises'fulfillmentvalues.Forrejection,youreceivejustthefirstpromiserejectionreasonvalue.Thispatternisclassicallycalleda"gate":allmustarrivebeforethegateopens.

ForPromise.race([..]),onlythefirstpromisetoresolve(fulfillmentorrejection)"wins,"andwhateverthatresolutionisbecomestheresolutionofthereturnedpromise.Thispatternisclassicallycalleda"latch":firstonetoopenthelatchgetsthrough.Consider:


varp2=Promise.resolve("HelloWorld");


Promise.race([p1,p2,p3])



});

Promise.all([p1,p2,p3])

.catch(function(err){

console.error(err);//"Oops"

});



console.log(msgs);//[42,"HelloWorld"]

});

Warning:Becareful!IfanemptyarrayispassedtoPromise.all([..]),itwillfulfillimmediately,butPromise.race([..])willhangforeverandneverresolve.

TheES6PromiseAPIisprettysimpleandstraightforward.It'satleastgoodenoughtoservethemostbasicofasynccases,andisagoodplacetostartwhenrearrangingyourcodefromcallbackhelltosomethingbetter.

Butthere'sawholelotofasyncsophisticationthatappsoftendemandwhichPromisesthemselveswillbelimitedinaddressing.Inthenextsection,we'lldiveintothoselimitationsasmotivationsforthebenefitofPromiselibraries.

Manyofthedetailswe'lldiscussinthissectionhavealreadybeenalludedtointhischapter,butwe'lljustmakesuretoreviewtheselimitationsspecifically.

WecoveredPromise-flavorederrorhandlingindetailearlierinthischapter.ThelimitationsofhowPromisesaredesigned--howtheychain,specifically--createsaveryeasypitfallwhereanerrorinaPromisechaincanbesilentlyignoredaccidentally.

Butthere'ssomethingelsetoconsiderwithPromiseerrors.BecauseaPromisechainisnothingmorethanitsconstituentPromiseswiredtogether,there'snoentitytorefertotheentirechainasasinglething,whichmeansthere'snoexternalwaytoobserveanyerrorsthatmayoccur.

IfyouconstructaPromisechainthathasnoerrorhandlinginit,anyerroranywhereinthechainwillpropagateindefinitelydownthechain,untilobserved(byregisteringarejectionhandleratsomestep).So,inthatspecificcase,havingareferencetothelastpromiseinthechainisenough(pinthefollowingsnippet),becauseyoucanregisterarejection

Promise.all([..])andPromise.race([..])

PromiseLimitations

SequenceErrorHandling

handlerthere,anditwillbenotifiedofanypropagatederrors:

//`foo(..)`,`STEP2(..)ànd`STEP3(..)àre

//allpromise-awareutilities

varp=foo(42)

.then(STEP2)

.then(STEP3);

Althoughitmayseemsneakilyconfusing,pheredoesn'tpointtothefirstpromiseinthechain(theonefromthefoo(42)call),butinsteadfromthelastpromise,theonethatcomesfromthethen(STEP3)call.

Also,nostepinthepromisechainisobservablydoingitsownerrorhandling.Thatmeansthatyoucouldthenregisterarejectionerrorhandleronp,anditwouldbenotifiedifanyerrorsoccuranywhereinthechain:

p.catch(handleErrors);

Butifanystepofthechaininfactdoesitsownerrorhandling(perhapshidden/abstractedawayfromwhatyoucansee),yourhandleErrors(..)won'tbenotified.Thismaybewhatyouwant--itwas,afterall,a"handledrejection"--butitalsomaynotbewhatyouwant.Thecompletelackofabilitytobenotified(of"alreadyhandled"rejectionerrors)isalimitationthatrestrictscapabilitiesinsomeusecases.

It'sbasicallythesamelimitationthatexistswithatry..catchthatcancatchanexceptionandsimplyswallowit.Sothisisn'talimitationuniquetoPromises,butitissomethingwemightwishtohaveaworkaroundfor.

Unfortunately,manytimesthereisnoreferencekeptfortheintermediatestepsinaPromise-chainsequence,sowithoutsuchreferences,youcannotattacherrorhandlerstoreliablyobservetheerrors.

Promisesbydefinitiononlyhaveasinglefulfillmentvalueorasinglerejectionreason.Insimpleexamples,thisisn'tthatbigofadeal,butinmoresophisticatedscenarios,youmayfindthislimiting.

Thetypicaladviceistoconstructavalueswrapper(suchasanobjectorarray)tocontainthesemultiplemessages.Thissolutionworks,butitcanbequiteawkwardandtedioustowrapandunwrapyourmessageswitheverysinglestepofyourPromisechain.

Sometimesyoucantakethisasasignalthatyoucould/shoulddecomposetheproblemintotwoormorePromises.

Imagineyouhaveautilityfoo(..)thatproducestwovalues(xandy)asynchronously:

functiongetY(x){



resolve((3*x)-1);

},100);

});

}

functionfoo(bar,baz){

varx=bar*baz;

returngetY(x)

.then(function(y){

//wrapbothvaluesintocontainer

return[x,y];

});

}

foo(10,20)

SingleValue

SplittingValues


varx=msgs[0];

vary=msgs[1];

console.log(x,y);//200599

});

First,let'srearrangewhatfoo(..)returnssothatwedon'thavetowrapxandyintoasinglearrayvaluetotransportthroughonePromise.Instead,wecanwrapeachvalueintoitsownpromise:

functionfoo(bar,baz){

varx=bar*baz;

//returnbothpromises

return[

Promise.resolve(x),

getY(x)

];

}

Promise.all(

foo(10,20)

)


varx=msgs[0];

vary=msgs[1];

console.log(x,y);

});

Isanarrayofpromisesreallybetterthananarrayofvaluespassedthroughasinglepromise?Syntactically,it'snotmuchofanimprovement.

ButthisapproachmorecloselyembracesthePromisedesigntheory.It'snoweasierinthefuturetorefactortosplitthecalculationofxandyintoseparatefunctions.It'scleanerandmoreflexibletoletthecallingcodedecidehowtoorchestratethetwopromises--usingPromise.all([..])here,butcertainlynottheonlyoption--ratherthantoabstractsuchdetailsawayinsideoffoo(..).

Thevarx=..andvary=..assignmentsarestillawkwardoverhead.Wecanemploysomefunctionaltrickery(hattiptoReginaldBraithwaite,@raganwaldonTwitter)inahelperutility:

functionspread(fn){

returnFunction.apply.bind(fn,null);

}

Promise.all(

foo(10,20)

)

.then(

spread(function(x,y){


})

)

That'sabitnicer!Ofcourse,youcouldinlinethefunctionalmagictoavoidtheextrahelper:

Promise.all(

foo(10,20)

)

.then(Function.apply.bind(

function(x,y){


},

null

));

Unwrap/SpreadArguments

Thesetricksmaybeneat,butES6hasanevenbetteranswerforus:destructuring.Thearraydestructuringassignmentformlookslikethis:

Promise.all(

foo(10,20)

)


var[x,y]=msgs;


});

Butbestofall,ES6offersthearrayparameterdestructuringform:

Promise.all(

foo(10,20)

)

.then(function([x,y]){


});

We'venowembracedtheone-value-per-Promisemantra,butkeptoursupportingboilerplatetoaminimum!

Note:FormoreinformationonES6destructuringforms,seetheES6&Beyondtitleofthisseries.

OneofthemostintrinsicbehaviorsofPromisesisthataPromisecanonlyberesolvedonce(fulfillmentorrejection).Formanyasyncusecases,you'reonlyretrievingavalueonce,sothisworksfine.

Butthere'salsoalotofasynccasesthatfitintoadifferentmodel--onethat'smoreakintoeventsand/orstreamsofdata.It'snotclearonthesurfacehowwellPromisescanfitintosuchusecases,ifatall.WithoutasignificantabstractionontopofPromises,theywillcompletelyfallshortforhandlingmultiplevalueresolution.

Imagineascenariowhereyoumightwanttofireoffasequenceofasyncstepsinresponsetoastimulus(likeanevent)thatcaninfacthappenmultipletimes,likeabuttonclick.

Thisprobablywon'tworkthewayyouwant:

//`click(..)`bindsthe`"click"èventtoaDOMelement

//`request(..)ìsthepreviouslydefinedPromise-awareAjax


click("#mybtn",resolve);

});

p.then(function(evt){

varbtnID=evt.currentTarget.id;

returnrequest("http://some.url.1/?id="+btnID);

})

.then(function(text){

console.log(text);

});

Thebehaviorhereonlyworksifyourapplicationcallsforthebuttontobeclickedjustonce.Ifthebuttonisclickedasecondtime,theppromisehasalreadybeenresolved,sothesecondresolve(..)callwouldbeignored.

Instead,you'dprobablyneedtoinverttheparadigm,creatingawholenewPromisechainforeacheventfiring:

SingleResolution

click("#mybtn",function(evt){

varbtnID=evt.currentTarget.id;

request("http://some.url.1/?id="+btnID)

.then(function(text){

console.log(text);

});

});

ThisapproachwillworkinthatawholenewPromisesequencewillbefiredoffforeach"click"eventonthebutton.

ButbeyondjusttheuglinessofhavingtodefinetheentirePromisechaininsidetheeventhandler,thisdesigninsomerespectsviolatestheideaofseparationofconcerns/capabilities(SoC).Youmightverywellwanttodefineyoureventhandlerinadifferentplaceinyourcodefromwhereyoudefinetheresponsetotheevent(thePromisechain).That'sprettyawkwardtodointhispattern,withouthelpermechanisms.

Note:Anotherwayofarticulatingthislimitationisthatit'dbeniceifwecouldconstructsomesortof"observable"thatwecansubscribeaPromisechainto.Therearelibrariesthathavecreatedtheseabstractions(suchasRxJS--http://rxjs.codeplex.com/),buttheabstractionscanseemsoheavythatyoucan'tevenseethenatureofPromisesanymore.Suchheavyabstractionbringsimportantquestionstomindsuchaswhether(sansPromises)thesemechanismsareastrustableasPromisesthemselveshavebeendesignedtobe.We'llrevisitthe"Observable"patterninAppendixB.

OneconcretebarriertostartingtousePromisesinyourowncodeisallthecodethatcurrentlyexistswhichisnotalreadyPromise-aware.Ifyouhavelotsofcallback-basedcode,it'sfareasiertojustkeepcodinginthatsamestyle.

"Acodebaseinmotion(withcallbacks)willremaininmotion(withcallbacks)unlessacteduponbyasmart,Promises-awaredeveloper."

Promisesofferadifferentparadigm,andassuch,theapproachtothecodecanbeanywherefromjustalittledifferentto,insomecases,radicallydifferent.Youhavetobeintentionalaboutit,becausePromiseswillnotjustnaturallyshakeoutfromthesameol'waysofdoingcodethathaveservedyouwellthusfar.

Consideracallback-basedscenariolikethefollowing:

functionfoo(x,y,cb){

ajax(

"http://some.url.1/?x="+x+"&y="+y,

cb

);

}

foo(11,31,function(err,text){

if(err){

console.error(err);

}

else{

console.log(text);

}

});

Isitimmediatelyobviouswhatthefirststepsaretoconvertthiscallback-basedcodetoPromise-awarecode?Dependsonyourexperience.Themorepracticeyouhavewithit,themorenaturalitwillfeel.Butcertainly,Promisesdon'tjustadvertiseonthelabelexactlyhowtodoit--there'snoone-size-fits-allanswer--sotheresponsibilityisuptoyou.

Aswe'vecoveredbefore,wedefinitelyneedanAjaxutilitythatisPromise-awareinsteadofcallback-based,whichwecouldcallrequest(..).Youcanmakeyourown,aswehavealready.ButtheoverheadofhavingtomanuallydefinePromise-awarewrappersforeverycallback-basedutilitymakesitlesslikelyyou'llchoosetorefactortoPromise-awarecodingatall.

Promisesoffernodirectanswertothatlimitation.MostPromiselibrariesdoofferahelper,however.Butevenwithouta

Inertia

http://rxjs.codeplex.com/

library,imagineahelperlikethis:


if(!Promise.wrap){

Promise.wrap=function(fn){

returnfunction(){

varargs=[].slice.call(arguments);


fn.apply(

null,

args.concat(function(err,v){

if(err){

reject(err);

}

else{

resolve(v);

}

})

);

});

};

};

}

OK,that'smorethanjustatinytrivialutility.However,althoughitmaylookabitintimidating,it'snotasbadasyou'dthink.Ittakesafunctionthatexpectsanerror-firststylecallbackasitslastparameter,andreturnsanewonethatautomaticallycreatesaPromisetoreturn,andsubstitutesthecallbackforyou,wireduptothePromisefulfillment/rejection.

RatherthanwastetoomuchtimetalkingabouthowthisPromise.wrap(..)helperworks,let'sjustlookathowweuseit:

varrequest=Promise.wrap(ajax);


.then(..)

..

Wow,thatwasprettyeasy!

Promise.wrap(..)doesnotproduceaPromise.ItproducesafunctionthatwillproducePromises.Inasense,aPromise-producingfunctioncouldbeseenasa"Promisefactory."Ipropose"promisory"asthenameforsuchathing("Promise"+"factory").

Theactofwrappingacallback-expectingfunctiontobeaPromise-awarefunctionissometimesreferredtoas"lifting"or"promisifying".Buttheredoesn'tseemtobeastandardtermforwhattocalltheresultantfunctionotherthana"liftedfunction",soIlike"promisory"betterasIthinkit'smoredescriptive.

Note:Promisoryisn'tamade-upterm.It'sarealword,anditsdefinitionmeanstocontainorconveyapromise.That'sexactlywhatthesefunctionsaredoing,soitturnsouttobeaprettyperfectterminologymatch!

So,Promise.wrap(ajax)producesanajax(..)promisorywecallrequest(..),andthatpromisoryproducesPromisesforAjaxresponses.

Ifallfunctionswerealreadypromisories,wewouldn'tneedtomakethemourselves,sotheextrastepisatadbitofashame.Butatleastthewrappingpatternis(usually)repeatablesowecanputitintoaPromise.wrap(..)helperasshowntoaidourpromisecoding.

Sobacktoourearlierexample,weneedapromisoryforbothajax(..)andfoo(..):

//makeapromisoryforàjax(..)`


//refactor`foo(..)`,butkeepitexternally

//callback-basedforcompatibilitywithother

//partsofthecodefornow--onlyuse

//`request(..)`'spromiseinternally.


request(

"http://some.url.1/?x="+x+"&y="+y

)

.then(

functionfulfilled(text){

cb(null,text);

},

cb

);

}

//now,forthiscode'spurposes,makea

//promisoryfor`foo(..)`

varbetterFoo=Promise.wrap(foo);

//andusethepromisory

betterFoo(11,31)

.then(

functionfulfilled(text){

console.log(text);

},


console.error(err);

}

);

Ofcourse,whilewe'rerefactoringfoo(..)touseournewrequest(..)promisory,wecouldjustmakefoo(..)apromisoryitself,insteadofremainingcallback-basedandneedingtomakeandusethesubsequentbetterFoo(..)promisory.Thisdecisionjustdependsonwhetherfoo(..)needstostaycallback-basedcompatiblewithotherpartsofthecodebaseornot.

Consider:

`foo(..)ìsnowalsoapromisorybecauseit

delegatestothe`request(..)`promisory

functionfoo(x,y){

returnrequest(


);

}

foo(11,31)

.then(..)

..

WhileES6Promisesdon'tnativelyshipwithhelpersforsuchpromisorywrapping,mostlibrariesprovidethem,oryoucanmakeyourown.Eitherway,thisparticularlimitationofPromisesisaddressablewithouttoomuchpain(certainlycomparedtothepainofcallbackhell!).

OnceyoucreateaPromiseandregisterafulfillmentand/orrejectionhandlerforit,there'snothingexternalyoucandotostopthatprogressionifsomethingelsehappenstomakethattaskmoot.

Note:ManyPromiseabstractionlibrariesprovidefacilitiestocancelPromises,butthisisaterribleidea!ManydeveloperswishPromiseshadnativelybeendesignedwithexternalcancelationcapability,buttheproblemisthatitwouldletoneconsumer/observerofaPromiseaffectsomeotherconsumer'sabilitytoobservethatsamePromise.Thisviolatesthefuture-value'strustability(externalimmutability),butmoreveristheembodimentofthe"actionatadistance"anti-pattern(http://en.wikipedia.org/wiki/Action_at_a_distance_%28computer_programming%29).Regardlessofhowusefulitseems,itwillactuallyleadyoustraightbackintothesamenightmaresascallbacks.

ConsiderourPromisetimeoutscenariofromearlier:

PromiseUncancelable

http://en.wikipedia.org/wiki/Action_at_a_distance_%28computer_programming%29

varp=foo(42);

Promise.race([

p,

timeoutPromise(3000)

])

.then(

doSomething,

handleError

);

p.then(function(){

//stillhappenseveninthetimeoutcase:(

});

The"timeout"wasexternaltothepromisep,sopitselfkeepsgoing,whichweprobablydon'twant.

Oneoptionistoinvasivelydefineyourresolutioncallbacks:

varOK=true;

varp=foo(42);

Promise.race([

p,

timeoutPromise(3000)

.catch(function(err){

OK=false;

throwerr;

})

])

.then(

doSomething,

handleError

);

p.then(function(){

if(OK){

//onlyhappensifnotimeout!:)

}

});

Thisisugly.Itworks,butit'sfarfromideal.Generally,youshouldtrytoavoidsuchscenarios.

Butifyoucan't,theuglinessofthissolutionshouldbeacluethatcancelationisafunctionalitythatbelongsatahigherlevelofabstractionontopofPromises.I'drecommendyoulooktoPromiseabstractionlibrariesforassistanceratherthanhackingityourself.

Note:MyasynquencePromiseabstractionlibraryprovidesjustsuchanabstractionandanabort()capabilityforthesequence,allofwhichwillbediscussedinAppendixA.

AsinglePromiseisnotreallyaflow-controlmechanism(atleastnotinaverymeaningfulsense),whichisexactlywhatcancelationrefersto;that'swhyPromisecancelationwouldfeelawkward.

Bycontrast,achainofPromisestakencollectivelytogether--whatIliketocalla"sequence"--isaflowcontrolexpression,andthusit'sappropriateforcancelationtobedefinedatthatlevelofabstraction.

NoindividualPromiseshouldbecancelable,butit'ssensibleforasequencetobecancelable,becauseyoudon'tpassaroundasequenceasasingleimmutablevaluelikeyoudowithaPromise.

Thisparticularlimitationisbothsimpleandcomplex.

Comparinghowmanypiecesaremovingwithabasiccallback-basedasynctaskchainversusaPromisechain,it'sclear

PromisePerformance

Promiseshaveafairbitmoregoingon,whichmeanstheyarenaturallyatleastatinybitslower.ThinkbacktojustthesimplelistoftrustguaranteesthatPromisesoffer,ascomparedtotheadhocsolutioncodeyou'dhavetolayerontopofcallbackstoachievethesameprotections.

Moreworktodo,moreguardstoprotect,meansthatPromisesareslowerascomparedtonaked,untrustablecallbacks.Thatmuchisobvious,andprobablysimpletowrapyourbrainaround.

Buthowmuchslower?Well...that'sactuallyprovingtobeanincrediblydifficultquestiontoanswerabsolutely,acrosstheboard.

Frankly,it'skindofanapples-to-orangescomparison,soit'sprobablythewrongquestiontoask.Youshouldactuallycomparewhetheranad-hoccallbacksystemwithallthesameprotectionsmanuallylayeredinisfasterthanaPromiseimplementation.

IfPromiseshavealegitimateperformancelimitation,it'smorethattheydon'treallyofferaline-itemchoiceastowhichtrustabilityprotectionsyouwant/needornot--yougetthemall,always.

Nevertheless,ifwegrantthataPromiseisgenerallyalittlebitslowerthanitsnon-Promise,non-trustablecallbackequivalent--assumingthereareplaceswhereyoufeelyoucanjustifythelackoftrustability--doesthatmeanthatPromisesshouldbeavoidedacrosstheboard,asifyourentireapplicationisdrivenbynothingbutmust-be-utterly-the-fastestcodepossible?

Sanitycheck:ifyourcodeislegitimatelylikethat,isJavaScripteventherightlanguageforsuchtasks?JavaScriptcanbeoptimizedtorunapplicationsveryperformantly(seeChapter5andChapter6).ButisobsessingovertinyperformancetradeoffswithPromises,inlightofallthebenefitstheyoffer,reallyappropriate?

AnothersubtleissueisthatPromisesmakeeverythingasync,whichmeansthatsomeimmediately(synchronously)completestepsstilldeferadvancementofthenextsteptoaJob(seeChapter1).Thatmeansthatit'spossiblethatasequenceofPromisetaskscouldcompleteever-so-slightlyslowerthanthesamesequencewiredupwithcallbacks.

Ofcourse,thequestionhereisthis:arethesepotentialslipsintinyfractionsofperformanceworthalltheotherarticulatedbenefitsofPromiseswe'velaidoutacrossthischapter?

MytakeisthatinvirtuallyallcaseswhereyoumightthinkPromiseperformanceisslowenoughtobeconcerned,it'sactuallyananti-patterntooptimizeawaythebenefitsofPromisetrustabilityandcomposabilitybyavoidingthemaltogether.

Instead,youshoulddefaulttousingthemacrossthecodebase,andthenprofileandanalyzeyourapplication'shot(critical)paths.ArePromisesreallyabottleneck,oraretheyjustatheoreticalslowdown?Onlythen,armedwithactualvalidbenchmarks(seeChapter6)isitresponsibleandprudenttofactoroutthePromisesinjustthoseidentifiedcriticalareas.

Promisesarealittleslower,butinexchangeyou'regettingalotoftrustability,non-Zalgopredictability,andcomposabilitybuiltin.Maybethelimitationisnotactuallytheirperformance,butyourlackofperceptionoftheirbenefits?

Promisesareawesome.Usethem.Theysolvetheinversionofcontrolissuesthatplagueuswithcallbacks-onlycode.

Theydon'tgetridofcallbacks,theyjustredirecttheorchestrationofthosecallbackstoatrustableintermediarymechanismthatsitsbetweenusandanotherutility.

Promisechainsalsobegintoaddress(thoughcertainlynotperfectly)abetterwayofexpressingasyncflowinsequentialfashion,whichhelpsourbrainsplanandmaintainasyncJScodebetter.We'llseeanevenbettersolutiontothatprobleminthenextchapter!

Review

InChapter2,weidentifiedtwokeydrawbackstoexpressingasyncflowcontrolwithcallbacks:

Callback-basedasyncdoesn'tfithowourbrainplansoutstepsofatask.Callbacksaren'ttrustableorcomposablebecauseofinversionofcontrol.

InChapter3,wedetailedhowPromisesuninverttheinversionofcontrolofcallbacks,restoringtrustability/composability.

Nowweturnourattentiontoexpressingasyncflowcontrolinasequential,synchronous-lookingfashion.The"magic"thatmakesitpossibleisES6generators.

InChapter1,weexplainedanexpectationthatJSdevelopersalmostuniversallyrelyonintheircode:onceafunctionstartsexecuting,itrunsuntilitcompletes,andnoothercodecaninterruptandruninbetween.

Asbizarreasitmayseem,ES6introducesanewtypeoffunctionthatdoesnotbehavewiththerun-to-completionbehavior.Thisnewtypeoffunctioniscalleda"generator."

Tounderstandtheimplications,let'sconsiderthisexample:

varx=1;

functionfoo(){

x++;

bar();//<--whataboutthisline?

console.log("x:",x);

}

functionbar(){

x++;

}

foo();//x:3

Inthisexample,weknowforsurethatbar()runsinbetweenx++andconsole.log(x).Butwhatifbar()wasn'tthere?Obviouslytheresultwouldbe2insteadof3.

Nowlet'stwistyourbrain.Whatifbar()wasn'tpresent,butitcouldstillsomehowrunbetweenthex++andconsole.log(x)statements?Howwouldthatbepossible?

Inpreemptivemultithreadedlanguages,itwouldessentiallybepossibleforbar()to"interrupt"andrunatexactlytherightmomentbetweenthosetwostatements.ButJSisnotpreemptive,norisit(currently)multithreaded.Andyet,acooperativeformofthis"interruption"(concurrency)ispossible,iffoo()itselfcouldsomehowindicatea"pause"atthatpartinthecode.

Note:Iusetheword"cooperative"notonlybecauseoftheconnectiontoclassicalconcurrencyterminology(seeChapter1),butbecauseasyou'llseeinthenextsnippet,theES6syntaxforindicatingapausepointincodeisyield--suggestingapolitelycooperativeyieldingofcontrol.

Here'stheES6codetoaccomplishsuchcooperativeconcurrency:

varx=1;


Chapter4:Generators

BreakingRun-to-Completion

function*foo(){

x++;

yield;//pause!

console.log("x:",x);

}

functionbar(){

x++;

}

Note:YouwilllikelyseemostotherJSdocumentation/codethatwillformatageneratordeclarationasfunction*foo(){..}insteadofasI'vedoneherewithfunction*foo(){..}--theonlydifferencebeingthestylisticpositioningofthe*.Thetwoformsarefunctionally/syntacticallyidentical,asisathirdfunction*foo(){..}(nospace)form.Thereareargumentsforbothstyles,butIbasicallypreferfunction*foo..becauseitthenmatcheswhenIreferenceageneratorinwritingwith*foo().IfIsaidonlyfoo(),youwouldn'tknowasclearlyifIwastalkingaboutageneratororaregularfunction.It'spurelyastylisticpreference.

Now,howcanwerunthecodeinthatprevioussnippetsuchthatbar()executesatthepointoftheyieldinsideof*foo()?

//constructaniteratorìt`tocontrolthegenerator

varit=foo();

//start`foo()`here!

it.next();

x;//2

bar();

x;//3

it.next();//x:3

OK,there'squiteabitofnewandpotentiallyconfusingstuffinthosetwocodesnippets,sowe'vegotplentytowadethrough.Butbeforeweexplainthedifferentmechanics/syntaxwithES6generators,let'swalkthroughthebehaviorflow:

1. Theit=foo()operationdoesnotexecutethe*foo()generatoryet,butitmerelyconstructsaniteratorthatwillcontrolitsexecution.Moreoniteratorsinabit.

2. Thefirstit.next()startsthe*foo()generator,andrunsthex++onthefirstlineof*foo().3. *foo()pausesattheyieldstatement,atwhichpointthatfirstit.next()callfinishes.Atthemoment,*foo()isstill

runningandactive,butit'sinapausedstate.4. Weinspectthevalueofx,andit'snow2.5. Wecallbar(),whichincrementsxagainwithx++.6. Weinspectthevalueofxagain,andit'snow3.7. Thefinalit.next()callresumesthe*foo()generatorfromwhereitwaspaused,andrunstheconsole.log(..)

statement,whichusesthecurrentvalueofxof3.

Clearly,foo()started,butdidnotrun-to-completion--itpausedattheyield.Weresumedfoo()later,andletitfinish,butthatwasn'tevenrequired.

So,ageneratorisaspecialkindoffunctionthatcanstartandstoponeormoretimes,anddoesn'tnecessarilyeverhavetofinish.Whileitwon'tbeterriblyobviousyetwhythat'ssopowerful,aswegothroughouttherestofthischapter,thatwillbeoneofthefundamentalbuildingblocksweusetoconstructgenerators-as-async-flow-controlasapatternforourcode.

Ageneratorfunctionisaspecialfunctionwiththenewprocessingmodelwejustalludedto.Butit'sstillafunction,whichmeansitstillhassomebasictenetsthathaven'tchanged--namely,thatitstillacceptsarguments(aka"input"),andthatitcanstillreturnavalue(aka"output"):

function*foo(x,y){

returnx*y;

}

InputandOutput

varit=foo(6,7);

varres=it.next();

res.value;//42

Wepassinthearguments6and7to*foo(..)astheparametersxandy,respectively.And*foo(..)returnsthevalue42backtothecallingcode.

Wenowseeadifferencewithhowthegeneratorisinvokedcomparedtoanormalfunction.foo(6,7)obviouslylooksfamiliar.Butsubtly,the*foo(..)generatorhasn'tactuallyrunyetasitwouldhavewithafunction.

Instead,we'rejustcreatinganiteratorobject,whichweassigntothevariableit,tocontrolthe*foo(..)generator.Thenwecallit.next(),whichinstructsthe*foo(..)generatortoadvancefromitscurrentlocation,stoppingeitheratthenextyieldorendofthegenerator.

Theresultofthatnext(..)callisanobjectwithavaluepropertyonitholdingwhatevervalue(ifanything)wasreturnedfrom*foo(..).Inotherwords,yieldcausedavaluetobesentoutfromthegeneratorduringthemiddleofitsexecution,kindoflikeanintermediatereturn.

Again,itwon'tbeobviousyetwhyweneedthiswholeindirectiteratorobjecttocontrolthegenerator.We'llgetthere,Ipromise.

Inadditiontogeneratorsacceptingargumentsandhavingreturnvalues,there'sevenmorepowerfulandcompellinginput/outputmessagingcapabilitybuiltintothem,viayieldandnext(..).

Consider:

function*foo(x){

vary=x*(yield);

returny;

}

varit=foo(6);

//start`foo(..)`

it.next();

varres=it.next(7);

res.value;//42

First,wepassin6astheparameterx.Thenwecallit.next(),anditstartsup*foo(..).

Inside*foo(..),thevary=x..statementstartstobeprocessed,butthenitrunsacrossayieldexpression.Atthatpoint,itpauses*foo(..)(inthemiddleoftheassignmentstatement!),andessentiallyrequeststhecallingcodetoprovidearesultvaluefortheyieldexpression.Next,wecallit.next(7),whichispassingthe7valuebackintobethatresultofthepausedyieldexpression.

So,atthispoint,theassignmentstatementisessentiallyvary=6*7.Now,returnyreturnsthat42valuebackastheresultoftheit.next(7)call.

Noticesomethingveryimportantbutalsoeasilyconfusing,eventoseasonedJSdevelopers:dependingonyourperspective,there'samismatchbetweentheyieldandthenext(..)call.Ingeneral,you'regoingtohaveonemorenext(..)callthanyouhaveyieldstatements--theprecedingsnippethasoneyieldandtwonext(..)calls.

Whythemismatch?

IterationMessaging

Becausethefirstnext(..)alwaysstartsagenerator,andrunstothefirstyield.Butit'sthesecondnext(..)callthatfulfillsthefirstpausedyieldexpression,andthethirdnext(..)wouldfulfillthesecondyield,andsoon.

Actually,whichcodeyou'rethinkingaboutprimarilywillaffectwhetherthere'saperceivedmismatchornot.

Consideronlythegeneratorcode:

vary=x*(yield);

returny;

Thisfirstyieldisbasicallyaskingaquestion:"WhatvalueshouldIinserthere?"

Who'sgoingtoanswerthatquestion?Well,thefirstnext()hasalreadyruntogetthegeneratoruptothispoint,soobviouslyitcan'tanswerthequestion.So,thesecondnext(..)callmustanswerthequestionposedbythefirstyield.

Seethemismatch--second-to-first?

Butlet'sflipourperspective.Let'slookatitnotfromthegenerator'spointofview,butfromtheiterator'spointofview.

Toproperlyillustratethisperspective,wealsoneedtoexplainthatmessagescangoinbothdirections--yield..asanexpressioncansendoutmessagesinresponsetonext(..)calls,andnext(..)cansendvaluestoapausedyieldexpression.Considerthisslightlyadjustedcode:

function*foo(x){

vary=x*(yield"Hello");//<--yieldavalue!

returny;

}

varit=foo(6);

varres=it.next();//first`next()`,don'tpassanything

res.value;//"Hello"

res=it.next(7);//pass`7`towaiting`yield`

res.value;//42

yield..andnext(..)pairtogetherasatwo-waymessagepassingsystemduringtheexecutionofthegenerator.

So,lookingonlyattheiteratorcode:

varres=it.next();//first`next()`,don'tpassanything

res.value;//"Hello"

res=it.next(7);//pass`7`towaiting`yield`

res.value;//42

Note:Wedon'tpassavaluetothefirstnext()call,andthat'sonpurpose.Onlyapausedyieldcouldacceptsuchavaluepassedbyanext(..),andatthebeginningofthegeneratorwhenwecallthefirstnext(),thereisnopausedyieldtoacceptsuchavalue.Thespecificationandallcompliantbrowsersjustsilentlydiscardanythingpassedtothefirstnext().It'sstillabadideatopassavalue,asyou'rejustcreatingsilently"failing"codethat'sconfusing.So,alwaysstartageneratorwithanargument-freenext().

Thefirstnext()call(withnothingpassedtoit)isbasicallyaskingaquestion:"Whatnextvaluedoesthe*foo(..)generatorhavetogiveme?"Andwhoanswersthisquestion?Thefirstyield"hello"expression.

See?Nomismatchthere.

Dependingonwhoyouthinkaboutaskingthequestion,thereiseitheramismatchbetweentheyieldandnext(..)calls,

TaleofTwoQuestions

ornot.

Butwait!There'sstillanextranext()comparedtothenumberofyieldstatements.So,thatfinalit.next(7)callisagainaskingthequestionaboutwhatnextvaluethegeneratorwillproduce.Butthere'snomoreyieldstatementslefttoanswer,isthere?Sowhoanswers?

Thereturnstatementanswersthequestion!

Andifthereisnoreturninyourgenerator--returniscertainlynotanymorerequiredingeneratorsthaninregularfunctions--there'salwaysanassumed/implicitreturn;(akareturnundefined;),whichservesthepurposeofdefaultansweringthequestionposedbythefinalit.next(7)call.

Thesequestionsandanswers--thetwo-waymessagepassingwithyieldandnext(..)--arequitepowerful,butit'snotobviousatallhowthesemechanismsareconnectedtoasyncflowcontrol.We'regettingthere!

Itmayappearfromthesyntacticusagethatwhenyouuseaniteratortocontrolagenerator,you'recontrollingthedeclaredgeneratorfunctionitself.Butthere'sasubtletythateasytomiss:eachtimeyouconstructaniterator,youareimplicitlyconstructinganinstanceofthegeneratorwhichthatiteratorwillcontrol.

Youcanhavemultipleinstancesofthesamegeneratorrunningatthesametime,andtheycaneveninteract:

function*foo(){

varx=yield2;

z++;

vary=yield(x*z);

console.log(x,y,z);

}

varz=1;

varit1=foo();

varit2=foo();

varval1=it1.next().value;//2<--yield2

varval2=it2.next().value;//2<--yield2

val1=it1.next(val2*10).value;//40<--x:20,z:2

val2=it2.next(val1*5).value;//600<--x:200,z:3

it1.next(val2/2);//y:300

//203003

it2.next(val1/4);//y:10

//200103

Warning:Themostcommonusageofmultipleinstancesofthesamegeneratorrunningconcurrentlyisnotsuchinteractions,butwhenthegeneratorisproducingitsownvalueswithoutinput,perhapsfromsomeindependentlyconnectedresource.We'lltalkmoreaboutvalueproductioninthenextsection.

Let'sbrieflywalkthroughtheprocessing:

1. Bothinstancesof*foo()arestartedatthesametime,andbothnext()callsrevealavalueof2fromtheyield2statements,respectively.

2. val2*10is2*10,whichissentintothefirstgeneratorinstanceit1,sothatxgetsvalue20.zisincrementedfrom1to2,andthen20*2isyieldedout,settingval1to40.

3. val1*5is40*5,whichissentintothesecondgeneratorinstanceit2,sothatxgetsvalue200.zisincrementedagain,from2to3,andthen200*3isyieldedout,settingval2to600.

4. val2/2is600/2,whichissentintothefirstgeneratorinstanceit1,sothatygetsvalue300,thenprintingout203003foritsxyzvalues,respectively.

5. val1/4is40/4,whichissentintothesecondgeneratorinstanceit2,sothatygetsvalue10,thenprintingout200103foritsxyzvalues,respectively.

MultipleIterators

That'sa"fun"exampletorunthroughinyourmind.Didyoukeepitstraight?

Recallthisscenariofromthe"Run-to-completion"sectionofChapter1:

vara=1;

varb=2;

functionfoo(){

a++;

b=b*a;

a=b+3;

}

functionbar(){

b--;

a=8+b;

b=a*2;

}

WithnormalJSfunctions,ofcourseeitherfoo()canruncompletelyfirst,orbar()canruncompletelyfirst,butfoo()cannotinterleaveitsindividualstatementswithbar().So,thereareonlytwopossibleoutcomestotheprecedingprogram.

However,withgenerators,clearlyinterleaving(eveninthemiddleofstatements!)ispossible:

vara=1;

varb=2;

function*foo(){

a++;

yield;

b=b*a;

a=(yieldb)+3;

}

function*bar(){

b--;

yield;

a=(yield8)+b;

b=a*(yield2);

}

Dependingonwhatrespectiveordertheiteratorscontrolling*foo()and*bar()arecalled,theprecedingprogramcouldproduceseveraldifferentresults.Inotherwords,wecanactuallyillustrate(inasortoffake-ishway)thetheoretical"threadedraceconditions"circumstancesdiscussedinChapter1,byinterleavingthetwogeneratorinterationsoverthesamesharedvariables.

First,let'smakeahelpercalledstep(..)thatcontrolsaniterator:

functionstep(gen){

varit=gen();

varlast;

returnfunction(){

//whateveris`yieldèdout,just

//senditrightbackinthenexttime!

last=it.next(last).value;

};

}

step(..)initializesageneratortocreateitsititerator,thenreturnsafunctionwhich,whencalled,advancestheiteratorbyonestep.Additionally,thepreviouslyyieldedoutvalueissentrightbackinatthenextstep.So,yield8willjustbecome8andyieldbwilljustbeb(whateveritwasatthetimeofyield).

Interleaving

Now,justforfun,let'sexperimenttoseetheeffectsofinterleavingthesedifferentchunksof*foo()and*bar().We'llstartwiththeboringbasecase,makingsure*foo()totallyfinishesbefore*bar()(justlikewedidinChapter1):

//makesuretoresetàànd`b`

a=1;

b=2;

vars1=step(foo);

vars2=step(bar);

//run`*foo()`completelyfirst

s1();

s1();

s1();

//nowrun`*bar()`

s2();

s2();

s2();

s2();

console.log(a,b);//1122

Theendresultis11and22,justasitwasintheChapter1version.Nowlet'smixuptheinterleavingorderingandseehowitchangesthefinalvaluesofaandb:

//makesuretoresetàànd`b`

a=1;

b=2;

vars1=step(foo);

vars2=step(bar);

s2();//b--;

s2();//yield8

s1();//a++;

s2();//a=8+b;

//yield2

s1();//b=b*a;

//yieldb

s1();//a=b+3;

s2();//b=a*2;

BeforeItellyoutheresults,canyoufigureoutwhataandbareaftertheprecedingprogram?Nocheating!

console.log(a,b);//1218

Note:Asanexerciseforthereader,trytoseehowmanyothercombinationsofresultsyoucangetbackrearrangingtheorderofthes1()ands2()calls.Don'tforgetyou'llalwaysneedthrees1()callsandfours2()calls.Recallthediscussionearlieraboutmatchingnext()withyieldforthereasonswhy.

Youalmostcertainlywon'twanttointentionallycreatethislevelofinterleavingconfusion,asitcreatesincrediblydifficulttounderstandcode.Buttheexerciseisinterestingandinstructivetounderstandmoreabouthowmultiplegeneratorscanrunconcurrentlyinthesamesharedscope,becausetherewillbeplaceswherethiscapabilityisquiteuseful.

We'lldiscussgeneratorconcurrencyinmoredetailattheendofthischapter.

Intheprevioussection,wementionedaninterestinguseforgenerators,asawaytoproducevalues.Thisisnotthemainfocusinthischapter,butwe'dberemissifwedidn'tcoverthebasics,especiallybecausethisusecaseisessentiallytheoriginofthename:generators.

Generator'ingValues

We'regoingtotakeaslightdiversionintothetopicofiteratorsforabit,butwe'llcirclebacktohowtheyrelatetogeneratorsandusingageneratortogeneratevalues.

Imagineyou'reproducingaseriesofvalueswhereeachvaluehasadefinablerelationshiptothepreviousvalue.Todothis,you'regoingtoneedastatefulproducerthatremembersthelastvalueitgaveout.

Youcanimplementsomethinglikethatstraightforwardlyusingafunctionclosure(seetheScope&Closurestitleofthisseries):

vargimmeSomething=(function(){

varnextVal;

returnfunction(){

if(nextVal===undefined){

nextVal=1;

}

else{

nextVal=(3*nextVal)+6;

}

returnnextVal;

};

})();

gimmeSomething();//1




Note:ThenextValcomputationlogicherecouldhavebeensimplified,butconceptually,wedon'twanttocalculatethenextvalue(akanextVal)untilthenextgimmeSomething()callhappens,becauseingeneralthatcouldbearesource-leakydesignforproducersofmorepersistentorresource-limitedvaluesthansimplenumbers.

Generatinganarbitrarynumberseriesisn'taterriblyrealisticexample.Butwhatifyouweregeneratingrecordsfromadatasource?Youcouldimaginemuchthesamecode.

Infact,thistaskisaverycommondesignpattern,usuallysolvedbyiterators.Aniteratorisawell-definedinterfaceforsteppingthroughaseriesofvaluesfromaproducer.TheJSinterfaceforiterators,asitisinmostlanguages,istocallnext()eachtimeyouwantthenextvaluefromtheproducer.

Wecouldimplementthestandarditeratorinterfaceforournumberseriesproducer:

varsomething=(function(){

varnextVal;

return{

//neededfor`for..of`loops

[Symbol.iterator]:function(){returnthis;},

//standarditeratorinterfacemethod

next:function(){


nextVal=1;

}

else{


}

return{done:false,value:nextVal};

}

};

})();

something.next().value;//1



ProducersandIterators


Note:We'llexplainwhyweneedthe[Symbol.iterator]:..partofthiscodesnippetinthe"Iterables"section.Syntacticallythough,twoES6featuresareatplay.First,the[..]syntaxiscalledacomputedpropertyname(seethethis&ObjectPrototypestitleofthisseries).It'sawayinanobjectliteraldefinitiontospecifyanexpressionandusetheresultofthatexpressionasthenamefortheproperty.Next,Symbol.iteratorisoneofES6'spredefinedspecialSymbolvalues(seetheES6&Beyondtitleofthisbookseries).

Thenext()callreturnsanobjectwithtwoproperties:doneisabooleanvaluesignalingtheiterator'scompletestatus;valueholdstheiterationvalue.

ES6alsoaddsthefor..ofloop,whichmeansthatastandarditeratorcanautomaticallybeconsumedwithnativeloopsyntax:

for(varvofsomething){

console.log(v);

//don'tletthelooprunforever!

if(v>500){

break;

}

}

//1933105321969

Note:Becauseoursomethingiteratoralwaysreturnsdone:false,thisfor..ofloopwouldrunforever,whichiswhyweputthebreakconditionalin.It'stotallyOKforiteratorstobenever-ending,buttherearealsocaseswheretheiteratorwillrunoverafinitesetofvaluesandeventuallyreturnadone:true.

Thefor..ofloopautomaticallycallsnext()foreachiteration--itdoesn'tpassanyvaluesintothenext()--anditwillautomaticallyterminateonreceivingadone:true.It'squitehandyforloopingoverasetofdata.

Ofcourse,youcouldmanuallyloopoveriterators,callingnext()andcheckingforthedone:trueconditiontoknowwhentostop:

for(

varret;

(ret=something.next())&&!ret.done;

){

console.log(ret.value);


if(ret.value>500){

break;

}

}

//1933105321969

Note:ThismanualforapproachiscertainlyuglierthantheES6for..ofloopsyntax,butitsadvantageisthatitaffordsyoutheopportunitytopassinvaluestothenext(..)callsifnecessary.

Inadditiontomakingyourowniterators,manybuilt-indatastructuresinJS(asofES6),likearrays,alsohavedefaultiterators:

vara=[1,3,5,7,9];

for(varvofa){

console.log(v);

}

//13579

Thefor..ofloopasksaforitsiterator,andautomaticallyusesittoiterateovera'svalues.

Note:ItmayseemastrangeomissionbyES6,butregularobjectsintentionallydonotcomewithadefaultiteratorthewayarraysdo.Thereasonsgodeeperthanwewillcoverhere.Ifallyouwantistoiterateoverthepropertiesofanobject(withnoparticularguaranteeofordering),Object.keys(..)returnsanarray,whichcanthenbeusedlikefor(varkofObject.keys(obj)){...Suchafor..ofloopoveranobject'skeyswouldbesimilartoafor..inloop,exceptthatObject.keys(..)doesnotincludepropertiesfromthe[[Prototype]]chainwhilefor..indoes(seethethis&ObjectPrototypestitleofthisseries).

Thesomethingobjectinourrunningexampleiscalledaniterator,asithasthenext()methodonitsinterface.Butacloselyrelatedtermisiterable,whichisanobjectthatcontainsaniteratorthatcaniterateoveritsvalues.

AsofES6,thewaytoretrieveaniteratorfromaniterableisthattheiterablemusthaveafunctiononit,withthenamebeingthespecialES6symbolvalueSymbol.iterator.Whenthisfunctioniscalled,itreturnsaniterator.Thoughnotrequired,generallyeachcallshouldreturnafreshnewiterator.

aintheprevioussnippetisaniterable.Thefor..ofloopautomaticallycallsitsSymbol.iteratorfunctiontoconstructaniterator.Butwecouldofcoursecallthefunctionmanually,andusetheiteratoritreturns:

vara=[1,3,5,7,9];

varit=a[Symbol.iterator]();

it.next().value;//1

it.next().value;//3

it.next().value;//5

..

Inthepreviouscodelistingthatdefinedsomething,youmayhavenoticedthisline:

[Symbol.iterator]:function(){returnthis;}

Thatlittlebitofconfusingcodeismakingthesomethingvalue--theinterfaceofthesomethingiterator--alsoaniterable;it'snowbothaniterableandaniterator.Then,wepasssomethingtothefor..ofloop:

for(varvofsomething){

..

}

Thefor..ofloopexpectssomethingtobeaniterable,soitlooksforandcallsitsSymbol.iteratorfunction.Wedefinedthatfunctiontosimplyreturnthis,soitjustgivesitselfback,andthefor..ofloopisnonethewiser.

Let'sturnourattentionbacktogenerators,inthecontextofiterators.Ageneratorcanbetreatedasaproducerofvaluesthatweextractoneatatimethroughaniteratorinterface'snext()calls.

So,ageneratoritselfisnottechnicallyaniterable,thoughit'sverysimilar--whenyouexecutethegenerator,yougetaniteratorback:

function*foo(){..}

varit=foo();

Wecanimplementthesomethinginfinitenumberseriesproducerfromearlierwithagenerator,likethis:

Iterables

GeneratorIterator

function*something(){

varnextVal;

while(true){


nextVal=1;

}

else{


}

yieldnextVal;

}

}

Note:Awhile..trueloopwouldnormallybeaverybadthingtoincludeinarealJSprogram,atleastifitdoesn'thaveabreakorreturninit,asitwouldlikelyrunforever,synchronously,andblock/lock-upthebrowserUI.However,inagenerator,suchaloopisgenerallytotallyOKifithasayieldinit,asthegeneratorwillpauseateachiteration,yieldingbacktothemainprogramand/ortotheeventloopqueue.Toputitglibly,"generatorsputthewhile..truebackinJSprogramming!"

That'safairbitcleanerandsimpler,right?Becausethegeneratorpausesateachyield,thestate(scope)ofthefunction*something()iskeptaround,meaningthere'snoneedfortheclosureboilerplatetopreservevariablestateacrosscalls.

Notonlyisitsimplercode--wedon'thavetomakeourowniteratorinterface--itactuallyismorereason-ablecode,becauseitmoreclearlyexpressestheintent.Forexample,thewhile..truelooptellsusthegeneratorisintendedtorunforever--tokeepgeneratingvaluesaslongaswekeepaskingforthem.

Andnowwecanuseourshinynew*something()generatorwithafor..ofloop,andyou'llseeitworksbasicallyidentically:

for(varvofsomething()){

console.log(v);


if(v>500){

break;

}

}

//1933105321969

Butdon'tskipoverfor(varvofsomething())..!Wedidn'tjustreferencesomethingasavaluelikeinearlierexamples,butinsteadcalledthe*something()generatortogetitsiteratorforthefor..oflooptouse.

Ifyou'repayingcloseattention,twoquestionsmayarisefromthisinteractionbetweenthegeneratorandtheloop:

Whycouldn'twesayfor(varvofsomething)..?Becausesomethinghereisagenerator,whichisnotaniterable.Wehavetocallsomething()toconstructaproducerforthefor..oflooptoiterateover.Thesomething()callproducesaniterator,butthefor..ofloopwantsaniterable,right?Yep.Thegenerator'siteratoralsohasaSymbol.iteratorfunctiononit,whichbasicallydoesareturnthis,justlikethesomethingiterablewedefinedearlier.Inotherwords,agenerator'siteratorisalsoaniterable!

Inthepreviousexample,itwouldappeartheiteratorinstanceforthe*something()generatorwasbasicallyleftinasuspendedstateforeverafterthebreakintheloopwascalled.

Butthere'sahiddenbehaviorthattakescareofthatforyou."Abnormalcompletion"(i.e.,"earlytermination")ofthefor..ofloop--generallycausedbyabreak,return,oranuncaughtexception--sendsasignaltothegenerator'siteratorforittoterminate.

Note:Technically,thefor..ofloopalsosendsthissignaltotheiteratoratthenormalcompletionoftheloop.Fora

StoppingtheGenerator

generator,that'sessentiallyamootoperation,asthegenerator'siteratorhadtocompletefirstsothefor..ofloopcompleted.However,customiteratorsmightdesiretoreceivethisadditionalsignalfromfor..ofloopconsumers.

Whileafor..ofloopwillautomaticallysendthissignal,youmaywishtosendthesignalmanuallytoaniterator;youdothisbycallingreturn(..).

Ifyouspecifyatry..finallyclauseinsidethegenerator,itwillalwaysberunevenwhenthegeneratorisexternallycompleted.Thisisusefulifyouneedtocleanupresources(databaseconnections,etc.):

function*something(){

try{

varnextVal;

while(true){


nextVal=1;

}

else{


}

yieldnextVal;

}

}

//cleanupclause

finally{

console.log("cleaningup!");

}

}

Theearlierexamplewithbreakinthefor..ofloopwilltriggerthefinallyclause.Butyoucouldinsteadmanuallyterminatethegenerator'siteratorinstancefromtheoutsidewithreturn(..):

varit=something();

for(varvofit){

console.log(v);


if(v>500){

console.log(

//completethegenerator'siterator

it.return("HelloWorld").value

);

//no`break`neededhere

}

}

//1933105321969

//cleaningup!

//HelloWorld

Whenwecallit.return(..),itimmediatelyterminatesthegenerator,whichofcourserunsthefinallyclause.Also,itsetsthereturnedvaluetowhateveryoupassedintoreturn(..),whichishow"HelloWorld"comesrightbackout.Wealsodon'tneedtoincludeabreaknowbecausethegenerator'siteratorissettodone:true,sothefor..ofloopwillterminateonitsnextiteration.

Generatorsowetheirnamesakemostlytothisconsumingproducedvaluesuse.Butagain,that'sjustoneoftheusesforgenerators,andfranklynoteventhemainonewe'reconcernedwithinthecontextofthisbook.

Butnowthatwemorefullyunderstandsomeofthemechanicsofhowtheywork,wecannextturnourattentiontohowgeneratorsapplytoasyncconcurrency.

Whatdogeneratorshavetodowithasynccodingpatterns,fixingproblemswithcallbacks,andthelike?Let'sgetto

IteratingGeneratorsAsynchronously

answeringthatimportantquestion.

WeshouldrevisitoneofourscenariosfromChapter3.Let'srecallthecallbackapproach:


ajax(


cb

);

}

foo(11,31,function(err,text){

if(err){

console.error(err);

}

else{

console.log(text);

}

});

Ifwewantedtoexpressthissametaskflowcontrolwithagenerator,wecoulddo:

functionfoo(x,y){

ajax(


function(err,data){

if(err){

//throwanerrorinto`*main()`

it.throw(err);

}

else{

//resume`*main()`withreceived`data`

it.next(data);

}

}

);

}

function*main(){

try{

vartext=yieldfoo(11,31);

console.log(text);

}

catch(err){

console.error(err);

}

}

varit=main();

//startitallup!

it.next();

Atfirstglance,thissnippetislonger,andperhapsalittlemorecomplexlooking,thanthecallbacksnippetbeforeit.Butdon'tletthatimpressiongetyouofftrack.Thegeneratorsnippetisactuallymuchbetter!Butthere'salotgoingonforustoexplain.

First,let'slookatthispartofthecode,whichisthemostimportant:


console.log(text);

Thinkabouthowthatcodeworksforamoment.We'recallinganormalfunctionfoo(..)andwe'reapparentlyabletogetbackthetextfromtheAjaxcall,eventhoughit'sasynchronous.

Howisthatpossible?IfyourecallthebeginningofChapter1,wehadalmostidenticalcode:

vardata=ajax("..url1..");

console.log(data);

Andthatcodedidn'twork!Canyouspotthedifference?It'stheyieldusedinagenerator.

That'sthemagic!That'swhatallowsustohavewhatappearstobeblocking,synchronouscode,butitdoesn'tactuallyblockthewholeprogram;itonlypauses/blocksthecodeinthegeneratoritself.

Inyieldfoo(11,31),firstthefoo(11,31)callismade,whichreturnsnothing(akaundefined),sowe'remakingacalltorequestdata,butwe'reactuallythendoingyieldundefined.That'sOK,becausethecodeisnotcurrentlyrelyingonayieldedvaluetodoanythinginteresting.We'llrevisitthispointlaterinthechapter.

We'renotusingyieldinamessagepassingsensehere,onlyinaflowcontrolsensetopause/block.Actually,itwillhavemessagepassing,butonlyinonedirection,afterthegeneratorisresumed.

So,thegeneratorpausesattheyield,essentiallyaskingthequestion,"whatvalueshouldIreturntoassigntothevariabletext?"Who'sgoingtoanswerthatquestion?

Lookatfoo(..).IftheAjaxrequestissuccessful,wecall:

it.next(data);

That'sresumingthegeneratorwiththeresponsedata,whichmeansthatourpausedyieldexpressionreceivesthatvaluedirectly,andthenasitrestartsthegeneratorcode,thatvaluegetsassignedtothelocalvariabletext.

Prettycool,huh?

Takeastepbackandconsidertheimplications.Wehavetotallysynchronous-lookingcodeinsidethegenerator(otherthantheyieldkeyworditself),buthiddenbehindthescenes,insideoffoo(..),theoperationscancompleteasynchronously.

That'shuge!That'sanearlyperfectsolutiontoourpreviouslystatedproblemwithcallbacksnotbeingabletoexpressasynchronyinasequential,synchronousfashionthatourbrainscanrelateto.

Inessence,weareabstractingtheasynchronyawayasanimplementationdetail,sothatwecanreasonsynchronously/sequentiallyaboutourflowcontrol:"MakeanAjaxrequest,andwhenitfinishesprintouttheresponse."Andofcourse,wejustexpressedtwostepsintheflowcontrol,butthissamecapabililtyextendswithoutbounds,toletusexpresshowevermanystepsweneedto.

Tip:Thisissuchanimportantrealization,justgobackandreadthelastthreeparagraphsagaintoletitsinkin!

Buttheprecedinggeneratorcodehasevenmoregoodnesstoyieldtous.Let'sturnourattentiontothetry..catchinsidethegenerator:

try{


console.log(text);

}

catch(err){

console.error(err);

}

Howdoesthiswork?Thefoo(..)callisasynchronouslycompleting,anddoesn'ttry..catchfailtocatchasynchronouserrors,aswelookedatinChapter3?

Wealreadysawhowtheyieldletstheassignmentstatementpausetowaitforfoo(..)tofinish,sothatthecompleted

SynchronousErrorHandling

responsecanbeassignedtotext.Theawesomepartisthatthisyieldpausingalsoallowsthegeneratortocatchanerror.Wethrowthaterrorintothegeneratorwiththispartoftheearliercodelisting:

if(err){

//throwanerrorinto`*main()`

it.throw(err);

}

Theyield-pausenatureofgeneratorsmeansthatnotonlydowegetsynchronous-lookingreturnvaluesfromasyncfunctioncalls,butwecanalsosynchronouslycatcherrorsfromthoseasyncfunctioncalls!

Sowe'veseenwecanthrowerrorsintoagenerator,butwhataboutthrowingerrorsoutofagenerator?Exactlyasyou'dexpect:

function*main(){

varx=yield"HelloWorld";

yieldx.toLowerCase();//causeanexception!

}

varit=main();

it.next().value;//HelloWorld

try{

it.next(42);

}

catch(err){

console.error(err);//TypeError

}

Ofcourse,wecouldhavemanuallythrownanerrorwiththrow..insteadofcausinganexception.

Wecanevencatchthesameerrorthatwethrow(..)intothegenerator,essentiallygivingthegeneratorachancetohandleitbutifitdoesn't,theiteratorcodemusthandleit:

function*main(){

varx=yield"HelloWorld";

//nevergetshere

console.log(x);

}

varit=main();

it.next();

try{

//will`*main()`handlethiserror?we'llsee!

it.throw("Oops");

}

catch(err){

//nope,didn'thandleit!

console.error(err);//Oops

}

Synchronous-lookingerrorhandling(viatry..catch)withasynccodeisahugewinforreadabilityandreason-ability.

Inourpreviousdiscussion,weshowedhowgeneratorscanbeiteratedasynchronously,whichisahugestepforwardinsequentialreason-abilityoverthespaghettimessofcallbacks.Butwelostsomethingveryimportant:thetrustabilityandcomposabilityofPromises(seeChapter3)!

Generators+Promises

Don'tworry--wecangetthatback.ThebestofallworldsinES6istocombinegenerators(synchronous-lookingasynccode)withPromises(trustableandcomposable).

Buthow?

RecallfromChapter3thePromise-basedapproachtoourrunningAjaxexample:

functionfoo(x,y){

returnrequest(


);

}

foo(11,31)

.then(

function(text){

console.log(text);

},

function(err){

console.error(err);

}

);

InourearliergeneratorcodefortherunningAjaxexample,foo(..)returnednothing(undefined),andouriteratorcontrolcodedidn'tcareaboutthatyieldedvalue.

ButherethePromise-awarefoo(..)returnsapromiseaftermakingtheAjaxcall.Thatsuggeststhatwecouldconstructapromisewithfoo(..)andthenyielditfromthegenerator,andthentheiteratorcontrolcodewouldreceivethatpromise.

Butwhatshouldtheiteratordowiththepromise?

Itshouldlistenforthepromisetoresolve(fulfillmentorrejection),andtheneitherresumethegeneratorwiththefulfillmentmessageorthrowanerrorintothegeneratorwiththerejectionreason.

Letmerepeatthat,becauseit'ssoimportant.ThenaturalwaytogetthemostoutofPromisesandgeneratorsistoyieldaPromise,andwirethatPromisetocontrolthegenerator'siterator.

Let'sgiveitatry!First,we'llputthePromise-awarefoo(..)togetherwiththegenerator*main():

functionfoo(x,y){

returnrequest(


);

}

function*main(){

try{


console.log(text);

}

catch(err){

console.error(err);

}

}

Themostpowerfulrevelationinthisrefactoristhatthecodeinside*main()didnothavetochangeatall!Insidethegenerator,whatevervaluesareyieldedoutisjustanopaqueimplementationdetail,sowe'renotevenawareit'shappening,nordoweneedtoworryaboutit.

Buthowarewegoingtorun*main()now?Westillhavesomeoftheimplementationplumbingworktodo,toreceiveandwireuptheyieldedpromisesothatitresumesthegeneratoruponresolution.We'llstartbytryingthatmanually:

varit=main();

varp=it.next().value;

//waitforthe`p`promisetoresolve

p.then(

function(text){

it.next(text);

},

function(err){

it.throw(err);

}

);

Actually,thatwasn'tsopainfulatall,wasit?

Thissnippetshouldlookverysimilartowhatwedidearlierwiththemanuallywiredgeneratorcontrolledbytheerror-firstcallback.Insteadofanif(err){it.throw..,thepromisealreadysplitsfulfillment(success)andrejection(failure)forus,butotherwisetheiteratorcontrolisidentical.

Now,we'veglossedoversomeimportantdetails.

Mostimportantly,wetookadvantageofthefactthatweknewthat*main()onlyhadonePromise-awarestepinit.WhatifwewantedtobeabletoPromise-driveageneratornomatterhowmanystepsithas?Wecertainlydon'twanttomanuallywriteoutthePromisechaindifferentlyforeachgenerator!Whatwouldbemuchnicerisiftherewasawaytorepeat(aka"loop"over)theiterationcontrol,andeachtimeaPromisecomesout,waitonitsresolutionbeforecontinuing.

Also,whatifthegeneratorthrowsoutanerror(intentionallyoraccidentally)duringtheit.next(..)call?Shouldwequit,orshouldwecatchitandsenditrightbackin?Similarly,whatifweit.throw(..)aPromiserejectionintothegenerator,butit'snothandled,andcomesrightbackout?

Themoreyoustarttoexplorethispath,themoreyourealize,"wow,it'dbegreatiftherewasjustsomeutilitytodoitforme."Andyou'reabsolutelycorrect.Thisissuchanimportantpattern,andyoudon'twanttogetitwrong(orexhaustyourselfrepeatingitoverandover),soyourbestbetistouseautilitythatisspecificallydesignedtorunPromise-yieldinggeneratorsinthemannerwe'veillustrated.

SeveralPromiseabstractionlibrariesprovidejustsuchautility,includingmyasynquencelibraryanditsrunner(..),whichwillbediscussedinAppendixAofthisbook.

Butforthesakeoflearningandillustration,let'sjustdefineourownstandaloneutilitythatwe'llcallrun(..):

//thankstoBenjaminGruenbaum(@benjamingronGitHub)for

//bigimprovementshere!

functionrun(gen){

varargs=[].slice.call(arguments,1),it;

//initializethegeneratorinthecurrentcontext

it=gen.apply(this,args);

//returnapromiseforthegeneratorcompleting

returnPromise.resolve()

.then(functionhandleNext(value){

//runtothenextyieldedvalue

varnext=it.next(value);

return(functionhandleResult(next){

//generatorhascompletedrunning?

if(next.done){

returnnext.value;

}

//otherwisekeepgoing

else{

returnPromise.resolve(next.value)

.then(

//resumetheasyncloopon

//success,sendingtheresolved

//valuebackintothegenerator

Promise-AwareGeneratorRunner

handleNext,

//if`valueìsarejected

//promise,propagateerrorback

//intothegeneratorforitsown

//errorhandling

functionhandleErr(err){

returnPromise.resolve(

it.throw(err)

)

.then(handleResult);

}

);

}

})(next);

});

}

Asyoucansee,it'saquiteabitmorecomplexthanyou'dprobablywanttoauthoryourself,andyouespeciallywouldn'twanttorepeatthiscodeforeachgeneratoryouuse.So,autility/libraryhelperisdefinitelythewaytogo.Nevertheless,Iencourageyoutospendafewminutesstudyingthatcodelistingtogetabettersenseofhowtomanagethegenerator+Promisenegotiation.

Howwouldyouuserun(..)with*main()inourrunningAjaxexample?

function*main(){

//..

}

run(main);

That'sit!Thewaywewiredrun(..),itwillautomaticallyadvancethegeneratoryoupasstoit,asynchronouslyuntilcompletion.

Note:Therun(..)wedefinedreturnsapromisewhichiswiredtoresolveoncethegeneratoriscomplete,orreceiveanuncaughtexceptionifthegeneratordoesn'thandleit.Wedon'tshowthatcapabilityhere,butwe'llcomebacktoitlaterinthechapter.

Theprecedingpattern--generatorsyieldingPromisesthatthencontrolthegenerator'siteratortoadvanceittocompletion--issuchapowerfulandusefulapproach,itwouldbenicerifwecoulddoitwithouttheclutterofthelibraryutilityhelper(akarun(..)).

There'sprobablygoodnewsonthatfront.Atthetimeofthiswriting,there'searlybutstrongsupportforaproposalformoresyntacticadditioninthisrealmforthepost-ES6,ES7-ishtimeframe.Obviously,it'stooearlytoguaranteethedetails,butthere'saprettydecentchanceitwillshakeoutsimilartothefollowing:

functionfoo(x,y){

returnrequest(


);

}

asyncfunctionmain(){

try{

vartext=awaitfoo(11,31);

console.log(text);

}

catch(err){

console.error(err);

}

}

main();

ES7:asyncandawait?

Asyoucansee,there'snorun(..)call(meaningnoneedforalibraryutility!)toinvokeanddrivemain()--it'sjustcalledasanormalfunction.Also,main()isn'tdeclaredasageneratorfunctionanymore;it'sanewkindoffunction:asyncfunction.Andfinally,insteadofyieldingaPromise,weawaitforittoresolve.

TheasyncfunctionautomaticallyknowswhattodoifyouawaitaPromise--itwillpausethefunction(justlikewithgenerators)untilthePromiseresolves.Wedidn'tillustrateitinthissnippet,butcallinganasyncfunctionlikemain()automaticallyreturnsapromisethat'sresolvedwheneverthefunctionfinishescompletely.

Tip:Theasync/awaitsyntaxshouldlookveryfamiliartoreaderswithexperienceinC#,becauseit'sbasicallyidentical.

Theproposalessentiallycodifiessupportforthepatternwe'vealreadyderived,intoasyntacticmechanism:combiningPromiseswithsync-lookingflowcontrolcode.That'sthebestofbothworldscombined,toeffectivelyaddresspracticallyallofthemajorconcernsweoutlinedwithcallbacks.

ThemerefactthatsuchaES7-ishproposalalreadyexistsandhasearlysupportandenthusiasmisamajorvoteofconfidenceinthefutureimportanceofthisasyncpattern.

Sofar,allwe'vedemonstratedisasingle-stepasyncflowwithPromises+generators.Butreal-worldcodewilloftenhavemanyasyncsteps.

Ifyou'renotcareful,thesync-lookingstyleofgeneratorsmaylullyouintocomplacencywithhowyoustructureyourasyncconcurrency,leadingtosuboptimalperformancepatterns.Sowewanttospendalittletimeexploringtheoptions.

Imagineascenariowhereyouneedtofetchdatafromtwodifferentsources,thencombinethoseresponsestomakeathirdrequest,andfinallyprintoutthelastresponse.WeexploredasimilarscenariowithPromisesinChapter3,butlet'sreconsideritinthecontextofgenerators.

Yourfirstinstinctmightbesomethinglike:

function*foo(){

varr1=yieldrequest("http://some.url.1");


varr3=yieldrequest(

"http://some.url.3/?v="+r1+","+r2

);

console.log(r3);

}

//usepreviouslydefined`run(..)ùtility

run(foo);

Thiscodewillwork,butinthespecificsofourscenario,it'snotoptimal.Canyouspotwhy?

Becausether1andr2requestscan--andforperformancereasons,should--runconcurrently,butinthiscodetheywillrunsequentially;the"http://some.url.2"URLisn'tAjaxfetcheduntilafterthe"http://some.url.1"requestisfinished.Thesetworequestsareindependent,sothebetterperformanceapproachwouldlikelybetohavethemrunatthesametime.

Buthowexactlywouldyoudothatwithageneratorandyield?Weknowthatyieldisonlyasinglepausepointinthecode,soyoucan'treallydotwopausesatthesametime.

ThemostnaturalandeffectiveansweristobasetheasyncflowonPromises,specificallyontheircapabilitytomanagestateinatime-independentfashion(see"FutureValue"inChapter3).

Thesimplestapproach:

PromiseConcurrencyinGenerators

function*foo(){

//makebothrequests"inparallel"

varp1=request("http://some.url.1");


//waituntilbothpromisesresolve

varr1=yieldp1;

varr2=yieldp2;

varr3=yieldrequest(


);

console.log(r3);

}


run(foo);

Whyisthisdifferentfromtheprevioussnippet?Lookatwheretheyieldisandisnot.p1andp2arepromisesforAjaxrequestsmadeconcurrently(aka"inparallel").Itdoesn'tmatterwhichonefinishesfirst,becausepromiseswillholdontotheirresolvedstateforaslongasnecessary.

Thenweusetwosubsequentyieldstatementstowaitforandretrievetheresolutionsfromthepromises(intor1andr2,respectively).Ifp1resolvesfirst,theyieldp1resumesfirstthenwaitsontheyieldp2toresume.Ifp2resolvesfirst,itwilljustpatientlyholdontothatresolutionvalueuntilasked,buttheyieldp1willholdonfirst,untilp1resolves.

Eitherway,bothp1andp2willrunconcurrently,andbothhavetofinish,ineitherorder,beforether3=yieldrequest..Ajaxrequestwillbemade.

Ifthatflowcontrolprocessingmodelsoundsfamiliar,it'sbasicallythesameaswhatweidentifiedinChapter3asthe"gate"pattern,enabledbythePromise.all([..])utility.So,wecouldalsoexpresstheflowcontrollikethis:

function*foo(){

//makebothrequests"inparallel,"and

//waituntilbothpromisesresolve

varresults=yieldPromise.all([

request("http://some.url.1"),

request("http://some.url.2")

]);

varr1=results[0];

varr2=results[1];

varr3=yieldrequest(


);

console.log(r3);

}


run(foo);

Note:AswediscussedinChapter3,wecanevenuseES6destructuringassignmenttosimplifythevarr1=..varr2=..assignments,withvar[r1,r2]=results.

Inotherwords,alloftheconcurrencycapabilitiesofPromisesareavailabletousinthegenerator+Promiseapproach.Soinanyplacewhereyouneedmorethansequentialthis-then-thatasyncflowcontrolsteps,Promisesarelikelyyourbestbet.

Asawordofstylisticcaution,becarefulabouthowmuchPromiselogicyouincludeinsideyourgenerators.Thewholepointofusinggeneratorsforasynchronyinthewaywe'vedescribedistocreatesimple,sequential,sync-lookingcode,andtohideasmuchofthedetailsofasynchronyawayfromthatcodeaspossible.

Promises,Hidden

Forexample,thismightbeacleanerapproach:

//note:normalfunction,notgenerator

functionbar(url1,url2){

returnPromise.all([

request(url1),

request(url2)

]);

}

function*foo(){

//hidethePromise-basedconcurrencydetails

//inside`bar(..)`

varresults=yieldbar(

"http://some.url.1",

"http://some.url.2"

);

varr1=results[0];

varr2=results[1];

varr3=yieldrequest(


);

console.log(r3);

}


run(foo);

Inside*foo(),it'scleanerandclearerthatallwe'redoingisjustaskingbar(..)togetussomeresults,andwe'llyield-waitonthattohappen.Wedon'thavetocarethatunderthecoversaPromise.all([..])Promisecompositionwillbeusedtomakethathappen.

Wetreatasynchrony,andindeedPromises,asanimplementationdetail.

HidingyourPromiselogicinsideafunctionthatyoumerelycallfromyourgeneratorisespeciallyusefulifyou'regoingtodoasophisticatedseriesflow-control.Forexample:

functionbar(){

Promise.all([

baz(..)

.then(..),

Promise.race([..])

])

.then(..)

}

Thatkindoflogicissometimesrequired,andifyoudumpitdirectlyinsideyourgenerator(s),you'vedefeatedmostofthereasonwhyyouwouldwanttousegeneratorsinthefirstplace.Weshouldintentionallyabstractsuchdetailsawayfromourgeneratorcodesothattheydon'tclutterupthehigherleveltaskexpression.

Beyondcreatingcodethatisbothfunctionalandperformant,youshouldalsostrivetomakecodethatisasreason-ableandmaintainableaspossible.

Note:Abstractionisnotalwaysahealthythingforprogramming--manytimesitcanincreasecomplexityinexchangeforterseness.Butinthiscase,Ibelieveit'smuchhealthierforyourgenerator+Promiseasynccodethanthealternatives.Aswithallsuchadvice,though,payattentiontoyourspecificsituationsandmakeproperdecisionsforyouandyourteam.

Intheprevioussection,weshowedcallingregularfunctionsfrominsideagenerator,andhowthatremainsausefultechniqueforabstractingawayimplementationdetails(likeasyncPromiseflow).Butthemaindrawbackofusinganormal

GeneratorDelegation

functionforthistaskisthatithastobehavebythenormalfunctionrules,whichmeansitcannotpauseitselfwithyieldlikeageneratorcan.

Itmaythenoccurtoyouthatyoumighttrytocallonegeneratorfromanothergenerator,usingourrun(..)helper,suchas:

function*foo(){


varr3=yieldrequest("http://some.url.3/?v="+r2);

returnr3;

}

function*bar(){


//"delegating"to`*foo()`via`run(..)`

varr3=yieldrun(foo);

console.log(r3);

}

run(bar);

Werun*foo()insideof*bar()byusingourrun(..)utilityagain.Wetakeadvantagehereofthefactthattherun(..)wedefinedearlierreturnsapromisewhichisresolvedwhenitsgeneratorisruntocompletion(orerrorsout),soifweyieldouttoarun(..)instancethepromisefromanotherrun(..)call,itautomaticallypauses*bar()until*foo()finishes.

Butthere'sanevenbetterwaytointegratecalling*foo()into*bar(),andit'scalledyield-delegation.Thespecialsyntaxforyield-delegationis:yield*__(noticetheextra*).Beforeweseeitworkinourpreviousexample,let'slookatasimplerscenario:

function*foo(){

console.log("`*foo()`starting");

yield3;

yield4;

console.log("`*foo()`finished");

}

function*bar(){

yield1;

yield2;

yield*foo();//`yield`-delegation!

yield5;

}

varit=bar();

it.next().value;//1

it.next().value;//2

it.next().value;//`*foo()`starting

//3

it.next().value;//4

it.next().value;//`*foo()`finished

//5

Note:SimilartoanoteearlierinthechapterwhereIexplainedwhyIpreferfunction*foo()..insteadoffunction*foo()..,Ialsoprefer--differingfrommostotherdocumentationonthetopic--tosayyield*foo()insteadofyield*foo().Theplacementofthe*ispurelystylisticanduptoyourbestjudgment.ButIfindtheconsistencyofstylingattractive.

Howdoestheyield*foo()delegationwork?

First,callingfoo()createsaniteratorexactlyaswe'vealreadyseen.Then,yield*delegates/transferstheiteratorinstancecontrol(ofthepresent*bar()generator)overtothisother*foo()iterator.

So,thefirsttwoit.next()callsarecontrolling*bar(),butwhenwemakethethirdit.next()call,now*foo()startsup,andnowwe'recontrolling*foo()insteadof*bar().That'swhyit'scalleddelegation--*bar()delegateditsiteration

controlto*foo().

Assoonastheititeratorcontrolexhauststheentire*foo()iterator,itautomaticallyreturnstocontrolling*bar().

SonowbacktothepreviousexamplewiththethreesequentialAjaxrequests:

function*foo(){



returnr3;

}

function*bar(){


//"delegating"to`*foo()`via`yield*`

varr3=yield*foo();

console.log(r3);

}

run(bar);

Theonlydifferencebetweenthissnippetandtheversionusedearlieristheuseofyield*foo()insteadofthepreviousyieldrun(foo).

Note:yield*yieldsiterationcontrol,notgeneratorcontrol;whenyouinvokethe*foo()generator,you'renowyield-delegatingtoitsiterator.Butyoucanactuallyyield-delegatetoanyiterable;yield*[1,2,3]wouldconsumethedefaultiteratorforthe[1,2,3]arrayvalue.

Thepurposeofyield-delegationismostlycodeorganization,andinthatwayissymmetricalwithnormalfunctioncalling.

Imaginetwomodulesthatrespectivelyprovidemethodsfoo()andbar(),wherebar()callsfoo().Thereasonthetwoareseparateisgenerallybecausetheproperorganizationofcodefortheprogramcallsforthemtobeinseparatefunctions.Forexample,theremaybecaseswherefoo()iscalledstandalone,andotherplaceswherebar()callsfoo().

Foralltheseexactsamereasons,keepinggeneratorsseparateaidsinprogramreadability,maintenance,anddebuggability.Inthatrespect,yield*isasyntacticshortcutformanuallyiteratingoverthestepsof*foo()whileinsideof*bar().

Suchmanualapproachwouldbeespeciallycomplexifthestepsin*foo()wereasynchronous,whichiswhyyou'dprobablyneedtousethatrun(..)utilitytodoit.Andaswe'veshown,yield*foo()eliminatestheneedforasub-instanceoftherun(..)utility(likerun(foo)).

Youmaywonderhowthisyield-delegationworksnotjustwithiteratorcontrolbutwiththetwo-waymessagepassing.Carefullyfollowtheflowofmessagesinandout,throughtheyield-delegation:

function*foo(){

console.log("inside`*foo()`:",yield"B");

console.log("inside`*foo()`:",yield"C");

return"D";

}

function*bar(){

console.log("inside`*bar()`:",yield"A");

WhyDelegation?

DelegatingMessages

//`yield`-delegation!

console.log("inside`*bar()`:",yield*foo());

console.log("inside`*bar()`:",yield"E");

return"F";

}

varit=bar();

console.log("outside:",it.next().value);

//outside:A

console.log("outside:",it.next(1).value);

//inside`*bar()`:1

//outside:B


//inside`*foo()`:2

//outside:C


//inside`*foo()`:3

//inside`*bar()`:D

//outside:E


//inside`*bar()`:4

//outside:F

Payparticularattentiontotheprocessingstepsaftertheit.next(3)call:

1. The3valueispassed(throughtheyield-delegationin*bar())intothewaitingyield"C"expressioninsideof*foo().

2. *foo()thencallsreturn"D",butthisvaluedoesn'tgetreturnedallthewaybacktotheoutsideit.next(3)call.3. Instead,the"D"valueissentastheresultofthewaitingyield*foo()expressioninsideof*bar()--thisyield-

delegationexpressionhasessentiallybeenpausedwhileallof*foo()wasexhausted.So"D"endsupinsideof*bar()forittoprintout.

4. yield"E"iscalledinsideof*bar(),andthe"E"valueisyieldedtotheoutsideastheresultoftheit.next(3)call.

Fromtheperspectiveoftheexternaliterator(it),itdoesn'tappearanydifferentlybetweencontrollingtheinitialgeneratororadelegatedone.

Infact,yield-delegationdoesn'tevenhavetobedirectedtoanothergenerator;itcanjustbedirectedtoanon-generator,generaliterable.Forexample:

function*bar(){

console.log("inside`*bar()`:",yield"A");

//`yield`-delegationtoanon-generator!

console.log("inside`*bar()`:",yield*["B","C","D"]);

console.log("inside`*bar()`:",yield"E");

return"F";

}

varit=bar();


//outside:A


//inside`*bar()`:1

//outside:B


//outside:C


//outside:D


//inside`*bar()`:undefined

//outside:E


//inside`*bar()`:5

//outside:F

Noticethedifferencesinwherethemessageswerereceived/reportedbetweenthisexampleandtheoneprevious.

Moststrikingly,thedefaultarrayiteratordoesn'tcareaboutanymessagessentinvianext(..)calls,sothevalues2,3,and4areessentiallyignored.Also,becausethatiteratorhasnoexplicitreturnvalue(unlikethepreviouslyused*foo()),theyield*expressiongetsanundefinedwhenitfinishes.

Inthesamewaythatyield-delegationtransparentlypassesmessagesthroughinbothdirections,errors/exceptionsalsopassinbothdirections:

function*foo(){

try{

yield"B";

}

catch(err){

console.log("errorcaughtinside`*foo()`:",err);

}

yield"C";

throw"D";

}

function*bar(){

yield"A";

try{

yield*foo();

}

catch(err){

console.log("errorcaughtinside`*bar()`:",err);

}

yield"E";

yield*baz();

//note:can'tgethere!

yield"G";

}

function*baz(){

throw"F";

}

varit=bar();


//outside:A


//outside:B

console.log("outside:",it.throw(2).value);

//errorcaughtinside`*foo()`:2

//outside:C


//errorcaughtinside`*bar()`:D

//outside:E

try{


}

catch(err){

console.log("errorcaughtoutside:",err);

}

ExceptionsDelegated,Too!

//errorcaughtoutside:F

Somethingstonotefromthissnippet:

1. Whenwecallit.throw(2),itsendstheerrormessage2into*bar(),whichdelegatesthatto*foo(),whichthencatchesitandhandlesitgracefully.Then,theyield"C"sends"C"backoutasthereturnvaluefromtheit.throw(2)call.

2. The"D"valuethat'snextthrownfrominside*foo()propagatesoutto*bar(),whichcatchesitandhandlesitgracefully.Thentheyield"E"sends"E"backoutasthereturnvaluefromtheit.next(3)call.

3. Next,theexceptionthrownfrom*baz()isn'tcaughtin*bar()--thoughwedidcatchitoutside--soboth*baz()and*bar()aresettoacompletedstate.Afterthissnippet,youwouldnotbeabletogetthe"G"valueoutwithanysubsequentnext(..)call(s)--theywilljustreturnundefinedforvalue.

Let'sfinallygetbacktoourearlieryield-delegationexamplewiththemultiplesequentialAjaxrequests:

function*foo(){



returnr3;

}

function*bar(){


varr3=yield*foo();

console.log(r3);

}

run(bar);

Insteadofcallingyieldrun(foo)insideof*bar(),wejustcallyield*foo().

Inthepreviousversionofthisexample,thePromisemechanism(controlledbyrun(..))wasusedtotransportthevaluefromreturnr3in*foo()tothelocalvariabler3inside*bar().Now,thatvalueisjustreturnedbackdirectlyviatheyield*mechanics.

Otherwise,thebehaviorisprettymuchidentical.

Ofcourse,yield-delegationcankeepfollowingasmanydelegationstepsasyouwireup.Youcouldevenuseyield-delegationforasync-capablegenerator"recursion"--ageneratoryield-delegatingtoitself:

function*foo(val){

if(val>1){

//generatorrecursion

val=yield*foo(val-1);

}

returnyieldrequest("http://some.url/?v="+val);

}

function*bar(){

varr1=yield*foo(3);

console.log(r1);

}

run(bar);

DelegatingAsynchrony

Delegating"Recursion"

Note:Ourrun(..)utilitycouldhavebeencalledwithrun(foo,3),becauseitsupportsadditionalparametersbeingpassedalongtotheinitializationofthegenerator.However,weusedaparameter-free*bar()heretohighlighttheflexibilityofyield*.

Whatprocessingstepsfollowfromthatcode?Hangon,thisisgoingtobequiteintricatetodescribeindetail:

1. run(bar)startsupthe*bar()generator.2. foo(3)createsaniteratorfor*foo(..)andpasses3asitsvalparameter.3. Because3>1,foo(2)createsanotheriteratorandpassesin2asitsvalparameter.4. Because2>1,foo(1)createsyetanotheriteratorandpassesin1asitsvalparameter.5. 1>1isfalse,sowenextcallrequest(..)withthe1value,andgetapromisebackforthatfirstAjaxcall.6. Thatpromiseisyieldedout,whichcomesbacktothe*foo(2)generatorinstance.7. Theyield*passesthatpromisebackouttothe*foo(3)generatorinstance.Anotheryield*passesthepromise

outtothe*bar()generatorinstance.Andyetagainanotheryield*passesthepromiseouttotherun(..)utility,whichwillwaitonthatpromise(forthefirstAjaxrequest)toproceed.

8. Whenthepromiseresolves,itsfulfillmentmessageissenttoresume*bar(),whichpassesthroughtheyield*intothe*foo(3)instance,whichthenpassesthroughtheyield*tothe*foo(2)generatorinstance,whichthenpassesthroughtheyield*tothenormalyieldthat'swaitinginthe*foo(3)generatorinstance.

9. Thatfirstcall'sAjaxresponseisnowimmediatelyreturnedfromthe*foo(3)generatorinstance,whichsendsthatvaluebackastheresultoftheyield*expressioninthe*foo(2)instance,andassignedtoitslocalvalvariable.

10. Inside*foo(2),asecondAjaxrequestismadewithrequest(..),whosepromiseisyieldedbacktothe*foo(1)instance,andthenyield*propagatesallthewayouttorun(..)(step7again).Whenthepromiseresolves,thesecondAjaxresponsepropagatesallthewaybackintothe*foo(2)generatorinstance,andisassignedtoitslocalvalvariable.

11. Finally,thethirdAjaxrequestismadewithrequest(..),itspromisegoesouttorun(..),andthenitsresolutionvaluecomesallthewayback,whichisthenreturnedsothatitcomesbacktothewaitingyield*expressionin*bar().

Phew!Alotofcrazymentaljuggling,huh?Youmightwanttoreadthroughthatafewmoretimes,andthengograbasnacktoclearyourhead!

AswediscussedinbothChapter1andearlierinthischapter,twosimultaneouslyrunning"processes"cancooperativelyinterleavetheiroperations,andmanytimesthiscanyield(punintended)verypowerfulasynchronyexpressions.

Frankly,ourearlierexamplesofconcurrencyinterleavingofmultiplegeneratorsshowedhowtomakeitreallyconfusing.Butwehintedthatthere'splaceswherethiscapabilityisquiteuseful.

RecallascenariowelookedatinChapter1,wheretwodifferentsimultaneousAjaxresponsehandlersneededtocoordinatewitheachothertomakesurethatthedatacommunicationwasnotaracecondition.Weslottedtheresponsesintotheresarraylikethis:


if(data.url=="http://some.url.1"){

res[0]=data;

}

elseif(data.url=="http://some.url.2"){

res[1]=data;

}

}

Buthowcanweusemultiplegeneratorsconcurrentlyforthisscenario?

//`request(..)ìsaPromise-awareAjaxutility

varres=[];

function*reqData(url){

GeneratorConcurrency

res.push(

yieldrequest(url)

);

}

Note:We'regoingtousetwoinstancesofthe*reqData(..)generatorhere,butthere'snodifferencetorunningasingleinstanceoftwodifferentgenerators;bothapproachesarereasonedaboutidentically.We'llseetwodifferentgeneratorscoordinatinginjustabit.

Insteadofhavingtomanuallysortoutres[0]andres[1]assignments,we'llusecoordinatedorderingsothatres.push(..)properlyslotsthevaluesintheexpectedandpredictableorder.Theexpressedlogicthusshouldfeelabitcleaner.

Buthowwillweactuallyorchestratethisinteraction?First,let'sjustdoitmanually,withPromises:

varit1=reqData("http://some.url.1");


varp1=it1.next();

varp2=it2.next();

p1

.then(function(data){

it1.next(data);

returnp2;

})

.then(function(data){

it2.next(data);

});

*reqData(..)'stwoinstancesarebothstartedtomaketheirAjaxrequests,thenpausedwithyield.Thenwechoosetoresumethefirstinstancewhenp1resolves,andthenp2'sresolutionwillrestartthesecondinstance.Inthisway,weusePromiseorchestrationtoensurethatres[0]willhavethefirstresponseandres[1]willhavethesecondresponse.

Butfrankly,thisisawfullymanual,anditdoesn'treallyletthegeneratorsorchestratethemselves,whichiswherethetruepowercanlie.Let'stryitadifferentway:


varres=[];

function*reqData(url){

vardata=yieldrequest(url);

//transfercontrol

yield;

res.push(data);

}



varp1=it.next();

varp2=it.next();

p1.then(function(data){

it1.next(data);

});

p2.then(function(data){

it2.next(data);

});


.then(function(){

it1.next();

it2.next();

});

OK,thisisabitbetter(thoughstillmanual!),becausenowthetwoinstancesof*reqData(..)runtrulyconcurrently,and(atleastforthefirstpart)independently.

Intheprevioussnippet,thesecondinstancewasnotgivenitsdatauntilafterthefirstinstancewastotallyfinished.Buthere,bothinstancesreceivetheirdataassoonastheirrespectiveresponsescomeback,andtheneachinstancedoesanotheryieldforcontroltransferpurposes.WethenchoosewhatordertoresumetheminthePromise.all([..])handler.

Whatmaynotbeasobviousisthatthisapproachhintsataneasierformforareusableutility,becauseofthesymmetry.Wecandoevenbetter.Let'simagineusingautilitycalledrunAll(..):


varres=[];

runAll(

function*(){


//transfercontrol

yield;

res.push(yieldp1);

},

function*(){


//transfercontrol

yield;

res.push(yieldp2);

}

);

Note:We'renotincludingacodelistingforrunAll(..)asitisnotonlylongenoughtobogdownthetext,butisanextensionofthelogicwe'vealreadyimplementedinrun(..)earlier.So,asagoodsupplementaryexerciseforthereader,tryyourhandatevolvingthecodefromrun(..)toworkliketheimaginedrunAll(..).Also,myasynquencelibraryprovidesapreviouslymentionedrunner(..)utilitywiththiskindofcapabilityalreadybuiltin,andwillbediscussedinAppendixAofthisbook.

Here'showtheprocessinginsiderunAll(..)wouldoperate:

1. ThefirstgeneratorgetsapromiseforthefirstAjaxresponsefrom"http://some.url.1",thenyieldscontrolbacktotherunAll(..)utility.

2. Thesecondgeneratorrunsanddoesthesamefor"http://some.url.2",yieldingcontrolbacktotherunAll(..)utility.

3. Thefirstgeneratorresumes,andthenyieldsoutitspromisep1.TherunAll(..)utilitydoesthesameinthiscaseasourpreviousrun(..),inthatitwaitsonthatpromisetoresolve,thenresumesthesamegenerator(nocontroltransfer!).Whenp1resolves,runAll(..)resumesthefirstgeneratoragainwiththatresolutionvalue,andthenres[0]isgivenitsvalue.Whenthefirstgeneratorthenfinishes,that'sanimplicittransferofcontrol.

4. Thesecondgeneratorresumes,yieldsoutitspromisep2,andwaitsforittoresolve.Onceitdoes,runAll(..)resumesthesecondgeneratorwiththatvalue,andres[1]isset.

Inthisrunningexample,weuseanoutervariablecalledrestostoretheresultsofthetwodifferentAjaxresponses--that'sourconcurrencycoordinationmakingthatpossible.

ButitmightbequitehelpfultofurtherextendrunAll(..)toprovideaninnervariablespaceforthemultiplegeneratorinstancestoshare,suchasanemptyobjectwe'llcalldatabelow.Also,itcouldtakenon-Promisevaluesthatareyieldedandhandthemofftothenextgenerator.

Consider:


runAll(

function*(data){

data.res=[];

//transfercontrol(andmessagepass)

varurl1=yield"http://some.url.2";

varp1=request(url1);//"http://some.url.1"

//transfercontrol

yield;

data.res.push(yieldp1);

},

function*(data){

//transfercontrol(andmessagepass)

varurl2=yield"http://some.url.1";

varp2=request(url2);//"http://some.url.2"

//transfercontrol

yield;

data.res.push(yieldp2);

}

);

Inthisformulation,thetwogeneratorsarenotjustcoordinatingcontroltransfer,butactuallycommunicatingwitheachother,boththroughdata.resandtheyieldedmessagesthattradeurl1andurl2values.That'sincrediblypowerful!

SuchrealizationalsoservesasaconceptualbaseforamoresophisticatedasynchronytechniquecalledCSP(CommunicatingSequentialProcesses),whichwewillcoverinAppendixBofthisbook.

Sofar,we'vemadetheassumptionthatyieldingaPromisefromagenerator--andhavingthatPromiseresumethegeneratorviaahelperutilitylikerun(..)--wasthebestpossiblewaytomanageasynchronywithgenerators.Tobeclear,itis.

Butweskippedoveranotherpatternthathassomemildlywidespreadadoption,sointheinterestofcompletenesswe'lltakeabrieflookatit.

Ingeneralcomputerscience,there'sanoldpre-JSconceptcalleda"thunk."Withoutgettingboggeddowninthehistoricalnature,anarrowexpressionofathunkinJSisafunctionthat--withoutanyparameters--iswiredtocallanotherfunction.

Inotherwords,youwrapafunctiondefinitionaroundfunctioncall--withanyparametersitneeds--todefertheexecutionofthatcall,andthatwrappingfunctionisathunk.Whenyoulaterexecutethethunk,youendupcallingtheoriginalfunction.

Forexample:

functionfoo(x,y){

returnx+y;

}

functionfooThunk(){

returnfoo(3,4);

}

//later

console.log(fooThunk());//7

Thunks

So,asynchronousthunkisprettystraightforward.Butwhataboutanasyncthunk?Wecanessentiallyextendthenarrowthunkdefinitiontoincludeitreceivingacallback.

Consider:



cb(x+y);

},1000);

}

functionfooThunk(cb){

foo(3,4,cb);

}

//later

fooThunk(function(sum){

console.log(sum);//7

});

Asyoucansee,fooThunk(..)onlyexpectsacb(..)parameter,asitalreadyhasvalues3and4(forxandy,respectively)pre-specifiedandreadytopasstofoo(..).Athunkisjustwaitingaroundpatientlyforthelastpieceitneedstodoitsjob:thecallback.

Youdon'twanttomakethunksmanually,though.So,let'sinventautilitythatdoesthiswrappingforus.

Consider:

functionthunkify(fn){

varargs=[].slice.call(arguments,1);

returnfunction(cb){

args.push(cb);

returnfn.apply(null,args);

};

}

varfooThunk=thunkify(foo,3,4);

//later



});

Tip:Hereweassumethattheoriginal(foo(..))functionsignatureexpectsitscallbackinthelastposition,withanyotherparameterscomingbeforeit.Thisisaprettyubiquitous"standard"forasyncJSfunctionstandards.Youmightcallit"callback-laststyle."Ifforsomereasonyouhadaneedtohandle"callback-firststyle"signatures,youwouldjustmakeautilitythatusedargs.unshift(..)insteadofargs.push(..).

Theprecedingformulationofthunkify(..)takesboththefoo(..)functionreference,andanyparametersitneeds,andreturnsbackthethunkitself(fooThunk(..)).However,that'snotthetypicalapproachyou'llfindtothunksinJS.

Insteadofthunkify(..)makingthethunkitself,typically--ifnotperplexingly--thethunkify(..)utilitywouldproduceafunctionthatproducesthunks.

Uhhhh...yeah.

Consider:

functionthunkify(fn){

returnfunction(){


returnfunction(cb){

args.push(cb);

returnfn.apply(null,args);

};

};

}

Themaindifferencehereistheextrareturnfunction(){..}layer.Here'showitsusagediffers:

varwhatIsThis=thunkify(foo);

varfooThunk=whatIsThis(3,4);

//later



});

Obviously,thebigquestionthissnippetimpliesiswhatiswhatIsThisproperlycalled?It'snotthethunk,it'sthethingthatwillproducethunksfromfoo(..)calls.It'skindoflikea"factory"for"thunks."Theredoesn'tseemtobeanykindofstandardagreementfornamingsuchathing.

So,myproposalis"thunkory"("thunk"+"factory").So,thunkify(..)producesathunkory,andathunkoryproducesthunks.Thatreasoningissymmetrictomyproposalfor"promisory"inChapter3:

varfooThunkory=thunkify(foo);

varfooThunk1=fooThunkory(3,4);


//later

fooThunk1(function(sum){


});

fooThunk2(function(sum){


});

Note:Therunningfoo(..)exampleexpectsastyleofcallbackthat'snot"error-firststyle."Ofcourse,"error-firststyle"ismuchmorecommon.Iffoo(..)hadsomesortoflegitimateerror-producingexpectation,wecouldchangeittoexpectanduseanerror-firstcallback.Noneofthesubsequentthunkify(..)machinerycareswhatstyleofcallbackisassumed.TheonlydifferenceinusagewouldbefooThunk1(function(err,sum){...

Exposingthethunkorymethod--insteadofhowtheearlierthunkify(..)hidesthisintermediarystep--mayseemlikeunnecessarycomplication.Butingeneral,it'squiteusefultomakethunkoriesatthebeginningofyourprogramtowrapexistingAPImethods,andthenbeabletopassaroundandcallthosethunkorieswhenyouneedthunks.Thetwodistinctstepspreserveacleanerseparationofcapability.

Toillustrate:

//cleaner:




//insteadof:

varfooThunk1=thunkify(foo,3,4);

varfooThunk2=thunkify(foo,5,6);

Regardlessofwhetheryouliketodealwiththethunkoriesexplicitlyornot,theusageofthunksfooThunk1(..)andfooThunk2(..)remainsthesame.

Sowhat'sallthisthunkstuffhavetodowithgenerators?

Comparingthunkstopromisesgenerally:they'renotdirectlyinterchangableasthey'renotequivalentinbehavior.Promisesarevastlymorecapableandtrustablethanbarethunks.

Butinanothersense,theybothcanbeseenasarequestforavalue,whichmaybeasyncinitsanswering.

RecallfromChapter3wedefinedautilityforpromisifyingafunction,whichwecalledPromise.wrap(..)--wecouldhavecalleditpromisify(..),too!ThisPromise-wrappingutilitydoesn'tproducePromises;itproducespromisoriesthatinturnproducePromises.Thisiscompletelysymmetrictothethunkoriesandthunkspresentlybeingdiscussed.

Toillustratethesymmetry,let'sfirstaltertherunningfoo(..)examplefromearliertoassumean"error-firststyle"callback:



//assume`cb(..)às"error-firststyle"

cb(null,x+y);

},1000);

}

Now,we'llcompareusingthunkify(..)andpromisify(..)(akaPromise.wrap(..)fromChapter3):

//symmetrical:constructingthequestionasker


varfooPromisory=promisify(foo);

//symmetrical:askingthequestion

varfooThunk=fooThunkory(3,4);

varfooPromise=fooPromisory(3,4);

//getthethunkanswer

fooThunk(function(err,sum){

if(err){

console.error(err);

}

else{


}

});

//getthepromiseanswer

fooPromise

.then(

function(sum){


},

function(err){

console.error(err);

}

);

Boththethunkoryandthepromisoryareessentiallyaskingaquestion(foravalue),andrespectivelythethunkfooThunkandpromisefooPromiserepresentthefutureanswerstothatquestion.Presentedinthatlight,thesymmetryisclear.

Withthatperspectiveinmind,wecanseethatgeneratorswhichyieldPromisesforasynchronycouldinsteadyieldthunksforasynchrony.Allwe'dneedisasmarterrun(..)utility(likefrombefore)thatcannotonlylookforandwireuptoayieldedPromisebutalsotoprovideacallbacktoayieldedthunk.

Consider:

function*foo(){

varval=yieldrequest("http://some.url.1");

console.log(val);

s/promise/thunk/

}

run(foo);

Inthisexample,request(..)couldeitherbeapromisorythatreturnsapromise,orathunkorythatreturnsathunk.Fromtheperspectiveofwhat'sgoingoninsidethegeneratorcodelogic,wedon'tcareaboutthatimplementationdetail,whichisquitepowerful!

So,request(..)couldbeeither:

//promisory`request(..)`(seeChapter3)


//vs.

//thunkory`request(..)`

varrequest=thunkify(ajax);

Finally,asathunk-awarepatchtoourearlierrun(..)utility,wewouldneedlogiclikethis:

//..

//didwereceiveathunkback?

elseif(typeofnext.value=="function"){


//callthethunkwithanerror-firstcallback

next.value(function(err,msg){

if(err){

reject(err);

}

else{

resolve(msg);

}

});

})

.then(

handleNext,

functionhandleErr(err){

returnPromise.resolve(

it.throw(err)

)

.then(handleResult);

}

);

}

Now,ourgeneratorscaneithercallpromisoriestoyieldPromises,orcallthunkoriestoyieldthunks,andineithercase,run(..)wouldhandlethatvalueanduseittowaitforthecompletiontoresumethegenerator.

Symmetrywise,thesetwoapproacheslookidentical.However,weshouldpointoutthat'strueonlyfromtheperspectiveofPromisesorthunksrepresentingthefuturevaluecontinuationofagenerator.

Fromthelargerperspective,thunksdonotinandofthemselveshavehardlyanyofthetrustabilityorcomposabilityguaranteesthatPromisesaredesignedwith.Usingathunkasastand-inforaPromiseinthisparticulargeneratorasynchronypatternisworkablebutshouldbeseenaslessthanidealwhencomparedtoallthebenefitsthatPromisesoffer(seeChapter3).

Ifyouhavetheoption,preferyieldprratherthanyieldth.Butthere'snothingwrongwithhavingarun(..)utilitywhichcanhandlebothvaluetypes.

Note:Therunner(..)utilityinmyasynquencelibrary,whichwillbediscussedinAppendixA,handlesyieldsofPromises,thunksandasynquencesequences.

Pre-ES6Generators

You'rehopefullyconvincednowthatgeneratorsareaveryimportantadditiontotheasyncprogrammingtoolbox.Butit'sanewsyntaxinES6,whichmeansyoucan'tjustpolyfillgeneratorslikeyoucanPromises(whicharejustanewAPI).SowhatcanwedotobringgeneratorstoourbrowserJSifwedon'thavetheluxuryofignoringpre-ES6browsers?

ForallnewsyntaxextensionsinES6,therearetools--themostcommontermforthemistranspilers,fortrans-compilers--whichcantakeyourES6syntaxandtransformitintoequivalent(butobviouslyuglier!)pre-ES6code.So,generatorscanbetranspiledintocodethatwillhavethesamebehaviorbutworkinES5andbelow.

Buthow?The"magic"ofyielddoesn'tobviouslysoundlikecodethat'seasytotranspile.Weactuallyhintedatasolutioninourearlierdiscussionofclosure-basediterators.

Beforewediscussthetranspilers,let'sderivehowmanualtranspilationwouldworkinthecaseofgenerators.Thisisn'tjustanacademicexercise,becausedoingsowillactuallyhelpfurtherreinforcehowtheywork.

Consider:


function*foo(url){

try{

console.log("requesting:",url);

varval=yieldrequest(url);

console.log(val);

}

catch(err){

console.log("Oops:",err);

returnfalse;

}

}

varit=foo("http://some.url.1");

Thefirstthingtoobserveisthatwe'llstillneedanormalfoo()functionthatcanbecalled,anditwillstillneedtoreturnaniterator.So,let'ssketchoutthenon-generatortransformation:

functionfoo(url){

//..

//makeandreturnaniterator

return{

next:function(v){

//..

},

throw:function(e){

//..

}

};

}

varit=foo("http://some.url.1");

Thenextthingtoobserveisthatageneratordoesits"magic"bysuspendingitsscope/state,butwecanemulatethatwithfunctionclosure(seetheScope&Closurestitleofthisseries).Tounderstandhowtowritesuchcode,we'llfirstannotatedifferentpartsofourgeneratorwithstatevalues:


function*foo(url){

//STATE*1*

try{


ManualTransformation

varTMP1=request(url);

//STATE*2*

varval=yieldTMP1;

console.log(val);

}

catch(err){

//STATE*3*


returnfalse;

}

}

Note:Formoreaccurateillustration,wesplituptheval=yieldrequest..statementintotwoparts,usingthetemporaryTMP1variable.request(..)happensinstate*1*,andtheassignmentofitscompletionvaluetovalhappensinstate*2*.We'llgetridofthatintermediateTMP1whenweconvertthecodetoitsnon-generatorequivalent.

Inotherwords,*1*isthebeginningstate,*2*isthestateiftherequest(..)succeeds,and*3*isthestateiftherequest(..)fails.Youcanprobablyimaginehowanyextrayieldstepswouldjustbeencodedasextrastates.

Backtoourtranspiledgenerator,let'sdefineavariablestateintheclosurewecanusetokeeptrackofthestate:

functionfoo(url){

//managegeneratorstate

varstate;

//..

}

Now,let'sdefineaninnerfunctioncalledprocess(..)insidetheclosurewhichhandleseachstate,usingaswitchstatement:


functionfoo(url){


varstate;

//generator-widevariabledeclarations

varval;

functionprocess(v){

switch(state){

case1:


returnrequest(url);

case2:

val=v;

console.log(val);

return;

case3:

varerr=v;


returnfalse;

}

}

//..

}

Eachstateinourgeneratorisrepresentedbyitsowncaseintheswitchstatement.process(..)willbecalledeachtimeweneedtoprocessanewstate.We'llcomebacktohowthatworksinjustamoment.

Foranygenerator-widevariabledeclarations(val),wemovethosetoavardeclarationoutsideofprocess(..)sotheycansurvivemultiplecallstoprocess(..).Butthe"blockscoped"errvariableisonlyneededforthe*3*state,soweleaveitinplace.

Instate*1*,insteadofyieldresolve(..),wedidreturnresolve(..).Interminalstate*2*,therewasnoexplicitreturn,sowejustdoareturn;whichisthesameasreturnundefined.Interminalstate*3*,therewasareturnfalse,sowepreservethat.

Nowweneedtodefinethecodeintheiteratorfunctionssotheycallprocess(..)appropriately:

functionfoo(url){


varstate;

//generator-widevariabledeclarations

varval;

functionprocess(v){

switch(state){

case1:


returnrequest(url);

case2:

val=v;

console.log(val);

return;

case3:

varerr=v;


returnfalse;

}

}

//makeandreturnaniterator

return{

next:function(v){

//initialstate

if(!state){

state=1;

return{

done:false,

value:process()

};

}

//yieldresumedsuccessfully

elseif(state==1){

state=2;

return{

done:true,

value:process(v)

};

}

//generatoralreadycompleted

else{

return{

done:true,

value:undefined

};

}

},

"throw":function(e){

//theonlyexpliciterrorhandlingisin

//state*1*

if(state==1){

state=3;

return{

done:true,

value:process(e)

};

}

//otherwise,anerrorwon'tbehandled,

//sojustthrowitrightbackout

else{

throwe;

}

}

};

}

Howdoesthiscodework?

1. Thefirstcalltotheiterator'snext()callwouldmovethegeneratorfromtheunitializedstatetostate1,andthencallprocess()tohandlethatstate.Thereturnvaluefromrequest(..),whichisthepromisefortheAjaxresponse,isreturnedbackasthevaluepropertyfromthenext()call.

2. IftheAjaxrequestsucceeds,thesecondcalltonext(..)shouldsendintheAjaxresponsevalue,whichmovesourstateto2.process(..)isagaincalled(thistimewiththepassedinAjaxresponsevalue),andthevaluepropertyreturnedfromnext(..)willbeundefined.

3. However,iftheAjaxrequestfails,throw(..)shouldbecalledwiththeerror,whichwouldmovethestatefrom1to3(insteadof2).Againprocess(..)iscalled,thistimewiththeerrorvalue.Thatcasereturnsfalse,whichissetasthevaluepropertyreturnedfromthethrow(..)call.

Fromtheoutside--thatis,interactingonlywiththeiterator--thisfoo(..)normalfunctionworksprettymuchthesameasthe*foo(..)generatorwouldhaveworked.Sowe'veeffectively"transpiled"ourES6generatortopre-ES6compatibility!

Wecouldthenmanuallyinstantiateourgeneratorandcontrolitsiterator--callingvarit=foo("..")andit.next(..)andsuch--orbetter,wecouldpassittoourpreviouslydefinedrun(..)utilityasrun(foo,"..").

TheprecedingexerciseofmanuallyderivingatransformationofourES6generatortopre-ES6equivalentteachesushowgeneratorsworkconceptually.Butthattransformationwasreallyintricateandverynon-portabletoothergeneratorsinourcode.Itwouldbequiteimpracticaltodothisworkbyhand,andwouldcompletelyobviateallthebenefitofgenerators.

Butluckily,severaltoolsalreadyexistthatcanautomaticallyconvertES6generatorstothingslikewhatwederivedintheprevioussection.Notonlydotheydotheheavyliftingworkforus,buttheyalsohandleseveralcomplicationsthatweglossedover.

Onesuchtoolisregenerator(https://facebook.github.io/regenerator/),fromthesmartfolksatFacebook.

Ifweuseregeneratortotranspileourpreviousgenerator,here'sthecodeproduced(atthetimeofthiswriting):


varfoo=regeneratorRuntime.mark(functionfoo(url){

varval;

returnregeneratorRuntime.wrap(functionfoo$(context$1$0){

while(1)switch(context$1$0.prev=context$1$0.next){

case0:

context$1$0.prev=0;


context$1$0.next=4;

returnrequest(url);

case4:

val=context$1$0.sent;

console.log(val);

context$1$0.next=12;

break;

case8:

context$1$0.prev=8;

context$1$0.t0=context$1$0.catch(0);

console.log("Oops:",context$1$0.t0);

returncontext$1$0.abrupt("return",false);

case12:

case"end":

returncontext$1$0.stop();

}

},foo,this,[[0,8]]);

});

There'ssomeobvioussimilaritiesheretoourmanualderivation,suchastheswitch/casestatements,andweevenseevalpulledoutoftheclosurejustaswedid.

Ofcourse,onetrade-offisthatregenerator'stranspilationrequiresahelperlibraryregeneratorRuntimethatholdsallthereusablelogicformanagingageneralgenerator/iterator.Alotofthatboilerplatelooksdifferentthanourversion,buteven

AutomaticTranspilation

https://facebook.github.io/regenerator/

then,theconceptscanbeseen,likewithcontext$1$0.next=4keepingtrackofthenextstateforthegenerator.

ThemaintakeawayisthatgeneratorsarenotrestrictedtoonlybeingusefulinES6+environments.Onceyouunderstandtheconcepts,youcanemploythemthroughoutyourcode,andusetoolstotransformthecodetobecompatiblewitholderenvironments.

ThisismoreworkthanjustusingaPromiseAPIpolyfillforpre-ES6Promises,buttheeffortistotallyworthit,becausegeneratorsaresomuchbetteratexpressingasyncflowcontrolinareason-able,sensible,synchronous-looking,sequentialfashion.

Onceyougethookedongenerators,you'llneverwanttogobacktothehellofasyncspaghetticallbacks!

GeneratorsareanewES6functiontypethatdoesnotrun-to-completionlikenormalfunctions.Instead,thegeneratorcanbepausedinmid-completion(entirelypreservingitsstate),anditcanlaterberesumedfromwhereitleftoff.

Thispause/resumeinterchangeiscooperativeratherthanpreemptive,whichmeansthatthegeneratorhasthesolecapabilitytopauseitself,usingtheyieldkeyword,andyettheiteratorthatcontrolsthegeneratorhasthesolecapability(vianext(..))toresumethegenerator.

Theyield/next(..)dualityisnotjustacontrolmechanism,it'sactuallyatwo-waymessagepassingmechanism.Ayield..expressionessentiallypauseswaitingforavalue,andthenextnext(..)callpassesavalue(orimplicitundefined)backtothatpausedyieldexpression.

Thekeybenefitofgeneratorsrelatedtoasyncflowcontrolisthatthecodeinsideageneratorexpressesasequenceofstepsforthetaskinanaturallysync/sequentialfashion.Thetrickisthatweessentiallyhidepotentialasynchronybehindtheyieldkeyword--movingtheasynchronytothecodewherethegenerator'siteratoriscontrolled.

Inotherwords,generatorspreserveasequential,synchronous,blockingcodepatternforasynccode,whichletsourbrainsreasonaboutthecodemuchmorenaturally,addressingoneofthetwokeydrawbacksofcallback-basedasync.

Review

Thisbooksofarhasbeenallabouthowtoleverageasynchronypatternsmoreeffectively.Butwehaven'tdirectlyaddressedwhyasynchronyreallymatterstoJS.Themostobviousexplicitreasonisperformance.

Forexample,ifyouhavetwoAjaxrequeststomake,andthey'reindependent,butyouneedtowaitonthembothtofinishbeforedoingthenexttask,youhavetwooptionsformodelingthatinteraction:serialandconcurrent.

Youcouldmakethefirstrequestandwaittostartthesecondrequestuntilthefirstfinishes.Or,aswe'veseenbothwithpromisesandgenerators,youcouldmakebothrequests"inparallel,"andexpressthe"gate"towaitonbothofthembeforemovingon.

Clearly,thelatterisusuallygoingtobemoreperformantthantheformer.Andbetterperformancegenerallyleadstobetteruserexperience.

It'sevenpossiblethatasynchrony(interleavedconcurrency)canimprovejusttheperceptionofperformance,eveniftheoverallprogramstilltakesthesameamountoftimetocomplete.Userperceptionofperformanceiseverybit--ifnotmore!--asimportantasactualmeasurableperformance.

Wewanttonowmovebeyondlocalizedasynchronypatternstotalkaboutsomebiggerpictureperformancedetailsattheprogramlevel.

Note:Youmaybewonderingaboutmicro-performanceissueslikeifa++or++aisfaster.We'lllookatthosesortsofperformancedetailsinthenextchapteron"Benchmarking&Tuning."

Ifyouhaveprocessing-intensivetasksbutyoudon'twantthemtorunonthemainthread(whichmayslowdownthebrowser/UI),youmighthavewishedthatJavaScriptcouldoperateinamultithreadedmanner.

InChapter1,wetalkedindetailabouthowJavaScriptissinglethreaded.Andthat'sstilltrue.Butasinglethreadisn'ttheonlywaytoorganizetheexecutionofyourprogram.

Imaginesplittingyourprogramintotwopieces,andrunningoneofthosepiecesonthemainUIthread,andrunningtheotherpieceonanentirelyseparatethread.

Whatkindsofconcernswouldsuchanarchitecturebringup?

Forone,you'dwanttoknowifrunningonaseparatethreadmeantthatitraninparallel(onsystemswithmultipleCPUs/cores)suchthatalong-runningprocessonthatsecondthreadwouldnotblockthemainprogramthread.Otherwise,"virtualthreading"wouldn'tbeofmuchbenefitoverwhatwealreadyhaveinJSwithasyncconcurrency.

Andyou'dwanttoknowifthesetwopiecesoftheprogramhaveaccesstothesamesharedscope/resources.Iftheydo,thenyouhaveallthequestionsthatmultithreadedlanguages(Java,C++,etc.)dealwith,suchasneedingcooperativeorpreemptivelocking(mutexes,etc.).That'salotofextrawork,andshouldn'tbeundertakenlightly.

Alternatively,you'dwanttoknowhowthesetwopiecescould"communicate"iftheycouldn'tsharescope/resources.

AllthesearegreatquestionstoconsiderasweexploreafeatureaddedtothewebplatformcircaHTML5called"WebWorkers."Thisisafeatureofthebrowser(akahostenvironment)andactuallyhasalmostnothingtodowiththeJSlanguageitself.Thatis,JavaScriptdoesnotcurrentlyhaveanyfeaturesthatsupportthreadedexecution.


Chapter5:ProgramPerformance

WebWorkers

ButanenvironmentlikeyourbrowsercaneasilyprovidemultipleinstancesoftheJavaScriptengine,eachonitsownthread,andletyourunadifferentprogramineachthread.Eachofthoseseparatethreadedpiecesofyourprogramiscalleda"(Web)Worker."Thistypeofparallelismiscalled"taskparallelism,"astheemphasisisonsplittingupchunksofyourprogramtoruninparallel.

FromyourmainJSprogram(oranotherWorker),youinstantiateaWorkerlikeso:

varw1=newWorker("http://some.url.1/mycoolworker.js");

TheURLshouldpointtothelocationofaJSfile(notanHTMLpage!)whichisintendedtobeloadedintoaWorker.Thebrowserwillthenspinupaseparatethreadandletthatfilerunasanindependentprograminthatthread.

Note:ThekindofWorkercreatedwithsuchaURLiscalleda"DedicatedWorker."ButinsteadofprovidingaURLtoanexternalfile,youcanalsocreatean"InlineWorker"byprovidingaBlobURL(anotherHTML5feature);essentiallyit'saninlinefilestoredinasingle(binary)value.However,Blobsarebeyondthescopeofwhatwe'lldiscusshere.

Workersdonotshareanyscopeorresourceswitheachotherorthemainprogram--thatwouldbringallthenightmaresoftheadedprogrammingtotheforefront--butinsteadhaveabasiceventmessagingmechanismconnectingthem.

Thew1Workerobjectisaneventlistenerandtrigger,whichletsyousubscribetoeventssentbytheWorkeraswellassendeventstotheWorker.

Here'showtolistenforevents(actually,thefixed"message"event):

w1.addEventListener("message",function(evt){

//evt.data

});

Andyoucansendthe"message"eventtotheWorker:

w1.postMessage("somethingcooltosay");

InsidetheWorker,themessagingistotallysymmetrical:

//"mycoolworker.js"

addEventListener("message",function(evt){

//evt.data

});

postMessage("areallycoolreply");

NoticethatadedicatedWorkerisinaone-to-onerelationshipwiththeprogramthatcreatedit.Thatis,the"message"eventdoesn'tneedanydisambiguationhere,becausewe'resurethatitcouldonlyhavecomefromthisone-to-onerelationship--eitheritcamefromtheWorkerorthemainpage.

UsuallythemainpageapplicationcreatestheWorkers,butaWorkercaninstantiateitsownchildWorker(s)--knownassubworkers--asnecessary.Sometimesthisisusefultodelegatesuchdetailstoasortof"master"WorkerthatspawnsotherWorkerstoprocesspartsofatask.Unfortunately,atthetimeofthiswriting,Chromestilldoesnotsupportsubworkers,whileFirefoxdoes.

TokillaWorkerimmediatelyfromtheprogramthatcreatedit,callterminate()ontheWorkerobject(likew1intheprevioussnippets).AbruptlyterminatingaWorkerthreaddoesnotgiveitanychancetofinishupitsworkorcleanupanyresources.It'sakintoyouclosingabrowsertabtokillapage.

Ifyouhavetwoormorepages(ormultipletabswiththesamepage!)inthebrowserthattrytocreateaWorkerfromthesamefileURL,thosewillactuallyendupascompletelyseparateWorkers.Shortly,we'lldiscussawayto"share"aWorker.

Note:ItmayseemlikeamaliciousorignorantJSprogramcouldeasilyperformadenial-of-serviceattackonasystembyspawninghundredsofWorkers,seeminglyeachwiththeirownthread.Whileit'struethatit'ssomewhatofaguaranteethataWorkerwillenduponaseparatethread,thisguaranteeisnotunlimited.Thesystemisfreetodecidehowmanyactualthreads/CPUs/coresitreallywantstocreate.There'snowaytopredictorguaranteehowmanyyou'llhaveaccessto,thoughmanypeopleassumeit'satleastasmanyasthenumberofCPUs/coresavailable.Ithinkthesafestassumptionisthatthere'satleastoneotherthreadbesidesthemainUIthread,butthat'saboutit.

InsidetheWorker,youdonothaveaccesstoanyofthemainprogram'sresources.Thatmeansyoucannotaccessanyofitsglobalvariables,norcanyouaccessthepage'sDOMorotherresources.Remember:it'satotallyseparatethread.

Youcan,however,performnetworkoperations(Ajax,WebSockets)andsettimers.Also,theWorkerhasaccesstoitsowncopyofseveralimportantglobalvariables/features,includingnavigator,location,JSON,andapplicationCache.

YoucanalsoloadextraJSscriptsintoyourWorker,usingimportScripts(..):

//insidetheWorker

importScripts("foo.js","bar.js");

Thesescriptsareloadedsynchronously,whichmeanstheimportScripts(..)callwillblocktherestoftheWorker'sexecutionuntilthefile(s)arefinishedloadingandexecuting.

Note:Therehavealsobeensomediscussionsaboutexposingthe<canvas>APItoWorkers,whichcombinedwithhavingcanvasesbeTransferables(seethe"DataTransfer"section),wouldallowWorkerstoperformmoresophisticatedoff-threadgraphicsprocessing,whichcanbeusefulforhigh-performancegaming(WebGL)andothersimilarapplications.Althoughthisdoesn'texistyetinanybrowsers,it'slikelytohappeninthenearfuture.

WhataresomecommonusesforWebWorkers?

ProcessingintensivemathcalculationsSortinglargedatasetsDataoperations(compression,audioanalysis,imagepixelmanipulations,etc.)High-trafficnetworkcommunications

Youmaynoticeacommoncharacteristicofmostofthoseuses,whichisthattheyrequirealargeamountofinformationtobetransferredacrossthebarrierbetweenthreadsusingtheeventmechanism,perhapsinbothdirections.

IntheearlydaysofWorkers,serializingalldatatoastringvaluewastheonlyoption.Inadditiontothespeedpenaltyofthetwo-wayserializations,theothermajornegativewasthatthedatawasbeingcopied,whichmeantadoublingofmemoryusage(andthesubsequentchurnofgarbagecollection).

Thankfully,wenowhaveafewbetteroptions.

Ifyoupassanobject,aso-called"StructuredCloningAlgorithm"(https://developer.mozilla.org/en-US/docs/Web/Guide/API/DOM/The_structured_clone_algorithm)isusedtocopy/duplicatetheobjectontheotherside.Thisalgorithmisfairlysophisticatedandcanevenhandleduplicatingobjectswithcircularreferences.Theto-string/from-stringperformancepenaltyisnotpaid,butwestillhaveduplicationofmemoryusingthisapproach.ThereissupportforthisinIE10andabove,aswellasalltheothermajorbrowsers.

Anevenbetteroption,especiallyforlargerdatasets,is"TransferableObjects"(http://updates.html5rocks.com/2011/12/Transferable-Objects-Lightning-Fast).Whathappensisthattheobject's

WorkerEnvironment

DataTransfer

https://developer.mozilla.org/en-US/docs/Web/Guide/API/DOM/The_structured_clone_algorithm

http://updates.html5rocks.com/2011/12/Transferable-Objects-Lightning-Fast

"ownership"istransferred,butthedataitselfisnotmoved.OnceyoutransferawayanobjecttoaWorker,it'semptyorinaccessibleinthetheoriginatinglocation--thateliminatesthehazardsofthreadedprogrammingoverasharedscope.Ofcourse,transferofownershipcangoinbothdirections.

Therereallyisn'tmuchyouneedtodotooptintoaTransferableObject;anydatastructurethatimplementstheTransferableinterface(https://developer.mozilla.org/en-US/docs/Web/API/Transferable)willautomaticallybetransferredthisway(supportFirefox&Chrome).

Forexample,typedarrayslikeUint8Array(seetheES6&Beyondtitleofthisseries)are"Transferables."Thisishowyou'dsendaTransferableObjectusingpostMessage(..):

//`fooìsaÙint8Array`forinstance

postMessage(foo.buffer,[foo.buffer]);

Thefirstparameteristherawbufferandthesecondparameterisalistofwhattotransfer.

Browsersthatdon'tsupportTransferableObjectssimplydegradetostructuredcloning,whichmeansperformancereductionratherthanoutrightfeaturebreakage.

Ifyoursiteorappallowsforloadingmultipletabsofthesamepage(acommonfeature),youmayverywellwanttoreducetheresourceusageoftheirsystembypreventingduplicatededicatedWorkers;themostcommonlimitedresourceinthisrespectisasocketnetworkconnection,asbrowserslimitthenumberofsimultaneousconnectionstoasinglehost.Ofcourse,limitingmultipleconnectionsfromaclientalsoeasesyourserverresourcerequirements.

Inthiscase,creatingasinglecentralizedWorkerthatallthepageinstancesofyoursiteorappcanshareisquiteuseful.

That'scalledaSharedWorker,whichyoucreatelikeso(supportforthisislimitedtoFirefoxandChrome):

varw1=newSharedWorker("http://some.url.1/mycoolworker.js");

BecauseasharedWorkercanbeconnectedtoorfrommorethanoneprograminstanceorpageonyoursite,theWorkerneedsawaytoknowwhichprogramamessagecomesfrom.Thisuniqueidentificationiscalleda"port"--thinknetworksocketports.SothecallingprogrammustusetheportobjectoftheWorkerforcommunication:

w1.port.addEventListener("message",handleMessages);

//..

w1.port.postMessage("somethingcool");

Also,theportconnectionmustbeinitialized,as:

w1.port.start();

InsidethesharedWorker,anextraeventmustbehandled:"connect".Thiseventprovidestheportobjectforthatparticularconnection.Themostconvenientwaytokeepmultipleconnectionsseparateistouseclosure(seeScope&Closurestitleofthisseries)overtheport,asshownnext,withtheeventlisteningandtransmittingforthatconnectiondefinedinsidethehandlerforthe"connect"event:

//insidethesharedWorker

addEventListener("connect",function(evt){

//theassignedportforthisconnection

SharedWorkers

https://developer.mozilla.org/en-US/docs/Web/API/Transferable

varport=evt.ports[0];

port.addEventListener("message",function(evt){

//..

port.postMessage(..);

//..

});

//initializetheportconnection

port.start();

});

Otherthanthatdifference,sharedanddedicatedWorkershavethesamecapabilitiesandsemantics.

Note:SharedWorkerssurvivetheterminationofaportconnectionifotherportconnectionsarestillalive,whereasdedicatedWorkersareterminatedwhenevertheconnectiontotheirinitiatingprogramisterminated.

WebWorkersareveryattractiveperformance-wiseforrunningJSprogramsinparallel.However,youmaybeinapositionwhereyourcodeneedstoruninolderbrowsersthatlacksupport.BecauseWorkersareanAPIandnotasyntax,theycanbepolyfilled,toanextent.

Ifabrowserdoesn'tsupportWorkers,there'ssimplynowaytofakemultithreadingfromtheperformanceperspective.Iframesarecommonlythoughtoftoprovideaparallelenvironment,butinallmodernbrowserstheyactuallyrunonthesamethreadasthemainpage,sothey'renotsufficientforfakingparallelism.

AswedetailedinChapter1,JS'sasynchronicity(notparallelism)comesfromtheeventloopqueue,soyoucanforcefakedWorkerstobeasynchronoususingtimers(setTimeout(..),etc.).ThenyoujustneedtoprovideapolyfillfortheWorkerAPI.Therearesomelistedhere(https://github.com/Modernizr/Modernizr/wiki/HTML5-Cross-Browser-Polyfills#web-workers),butfranklynoneofthemlookgreat.

I'vewrittenasketchofapolyfillforWorkerhere(https://gist.github.com/getify/1b26accb1a09aa53ad25).It'sbasic,butitshouldgetthejobdoneforsimpleWorkersupport,giventhatthetwo-waymessagingworkscorrectlyaswellas"onerror"handling.Youcouldprobablyalsoextenditwithmorefeatures,suchasterminate()orfakedSharedWorkers,asyouseefit.

Note:Youcan'tfakesynchronousblocking,sothispolyfilljustdisallowsuseofimportScripts(..).AnotheroptionmighthavebeentoparseandtransformtheWorker'scode(onceAjaxloaded)tohandlerewritingtosomeasynchronousformofanimportScripts(..)polyfill,perhapswithapromise-awareinterface.

Singleinstruction,multipledata(SIMD)isaformof"dataparallelism,"ascontrastedto"taskparallelism"withWebWorkers,becausetheemphasisisnotreallyonprogramlogicchunksbeingparallelized,butrathermultiplebitsofdatabeingprocessedinparallel.

WithSIMD,threadsdon'tprovidetheparallelism.Instead,modernCPUsprovideSIMDcapabilitywith"vectors"ofnumbers--think:typespecializedarrays--aswellasinstructionsthatcanoperateinparallelacrossallthenumbers;thesearelow-leveloperationsleveraginginstruction-levelparallelism.

TheefforttoexposeSIMDcapabilitytoJavaScriptisprimarilyspearheadedbyIntel(https://01.org/node/1495),namelybyMohammadHaghighat(atthetimeofthiswriting),incooperationwithFirefoxandChrometeams.SIMDisonanearlystandardstrackwithagoodchanceofmakingitintoafuturerevisionofJavaScript,likelyintheES7timeframe.

SIMDJavaScriptproposestoexposeshortvectortypesandAPIstoJScode,whichonthoseSIMD-enabledsystemswouldmaptheoperationsdirectlythroughtotheCPUequivalents,withfallbacktonon-parallelizedoperation"shims"onnon-SIMDsystems.

PolyfillingWebWorkers

SIMD

https://github.com/Modernizr/Modernizr/wiki/HTML5-Cross-Browser-Polyfills#web-workers

https://gist.github.com/getify/1b26accb1a09aa53ad25

https://01.org/node/1495

Theperformancebenefitsfordata-intensiveapplications(signalanalysis,matrixoperationsongraphics,etc.)withsuchparallelmathprocessingarequiteobvious!

EarlyproposalformsoftheSIMDAPIatthetimeofthiswritinglooklikethis:

varv1=SIMD.float32x4(3.14159,21.0,32.3,55.55);

varv2=SIMD.float32x4(2.1,3.2,4.3,5.4);

varv3=SIMD.int32x4(10,101,1001,10001);

varv4=SIMD.int32x4(10,20,30,40);

SIMD.float32x4.mul(v1,v2);//[6.597339,67.2,138.89,299.97]

SIMD.int32x4.add(v3,v4);//[20,121,1031,10041]

Shownherearetwodifferentvectordatatypes,32-bitfloating-pointnumbersand32-bitintegernumbers.Youcanseethatthesevectorsaresizedexactlytofour32-bitelements,asthismatchestheSIMDvectorsizes(128-bit)availableinmostmodernCPUs.It'salsopossiblewemayseeanx8(orlarger!)versionoftheseAPIsinthefuture.

Besidesmul()andadd(),manyotheroperationsarelikelytobeincluded,suchassub(),div(),abs(),neg(),sqrt(),reciprocal(),reciprocalSqrt()(arithmetic),shuffle()(rearrangevectorelements),and(),or(),xor(),not()(logical),equal(),greaterThan(),lessThan()(comparison),shiftLeft(),shiftRightLogical(),shiftRightArithmetic()(shifts),fromFloat32x4(),andfromInt32x4()(conversions).

Note:There'sanofficial"prollyfill"(hopeful,expectant,future-leaningpolyfill)fortheSIMDfunctionalityavailable(https://github.com/johnmccutchan/ecmascript_simd),whichillustratesalotmoreoftheplannedSIMDcapabilitythanwe'veillustratedinthissection.

"asm.js"(http://asmjs.org/)isalabelforahighlyoptimizablesubsetoftheJavaScriptlanguage.Bycarefullyavoidingcertainmechanismsandpatternsthatarehardtooptimize(garbagecollection,coercion,etc.),asm.js-styledcodecanberecognizedbytheJSengineandgivenspecialattentionwithaggressivelow-leveloptimizations.

Distinctfromotherprogramperfomancemechanismsdiscussedinthischapter,asm.jsisn'tnecessarilysomethingthatneedstobeadoptedintotheJSlanguagespecification.Thereisanasm.jsspecification(http://asmjs.org/spec/latest/),butit'smostlyfortrackinganagreeduponsetofcandidateinferencesforoptimizationratherthanasetofrequirementsofJSengines.

There'snotcurrentlyanynewsyntaxbeingproposed.Instead,asm.jssuggestswaystorecognizeexistingstandardJSsyntaxthatconformstotherulesofasm.jsandletenginesimplementtheirownoptimizationsaccordingly.

There'sbeensomedisagreementbetweenbrowservendorsoverexactlyhowasm.jsshouldbeactivatedinaprogram.Earlyversionsoftheasm.jsexperimentrequireda"useasm";pragma(similartostrictmode's"usestrict";)tohelpcluetheJSenginetobelookingforasm.jsoptimizationopportunitiesandhints.Othershaveassertedthatasm.jsshouldjustbeasetofheuristicsthatenginesautomaticallyrecognizewithouttheauthorhavingtodoanythingextra,meaningthatexistingprogramscouldtheoreticallybenefitfromasm.js-styleoptimizationswithoutdoinganythingspecial.

Thefirstthingtounderstandaboutasm.jsoptimizationsisaroundtypesandcoercion(seetheTypes&Grammartitleofthisseries).IftheJSenginehastotrackmultipledifferenttypesofvaluesinavariablethroughvariousoperations,sothatitcanhandlecoercionsbetweentypesasnecessary,that'salotofextraworkthatkeepstheprogramoptimizationsuboptimal.

Note:We'regoingtouseasm.js-stylecodehereforillustrationpurposes,butbeawarethatit'snotcommonlyexpectedthatyou'llauthorsuchcodebyhand.asm.jsismoreintendedtoacompliationtargetfromothertools,suchasEmscripten(https://github.com/kripken/emscripten/wiki).It'sofcoursepossibletowriteyourownasm.jscode,butthat'susuallyabadideabecausethecodeisverylowlevelandmanagingitcanbeverytimeconsuminganderrorprone.Nevertheless,there

asm.js

HowtoOptimizewithasm.js

https://github.com/johnmccutchan/ecmascript_simd

http://asmjs.org/

http://asmjs.org/spec/latest/

https://github.com/kripken/emscripten/wiki

maybecaseswhereyou'dwanttohandtweakyourcodeforasm.jsoptimizationpurposes.

Therearesome"tricks"youcanusetohinttoanasm.js-awareJSenginewhattheintendedtypeisforvariables/operations,sothatitcanskipthesecoerciontrackingsteps.

Forexample:

vara=42;

//..

varb=a;

Inthatprogram,theb=aassignmentleavesthedooropenfortypedivergenceinvariables.However,itcouldinsteadbewrittenas:

vara=42;

//..

varb=a|0;

Here,we'veusedthe|("binaryOR")withvalue0,whichhasnoeffectonthevalueotherthantomakesureit'sa32-bitinteger.ThatcoderuninanormalJSengineworksjustfine,butwhenruninanasm.js-awareJSengineitcansignalthatbshouldalwaysbetreatedasa32-bitinteger,sothecoerciontrackingcanbeskipped.

Similarly,theadditionoperationbetweentwovariablescanberestrictedtoamoreperformantintegeraddition(insteadoffloatingpoint):

(a+b)|0

Again,theasm.js-awareJSenginecanseethathintandinferthatthe+operationshouldbe32-bitintegeradditionbecausetheendresultofthewholeexpressionwouldautomaticallybe32-bitintegerconformedanyway.

OneofthebiggestdetractorstoperformanceinJSisaroundmemoryallocation,garbagecollection,andscopeaccess.asm.jssuggestsoneofthewaysaroundtheseissuesistodeclareamoreformalizedasm.js"module"--donotconfusethesewithES6modules;seetheES6&Beyondtitleofthisseries.

Foranasm.jsmodule,youneedtoexplicitlypassinatightlyconformednamespace--thisisreferredtointhespecasstdlib,asitshouldrepresentstandardlibrariesneeded--toimportnecessarysymbols,ratherthanjustusingglobalsvialexicalscope.Inthebasecase,thewindowobjectisanacceptablestdlibobjectforasm.jsmodulepurposes,butyoucouldandperhapsshouldconstructanevenmorerestrictedone.

Youalsomustdeclarea"heap"--whichisjustafancytermforareservedspotinmemorywherevariablescanalreadybeusedwithoutaskingformorememoryorreleasingpreviouslyusedmemory--andpassthatin,sothattheasm.jsmodulewon'tneedtodoanythingthatwouldcausememorychurn;itcanjustusethepre-reservedspace.

A"heap"islikelyatypedArrayBuffer,suchas:

varheap=newArrayBuffer(0x10000);//64kheap

Usingthatpre-reserved64kofbinaryspace,anasm.jsmodulecanstoreandretrievevaluesinthatbufferwithoutanymemoryallocationorgarbagecollectionpenalties.Forexample,theheapbuffercouldbeusedinsidethemoduletoback

asm.jsModules

anarrayof64-bitfloatvalueslikethis:

vararr=newFloat64Array(heap);

OK,solet'smakeaquick,sillyexampleofanasm.js-styledmoduletoillustratehowthesepiecesfittogether.We'lldefineafoo(..)thattakesastart(x)andend(y)integerforarange,andcalculatesalltheinneradjacentmultiplicationsofthevaluesintherange,andthenfinallyaveragesthosevaluestogether:

functionfooASM(stdlib,foreign,heap){

"useasm";

vararr=newstdlib.Int32Array(heap);

functionfoo(x,y){

x=x|0;

y=y|0;

vari=0;

varp=0;

varsum=0;

varcount=((y|0)-(x|0))|0;

//calculatealltheinneradjacentmultiplications

for(i=x|0;

(i|0)<(y|0);

p=(p+8)|0,i=(i+1)|0

){

//storeresult

arr[p>>3]=(i*(i+1))|0;

}

//calculateaverageofallintermediatevalues

for(i=0,p=0;

(i|0)<(count|0);

p=(p+8)|0,i=(i+1)|0

){

sum=(sum+arr[p>>3])|0;

}

return+(sum/count);

}

return{

foo:foo

};

}

varheap=newArrayBuffer(0x1000);

varfoo=fooASM(window,null,heap).foo;

foo(10,20);//233

Note:Thisasm.jsexampleishandauthoredforillustrationpurposes,soitdoesn'trepresentthesamecodethatwouldbeproducedfromacompilationtooltargetingasm.js.Butitdoesshowthetypicalnatureofasm.jscode,especiallythetypehintinganduseoftheheapbufferfortemporaryvariablestorage.

ThefirstcalltofooASM(..)iswhatsetsupourasm.jsmodulewithitsheapallocation.Theresultisafoo(..)functionwecancallasmanytimesasnecessary.Thosefoo(..)callsshouldbespeciallyoptimizedbyanasm.js-awareJSengine.Importantly,theprecedingcodeiscompletelystandardJSandwouldrunjustfine(withoutspecialoptimization)inanon-asm.jsengine.

Obviously,thenatureofrestrictionsthatmakeasm.jscodesooptimizablereducesthepossibleusesforsuchcodesignificantly.asm.jswon'tnecessarilybeageneraloptimizationsetforanygivenJSprogram.Instead,it'sintendedtoprovideanoptimizedwayofhandlingspecializedtaskssuchasintensivemathoperations(e.g.,thoseusedingraphicsprocessingforgames).

Review

Thefirstfourchaptersofthisbookarebasedonthepremisethatasynccodingpatternsgiveyoutheabilitytowritemoreperformantcode,whichisgenerallyaveryimportantimprovement.Butasyncbehavioronlygetsyousofar,becauseit'sstillfundamentallyboundtoasingleeventloopthread.

Sointhischapterwe'vecoveredseveralprogram-levelmechanismsforimprovingperformanceevenfurther.

WebWorkersletyourunaJSfile(akaprogram)inaseparatethreadusingasynceventstomessagebetweenthethreads.They'rewonderfulforoffloadinglong-runningorresource-intensivetaskstoadifferentthread,leavingthemainUIthreadmoreresposive.

SIMDproposestomapCPU-levelparallelmathoperationstoJavaScriptAPIsforhigh-performancedata-paralleloperations,likenumberprocessingonlargedatasets.

Finally,asm.jsdescribesasmallsubsetofJavaScriptthatavoidsthehard-to-optimizepartsofJS(likegarbagecollectionandcoercion)andletstheJSenginerecognizeandrunsuchcodethroughaggressiveoptimizations.asm.jscouldbehandauthored,butthat'sextremelytediousanderrorprone,akintohandauthoringassemblylanguage(hencethename).Instead,themainintentisthatasm.jswouldbeagoodtargetforcross-compilationfromotherhighlyoptimizedprogramlanguages--forexample,Emscripten(https://github.com/kripken/emscripten/wiki)transpilingC/C++toJavaScript.

Whilenotcoveredexplicitlyinthischapter,thereareevenmoreradicalideasunderveryearlydiscussionforJavaScript,includingapproximationsofdirectthreadedfunctionality(notjusthiddenbehinddatastructureAPIs).Whetherthathappensexplicitly,orwejustseemoreparallelismcreepintoJSbehindthescenes,thefutureofmoreoptimizedprogram-levelperformanceinJSlooksreallypromising.

https://github.com/kripken/emscripten/wiki

Asthefirstfourchaptersofthisbookwereallaboutperformanceasacodingpattern(asynchronyandconcurrency),andChapter5wasaboutperformanceatthemacroprogramarchitecturelevel,thischaptergoesafterthetopicofperformanceatthemicrolevel,focusingonsingleexpressions/statements.

Oneofthemostcommonareasofcuriosity--indeed,somedeveloperscangetquiteobsessedaboutit--isinanalyzingandtestingvariousoptionsforhowtowritealineorchunkofcode,andwhichoneisfaster.

We'regoingtolookatsomeoftheseissues,butit'simportanttounderstandfromtheoutsetthatthischapterisnotaboutfeedingtheobsessionofmicro-performancetuning,likewhethersomegivenJSenginecanrun++afasterthana++.ThemoreimportantgoalofthischapteristofigureoutwhatkindsofJSperformancematterandwhichonesdon't,andhowtotellthedifference.

Butevenbeforewegetthere,weneedtoexplorehowtomostaccuratelyandreliablytestJSperformance,becausethere'stonsofmisconceptionsandmythsthathavefloodedourcollectivecultknowledgebase.We'vegottosiftthroughallthatjunktofindsomeclarity.

OK,timetostartdispellingsomemisconceptions.I'dwagerthevastmajorityofJSdevelopers,ifaskedtobenchmarkthespeed(executiontime)ofacertainoperation,wouldinitiallygoaboutitsomethinglikethis:

varstart=(newDate()).getTime();//or`Date.now()`

//dosomeoperation

varend=(newDate()).getTime();

console.log("Duration:",(end-start));

Raiseyourhandifthat'sroughlywhatcametoyourmind.Yep,Ithoughtso.There'salotwrongwiththisapproach,butdon'tfeelbad;we'veallbeenthere.

Whatdidthatmeasurementtellyou,exactly?Understandingwhatitdoesanddoesn'tsayabouttheexecutiontimeoftheoperationinquestioniskeytolearninghowtoappropriatelybenchmarkperformanceinJavaScript.

Ifthedurationreportedis0,youmaybetemptedtobelievethatittooklessthanamillisecond.Butthat'snotveryaccurate.Someplatformsdon'thavesinglemillisecondprecision,butinsteadonlyupdatethetimerinlargerincrements.Forexample,olderversionsofwindows(andthusIE)hadonly15msprecision,whichmeanstheoperationhastotakeatleastthatlongforanythingotherthan0tobereported!

Moreover,whateverdurationisreported,theonlythingyoureallyknowisthattheoperationtookapproximatelythatlongonthatexactsinglerun.Youhavenear-zeroconfidencethatitwillalwaysrunatthatspeed.Youhavenoideaiftheengineorsystemhadsomesortofinterferenceatthatexactmoment,andthatatothertimestheoperationcouldrunfaster.

Whatifthedurationreportedis4?Areyoumoresureittookaboutfourmilliseconds?Nope.Itmighthavetakenlesstime,andtheremayhavebeensomeotherdelayingettingeitherstartorendtimestamps.

Moretroublingly,youalsodon'tknowthatthecircumstancesofthisoperationtestaren'toverlyoptimistic.It'spossiblethattheJSenginefiguredoutawaytooptimizeyourisolatedtestcase,butinamorerealprogramsuchoptimizationwouldbedilutedorimpossible,suchthattheoperationwouldrunslowerthanyourtest.


Chapter6:Benchmarking&Tuning

Benchmarking

So...whatdoweknow?Unfortunately,withthoserealizationsstated,weknowverylittle.Somethingofsuchlowconfidenceisn'tevenremotelygoodenoughtobuildyourdeterminationson.Your"benchmark"isbasicallyuseless.Andworse,it'sdangerousinthatitimpliesfalseconfidence,notjusttoyoubutalsotootherswhodon'tthinkcriticallyabouttheconditionsthatledtothoseresults.

"OK,"younowsay,"Justputalooparounditsothewholetesttakeslonger."Ifyourepeatanoperation100times,andthatwholeloopreportedlytakesatotalof137ms,thenyoucanjustdivideby100andgetanaveragedurationof1.37msforeachoperation,right?

Well,notexactly.

Astraightmathematicalaveragebyitselfisdefinitelynotsufficientformakingjudgmentsaboutperformancewhichyouplantoextrapolatetothebreadthofyourentireapplication.Withahundrediterations,evenacoupleofoutliers(highorlow)canskewtheaverage,andthenwhenyouapplythatconclusionrepeatedly,youevenfurtherinflatetheskewbeyondcredulity.

Insteadofjustrunningforafixednumberofiterations,youcaninsteadchoosetoruntheloopoftestsuntilacertainamountoftimehaspassed.Thatmightbemorereliable,buthowdoyoudecidehowlongtorun?Youmightguessthatitshouldbesomemultipleofhowlongyouroperationshouldtaketorunonce.Wrong.

Actually,thelengthoftimetorepeatacrossshouldbebasedontheaccuracyofthetimeryou'reusing,specificallytominimizethechancesofinaccuracy.Thelesspreciseyourtimer,thelongeryouneedtoruntomakesureyou'veminimizedtheerrorpercentage.A15mstimerisprettybadforaccuratebenchmarking;tominimizeitsuncertainty(aka"errorrate")tolessthan1%,youneedtorunyoureachcycleoftestiterationsfor750ms.A1mstimeronlyneedsacycletorunfor50mstogetthesameconfidence.

Butthen,that'sjustasinglesample.Tobesureyou'refactoringouttheskew,you'llwantlotsofsamplestoaverageacross.You'llalsowanttounderstandsomethingaboutjusthowslowtheworstsampleis,howfastthebestsampleis,howfarapartthosebestandworsecaseswere,andsoon.You'llwanttoknownotjustanumberthattellsyouhowfastsomethingran,butalsotohavesomequantifiablemeasureofhowtrustablethatnumberis.

Also,youprobablywanttocombinethesedifferenttechniques(aswellasothers),sothatyougetthebestbalanceofallthepossibleapproaches.

That'sallbareminimumjusttogetstarted.Ifyou'vebeenapproachingperformancebenchmarkingwithanythinglessseriousthanwhatIjustglossedover,well..."youdon'tknow:properbenchmarking."

Anyrelevantandreliablebenchmarkshouldbebasedonstatisticallysoundpractices.Iamnotgoingtowriteachapteronstatisticshere,soI'llhandwavearoundsometerms:standarddeviation,variance,marginoferror.Ifyoudon'tknowwhatthosetermsreallymean--ItookastatsclassbackincollegeandI'mstillalittlefuzzyonthem--youarenotactuallyqualifiedtowriteyourownbenchmarkinglogic.

Luckily,smartfolkslikeJohn-DavidDaltonandMathiasBynensdounderstandtheseconcepts,andwroteastatisticallysoundbenchmarkingtoolcalledBenchmark.js(http://benchmarkjs.com/).SoIcanendthesuspensebysimplysaying:"justusethattool."

Iwon'trepeattheirwholedocumentationforhowBenchmark.jsworks;theyhavefantasticAPIDocs(http://benchmarkjs.com/docs)youshouldread.Alsotherearesomegreat(http://calendar.perfplanet.com/2010/bulletproof-javascript-benchmarks/)writeups(http://monsur.hossa.in/2012/12/11/benchmarkjs.html)onmoreofthedetailsandmethodology.

Butjustforquickillustrationpurposes,here'showyoucoulduseBenchmark.jstorunaquickperformancetest:

functionfoo(){

Repetition

Benchmark.js

http://benchmarkjs.com/

http://benchmarkjs.com/docs

http://calendar.perfplanet.com/2010/bulletproof-javascript-benchmarks/

http://monsur.hossa.in/2012/12/11/benchmarkjs.html

//operation(s)totest

}

varbench=newBenchmark(

"footest",//testname

foo,//functiontotest(justcontents)

{

//..//optionalextraoptions(seedocs)

}

);

bench.hz;//numberofoperationspersecond

bench.stats.moe;//marginoferror

bench.stats.variance;//varianceacrosssamples

//..

There'slotsmoretolearnaboutusingBenchmark.jsbesidesthisglanceI'mincludinghere.Butthepointisthatit'shandlingallofthecomplexitiesofsettingupafair,reliable,andvalidperformancebenchmarkforagivenpieceofJavaScriptcode.Ifyou'regoingtotrytotestandbenchmarkyourcode,thislibraryisthefirstplaceyoushouldturn.

We'reshowingheretheusagetotestasingleoperationlikeX,butit'sfairlycommonthatyouwanttocompareXtoY.Thisiseasytodobysimplysettinguptwodifferenttestsina"Suite"(aBenchmark.jsorganizationalfeature).Then,yourunthemhead-to-head,andcomparethestatisticstoconcludewhetherXorYwasfaster.

Benchmark.jscanofcoursebeusedtotestJavaScriptinabrowser(seethe"jsPerf.com"sectionlaterinthischapter),butitcanalsoruninnon-browserenvironments(Node.js,etc.).

Onelargelyuntappedpotentialuse-caseforBenchmark.jsistouseitinyourDevorQAenvironmentstorunautomatedperformanceregressiontestsagainstcriticalpathpartsofyourapplication'sJavaScript.Similartohowyoumightrununittestsuitesbeforedeployment,youcanalsocomparetheperformanceagainstpreviousbenchmarkstomonitorifyouareimprovingordegradingapplicationperformance.

Inthepreviouscodesnippet,weglossedoverthe"extraoptions"{..}object.Buttherearetwooptionsweshoulddiscuss:setupandteardown.

Thesetwooptionsletyoudefinefunctionstobecalledbeforeandafteryourtestcaseruns.

It'sincrediblyimportanttounderstandthatyoursetupandteardowncodedoesnotrunforeachtestiteration.Thebestwaytothinkaboutitisthatthere'sanouterloop(repeatingcycles),andaninnerloop(repeatingtestiterations).setupandteardownarerunatthebeginningandendofeachouterloop(akacycle)iteration,butnotinsidetheinnerloop.

Whydoesthismatter?Let'simagineyouhaveatestcasethatlookslikethis:

a=a+"w";

b=a.charAt(1);

Then,yousetupyourtestsetupasfollows:

vara="x";

Yourtemptationisprobablytobelievethataisstartingoutas"x"foreachtestiteration.

Butit'snot!It'sstartingaat"x"foreachtestcycle,andthenyourrepeated+"w"concatenationswillbemakingalargerandlargeravalue,eventhoughyou'reonlyeveraccessingthecharacter"w"atthe1position.

WherethismostcommonlybitesyouiswhenyoumakesideeffectchangestosomethingliketheDOM,likeappendingachildelement.Youmaythinkyourparentelementissetasemptyeachtime,butit'sactuallygettinglotsofelementsadded,

Setup/Teardown

andthatcansignificantlyswaytheresultsofyourtests.

Don'tforgettocheckthecontextofaparticularperformancebenchmark,especiallyacomparisonbetweenXandYtasks.JustbecauseyourtestrevealsthatXisfasterthanYdoesn'tmeanthattheconclusion"XisfasterthanY"isactuallyrelevant.

Forexample,let'ssayaperformancetestrevealsthatXruns10,000,000operationspersecond,andYrunsat8,000,000operationspersecond.YoucouldclaimthatYis20%slowerthanX,andyou'dbemathematicallycorrect,butyourassertiondoesn'tholdasmuchwaterasyou'dthink.

Let'sthinkabouttheresultsmorecritically:10,000,000operationspersecondis10,000operationspermillisecond,and10operationspermicrosecond.Inotherwords,asingleoperationtakes0.1microseconds,or100nanoseconds.It'shardtofathomjusthowsmall100nsis,butforcomparison,it'softencitedthatthehumaneyeisn'tgenerallycapableofdistinguishinganythinglessthan100ms,whichisonemilliontimesslowerthanthe100nsspeedoftheXoperation.

Evenrecentscientificstudiesshowingthatmaybethebraincanprocessasquickas13ms(about8xfasterthanpreviouslyasserted)wouldmeanthatXisstillrunning125,000timesfasterthanthehumanbraincanperceiveadistinctthinghappening.Xisgoingreally,reallyfast.

Butmoreimportantly,let'stalkaboutthedifferencebetweenXandY,the2,000,000operationsperseconddifference.IfXtakes100ns,andYtakes80ns,thedifferenceis20ns,whichinthebestcaseisstillone650-thousandthoftheintervalthehumanbraincanperceive.

What'smypoint?Noneofthisperformancedifferencematters,atall!

Butwait,whatifthisoperationisgoingtohappenawholebunchoftimesinarow?Thenthedifferencecouldaddup,right?

OK,sowhatwe'reaskingthenis,howlikelyisitthatoperationXisgoingtoberunoverandoveragain,onerightaftertheother,andthatthishastohappen650,000timesjusttogetasliverofahopethehumanbraincouldperceiveit.Morelikely,it'dhavetohappen5,000,000to10,000,000timestogetherinatightlooptoevenapproachrelevance.

Whilethecomputerscientistinyoumightprotestthatthisispossible,theloudervoiceofrealisminyoushouldsanitycheckjusthowlikelyorunlikelythatreallyis.Evenifitisrelevantinrareoccasions,it'sirrelevantinmostsituations.

Thevastmajorityofyourbenchmarkresultsontinyoperations--likethe++xvsx++myth--arejusttotallybogusforsupportingtheconclusionthatXshouldbefavoredoverYonaperformancebasis.

YousimplycannotreliablyextrapolatethatifXwas10microsecondsfasterthanYinyourisolatedtest,thatmeansXisalwaysfasterthanYandshouldalwaysbeused.That'snothowperformanceworks.It'svastlymorecomplicated.

Forexample,let'simagine(purelyhypothetical)thatyoutestsomemicroperformancebehaviorsuchascomparing:

vartwelve="12";

varfoo="foo";

//test1

varX1=parseInt(twelve);

varX2=parseInt(foo);

//test2

varY1=Number(twelve);

varY2=Number(foo);

IfyouunderstandwhatparseInt(..)doescomparedtoNumber(..),youmightintuitthatparseInt(..)potentiallyhas

ContextIsKing

EngineOptimizations

"morework"todo,especiallyinthefoocase.Oryoumightintuitthattheyshouldhavethesameamountofworktodointhefoocase,asbothshouldbeabletostopatthefirstcharacter"f".

Whichintuitioniscorrect?Ihonestlydon'tknow.ButI'llmakethecaseitdoesn'tmatterwhatyourintuitionis.Whatmighttheresultsbewhenyoutestit?Again,I'mmakingupapurehypotheticalhere,Ihaven'tactuallytried,nordoIcare.

Let'spretendthetestcomesbackthatXandYarestatisticallyidentical.Haveyouthenconfirmedyourintuitionaboutthe"f"characterthing?Nope.

It'spossibleinourhypotheticalthattheenginemightrecognizethatthevariablestwelveandfooareonlybeingusedinoneplaceineachtest,andsoitmightdecidetoinlinethosevalues.ThenitmayrealizethatNumber("12")canjustbereplacedby12.AndmaybeitcomestothesameconclusionwithparseInt(..),ormaybenot.

Oranengine'sdead-coderemovalheuristiccouldkickin,anditcouldrealizethatvariablesXandYaren'tbeingused,sodeclaringthemisirrelevant,soitdoesn'tendupdoinganythingatallineithertest.

Andallthat'sjustmadewiththemindsetofassumptionsaboutasingletestrun.Modernenginesarefantasticallymorecomplicatedthanwhatwe'reintuitinghere.Theydoallsortsoftricks,liketracingandtrackinghowapieceofcodebehavesoverashortperiodoftime,orwithaparticularlyconstrainedsetofinputs.

Whatiftheengineoptimizesacertainwaybecauseofthefixedinput,butinyourrealprogramyougivemorevariedinputandtheoptimizationdecisionsshakeoutdifferently(ornotatall!)?Orwhatiftheenginekicksinoptimizationsbecauseitseesthecodebeingruntensofthousandsoftimesbythebenchmarkingutility,butinyourrealprogramitwillonlyrunahundredtimesinnearproximity,andunderthoseconditionstheenginedeterminestheoptimizationsarenotworthit?

Andallthoseoptimizationswejusthypothesizedaboutmighthappeninourconstrainedtestbutmaybetheenginewouldn'tdotheminamorecomplexprogram(forvariousreasons).Oritcouldbereversed--theenginemightnotoptimizesuchtrivialcodebutmaybemoreinclinedtooptimizeitmoreaggressivelywhenthesystemisalreadymoretaxedbyamoresophisticatedprogram.

ThepointI'mtryingtomakeisthatyoureallydon'tknowforsureexactlywhat'sgoingonunderthecovers.Alltheguessesandhypothesisyoucanmusterdon'tamounttohardlyanythingconcreteforreallymakingsuchdecisions.

Doesthatmeanyoucan'treallydoanyusefultesting?Definitelynot!

Whatthisboilsdowntoisthattestingnotrealcodegivesyounotrealresults.Insomuchasispossibleandpractical,youshouldtestactualreal,non-trivialsnippetsofyourcode,andunderasbestofrealconditionsasyoucanactuallyhopeto.Onlythenwilltheresultsyougethaveachancetoapproximatereality.

Microbenchmarkslike++xvsx++aresoincrediblylikelytobebogus,wemightaswelljustflatlyassumethemassuch.

WhileBenchmark.jsisusefulfortestingtheperformanceofyourcodeinwhateverJSenvironmentyou'rerunning,itcannotbestressedenoughthatyouneedtocompiletestresultsfromlotsofdifferentenvironments(desktopbrowsers,mobiledevices,etc.)ifyouwanttohaveanyhopeofreliabletestconclusions.

Forexample,Chromeonahigh-enddesktopmachineisnotlikelytoperformanywherenearthesameasChromemobileonasmartphone.Andasmartphonewithafullbatterychargeisnotlikelytoperformanywherenearthesameasasmartphonewith2%batterylifeleft,whenthedeviceisstartingtopowerdowntheradioandprocessor.

Ifyouwanttomakeassertionslike"XisfasterthanY"inanyreasonablesenseacrossmorethanjustasingleenvironment,you'regoingtoneedtoactuallytestasmanyofthoserealworldenvironmentsaspossible.JustbecauseChromeexecutessomeXoperationfasterthanYdoesn'tmeanthatallbrowsersdo.Andofcourseyoualsoprobablywillwanttocross-referencetheresultsofmultiplebrowsertestrunswiththedemographicsofyourusers.

There'sanawesomewebsiteforthispurposecalledjsPerf(http://jsperf.com).ItusestheBenchmark.jslibrarywetalked

jsPerf.com

http://jsperf.com

aboutearliertorunstatisticallyaccurateandreliabletests,andmakesthetestonanopenlyavailableURLthatyoucanpassaroundtoothers.

Eachtimeatestisrun,theresultsarecollectedandpersistedwiththetest,andthecumulativetestresultsaregraphedonthepageforanyonetosee.

Whencreatingatestonthesite,youstartoutwithtwotestcasestofillin,butyoucanaddasmanyasyouneed.Youalsohavetheabilitytosetupsetupcodethatisrunatthebeginningofeachtestcycleandteardowncoderunattheendofeachcycle.

Note:Atrickfordoingjustonetestcase(ifyou'rebenchmarkingasingleapproachinsteadofahead-to-head)istofillinthesecondtestinputboxeswithplaceholdertextonfirstcreation,theneditthetestandleavethesecondtestblank,whichwilldeleteit.Youcanalwaysaddmoretestcaseslater.

Youcandefinetheinitialpagesetup(importinglibraries,definingutilityhelperfunctions,declaringvariables,etc.).Therearealsooptionsfordefiningsetupandteardownbehaviorifneeded--consultthe"Setup/Teardown"sectionintheBenchmark.jsdiscussionearlier.

jsPerfisafantasticresource,butthere'sanawfullotoftestspublishedthatwhenyouanalyzethemarequiteflawedorbogus,foranyofavarietyofreasonsasoutlinedsofarinthischapter.

Consider:

//Case1

varx=[];

for(vari=0;i<10;i++){

x[i]="x";

}

//Case2

varx=[];


x[x.length]="x";

}

//Case3

varx=[];


x.push("x");

}

Someobservationstoponderaboutthistestscenario:

It'sextremelycommonfordevstoputtheirownloopsintotestcases,andtheyforgetthatBenchmark.jsalreadydoesalltherepetitionyouneed.There'sareallystrongchancethattheforloopsinthesecasesaretotallyunnecessarynoise.Thedeclaringandinitializingofxisincludedineachtestcase,possiblyunnecessarily.Recallfromearlierthatifx=[]wereinthesetupcode,itwouldn'tactuallyberunbeforeeachtestiteration,butinsteadonceatthebeginningofeachcycle.Thatmeansxwouldcontinuegrowingquitelarge,notjustthesize10impliedbytheforloops.

SoistheintenttomakesurethetestsareconstrainedonlytohowtheJSenginebehaveswithverysmallarrays(size10)?Thatcouldbetheintent,butifitis,youhavetoconsiderifthat'snotfocusingfartoomuchonnuancedinternalimplementationdetails.

Ontheotherhand,doestheintentofthetestembracethecontextthatthearrayswillactuallybegrowingquitelarge?IstheJSengines'behaviorwithlargerarraysrelevantandaccuratewhencomparedwiththeintendedrealworldusage?

Istheintenttofindouthowmuchx.lengthorx.push(..)addtotheperformanceoftheoperationtoappendtothe

SanityCheck

xarray?OK,thatmightbeavalidthingtotest.Butthenagain,push(..)isafunctioncall,soofcourseit'sgoingtobeslowerthan[..]access.Arguably,cases1and2arefairerthancase3.

Here'sanotherexamplethatillustratesacommonapples-to-orangesflaw:

//Case1

varx=["John","Albert","Sue","Frank","Bob"];

x.sort();

//Case2

varx=["John","Albert","Sue","Frank","Bob"];

x.sort(functionmySort(a,b){

if(a<b)return-1;

if(a>b)return1;

return0;

});

Here,theobviousintentistofindouthowmuchslowerthecustommySort(..)comparatoristhanthebuilt-indefaultcomparator.ButbyspecifyingthefunctionmySort(..)asinlinefunctionexpression,you'vecreatedanunfair/bogustest.Here,thesecondcaseisnotonlytestingacustomuserJSfunction,butit'salsotestingcreatinganewfunctionexpressionforeachiteration.

Woulditsurpriseyoutofindoutthatifyourunasimilartestbutupdateittoisolateonlyforcreatinganinlinefunctionexpressionversususingapre-declaredfunction,theinlinefunctionexpressioncreationcanbefrom2%to20%slower!?

Unlessyourintentwiththistestistoconsidertheinlinefunctionexpressioncreation"cost,"abetter/fairertestwouldputmySort(..)'sdeclarationinthepagesetup--don'tputitinthetestsetupasthat'sunnecessaryredeclarationforeachcycle--andsimplyreferenceitbynameinthetestcase:x.sort(mySort).

Buildingonthepreviousexample,anotherpitfallisinopaquelyavoidingoradding"extrawork"toonetestcasethatcreatesanapples-to-orangesscenario:

//Case1

varx=[12,-14,0,3,18,0,2.9];

x.sort();

//Case2

varx=[12,-14,0,3,18,0,2.9];

x.sort(functionmySort(a,b){

returna-b;

});

Settingasidethepreviouslymentionedinlinefunctionexpressionpitfall,thesecondcase'smySort(..)worksinthiscasebecauseyouhaveprovideditnumbers,butwouldhaveofcoursefailedwithstrings.Thefirstcasedoesn'tthrowanerror,butitactuallybehavesdifferentlyandhasadifferentoutcome!Itshouldbeobvious,but:adifferentoutcomebetweentwotestcasesalmostcertainlyinvalidatestheentiretest!

Butbeyondthedifferentoutcomes,inthiscase,thebuiltinsort(..)'scomparatorisactuallydoing"extrawork"thatmySort()doesnot,inthatthebuilt-inonecoercesthecomparedvaluestostringsanddoeslexicographiccomparison.Thefirstsnippetresultsin[-14,0,0,12,18,2.9,3]whilethesecondsnippetresults(likelymoreaccuratelybasedonintent)in[-14,0,0,2.9,3,12,18].

Sothattestisunfairbecauseit'snotactuallydoingthesametaskbetweenthecases.Anyresultsyougetarebogus.

Thesesamepitfallscanevenbemuchmoresubtle:

//Case1

varx=false;

vary=x?1:2;

//Case2

varx;

vary=x?1:2;

Here,theintentmightbetotesttheperformanceimpactofthecoerciontoaBooleanthatthe?:operatorwilldoifthexexpressionisnotalreadyaBoolean(seetheTypes&Grammartitleofthisbookseries).So,you'reapparentlyOKwiththefactthatthereisextraworktodothecoercioninthesecondcase.

Thesubtleproblem?You'resettingx'svalueinthefirstcaseandnotsettingitintheother,soyou'reactuallydoingworkinthefirstcasethatyou'renotdoinginthesecond.Toeliminateanypotential(albeitminor)skew,try:

//Case1

varx=false;

vary=x?1:2;

//Case2

varx=undefined;

vary=x?1:2;

Nowthere'sanassignmentinbothcases,sothethingyouwanttotest--thecoercionofxornot--haslikelybeenmoreaccuratelyisolatedandtested.

LetmeseeifIcanarticulatethebiggerpointI'mtryingtomakehere.

Goodtestauthoringrequirescarefulanalyticalthinkingaboutwhatdifferencesexistbetweentwotestcasesandwhetherthedifferencesbetweenthemareintentionalorunintentional.

IntentionaldifferencesareofcoursenormalandOK,butit'stooeasytocreateunintentionaldifferencesthatskewyourresults.Youhavetobereally,reallycarefultoavoidthatskew.Moreover,youmayintendadifferencebutitmaynotbeobvioustootherreadersofyourtestwhatyourintentwas,sotheymaydoubt(ortrust!)yourtestincorrectly.Howdoyoufixthat?

Writebetter,clearertests.Butalso,takethetimetodocument(usingthejsPerf.com"Description"fieldand/orcodecomments)exactlywhattheintentofyourtestis,eventothenuanceddetail.Callouttheintentionaldifferences,whichwillhelpothersandyourfutureselftobetteridentifyunintentionaldifferencesthatcouldbeskewingthetestresults.

Isolatethingswhicharen'trelevanttoyourtestbypre-declaringtheminthepageortestsetupsettingssothey'reoutsidethetimedpartsofthetest.

Insteadoftryingtonarrowinonatinysnippetofyourrealcodeandbenchmarkingjustthatpieceoutofcontext,testsandbenchmarksarebetterwhentheyincludealarger(whilestillrelevant)context.Thosetestsalsotendtorunslower,whichmeansanydifferencesyouspotaremorerelevantincontext.

OK,untilnowwe'vebeendancingaroundvariousmicroperformanceissuesandgenerallylookingdisfavorablyuponobsessingaboutthem.Iwanttotakejustamomenttoaddressthemdirectly.

Thefirstthingyouneedtogetmorecomfortablewithwhenthinkingaboutperformancebenchmarkingyourcodeisthatthecodeyouwriteisnotalwaysthecodetheengineactuallyruns.WebrieflylookedatthattopicbackinChapter1whenwediscussedstatementreorderingbythecompiler,butherewe'regoingtosuggestthecompilercansometimesdecidetorundifferentcodethanyouwrote,notjustindifferentordersbutdifferentinsubstance.

Let'sconsiderthispieceofcode:

varfoo=41;

WritingGoodTests

Microperformance

(function(){

(function(){

(function(baz){

varbar=foo+baz;

//..

})(1);

})();

})();

Youmaythinkaboutthefooreferenceintheinnermostfunctionasneedingtodoathree-levelscopelookup.WecoveredintheScope&Closurestitleofthisbookserieshowlexicalscopeworks,andthefactthatthecompilergenerallycachessuchlookupssothatreferencingfoofromdifferentscopesdoesn'treallypractically"cost"anythingextra.

Butthere'ssomethingdeepertoconsider.Whatifthecompilerrealizesthatfooisn'treferencedanywhereelsebutthatonelocation,anditfurthernoticesthatthevalueneverisanythingexceptthe41asshown?

Isn'titquitepossibleandacceptablethattheJScompilercoulddecidetojustremovethefoovariableentirely,andinlinethevalue,suchasthis:

(function(){

(function(){

(function(baz){

varbar=41+baz;

//..

})(1);

})();

})();

Note:Ofcourse,thecompilercouldprobablyalsodoasimilaranalysisandrewritewiththebazvariablehere,too.

WhenyoubegintothinkaboutyourJScodeasbeingahintorsuggestiontotheengineofwhattodo,ratherthanaliteralrequirement,yourealizethatalotoftheobsessionoverdiscretesyntacticminutiaismostlikelyunfounded.

Anotherexample:

functionfactorial(n){

if(n<2)return1;

returnn*factorial(n-1);

}

factorial(5);//120

Ah,thegoodol'fashioned"factorial"algorithm!YoumightassumethattheJSenginewillrunthatcodemostlyasis.Andtobehonest,itmight--I'mnotreallysure.

Butasananecdote,thesamecodeexpressedinCandcompiledwithadvancedoptimizationswouldresultinthecompilerrealizingthatthecallfactorial(5)canjustbereplacedwiththeconstantvalue120,eliminatingthefunctionandcallentirely!

Moreover,someengineshaveapracticecalled"unrollingrecursion,"whereitcanrealizethattherecursionyou'veexpressedcanactuallybedone"easier"(i.e.,moreoptimally)withaloop.It'spossibletheprecedingcodecouldberewrittenbyaJSenginetorunas:


if(n<2)return1;

varres=1;

for(vari=n;i>1;i--){

res*=i;

}

returnres;

}

factorial(5);//120

Now,let'simaginethatintheearliersnippetyouhadbeenworriedaboutwhethern*factorial(n-1)orn*=factorial(--n)runsfaster.Maybeyouevendidaperformancebenchmarktotrytofigureoutwhichwasbetter.Butyoumissthefactthatinthebiggercontext,theenginemaynotruneitherlineofcodebecauseitmayunrolltherecursion!

Speakingof--,--nversusn--isoftencitedasoneofthoseplaceswhereyoucanoptimizebychoosingthe--nversion,becausetheoreticallyitrequireslesseffortdownattheassemblylevelofprocessing.

ThatsortofobsessionisbasicallynonsenseinmodernJavaScript.That'sthekindofthingyoushouldbelettingtheenginetakecareof.Youshouldwritethecodethatmakesthemostsense.Comparethesethreeforloops:

//Option1


console.log(i);

}

//Option2

for(vari=0;i<10;++i){

console.log(i);

}

//Option3

for(vari=-1;++i<10;){

console.log(i);

}

Evenifyouhavesometheorywherethesecondorthirdoptionismoreperformantthanthefirstoptionbyatinybit,whichisdubiousatbest,thethirdloopismoreconfusingbecauseyouhavetostartwith-1foritoaccountforthefactthat++ipre-incrementisused.Andthedifferencebetweenthefirstandsecondoptionsisreallyquiteirrelevant.

It'sentirelypossiblethataJSenginemayseeaplacewherei++isusedandrealizethatitcansafelyreplaceitwiththe++iequivalent,whichmeansyourtimespentdecidingwhichonetopickwascompletelywastedandtheoutcomemoot.

Here'sanothercommonexampleofsillymicroperformanceobsession:

varx=[..];

//Option1

for(vari=0;i<x.length;i++){

//..

}

//Option2

for(vari=0,len=x.length;i<len;i++){

//..

}

Thetheoryheregoesthatyoushouldcachethelengthofthexarrayinthevariablelen,becauseostensiblyitdoesn'tchange,toavoidpayingthepriceofx.lengthbeingconsultedforeachiterationoftheloop.

Ifyourunperformancebenchmarksaroundx.lengthusagecomparedtocachingitinalenvariable,you'llfindthatwhilethetheorysoundsnice,inpracticeanymeasureddifferencesarestatisticallycompletelyirrelevant.

Infact,insomeengineslikev8,itcanbeshown(http://mrale.ph/blog/2014/12/24/array-length-caching.html)thatyoucouldmakethingsslightlyworsebypre-cachingthelengthinsteadoflettingtheenginefigureitoutforyou.Don'ttrytooutsmartyourJavaScriptengine,you'llprobablylosewhenitcomestoperformanceoptimizations.

ThedifferentJSenginesinvariousbrowserscanallbe"speccompliant"whilehavingradicallydifferentwaysofhandling

NotAllEnginesAreAlike

http://mrale.ph/blog/2014/12/24/array-length-caching.html

code.TheJSspecificationdoesn'trequireanythingperformancerelated--well,exceptES6's"TailCallOptimization"coveredlaterinthischapter.

Theenginesarefreetodecidethatoneoperationwillreceiveitsattentiontooptimize,perhapstradingoffforlesserperformanceonanotheroperation.Itcanbeverytenuoustofindanapproachforanoperationthatalwaysrunsfasterinallbrowsers.

There'samovementamongsomeintheJSdevcommunity,especiallythosewhoworkwithNode.js,toanalyzethespecificinternalimplementationdetailsofthev8JavaScriptengineandmakedecisionsaboutwritingJScodethatistailoredtotakebestadvantageofhowv8works.Youcanactuallyachieveasurprisinglyhighdegreeofperformanceoptimizationwithsuchendeavors,sothepayofffortheeffortcanbequitehigh.

Somecommonlycitedexamples(https://github.com/petkaantonov/bluebird/wiki/Optimization-killers)forv8:

Don'tpasstheargumentsvariablefromonefunctiontoanyotherfunction,assuch"leakage"slowsdownthefunctionimplementation.Isolateatry..catchinitsownfunction.Browsersstrugglewithoptimizinganyfunctionwithatry..catchinit,somovingthatconstructtoitsownfunctionmeansyoucontainthede-optimizationharmwhilelettingthesurroundingcodebeoptimizable.

Butratherthanfocusonthosetipsspecifically,let'ssanitycheckthev8-onlyoptimizationapproachinageneralsense.

AreyougenuinelywritingcodethatonlyneedstoruninoneJSengine?EvenifyourcodeisentirelyintendedforNode.jsrightnow,istheassumptionthatv8willalwaysbetheusedJSenginereliable?Isitpossiblethatsomedayafewyearsfromnow,there'sanotherserver-sideJSplatformbesidesNode.jsthatyouchoosetorunyourcodeon?Whatifwhatyouoptimizedforbeforeisnowamuchslowerwayofdoingthatoperationonthenewengine?

Orwhatifyourcodealwaysstaysrunningonv8fromhereonout,butv8decidesatsomepointtochangethewaysomesetofoperationsworkssuchthatwhatusedtobefastisnowslow,andviceversa?

Thesescenariosaren'tjusttheoretical,either.Itusedtobethatitwasfastertoputmultiplestringvaluesintoanarrayandthencalljoin("")onthearraytoconcatenatethevaluesthantojustuse+concatenationdirectlywiththevalues.Thehistoricalreasonforthisisnuanced,butithastodowithinternalimplementationdetailsabouthowstringvalueswerestoredandmanagedinmemory.

Asaresult,"bestpractice"adviceatthetimedisseminatedacrosstheindustrysuggestingdevelopersalwaysusethearrayjoin(..)approach.Andmanyfollowed.

Except,somewherealongtheway,theJSengineschangedapproachesforinternallymanagingstrings,andspecificallyputinoptimizationsfor+concatenation.Theydidn'tslowdownjoin(..)perse,buttheyputmoreeffortintohelping+usage,asitwasstillquiteabitmorewidespread.

Note:Thepracticeofstandardizingoroptimizingsomeparticularapproachbasedmostlyonitsexistingwidespreadusageisoftencalled(metaphorically)"pavingthecowpath."

Oncethatnewapproachtohandlingstringsandconcatenationtookhold,unfortunatelyallthecodeoutinthewildthatwasusingarrayjoin(..)toconcatenatestringswasthensub-optimal.

Anotherexample:atonetime,theOperabrowserdifferedfromotherbrowsersinhowithandledtheboxing/unboxingofprimitivewrapperobjects(seetheTypes&Grammartitleofthisbookseries).Assuch,theiradvicetodeveloperswastouseaStringobjectinsteadoftheprimitivestringvalueifpropertieslikelengthormethodslikecharAt(..)neededtobeaccessed.ThisadvicemayhavebeencorrectforOperaatthetime,butitwasliterallycompletelyoppositeforothermajorcontemporarybrowsers,astheyhadoptimizationsspecificallyforthestringprimitivesandnottheirobjectwrappercounterparts.

Ithinkthesevariousgotchasareatleastpossible,ifnotlikely,forcodeeventoday.SoI'mverycautiousaboutmakingwiderangingperformanceoptimizationsinmyJScodebasedpurelyonengineimplementationdetails,especiallyifthosedetailsareonlytrueofasingleengine.

https://github.com/petkaantonov/bluebird/wiki/Optimization-killers

Thereverseisalsosomethingtobewaryof:youshouldn'tnecessarilychangeapieceofcodetoworkaroundoneengine'sdifficultywithrunningapieceofcodeinanacceptablyperformantway.

Historically,IEhasbeenthebruntofmanysuchfrustrations,giventhattherehavebeenplentyofscenariosinolderIEversionswhereitstruggledwithsomeperformanceaspectthatothermajorbrowsersofthetimeseemednottohavemuchtroublewith.ThestringconcatenationdiscussionwejusthadwasactuallyarealconcernbackintheIE6andIE7days,whereitwaspossibletogetbetterperformanceoutofjoin(..)than+.

Butit'stroublesometosuggestthatjustonebrowser'stroublewithperformanceisjustifcationforusingacodeapproachthatquitepossiblycouldbesub-optimialinallotherbrowsers.Evenifthebrowserinquestionhasalargemarketshareforyoursite'saudience,itmaybemorepracticaltowritethepropercodeandrelyonthebrowsertoupdateitselfwithbetteroptimizationseventually.

"Thereisnothingmorepermanentthanatemporaryhack."Chancesare,thecodeyouwritenowtoworkaroundsomeperformancebugwillprobablyoutlivetheperformancebuginthebrowseritself.

Inthedayswhenabrowseronlyupdatedonceeveryfiveyears,thatwasatoughercalltomake.Butasitstandsnow,browsersacrosstheboardareupdatingatamuchmorerapidinterval(thoughobviouslythemobileworldstilllags),andthey'reallcompetingtooptimizewebfeaturesbetterandbetter.

Ifyourunacrossacasewhereabrowserdoeshaveaperformancewartthatothersdon'tsufferfrom,makesuretoreportittothemthroughwhatevermeansyouhaveavailable.Mostbrowsershaveopenpublicbugtrackerssuitableforthispurpose.

Tip:I'donlysuggestworkingaroundaperformanceissueinabrowserifitwasareallydrasticshow-stopper,notjustanannoyanceorfrustration.AndI'dbeverycarefultocheckthattheperformancehackdidn'thavenoticeablenegativesideeffectsinanotherbrowser.

Insteadofworryingaboutallthesemicroperformancenuances,weshouldinsteadbelookingatbig-picturetypesofoptimizations.

Howdoyouknowwhat'sbigpictureornot?Youhavetofirstunderstandifyourcodeisrunningonacriticalpathornot.Ifit'snotonthecriticalpath,chancesareyouroptimizationsarenotworthmuch.

Everheardtheadmonition,"that'sprematureoptimization!"?ItcomesfromafamousquotefromDonaldKnuth:"prematureoptimizationistherootofallevil.".Manydeveloperscitethisquotetosuggestthatmostoptimizationsare"premature"andarethusawasteofeffort.Thetruthis,asusual,morenuanced.

HereisKnuth'squote,incontext:

Programmerswasteenormousamountsoftimethinkingabout,orworryingabout,thespeedofnoncriticalpartsoftheirprograms,andtheseattemptsatefficiencyactuallyhaveastrongnegativeimpactwhendebuggingandmaintenanceareconsidered.Weshouldforgetaboutsmallefficiencies,sayabout97%ofthetime:prematureoptimizationistherootofallevil.Yetweshouldnotpassupouropportunitiesinthatcritical3%.[emphasisadded]

(http://web.archive.org/web/20130731202547/http://pplab.snu.ac.kr/courses/adv_pl05/papers/p261-knuth.pdf,ComputingSurveys,Vol6,No4,December1974)

Ibelieveit'safairparaphrasingtosaythatKnuthmeant:"non-criticalpathoptimizationistherootofallevil."Sothekeyistofigureoutifyourcodeisonthecriticalpath--youshouldoptimizeit!--ornot.

I'devengosofarastosaythis:noamountoftimespentoptimizingcriticalpathsiswasted,nomatterhowlittleissaved;butnoamountofoptimizationonnoncriticalpathsisjustified,nomatterhowmuchissaved.

Ifyourcodeisonthecriticalpath,suchasa"hot"pieceofcodethat'sgoingtoberunoverandoveragain,orinUXcriticalplaceswhereuserswillnotice,likeananimationlooporCSSstyleupdates,thenyoushouldsparenoeffortintryingto

BigPicture

http://web.archive.org/web/20130731202547/http://pplab.snu.ac.kr/courses/adv_pl05/papers/p261-knuth.pdf

employrelevant,measurablysignificantoptimizations.

Forexample,consideracriticalpathanimationloopthatneedstocoerceastringvaluetoanumber.Thereareofcoursemultiplewaystodothat(seetheTypes&Grammartitleofthisbookseries),butwhichoneifanyisthefastest?

varx="42";//neednumber`42`

//Option1:letimplicitcoercionautomaticallyhappen

vary=x/2;

//Option2:use`parseInt(..)`

vary=parseInt(x,0)/2;

//Option3:use`Number(..)`

vary=Number(x)/2;

//Option4:use`+ùnaryoperator

vary=+x/2;

//Option5:use`|ùnaryoperator

vary=(x|0)/2;

Note:Iwillleaveitasanexercisetothereadertosetupatestifyou'reinterestedinexaminingtheminutedifferencesinperformanceamongtheseoptions.

Whenconsideringthesedifferentoptions,astheysay,"Oneofthesethingsisnotliketheothers."parseInt(..)doesthejob,butitalsodoesalotmore--itparsesthestringratherthanjustcoercing.Youcanprobablyguess,correctly,thatparseInt(..)isasloweroption,andyoushouldprobablyavoidit.

Ofcourse,ifxcaneverbeavaluethatneedsparsing,suchas"42px"(likefromaCSSstylelookup),thenparseInt(..)reallyistheonlysuitableoption!

Number(..)isalsoafunctioncall.Fromabehavioralperspective,it'sidenticaltothe+unaryoperatoroption,butitmayinfactbealittleslower,requiringmoremachinerytoexecutethefunction.Ofcourse,it'salsopossiblethattheJSenginerecognizesthisbehavioralsymmetryandjusthandlestheinliningofNumber(..)'sbehavior(aka+x)foryou!

Butremember,obsessingabout+xversusx|0isinmostcaseslikelyawasteofeffort.Thisisamicroperformanceissue,andonethatyoushouldn'tletdictate/degradethereadabilityofyourprogram.

Whileperformanceisveryimportantincriticalpathsofyourprogram,it'snottheonlyfactor.Amongseveraloptionsthatareroughlysimilarinperformance,readabilityshouldbeanotherimportantconcern.

Aswebrieflymentionedearlier,ES6includesaspecificrequirementthatventuresintotheworldofperformance.It'srelatedtoaspecificformofoptimizationthatcanoccurwithfunctioncalls:tailcalloptimization.

Briefly,a"tailcall"isafunctioncallthatappearsatthe"tail"ofanotherfunction,suchthatafterthecallfinishes,there'snothinglefttodo(exceptperhapsreturnitsresultvalue).

Forexample,here'sanon-recursivesetupwithtailcalls:

functionfoo(x){

returnx;

}

functionbar(y){

returnfoo(y+1);//tailcall

}

functionbaz(){

return1+bar(40);//nottailcall

}

TailCallOptimization(TCO)

baz();//42

foo(y+1)isatailcallinbar(..)becauseafterfoo(..)finishes,bar(..)isalsofinishedexceptinthiscasereturningtheresultofthefoo(..)call.However,bar(40)isnotatailcallbecauseafteritcompletes,itsresultvaluemustbeaddedto1beforebaz()canreturnit.

Withoutgettingintotoomuchnitty-grittydetail,callinganewfunctionrequiresanextraamountofreservedmemorytomanagethecallstack,calleda"stackframe."Sotheprecedingsnippetwouldgenerallyrequireastackframeforeachofbaz(),bar(..),andfoo(..)allatthesametime.

However,ifaTCO-capableenginecanrealizethatthefoo(y+1)callisintailpositionmeaningbar(..)isbasicallycomplete,thenwhencallingfoo(..),itdoesn'tneedtocreateanewstackframe,butcaninsteadreusetheexistingstackframefrombar(..).That'snotonlyfaster,butitalsouseslessmemory.

Thatsortofoptimizationisn'tabigdealinasimplesnippet,butitbecomesamuchbiggerdealwhendealingwithrecursion,especiallyiftherecursioncouldhaveresultedinhundredsorthousandsofstackframes.WithTCOtheenginecanperformallthosecallswithasinglestackframe!

RecursionisahairytopicinJSbecausewithoutTCO,engineshavehadtoimplementarbitrary(anddifferent!)limitstohowdeeptheywilllettherecursionstackgetbeforetheystopit,topreventrunningoutofmemory.WithTCO,recursivefunctionswithtailpositioncallscanessentiallyrununbounded,becausethere'sneveranyextrausageofmemory!

Considerthatrecursivefactorial(..)frombefore,butrewrittentomakeitTCOfriendly:


functionfact(n,res){

if(n<2)returnres;

returnfact(n-1,n*res);

}

returnfact(n,1);

}

factorial(5);//120

Thisversionoffactorial(..)isstillrecursive,butit'salsooptimizablewithTCO,becausebothinnerfact(..)callsareintailposition.

Note:It'simportanttonotethatTCOonlyappliesifthere'sactuallyatailcall.Ifyouwriterecursivefunctionswithouttailcalls,theperformancewillstillfallbacktonormalstackframeallocation,andtheengines'limitsonsuchrecursivecallstackswillstillapply.Manyrecursivefunctionscanberewrittenaswejustshowedwithfactorial(..),butittakescarefulattentiontodetail.

OnereasonthatES6requiresenginestoimplementTCOratherthanleavingituptotheirdiscretionisbecausethelackofTCOactuallytendstoreducethechancesthatcertainalgorithmswillbeimplementedinJSusingrecursion,forfearofthecallstacklimits.

IfthelackofTCOintheenginewouldjustgracefullydegradetoslowerperformanceinallcases,itwouldn'tprobablyhavebeensomethingthatES6neededtorequire.ButbecausethelackofTCOcanactuallymakecertainprogramsimpractical,it'smoreanimportantfeatureofthelanguagethanjustahiddenimplementationdetail.

ES6guaranteesthatfromnowon,JSdeveloperswillbeabletorelyonthisoptimizationacrossallES6+compliantbrowsers.That'sawinforJSperformance!

Effectivelybenchmarkingperformanceofapieceofcode,especiallytocompareittoanotheroptionforthatsamecodeto

Review

seewhichapproachisfaster,requirescarefulattentiontodetail.

Ratherthanrollingyourownstatisticallyvalidbenchmarkinglogic,justusetheBenchmark.jslibrary,whichdoesthatforyou.Butbecarefulabouthowyouauthortests,becauseit'sfartooeasytoconstructatestthatseemsvalidbutthat'sactuallyflawed--eventinydifferencescanskewtheresultstobecompletelyunreliable.

It'simportanttogetasmanytestresultsfromasmanydifferentenvironmentsaspossibletoeliminatehardware/devicebias.jsPerf.comisafantasticwebsiteforcrowdsourcingperformancebenchmarktestruns.

Manycommonperformancetestsunfortunatelyobsessaboutirrelevantmicroperformancedetailslikex++versus++x.Writinggoodtestsmeansunderstandinghowtofocusonbigpictureconcerns,likeoptimizingonthecriticalpath,andavoidingfallingintotrapslikedifferentJSengines'implementationdetails.

Tailcalloptimization(TCO)isarequiredoptimizationasofES6thatwillmakesomerecursivepatternspracticalinJSwheretheywouldhavebeenimpossibleotherwise.TCOallowsafunctioncallinthetailpositionofanotherfunctiontoexecutewithoutneedinganyextraresources,whichmeanstheenginenolongerneedstoplacearbitraryrestrictionsoncallstackdepthforrecursivealgorithms.

Chapters1and2wentintoquiteabitofdetailabouttypicalasynchronousprogrammingpatternsandhowthey'recommonlysolvedwithcallbacks.Butwealsosawwhycallbacksarefatallylimitedincapability,whichledustoChapters3and4,withPromisesandgeneratorsofferingamuchmoresolid,trustable,andreason-ablebasetobuildyourasynchronyon.

Ireferencedmyownasynchronouslibraryasynquence(http://github.com/getify/asynquence)--"async"+"sequence"="asynquence"--severaltimesinthisbook,andIwanttonowbrieflyexplainhowitworksandwhyitsuniquedesignisimportantandhelpful.

Inthenextappendix,we'llexploresomeadvancedasyncpatterns,butyou'llprobablywantalibrarytomakethosepalatableenoughtobeuseful.We'lluseasynquencetoexpressthosepatterns,soyou'llwanttospendalittletimeheregettingtoknowthelibraryfirst.

asynquenceisobviouslynottheonlyoptionforgoodasynccoding;certainlytherearemanygreatlibrariesinthisspace.Butasynquenceprovidesauniqueperspectivebycombiningthebestofallthesepatternsintoasinglelibrary,andmoreoverisbuiltonasinglebasicabstraction:the(async)sequence.

MypremiseisthatsophisticatedJSprogramsoftenneedbitsandpiecesofvariousdifferentasynchronouspatternswoventogether,andthisisusuallyleftentirelyuptoeachdevelopertofigureout.Insteadofhavingtobringintwoormoredifferentasynclibrariesthatfocusondifferentaspectsofasynchrony,asynquenceunifiesthemintovariatedsequencesteps,withjustonecorelibrarytolearnanddeploy.

IbelievethevalueisstrongenoughwithasynquencetomakeasyncflowcontrolprogrammingwithPromise-stylesemanticssupereasytoaccomplish,sothat'swhywe'llexclusivelyfocusonthatlibraryhere.

Tobegin,I'llexplainthedesignprinciplesbehindasynquence,andthenwe'llillustratehowitsAPIworkswithcodeexamples.

Understandingasynquencebeginswithunderstandingafundamentalabstraction:anyseriesofstepsforatask,whethertheyseparatelyaresynchronousorasynchronous,canbecollectivelythoughtofasa"sequence".Inotherwords,asequenceisacontainerthatrepresentsatask,andiscomprisedofindividual(potentiallyasync)stepstocompletethattask.

EachstepinthesequenceiscontrolledunderthecoversbyaPromise(seeChapter3).Thatis,everystepyouaddtoasequenceimplicitlycreatesaPromisethatiswiredtothepreviousendofthesequence.BecauseofthesemanticsofPromises,everysinglestepadvancementinasequenceisasynchronous,evenifyousynchronouslycompletethestep.

Moreover,asequencewillalwaysproceedlinearlyfromsteptostep,meaningthatstep2alwayscomesafterstep1finishes,andsoon.

Ofcourse,anewsequencecanbeforkedoffanexistingsequence,meaningtheforkonlyoccursoncethemainsequencereachesthatpointintheflow.Sequencescanalsobecombinedinvariousways,includinghavingonesequencesubsumedbyanothersequenceataparticularpointintheflow.

AsequenceiskindoflikeaPromisechain.However,withPromisechains,thereisno"handle"tograbthatreferencestheentirechain.WhicheverPromiseyouhaveareferencetoonlyrepresentsthecurrentstepinthechainplusanyotherstepshangingoffit.Essentially,youcannotholdareferencetoaPromisechainunlessyouholdareferencetothefirstPromiseinthechain.


AppendixA:asynquenceLibrary

Sequences,AbstractionDesign

http://github.com/getify/asynquence

Therearemanycaseswhereitturnsouttobequiteusefultohaveahandlethatreferencestheentiresequencecollectively.Themostimportantofthosecasesiswithsequenceabort/cancel.AswecoveredextensivelyinChapter3,Promisesthemselvesshouldneverbeabletobecanceled,asthisviolatesafundamentaldesignimperative:externalimmutability.

Butsequenceshavenosuchimmutabilitydesignprinciple,mostlybecausesequencesarenotpassedaroundasfuture-valuecontainersthatneedimmutablevaluesemantics.Sosequencesaretheproperlevelofabstractiontohandleabort/cancelbehavior.asynquencesequencescanbeabort()edatanytime,andthesequencewillstopatthatpointandnotgoforanyreason.

There'splentymorereasonstopreferasequenceabstractionontopofPromisechains,forflowcontrolpurposes.

First,Promisechainingisarathermanualprocess--onethatcangetprettytediousonceyoustartcreatingandchainingPromisesacrossawideswathofyourprograms--andthistediumcanactcounterproductivelytodissuadethedeveloperfromusingPromisesinplaceswheretheyarequiteappropriate.

Abstractionsaremeanttoreduceboilerplateandtedium,sothesequenceabstractionisagoodsolutiontothisproblem.WithPromises,yourfocusisontheindividualstep,andthere'slittleassumptionthatyouwillkeepthechaingoing.Withsequences,theoppositeapproachistaken,assumingthesequencewillkeephavingmorestepsaddedindefinitely.

Thisabstractioncomplexityreductionisespeciallypowerfulwhenyoustartthinkingabouthigher-orderPromisepatterns(beyondrace([..])andall([..]).

Forexample,inthemiddleofasequence,youmaywanttoexpressastepthatisconceptuallylikeatry..catchinthatthestepwillalwaysresultinsuccess,eithertheintendedmainsuccessresolutionorapositivenonerrorsignalforthecaughterror.Or,youmightwanttoexpressastepthatislikearetry/untilloop,whereitkeepstryingthesamestepoverandoveruntilsuccessoccurs.

ThesesortsofabstractionsarequitenontrivialtoexpressusingonlyPromiseprimitives,anddoingsointhemiddleofanexistingPromisechainisnotpretty.Butifyouabstractyourthinkingtoasequence,andconsiderastepasawrapperaroundaPromise,thatstepwrappercanhidesuchdetails,freeingyoutothinkabouttheflowcontrolinthemostsensiblewaywithoutbeingbotheredbythedetails.

Second,andperhapsmoreimportantly,thinkingofasyncflowcontrolintermsofstepsinasequenceallowsyoutoabstractoutthedetailsofwhattypesofasynchronicityareinvolvedwitheachindividualstep.Underthecovers,aPromisewillalwayscontrolthestep,butabovethecovers,thatstepcanlookeitherlikeacontinuationcallback(thesimpledefault),orlikearealPromise,orasarun-to-completiongenerator,or...Hopefully,yougetthepicture.

Third,sequencescanmoreeasilybetwistedtoadapttodifferentmodesofthinking,suchasevent-,stream-,orreactive-basedcoding.asynquenceprovidesapatternIcall"reactivesequences"(whichwe'llcoverlater)asavariationonthe"reactiveobservable"ideasinRxJS("ReactiveExtensions"),thatletsarepeatableeventfireoffanewsequenceinstanceeachtime.Promisesareone-shot-only,soit'squiteawkwardtoexpressrepetitiousasynchronywithPromisesalone.

Anotheralternatemodeofthinkinginvertstheresolution/controlcapabilityinapatternIcall"iterablesequences".Insteadofeachindividualstepinternallycontrollingitsowncompletion(andthusadvancementofthesequence),thesequenceisinvertedsotheadvancementcontrolisthroughanexternaliterator,andeachstepintheiterablesequencejustrespondstothenext(..)iteratorcontrol.

We'llexploreallofthesedifferentvariationsaswegothroughouttherestofthisappendix,sodon'tworryifweranoverthosebitsfartooquicklyjustnow.

ThetakeawayisthatsequencesareamorepowerfulandsensibleabstractionforcomplexasynchronythanjustPromises(Promisechains)orjustgenerators,andasynquenceisdesignedtoexpressthatabstractionwithjusttherightlevelofsugartomakeasyncprogrammingmoreunderstandableandmoreenjoyable.

asynquenceAPI

Tostartoff,thewayyoucreateasequence(anasynquenceinstance)iswiththeASQ(..)function.AnASQ()callwithnoparameterscreatesanemptyinitialsequence,whereaspassingoneormorevaluesorfunctionstoASQ(..)setsupthesequencewitheachargumentrepresentingtheinitialstepsofthesequence.

Note:Forthepurposesofallcodeexampleshere,Iwillusetheasynquencetop-levelidentifieringlobalbrowserusage:ASQ.Ifyouincludeanduseasynquencethroughamodulesystem(browserorserver),youofcoursecandefinewhicheversymbolyouprefer,andasynquencewon'tcare!

ManyoftheAPImethodsdiscussedherearebuiltintothecoreofasynquence,butothersareprovidedthroughincludingtheoptional"contrib"plug-inspackage.Seethedocumentationforasynquenceforwhetheramethodisbuiltinordefinedviaplug-in:http://github.com/getify/asynquence

Ifafunctionrepresentsanormalstepinthesequence,thatfunctionisinvokedwiththefirstparameterbeingthecontinuationcallback,andanysubsequentparametersbeinganymessagespassedonfromthepreviousstep.Thestepwillnotcompleteuntilthecontinuationcallbackiscalled.Onceit'scalled,anyargumentsyoupasstoitwillbesentalongasmessagestothenextstepinthesequence.

Toaddanadditionalnormalsteptothesequence,callthen(..)(whichhasessentiallytheexactsamesemanticsastheASQ(..)call):

ASQ(

//step1

function(done){


done("Hello");

},100);

},

//step2

function(done,greeting){


done(greeting+"World");

},100);

}

)

//step3

.then(function(done,msg){


done(msg.toUpperCase());

},100);

})

//step4


console.log(msg);//HELLOWORLD

});

Note:Thoughthenamethen(..)isidenticaltothenativePromisesAPI,thisthen(..)isdifferent.Youcanpassasfeworasmanyfunctionsorvaluestothen(..)asyou'dlike,andeachistakenasaseparatestep.There'snotwo-callbackfulfilled/rejectedsemanticsinvolved.

UnlikewithPromises,wheretochainonePromisetothenextyouhavetocreateandreturnthatPromisefromathen(..)fulfillmenthandler,withasynquence,allyouneedtodoiscallthecontinuationcallback--Ialwayscallitdone()butyoucannameitwhateversuitsyou--andoptionallypassitcompletionmessagesasarguments.

Eachstepdefinedbythen(..)isassumedtobeasynchronous.Ifyouhaveastepthat'ssynchronous,youcaneitherjustcalldone(..)rightaway,oryoucanusethesimplerval(..)stephelper:

//step1(sync)

ASQ(function(done){

done("Hello");//manuallysynchronous

})

//step2(sync)

.val(function(greeting){

Steps

http://github.com/getify/asynquence

returngreeting+"World";

})

//step3(async)



done(msg.toUpperCase());

},100);

})

//step4(sync)

.val(function(msg){

console.log(msg);

});

Asyoucansee,val(..)-invokedstepsdon'treceiveacontinuationcallback,asthatpartisassumedforyou--andtheparameterlistislessclutteredasaresult!Tosendamessagealongtothenextstep,yousimplyusereturn.

Thinkofval(..)asrepresentingasynchronous"value-only"step,whichisusefulforsynchronousvalueoperations,logging,andthelike.

OneimportantdifferencewithasynquencecomparedtoPromisesiswitherrorhandling.

WithPromises,eachindividualPromise(step)inachaincanhaveitsownindependenterror,andeachsubsequentstephastheabilitytohandletheerrorornot.Themainreasonforthissemanticcomes(again)fromthefocusonindividualPromisesratherthanonthechain(sequence)asawhole.

Ibelievethatmostofthetime,anerrorinonepartofasequenceisgenerallynotrecoverable,sothesubsequentstepsinthesequencearemootandshouldbeskipped.So,bydefault,anerroratanystepofasequencethrowstheentiresequenceintoerrormode,andtherestofthenormalstepsareignored.

Ifyoudoneedtohaveastepwhereitserrorisrecoverable,thereareseveraldifferentAPImethodsthatcanaccomodate,suchastry(..)--previouslymentionedasakindoftry..catchstep--oruntil(..)--aretryloopthatkeepsattemptingthestepuntilitsucceedsoryoumanuallybreak()theloop.asynquenceevenhaspThen(..)andpCatch(..)methods,whichworkidenticallytohownormalPromisethen(..)andcatch(..)work(seeChapter3),soyoucandolocalizedmid-sequenceerrorhandlingifyousochoose.

Thepointis,youhavebothoptions,butthemorecommononeinmyexperienceisthedefault.WithPromises,togetachainofstepstoignoreallstepsonceanerroroccurs,youhavetotakecarenottoregisterarejectionhandleratanystep;otherwise,thaterrorgetsswallowedashandled,andthesequencemaycontinue(perhapsunexpectedly).Thiskindofdesiredbehaviorisabitawkwardtoproperlyandreliablyhandle.

Toregisterasequenceerrornotificationhandler,asynquenceprovidesanor(..)sequencemethod,whichalsohasanaliasofonerror(..).Youcancallthismethodanywhereinthesequence,andyoucanregisterasmanyhandlersasyou'dlike.Thatmakesiteasyformultipledifferentconsumerstolisteninonasequencetoknowifitfailedornot;it'skindoflikeanerroreventhandlerinthatrespect.

JustlikewithPromises,allJSexceptionsbecomesequenceerrors,oryoucanprogrammaticallysignalasequenceerror:

varsq=ASQ(function(done){


//signalanerrorforthesequence

done.fail("Oops");

},100);

})

.then(function(done){

//willnevergethere

})

.or(function(err){

console.log(err);//Oops

})


//won'tgethereeither

});

Errors

//later

sq.or(function(err){


});

AnotherreallyimportantdifferencewitherrorhandlinginasynquencecomparedtonativePromisesisthedefaultbehaviorof"unhandledexceptions".AswediscussedatlengthinChapter3,arejectedPromisewithoutaregisteredrejectionhandlerwilljustsilentlyhold(akaswallow)theerror;youhavetoremembertoalwaysendachainwithafinalcatch(..).

Inasynquence,theassumptionisreversed.

Ifanerroroccursonasequence,anditatthatmomenthasnoerrorhandlersregistered,theerrorisreportedtotheconsole.Inotherwords,unhandledrejectionsarebydefaultalwaysreportedsoasnottobeswallowedandmissed.

Assoonasyouregisteranerrorhandleragainstasequence,itoptsthatsequenceoutofsuchreporting,topreventduplicatenoise.

Theremay,infact,becaseswhereyouwanttocreateasequencethatmaygointotheerrorstatebeforeyouhaveachancetoregisterthehandler.Thisisn'tcommon,butitcanhappenfromtimetotime.

Inthosecases,youcanalsooptasequenceinstanceoutoferrorreportingbycallingdefer()onthesequence.Youshouldonlyoptoutoferrorreportingifyouaresurethatyou'regoingtoeventuallyhandlesucherrors:

varsq1=ASQ(function(done){

doesnt.Exist();//willthrowexceptiontoconsole

});

varsq2=ASQ(function(done){

doesnt.Exist();//willthrowonlyasequenceerror

})

//opt-outoferrorreporting

.defer();


sq1.or(function(err){

console.log(err);//ReferenceError

});



});

},100);

//ReferenceError(fromsq1)

ThisisbettererrorhandlingbehaviorthanPromisesthemselveshave,becauseit'sthePitofSuccess,notthePitofFailure(seeChapter3).

Note:Ifasequenceispipedinto(akasubsumedby)anothersequence--see"CombiningSequences"foracompletedescription--thenthesourcesequenceisoptedoutoferrorreporting,butnowthetargetsequence'serrorreportingorlackthereofmustbeconsidered.

Notallstepsinyoursequenceswillhavejustasingle(async)tasktoperform;somewillneedtoperformmultiplesteps"inparallel"(concurrently).Astepinasequenceinwhichmultiplesubstepsareprocessingconcurrentlyiscalledagate(..)--there'sanall(..)aliasifyouprefer--andisdirectlysymmetrictonativePromise.all([..]).

Ifallthestepsinthegate(..)completesuccessfully,allsuccessmessageswillbepassedtothenextsequencestep.Ifanyofthemgenerateerrors,thewholesequenceimmediatelygoesintoanerrorstate.

Consider:

ParallelSteps

ASQ(function(done){

setTimeout(done,100);

})

.gate(

function(done){


done("Hello");

},100);

},

function(done){


done("World","!");

},100);

}

)

.val(function(msg1,msg2){

console.log(msg1);//Hello

console.log(msg2);//["World","!"]

});

Forillustration,let'scomparethatexampletonativePromises:

newPromise(function(resolve,reject){

setTimeout(resolve,100);

})

.then(function(){

returnPromise.all([



resolve("Hello");

},100);

}),



//note:weneeda[]arrayhere

resolve(["World","!"]);

},100);

})

]);

})


console.log(msgs[0]);//Hello

console.log(msgs[1]);//["World","!"]

});

Yuck.Promisesrequirealotmoreboilerplateoverheadtoexpressthesameasynchronousflowcontrol.That'sagreatillustrationofwhytheasynquenceAPIandabstractionmakedealingwithPromisestepsalotnicer.Theimprovementonlygoeshigherthemorecomplexyourasynchronyis.

Thereareseveralvariationsinthecontribplug-insonasynquence'sgate(..)steptypethatcanbequitehelpful:

any(..)islikegate(..),exceptjustonesegmenthastoeventuallysucceedtoproceedonthemainsequence.first(..)islikeany(..),exceptassoonasanysegmentsucceeds,themainsequenceproceeds(ignoringsubsequentresultsfromothersegments).race(..)(symmetricwithPromise.race([..]))islikefirst(..),exceptthemainsequenceproceedsassoonasanysegmentcompletes(eithersuccessorfailure).last(..)islikeany(..),exceptonlythelatestsegmenttocompletesuccessfullysendsitsmessage(s)alongtothemainsequence.none(..)istheinverseofgate(..):themainsequenceproceedsonlyifallthesegmentsfail(withallsegmenterrormessage(s)transposedassuccessmessage(s)andviceversa).

Let'sfirstdefinesomehelperstomakeillustrationcleaner:

functionsuccess1(done){

StepVariations


done(1);

},100);

}

functionsuccess2(done){


done(2);

},100);

}

functionfailure3(done){


done.fail(3);

},100);

}

functionoutput(msg){

console.log(msg);

}

Now,let'sdemonstratethesegate(..)stepvariations:

ASQ().race(

failure3,

success1

)

.or(output);//3

ASQ().any(

success1,

failure3,

success2

)

.val(function(){


console.log(

args//[1,undefined,2]

);

});

ASQ().first(

failure3,

success1,

success2

)

.val(output);//1

ASQ().last(

failure3,

success1,

success2

)

.val(output);//2

ASQ().none(

failure3

)

.val(output)//3

.none(

failure3

success1

)

.or(output);//1

Anotherstepvariationismap(..),whichletsyouasynchronouslymapelementsofanarraytodifferentvalues,andthestepdoesn'tproceeduntilallthemappingsarecomplete.map(..)isverysimilartogate(..),exceptitgetstheinitialvaluesfromanarrayinsteadoffromseparatelyspecifiedfunctions,andalsobecauseyoudefineasinglefunctioncallbacktooperateoneachvalue:

functiondouble(x,done){


done(x*2);

},100);

}

ASQ().map([1,2,3],double)

.val(output);//[2,4,6]

Also,map(..)canreceiveeitherofitsparameters(thearrayorthecallback)frommessagespassedfromthepreviousstep:

functionplusOne(x,done){


done(x+1);

},100);

}

ASQ([1,2,3])

.map(double)//message`[1,2,3]`comesin

.map(plusOne)//message`[2,4,6]`comesin

.val(output);//[3,5,7]

Anothervariationiswaterfall(..),whichiskindoflikeamixturebetweengate(..)'smessagecollectionbehaviorbutthen(..)'ssequentialprocessing.

Step1isfirstexecuted,thenthesuccessmessagefromstep1isgiventostep2,andthenbothsuccessmessagesgotostep3,andthenallthreesuccessmessagesgotostep4,andsoon,suchthatthemessagessortofcollectandcascadedownthe"waterfall".

Consider:

functiondouble(done){

varargs=[].slice.call(arguments,1);

console.log(args);


done(args[args.length-1]*2);

},100);

}

ASQ(3)

.waterfall(

double,//[3]

double,//[6]

double,//[6,12]

double//[6,12,24]

)

.val(function(){


console.log(args);//[6,12,24,48]

});

Ifatanypointinthe"waterfall"anerroroccurs,thewholesequenceimmediatelygoesintoanerrorstate.

Sometimesyouwanttomanageerrorsatthesteplevelandnotletthemnecessarilysendthewholesequenceintotheerrorstate.asynquenceofferstwostepvariationsforthatpurpose.

try(..)attemptsastep,andifitsucceeds,thesequenceproceedsasnormal,butifthestepfails,thefailureisturnedintoasuccessmessageformatedas{catch:..}withtheerrormessage(s)filledin:

ASQ()

.try(success1)

.val(output)//1

ErrorTolerance

.try(failure3)

.val(output)//{catch:3}

.or(function(err){

//nevergetshere

});

Youcouldinsteadsetuparetryloopusinguntil(..),whichtriesthestepandifitfails,retriesthestepagainonthenexteventlooptick,andsoon.

Thisretryloopcancontinueindefinitely,butifyouwanttobreakoutoftheloop,youcancallthebreak()flagonthecompletiontrigger,whichsendsthemainsequenceintoanerrorstate:

varcount=0;

ASQ(3)

.until(double)

.val(output)//6

.until(function(done){

count++;


if(count<5){

done.fail();

}

else{

//breakoutoftheùntil(..)`retryloop

done.break("Oops");

}

},100);

})

.or(output);//Oops

Ifyouwouldprefertohave,inlineinyoursequence,Promise-stylesemanticslikePromises'then(..)andcatch(..)(seeChapter3),youcanusethepThenandpCatchplug-ins:

ASQ(21)

.pThen(function(msg){

returnmsg*2;

})

.pThen(output)//42

.pThen(function(){

//throwanexception

doesnt.Exist();

})

.pCatch(function(err){

//caughttheexception(rejection)


})

.val(function(){

//mainsequenceisbackina

//successstatebecauseprevious

//exceptionwascaughtby

//`pCatch(..)`

});

pThen(..)andpCatch(..)aredesignedtoruninthesequence,butbehaveasifitwasanormalPromisechain.Assuch,youcaneitherresolvegenuinePromisesorasynquencesequencesfromthe"fulfillment"handlerpassedtopThen(..)(seeChapter3).

OnefeaturethatcanbequiteusefulaboutPromisesisthatyoucanattachmultiplethen(..)handlerregistrationstothesamepromise,effectively"forking"theflow-controlatthatpromise:

Promise-StyleSteps

ForkingSequences


//fork1(from`p`)

p.then(function(msg){

returnmsg*2;

})



})

//fork2(from`p`)

p.then(function(msg){


});

Thesame"forking"iseasyinasynquencewithfork():

varsq=ASQ(..).then(..).then(..);

varsq2=sq.fork();

//fork1

sq.then(..)..;

//fork2

sq2.then(..)..;

Thereverseoffork()ing,youcancombinetwosequencesbysubsumingoneintoanother,usingtheseq(..)instancemethod:



done("HelloWorld");

},200);

});

ASQ(function(done){

setTimeout(done,100);

})

//subsume`sq`sequenceintothissequence

.seq(sq)

.val(function(msg){

console.log(msg);//HelloWorld

})

seq(..)caneitheracceptasequenceitself,asshownhere,orafunction.Ifafunction,it'sexpectedthatthefunctionwhencalledwillreturnasequence,sotheprecedingcodecouldhavebeendonewith:

//..

.seq(function(){

returnsq;

})

//..

Also,thatstepcouldinsteadhavebeenaccomplishedwithapipe(..):

//..


//pipe`sqìntothe`done`continuationcallback

sq.pipe(done);

})

//..

CombiningSequences

Whenasequenceissubsumed,bothitssuccessmessagestreamanditserrorstreamarepipedin.

Note:Asmentionedinanearliernote,piping(manuallywithpipe(..)orautomaticallywithseq(..))optsthesourcesequenceoutoferror-reporting,butdoesn'taffecttheerrorreportingstatusofthetargetsequence.

Ifanystepofasequenceisjustanormalvalue,thatvalueisjustmappedtothatstep'scompletionmessage:

varsq=ASQ(42);

sq.val(function(msg){


});

Ifyouwanttomakeasequencethat'sautomaticallyerrored:

varsq=ASQ.failed("Oops");

ASQ()

.seq(sq)

.val(function(msg){

//won'tgethere

})

.or(function(err){


});

Youalsomaywanttoautomaticallycreateadelayed-valueoradelayed-errorsequence.UsingtheafterandfailAftercontribplug-ins,thisiseasy:

varsq1=ASQ.after(100,"Hello","World");

varsq2=ASQ.failAfter(100,"Oops");

sq1.val(function(msg1,msg2){

console.log(msg1,msg2);//HelloWorld

});



});

Youcanalsoinsertadelayinthemiddleofasequenceusingafter(..):

ASQ(42)

//insertadelayintothesequence

.after(100)

.val(function(msg){


});

IthinkasynquencesequencesprovidealotofvalueontopofnativePromises,andforthemostpartyou'llfinditmorepleasantandmorepowerfultoworkatthatlevelofabstration.However,integratingasynquencewithothernon-asynquencecodewillbeareality.

Youcaneasilysubsumeapromise(e.g.,thenable--seeChapter3)intoasequenceusingthepromise(..)instancemethod:

ValueandErrorSequences

PromisesandCallbacks


ASQ()

.promise(p)//couldalso:`function(){returnp;}`

.val(function(msg){


});

Andtogotheoppositedirectionandfork/vendapromisefromasequenceatacertainstep,usethetoPromisecontribplug-in:

varsq=ASQ.after(100,"HelloWorld");

sq.toPromise()

//thisisastandardpromisechainnow


returnmsg.toUpperCase();

})


console.log(msg);//HELLOWORLD

});

Toadaptasynquencetosystemsusingcallbacks,thereareseveralhelperfacilities.Toautomaticallygeneratean"error-firststyle"callbackfromyoursequencetowireintoacallback-orientedutility,useerrfcb:


//note:expecting"error-firststyle"callback

someAsyncFuncWithCB(1,2,done.errfcb)

})

.val(function(msg){

//..

})

.or(function(err){

//..

});

//note:expecting"error-firststyle"callback

anotherAsyncFuncWithCB(1,2,sq.errfcb());

Youalsomaywanttocreateasequence-wrappedversionofautility--compareto"promisory"inChapter3and"thunkory"inChapter4--andasynquenceprovidesASQ.wrap(..)forthatpurpose:

varcoolUtility=ASQ.wrap(someAsyncFuncWithCB);

coolUtility(1,2)

.val(function(msg){

//..

})

.or(function(err){

//..

});

Note:Forthesakeofclarity(andforfun!),let'scoinyetanotherterm,forasequence-producingfunctionthatcomesfromASQ.wrap(..),likecoolUtilityhere.Ipropose"sequory"("sequence"+"factory").

Thenormalparadigmforasequenceisthateachstepisresponsibleforcompletingitself,whichiswhatadvancesthesequence.Promisesworkthesameway.

TheunfortunatepartisthatsometimesyouneedexternalcontroloveraPromise/step,whichleadstoawkward"capabilityextraction".

IterableSequences

ConsiderthisPromisesexample:

vardomready=newPromise(function(resolve,reject){

//don'twanttoputthishere,because

//itbelongslogicallyinanotherpart

//ofthecode

document.addEventListener("DOMContentLoaded",resolve);

});

//..

domready.then(function(){

//DOMisready!

});

The"capabilityextraction"anti-patternwithPromiseslookslikethis:

varready;

vardomready=newPromise(function(resolve,reject){

//extractthe`resolve()`capability

ready=resolve;

});

//..

domready.then(function(){

//DOMisready!

});

//..

document.addEventListener("DOMContentLoaded",ready);

Note:Thisanti-patternisanawkwardcodesmell,inmyopinion,butsomedeveloperslikeit,forreasonsIcan'tgrasp.

asynquenceoffersaninvertedsequencetypeIcall"iterablesequences",whichexternalizesthecontrolcapability(it'squiteusefulinusecaseslikethedomready):

//note:`domready`hereisan*iterator*that

//controlsthesequence

vardomready=ASQ.iterable();

//..

domready.val(function(){

//DOMisready

});

//..

document.addEventListener("DOMContentLoaded",domready.next);

There'smoretoiterablesequencesthanwhatweseeinthisscenario.We'llcomebacktotheminAppendixB.

InChapter4,wederivedautilitycalledrun(..)whichcanrungeneratorstocompletion,listeningforyieldedPromisesandusingthemtoasyncresumethegenerator.asynquencehasjustsuchautilitybuiltin,calledrunner(..).

Let'sfirstsetupsomehelpersforillustration:

functiondoublePr(x){



RunningGenerators

resolve(x*2);

},100);

});

}

functiondoubleSeq(x){

returnASQ(function(done){


done(x*2)

},100);

});

}

Now,wecanuserunner(..)asastepinthemiddleofasequence:

ASQ(10,11)

.runner(function*(token){

varx=token.messages[0]+token.messages[1];

//yieldarealpromise

x=yielddoublePr(x);

//yieldasequence

x=yielddoubleSeq(x);

returnx;

})

.val(function(msg){


});

Youcanalsocreateaself-packagedgenerator--thatis,anormalfunctionthatrunsyourspecifiedgeneratorandreturnsasequenceforitscompletion--byASQ.wrap(..)ingit:

varfoo=ASQ.wrap(function*(token){

varx=token.messages[0]+token.messages[1];

//yieldarealpromise

x=yielddoublePr(x);

//yieldasequence

x=yielddoubleSeq(x);

returnx;

},{gen:true});

//..

foo(8,9)

.val(function(msg){


});

There'salotmoreawesomethatrunner(..)iscapableof,butwe'llcomebacktothatinAppendixB.

asynquenceisasimpleabstraction--asequenceisaseriesof(async)steps--ontopofPromises,aimedatmakingworkingwithvariousasynchronouspatternsmucheasier,withoutanycompromiseincapability.

ThereareothergoodiesintheasynquencecoreAPIanditscontribplug-insbeyondwhatwesawinthisappendix,butwe'llleavethatasanexerciseforthereadertogochecktherestofthecapabilitiesout.

You'venowseentheessenceandspiritofasynquence.Thekeytakeawayisthatasequenceiscomprisedofsteps,and

WrappedGenerators

Review

thosestepscanbeanyofdozensofdifferentvariationsonPromises,ortheycanbeagenerator-run,or...Thechoiceisuptoyou,youhaveallthefreedomtoweavetogetherwhateverasyncflowcontrollogicisappropriateforyourtasks.Nomorelibraryswitchingtocatchdifferentasyncpatterns.

Iftheseasynquencesnippetshavemadesensetoyou,you'renowprettywelluptospeedonthelibrary;itdoesn'ttakethatmuchtolearn,actually!

Ifyou'restillalittlefuzzyonhowitworks(orwhy!),you'llwanttospendalittlemoretimeexaminingthepreviousexamplesandplayingaroundwithasynquenceyourself,beforegoingontothenextappendix.AppendixBwillpushasynquenceintoseveralmoreadvancedandpowerfulasyncpatterns.

AppendixAintroducedtheasynquencelibraryforsequence-orientedasyncflowcontrol,primarilybasedonPromisesandgenerators.

Nowwe'llexploreotheradvancedasynchronouspatternsbuiltontopofthatexistingunderstandingandfunctionality,andseehowasynquencemakesthosesophisticatedasynctechniqueseasytomixandmatchinourprogramswithoutneedinglotsofseparatelibraries.

Weintroducedasynquence'siterablesequencesinthepreviousappendix,butwewanttorevisittheminmoredetail.

Torefresh,recall:

vardomready=ASQ.iterable();

//..

domready.val(function(){

//DOMisready

});

//..

document.addEventListener("DOMContentLoaded",domready.next);

Now,let'sdefineasequenceofmultiplestepsasaniterablesequence:

varsteps=ASQ.iterable();

steps

.then(functionSTEP1(x){

returnx*2;

})

.steps(functionSTEP2(x){

returnx+3;

})

.steps(functionSTEP3(x){

returnx*4;

});

steps.next(8).value;//16



steps.next().done;//true

Asyoucansee,aniterablesequenceisastandard-compliantiterator(seeChapter4).So,itcanbeiteratedwithanES6for..ofloop,justlikeagenerator(oranyotheriterable)can:

varsteps=ASQ.iterable();

steps

.then(functionSTEP1(){return2;})




.then(functionSTEP5(){return10;});


AppendixB:AdvancedAsyncPatterns

IterableSequences

for(varvofsteps){

console.log(v);

}

//246810

Beyondtheeventtriggeringexampleshowninthepreviousappendix,iterablesequencesareinterestingbecauseinessencetheycanbeseenasastand-inforgeneratorsorPromisechains,butwithevenmoreflexibility.

ConsideramultipleAjaxrequestexample--we'veseenthesamescenarioinChapters3and4,bothasaPromisechainandasagenerator,respectively--expressedasaniterablesequence:

//sequence-awareajax

varrequest=ASQ.wrap(ajax);

ASQ("http://some.url.1")

.runner(

ASQ.iterable()

.then(functionSTEP1(token){

varurl=token.messages[0];

returnrequest(url);

})

.then(functionSTEP2(resp){

returnASQ().gate(

request("http://some.url.2/?v="+resp),

request("http://some.url.3/?v="+resp)

);

})

.then(functionSTEP3(r1,r2){returnr1+r2;})

)

.val(function(msg){

console.log(msg);

});

Theiterablesequenceexpressesasequentialseriesof(syncorasync)stepsthatlooksawfullysimilartoaPromisechain--inotherwords,it'smuchcleanerlookingthanjustplainnestedcallbacks,butnotquiteasniceastheyield-basedsequentialsyntaxofgenerators.

ButwepasstheiterablesequenceintoASQ#runner(..),whichrunsittocompletionthesameasifitwasagenerator.Thefactthataniterablesequencebehavesessentiallythesameasageneratorisnotableforacoupleofreasons.

First,iterablesequencesarekindofapre-ES6equivalenttoacertainsubsetofES6generators,whichmeansyoucaneitherauthorthemdirectly(torunanywhere),oryoucanauthorES6generatorsandtranspile/convertthemtoiterablesequences(orPromisechainsforthatmatter!).

Thinkingofanasync-run-to-completiongeneratorasjustsyntacticsugarforaPromisechainisanimportantrecognitionoftheirisomorphicrelationship.

Beforewemoveon,weshouldnotethattheprevioussnippetcouldhavebeenexpressedinasynquenceas:

ASQ("http://some.url.1")

.seq(/*STEP1*/request)

.seq(functionSTEP2(resp){

returnASQ().gate(



);

})

.val(functionSTEP3(r1,r2){returnr1+r2;})

.val(function(msg){

console.log(msg);

});

Moreover,step2couldhaveevenbeenexpressedas:

.gate(

functionSTEP2a(done,resp){


.pipe(done);

},

functionSTEP2b(done,resp){


.pipe(done);

}

)

So,whywouldwegotothetroubleofexpressingourflowcontrolasaniterablesequenceinaASQ#runner(..)step,whenitseemslikeasimpler/flatterasyquencechaindoesthejobwell?

Becausetheiterablesequenceformhasanimportanttrickupitssleevethatgivesusmorecapability.Readon.

Generators,normalasynquencesequences,andPromisechains,arealleagerlyevaluated--whateverflowcontrolisexpressedinitiallyisthefixedflowthatwillbefollowed.

However,iterablesequencesarelazilyevaluated,whichmeansthatduringexecutionoftheiterablesequence,youcanextendthesequencewithmorestepsifdesired.

Note:Youcanonlyappendtotheendofaniterablesequence,notinjectintothemiddleofthesequence.

Let'sfirstlookatasimpler(synchronous)exampleofthatcapabilitytogetfamiliarwithit:

functiondouble(x){

x*=2;

//shouldwekeepextending?

if(x<500){

isq.then(double);

}

returnx;

}

//setupsingle-stepiterablesequence

varisq=ASQ.iterable().then(double);

for(varv=10,ret;

(ret=isq.next(v))&&!ret.done;

){

v=ret.value;

console.log(v);

}

Theiterablesequencestartsoutwithonlyonedefinedstep(isq.then(double)),butthesequencekeepsextendingitselfundercertainconditions(x<500).BothasynquencesequencesandPromisechainstechnicallycandosomethingsimilar,butwe'llseeinalittlebitwhytheircapabilityisinsufficient.

Thoughthisexampleisrathertrivialandcouldotherwisebeexpressedwithawhileloopinagenerator,we'llconsidermoresophisticatedcases.

Forinstance,youcouldexaminetheresponsefromanAjaxrequestandifitindicatesthatmoredataisneeded,youconditionallyinsertmorestepsintotheiterablesequencetomaketheadditionalrequest(s).Oryoucouldconditionallyaddavalue-formattingsteptotheendofyourAjaxhandling.

Consider:

varsteps=ASQ.iterable()

ExtendingIterableSequences

.then(functionSTEP1(token){

varurl=token.messages[0].url;

//wasanadditionalformattingstepprovided?

if(token.messages[0].format){

steps.then(token.messages[0].format);

}

returnrequest(url);

})

.then(functionSTEP2(resp){

//addanotherAjaxrequesttothesequence?

if(/x1/.test(resp)){

steps.then(functionSTEP5(text){

returnrequest(

"http://some.url.4/?v="+text

);

});

}

returnASQ().gate(



);

})

.then(functionSTEP3(r1,r2){returnr1+r2;});

Youcanseeintwodifferentplaceswhereweconditionallyextendstepswithsteps.then(..).Andtorunthisstepsiterablesequence,wejustwireitintoourmainprogramflowwithanasynquencesequence(calledmainhere)usingASQ#runner(..):

varmain=ASQ({

url:"http://some.url.1",

format:functionSTEP4(text){

returntext.toUpperCase();

}

})

.runner(steps)

.val(function(msg){

console.log(msg);

});

Cantheflexibility(conditionalbehavior)ofthestepsiterablesequencebeexpressedwithagenerator?Kindof,butwehavetorearrangethelogicinaslightlyawkwardway:

function*steps(token){

//**STEP1**

varresp=yieldrequest(token.messages[0].url);

//**STEP2**

varrvals=yieldASQ().gate(



);

//**STEP3**

vartext=rvals[0]+rvals[1];

//**STEP4**

//wasanadditionalformattingstepprovided?

if(token.messages[0].format){

text=yieldtoken.messages[0].format(text);

}

//**STEP5**

//needanotherAjaxrequestaddedtothesequence?

if(/foobar/.test(resp)){

text=yieldrequest(

"http://some.url.4/?v="+text

);

}

returntext;

}

//note:`*steps()`canberunbythesameÀSQ`sequence

//as`steps`waspreviously

Settingasidethealreadyidentifiedbenefitsofthesequential,synchronous-lookingsyntaxofgenerators(seeChapter4),thestepslogichadtobereorderedinthe*steps()generatorform,tofakethedynamicismoftheextendableiterablesequencesteps.

WhataboutexpressingthefunctionalitywithPromisesorsequences,though?Youcandosomethinglikethis:

varsteps=something(..)

.then(..)

.then(function(..){

//..

//extendingthechain,right?

steps=steps.then(..);

//..

})

.then(..);

Theproblemissubtlebutimportanttograsp.So,considertryingtowireupourstepsPromisechainintoourmainprogramflow--thistimeexpressedwithPromisesinsteadofasynquence:

varmain=Promise.resolve({

url:"http://some.url.1",

format:functionSTEP4(text){

returntext.toUpperCase();

}

})

.then(function(..){

returnsteps;//hint!

})

.val(function(msg){

console.log(msg);

});

Canyouspottheproblemnow?Lookclosely!

There'saraceconditionforsequencestepsordering.Whenyoureturnsteps,atthatmomentstepsmightbetheoriginallydefinedpromisechain,oritmightnowpointtotheextendedpromisechainviathesteps=steps.then(..)call,dependingonwhatorderthingshappen.

Herearethetwopossibleoutcomes:

Ifstepsisstilltheoriginalpromisechain,onceit'slater"extended"bysteps=steps.then(..),thatextendedpromiseontheendofthechainisnotconsideredbythemainflow,asit'salreadytappedthestepschain.Thisistheunfortunatelylimitingeagerevaluation.Ifstepsisalreadytheextendedpromisechain,itworksasweexpectinthattheextendedpromiseiswhatmaintaps.

Otherthantheobviousfactthataraceconditionisintolerable,thefirstcaseistheconcern;itillustrateseagerevaluationofthepromisechain.Bycontrast,weeasilyextendedtheiterablesequencewithoutsuchissues,becauseiterablesequencesarelazilyevaluated.

Themoredynamicyouneedyourflowcontrol,themoreiterablesequenceswillshine.

Tip:Checkoutmoreinformationandexamplesofiterablesequencesontheasynquencesite(https://github.com/getify/asynquence/blob/master/README.md#iterable-sequences).

https://github.com/getify/asynquence/blob/master/README.md#iterable-sequences

Itshouldbeobviousfrom(atleast!)Chapter3thatPromisesareaverypowerfultoolinyourasynctoolbox.Butonethingthat'sclearlylackingisintheircapabilitytohandlestreamsofevents,asaPromisecanonlyberesolvedonce.Andfrankly,thisexactsameweaknessistrueofplainasynquencesequences,aswell.

Considerascenariowhereyouwanttofireoffaseriesofstepseverytimeacertaineventisfired.AsinglePromiseorsequencecannotrepresentalloccurrencesofthatevent.So,youhavetocreateawholenewPromisechain(orsequence)foreacheventoccurrence,suchas:

listener.on("foobar",function(data){

//createaneweventhandlingpromisechain


//..

})

.then(..)

.then(..);

});

Thebasefunctionalityweneedispresentinthisapproach,butit'sfarfromadesirablewaytoexpressourintendedlogic.Therearetwoseparatecapabilitiesconflatedinthisparadigm:theeventlistening,andrespondingtotheevent;separationofconcernswouldimploreustoseparateoutthesecapabilities.

ThecarefullyobservantreaderwillseethisproblemassomewhatsymmetricaltotheproblemswedetailedwithcallbacksinChapter2;it'skindofaninversionofcontrolproblem.

Imagineuninvertingthisparadigm,likeso:

varobservable=listener.on("foobar");

//later

observable

.then(..)

.then(..);

//elsewhere

observable

.then(..)

.then(..);

TheobservablevalueisnotexactlyaPromise,butyoucanobserveitmuchlikeyoucanobserveaPromise,soit'scloselyrelated.Infact,itcanbeobservedmanytimes,anditwillsendoutnotificationseverytimeitsevent("foobar")occurs.

Tip:ThispatternI'vejustillustratedisamassivesimplificationoftheconceptsandmotivationsbehindreactiveprogramming(akaRP),whichhasbeenimplemented/expoundeduponbyseveralgreatprojectsandlanguages.AvariationonRPisfunctionalreactiveprogramming(FRP),whichreferstoapplyingfunctionalprogrammingtechniques(immutability,referentialintegrity,etc.)tostreamsofdata."Reactive"referstospreadingthisfunctionalityoutovertimeinresponsetoevents.Theinterestedreadershouldconsiderstudying"ReactiveObservables"inthefantastic"ReactiveExtensions"library("RxJS"forJavaScript)byMicrosoft(http://reactive-extensions.github.io/RxJS/);it'smuchmoresophisticatedandpowerfulthanI'vejustshown.Also,AndreStaltzhasanexcellentwrite-up(https://gist.github.com/staltz/868e7e9bc2a7b8c1f754)thatpragmaticallylaysoutRPinconcreteexamples.

Atthetimeofthiswriting,there'sanearlyES7proposalforanewdatatypecalled"Observable"(https://github.com/jhusain/asyncgenerator#introducing-observable),whichinspiritissimilartowhatwe'velaidouthere,butisdefinitelymoresophisticated.

EventReactive

ES7Observables

http://reactive-extensions.github.io/RxJS/

https://gist.github.com/staltz/868e7e9bc2a7b8c1f754

https://github.com/jhusain/asyncgenerator#introducing-observable

ThenotionofthiskindofObservableisthatthewayyou"subscribe"totheeventsfromastreamistopassinagenerator--actuallytheiteratoristheinterestedparty--whosenext(..)methodwillbecalledforeachevent.

Youcouldimagineitsortoflikethis:

//`someEventStreamìsastreamofevents,likefrom

//mouseclicks,andthelike.

varobserver=newObserver(someEventStream,function*(){

while(varevt=yield){

console.log(evt);

}

});

Thegeneratoryoupassinwillyieldpausethewhileloopwaitingforthenextevent.Theiteratorattachedtothegeneratorinstancewillhaveitsnext(..)calledeachtimesomeEventStreamhasaneweventpublished,andsothateventdatawillresumeyourgenerator/iteratorwiththeevtdata.

Inthesubscriptiontoeventsfunctionalityhere,it'stheiteratorpartthatmatters,notthegenerator.Soconceptuallyyoucouldpassinpracticallyanyiterable,includingASQ.iterable()iterablesequences.

Interestingly,therearealsoproposedadapterstomakeiteasytoconstructObservablesfromcertaintypesofstreams,suchasfromEvent(..)forDOMevents.IfyoulookatasuggestedimplementationoffromEvent(..)intheearlierlinkedES7proposal,itlooksanawfullotliketheASQ.react(..)we'llseeinthenextsection.

Ofcourse,theseareallearlyproposals,sowhatshakesoutmayverywelllook/behavedifferentlythanshownhere.Butit'sexcitingtoseetheearlyalignmentsofconceptsacrossdifferentlibrariesandlanguageproposals!

WiththatcrazybriefsummaryofObservables(andF/RP)asourinspirationandmotivation,Iwillnowillustrateanadaptationofasmallsubsetof"ReactiveObservables,"whichIcall"ReactiveSequences."

First,let'sstartwithhowtocreateanObservable,usinganasynquenceplug-inutilitycalledreact(..):

varobservable=ASQ.react(functionsetup(next){

listener.on("foobar",next);

});

Now,let'sseehowtodefineasequencethat"reacts"--inF/RP,thisistypicallycalled"subscribing"--tothatobservable:

observable

.seq(..)

.then(..)

.val(..);

So,youjustdefinethesequencebychainingofftheObservable.That'seasy,huh?

InF/RP,thestreamofeventstypicallychannelsthroughasetoffunctionaltransforms,likescan(..),map(..),reduce(..),andsoon.Withreactivesequences,eacheventchannelsthroughanewinstanceofthesequence.Let'slookatamoreconcreteexample:

ASQ.react(functionsetup(next){

document.getElementById("mybtn")

.addEventListener("click",next,false);

})

.seq(function(evt){

varbtnID=evt.target.id;

returnrequest(

ReactiveSequences

"http://some.url.1/?id="+btnID

);

})

.val(function(text){

console.log(text);

});

The"reactive"portionofthereactivesequencecomesfromassigningoneormoreeventhandlerstoinvoketheeventtrigger(callingnext(..)).

The"sequence"portionofthereactivesequenceisexactlylikethesequenceswe'vealreadyexplored:eachstepcanbewhateverasynchronoustechniquemakessense,fromcontinuationcallbacktoPromisetogenerator.

Onceyousetupareactivesequence,itwillcontinuetoinitiateinstancesofthesequenceaslongastheeventskeepfiring.Ifyouwanttostopareactivesequence,youcancallstop().

Ifareactivesequenceisstop()'d,youlikelywanttheeventhandler(s)tobeunregisteredaswell;youcanregisterateardownhandlerforthispurpose:

varsq=ASQ.react(functionsetup(next,registerTeardown){

varbtn=document.getElementById("mybtn");

btn.addEventListener("click",next,false);

//willbecalledonce`sq.stop()ìscalled

registerTeardown(function(){

btn.removeEventListener("click",next,false);

});

})

.seq(..)

.then(..)

.val(..);

//later

sq.stop();

Note:Thethisbindingreferenceinsidethesetup(..)handleristhesamesqreactivesequence,soyoucanusethethisreferencetoaddtothereactivesequencedefinition,callmethodslikestop(),andsoon.

Here'sanexamplefromtheNode.jsworld,usingreactivesequencestohandleincomingHTTPrequests:

varserver=http.createServer();

server.listen(8000);

//reactiveobserver

varrequest=ASQ.react(functionsetup(next,registerTeardown){

server.addListener("request",next);

server.addListener("close",this.stop);

registerTeardown(function(){

server.removeListener("request",next);

server.removeListener("close",request.stop);

});

});

//respondtorequests

request

.seq(pullFromDatabase)

.val(function(data,res){

res.end(data);

});

//nodeteardown

process.on("SIGINT",request.stop);

Thenext(..)triggercanalsoadapttonodestreamseasily,usingonStream(..)andunStream(..):

ASQ.react(functionsetup(next){

varfstream=fs.createReadStream("/some/file");

//pipethestream's"data"eventto`next(..)`

next.onStream(fstream);

//listenfortheendofthestream

fstream.on("end",function(){

next.unStream(fstream);

});

})

.seq(..)

.then(..)

.val(..);

Youcanalsousesequencecombinationstocomposemultiplereactivesequencestreams:

varsq1=ASQ.react(..).seq(..).then(..);

varsq2=ASQ.react(..).seq(..).then(..);

varsq3=ASQ.react(..)

.gate(

sq1,

sq2

)

.then(..);

ThemaintakeawayisthatASQ.react(..)isalightweightadaptationofF/RPconcepts,enablingthewiringofaneventstreamtoasequence,hencetheterm"reactivesequence."Reactivesequencesaregenerallycapableenoughforbasicreactiveuses.

Note:Here'sanexampleofusingASQ.react(..)inmanagingUIstate(http://jsbin.com/rozipaki/6/edit?js,output),andanotherexampleofhandlingHTTPrequest/responsestreamswithASQ.react(..)(https://gist.github.com/getify/bba5ec0de9d6047b720e).

HopefullyChapter4helpedyougetprettyfamiliarwithES6generators.Inparticular,wewanttorevisitthe"GeneratorConcurrency"discussion,andpushitevenfurther.

WeimaginedarunAll(..)utilitythatcouldtaketwoormoregeneratorsandrunthemconcurrently,lettingthemcooperativelyyieldcontrolfromonetothenext,withoptionalmessagepassing.

Inadditiontobeingabletorunasinglegeneratortocompletion,theASQ#runner(..)wediscussedinAppendixAisasimilarimplementationoftheconceptsofrunAll(..),whichcanrunmultiplegeneratorsconcurrentlytocompletion.

Solet'sseehowwecanimplementtheconcurrentAjaxscenariofromChapter4:

ASQ(

"http://some.url.2"

)

.runner(

function*(token){

//transfercontrol

yieldtoken;

varurl1=token.messages[0];//"http://some.url.1"

//clearoutmessagestostartfresh

token.messages=[];

varp1=request(url1);

//transfercontrol

yieldtoken;

GeneratorCoroutine

http://jsbin.com/rozipaki/6/edit?js,output

https://gist.github.com/getify/bba5ec0de9d6047b720e

token.messages.push(yieldp1);

},

function*(token){

varurl2=token.messages[0];//"http://some.url.2"

//messagepassandtransfercontrol

token.messages[0]="http://some.url.1";

yieldtoken;

varp2=request(url2);

//transfercontrol

yieldtoken;

token.messages.push(yieldp2);

//passalongresultstonextsequencestep

returntoken.messages;

}

)

.val(function(res){

//`res[0]`comesfrom"http://some.url.1"

//`res[1]`comesfrom"http://some.url.2"

});

ThemaindifferencesbetweenASQ#runner(..)andrunAll(..)areasfollows:

Eachgenerator(coroutine)isprovidedanargumentwecalltoken,whichisthespecialvaluetoyieldwhenyouwanttoexplicitlytransfercontroltothenextcoroutine.token.messagesisanarraythatholdsanymessagespassedinfromtheprevioussequencestep.It'salsoadatastructurethatyoucanusetosharemessagesbetweencoroutines.yieldingaPromise(orsequence)valuedoesnottransfercontrol,butinsteadpausesthecoroutineprocessinguntilthatvalueisready.Thelastreturnedoryieldedvaluefromthecoroutineprocessingrunwillbeforwardpassedtothenextstepinthesequence.

It'salsoeasytolayerhelpersontopofthebaseASQ#runner(..)functionalitytosuitdifferentuses.

Oneexamplethatmaybefamiliartomanyprogrammersisstatemachines.Youcan,withthehelpofasimplecosmeticutility,createaneasy-to-expressstatemachineprocessor.

Let'simaginesuchautility.We'llcallitstate(..),andwillpassittwoarguments:astatevalueandageneratorthathandlesthatstate.state(..)willdothedirtyworkofcreatingandreturninganadaptergeneratortopasstoASQ#runner(..).

Consider:

functionstate(val,handler){

//makeacoroutinehandlerforthisstate

returnfunction*(token){

//statetransitionhandler

functiontransition(to){

token.messages[0]=to;

}

//setinitialstate(ifnonesetyet)

if(token.messages.length<1){

token.messages[0]=val;

}

//keepgoinguntilfinalstate(false)isreached

while(token.messages[0]!==false){

//currentstatematchesthishandler?

if(token.messages[0]===val){

//delegatetostatehandler

yield*handler(transition);

StateMachines

}

//transfercontroltoanotherstatehandler?

if(token.messages[0]!==false){

yieldtoken;

}

}

};

}

Ifyoulookclosely,you'llseethatstate(..)returnsbackageneratorthatacceptsatoken,andthenitsetsupawhileloopthatwillrununtilthestatemachinereachesitsfinalstate(whichwearbitrarilypickasthefalsevalue);that'sexactlythekindofgeneratorwewanttopasstoASQ#runner(..)!

Wealsoarbitrarilyreservethetoken.messages[0]slotastheplacewherethecurrentstateofourstatemachinewillbetracked,whichmeanswecanevenseedtheinitialstateasthevaluepassedinfromthepreviousstepinthesequence.

Howdoweusethestate(..)helperalongwithASQ#runner(..)?

varprevState;

ASQ(

/*optional:initialstatevalue*/

2

)

//runourstatemachine

//transitions:2->3->1->3->false

.runner(

//state`1`handler

state(1,function*stateOne(transition){

console.log("instate1");

prevState=1;

yieldtransition(3);//gotostate`3`

}),

//state`2`handler

state(2,function*stateTwo(transition){


prevState=2;


}),

//state`3`handler

state(3,function*stateThree(transition){


if(prevState===2){

prevState=3;


}

//alldone!

else{

yield"That'sallfolks!";

prevState=3;

yieldtransition(false);//terminalstate

}

})

)

//statemachinecomplete,somoveon

.val(function(msg){

console.log(msg);//That'sallfolks!

});

It'simportanttonotethatthe*stateOne(..),*stateTwo(..),and*stateThree(..)generatorsthemselvesarereinvokedeachtimethatstateisentered,andtheyfinishwhenyoutransition(..)toanothervalue.Whilenotshownhere,ofcoursethesestategeneratorhandlerscanbeasynchronouslypausedbyyieldingPromises/sequences/thunks.

Theunderneathhiddengeneratorsproducedbythestate(..)helperandactuallypassedtoASQ#runner(..)aretheonesthatcontinuetorunconcurrentlyforthelengthofthestatemachine,andeachofthemhandlescooperativelyyielding

controltothenext,andsoon.

Note:Seethis"pingpong"example(http://jsbin.com/qutabu/1/edit?js,output)formoreillustrationofusingcooperativeconcurrencywithgeneratorsdrivenbyASQ#runner(..).

"CommunicatingSequentialProcesses"(CSP)wasfirstdescribedbyC.A.R.Hoareina1978academicpaper(http://dl.acm.org/citation.cfm?doid=359576.359585),andlaterina1985book(http://www.usingcsp.com/)ofthesamename.CSPdescribesaformalmethodforconcurrent"processes"tointeract(aka"communicate")duringprocessing.

Youmayrecallthatweexaminedconcurrent"processes"backinChapter1,soourexplorationofCSPherewillbuilduponthatunderstanding.

Likemostgreatconceptsincomputerscience,CSPisheavilysteepedinacademicformalism,expressedasaprocessalgebra.However,Isuspectsymbolicalgebratheoremswon'tmakemuchpracticaldifferencetothereader,sowewillwanttofindsomeotherwayofwrappingourbrainsaroundCSP.

IwillleavemuchoftheformaldescriptionandproofofCSPtoHoare'swriting,andtomanyotherfantasticwritingssince.Instead,wewilltrytojustbrieflyexplaintheideaofCSPinasun-academicandhopefullyintuitivelyunderstandableawayaspossible.

ThecoreprincipleinCSPisthatallcommunication/interactionbetweenotherwiseindependentprocessesmustbethroughformalmessagepassing.Perhapscountertoyourexpectations,CSPmessagepassingisdescribedasasynchronousaction,wherethesenderprocessandthereceiverprocesshavetomutuallybereadyforthemessagetobepassed.

HowcouldsuchsynchronousmessagingpossiblyberelatedtoasynchronousprogramminginJavaScript?

TheconcretenessofrelationshipcomesfromthenatureofhowES6generatorsareusedtoproducesynchronous-lookingactionsthatunderthecoverscanindeedeitherbesynchronousor(morelikely)asynchronous.

Inotherwords,twoormoreconcurrentlyrunninggeneratorscanappeartosynchronouslymessageeachotherwhilepreservingthefundamentalasynchronyofthesystembecauseeachgenerator'scodeispaused(aka"blocked")waitingonresumptionofanasynchronousaction.

Howdoesthiswork?

Imagineagenerator(aka"process")called"A"thatwantstosendamessagetogenerator"B."First,"A"yieldsthemessage(thuspausing"A")tobesentto"B."When"B"isreadyandtakesthemessage,"A"isthenresumed(unblocked).

Symmetrically,imagineagenerator"A"thatwantsamessagefrom"B.""A"yieldsitsrequest(thuspausing"A")forthemessagefrom"B,"andonce"B"sendsamessage,"A"takesthemessageandisresumed.

OneofthemorepopularexpressionsofthisCSPmessagepassingtheorycomesfromClojureScript'score.asynclibrary,andalsofromthegolanguage.ThesetakesonCSPembodythedescribedcommunicationsemanticsinaconduitthatisopenedbetweenprocessescalleda"channel."

Note:Thetermchannelisusedinpartbecausetherearemodesinwhichmorethanonevaluecanbesentatonceintothe"buffer"ofthechannel;thisissimilartowhatyoumaythinkofasastream.Wewon'tgointodepthaboutithere,butitcanbeaverypowerfultechniqueformanagingstreamsofdata.

InthesimplestnotionofCSP,achannelthatwecreatebetween"A"and"B"wouldhaveamethodcalledtake(..)forblockingtoreceiveavalue,andamethodcalledput(..)forblockingtosendavalue.

Thismightlooklike:

CommunicatingSequentialProcesses(CSP)

MessagePassing

http://jsbin.com/qutabu/1/edit?js,output

http://dl.acm.org/citation.cfm?doid=359576.359585

http://www.usingcsp.com/

varch=channel();

function*foo(){

varmsg=yieldtake(ch);

console.log(msg);

}

function*bar(){

yieldput(ch,"HelloWorld");

console.log("messagesent");

}

run(foo);

run(bar);

//HelloWorld

//"messagesent"

Comparethisstructured,synchronous(-looking)messagepassinginteractiontotheinformalandunstructuredmessagesharingthatASQ#runner(..)providesthroughthetoken.messagesarrayandcooperativeyielding.Inessence,yieldput(..)isasingleoperationthatbothsendsthevalueandpausesexecutiontotransfercontrol,whereasinearlierexampleswedidthoseasseparatesteps.

Moreover,CSPstressesthatyoudon'treallyexplicitly"transfercontrol,"butratheryoudesignyourconcurrentroutinestoblockexpectingeitheravaluereceivedfromthechannel,ortoblockexpectingtotrytosendamessageonthechannel.Theblockingaroundreceivingorsendingmessagesishowyoucoordinatesequencingofbehaviorbetweenthecoroutines.

Note:Fairwarning:thispatternisverypowerfulbutit'salsoalittlemindtwistingtogetusedtoatfirst.Youwillwanttopracticethisabittogetusedtothisnewwayofthinkingaboutcoordinatingyourconcurrency.

ThereareseveralgreatlibrariesthathaveimplementedthisflavorofCSPinJavaScript,mostnotably"js-csp"(https://github.com/ubolonton/js-csp),whichJamesLong(http://twitter.com/jlongster)forked(https://github.com/jlongster/js-csp)andhaswrittenextensivelyabout(http://jlongster.com/Taming-the-Asynchronous-Beast-with-CSP-in-JavaScript).Also,itcannotbestressedenoughhowamazingthemanywritingsofDavidNolen(http://twitter.com/swannodette)areonthetopicofadaptingClojureScript'sgo-stylecore.asyncCSPintoJSgenerators(http://swannodette.github.io/2013/08/24/es6-generators-and-csp/).

Becausewe'vebeendiscussingasyncpatternshereinthecontextofmyasynquencelibrary,youmightbeinterestedtoseethatwecanfairlyeasilyaddanemulationlayerontopofASQ#runner(..)generatorhandlingasanearlyperfectportingoftheCSPAPIandbehavior.Thisemulationlayershipsasanoptionalpartofthe"asynquence-contrib"packagealongsideasynquence.

Verysimilartothestate(..)helperfromearlier,ASQ.csp.go(..)takesagenerator--ingo/core.asyncterms,it'sknownasagoroutine--andadaptsittousewithASQ#runner(..)byreturninganewgenerator.

Insteadofbeingpassedatoken,yourgoroutinereceivesaninitiallycreatedchannel(chbelow)thatallgoroutinesinthisrunwillshare.Youcancreatemorechannels(whichisoftenquitehelpful!)withASQ.csp.chan(..).

InCSP,wemodelallasynchronyintermsofblockingonchannelmessages,ratherthanblockingwaitingforaPromise/sequence/thunktocomplete.

So,insteadofyieldingthePromisereturnedfromrequest(..),request(..)shouldreturnachannelthatyoutake(..)avaluefrom.Inotherwords,asingle-valuechannelisroughlyequivalentinthiscontext/usagetoaPromise/sequence.

Let'sfirstmakeachannel-awareversionofrequest(..):

functionrequest(url){

varch=ASQ.csp.channel();

ajax(url).then(function(content){

asynquenceCSPemulation

https://github.com/ubolonton/js-csp

http://twitter.com/jlongster

https://github.com/jlongster/js-csp

http://jlongster.com/Taming-the-Asynchronous-Beast-with-CSP-in-JavaScript

http://twitter.com/swannodette

http://swannodette.github.io/2013/08/24/es6-generators-and-csp/

//`putAsync(..)ìsaversionof`put(..)`that

//canbeusedoutsideofagenerator.Itreturns

//apromisefortheoperation'scompletion.We

//don'tusethatpromisehere,butwecouldif

//weneededtobenotifiedwhenthevaluehad

//been`take(..)`n.

ASQ.csp.putAsync(ch,content);

});

returnch;

}

FromChapter3,"promisory"isaPromise-producingutility,"thunkory"fromChapter4isathunk-producingutility,andfinally,inAppendixAweinvented"sequory"forasequence-producingutility.

Naturally,weneedtocoinasymmetrictermhereforachannel-producingutility.Solet'sunsurprisinglycallita"chanory"("channel"+"factory").Asanexerciseforthereader,tryyourhandatdefiningachannelify(..)utilitysimilartoPromise.wrap(..)/promisify(..)(Chapter3),thunkify(..)(Chapter4),andASQ.wrap(..)(AppendixA).

NowconsidertheconcurrentAjaxexampleusingasyquence-flavoredCSP:

ASQ()

.runner(

ASQ.csp.go(function*(ch){

yieldASQ.csp.put(ch,"http://some.url.2");

varurl1=yieldASQ.csp.take(ch);

//"http://some.url.1"

varres1=yieldASQ.csp.take(request(url1));

yieldASQ.csp.put(ch,res1);

}),

ASQ.csp.go(function*(ch){

varurl2=yieldASQ.csp.take(ch);

//"http://some.url.2"

yieldASQ.csp.put(ch,"http://some.url.1");

varres2=yieldASQ.csp.take(request(url2));

varres1=yieldASQ.csp.take(ch);

//passalongresultstonextsequencestep

ch.buffer_size=2;

ASQ.csp.put(ch,res1);

ASQ.csp.put(ch,res2);

})

)

.val(function(res1,res2){

//`res1`comesfrom"http://some.url.1"

//`res2`comesfrom"http://some.url.2"

});

ThemessagepassingthattradestheURLstringsbetweenthetwogoroutinesisprettystraightforward.ThefirstgoroutinemakesanAjaxrequesttothefirstURL,andthatresponseisputontothechchannel.ThesecondgoroutinemakesanAjaxrequesttothesecondURL,thengetsthefirstresponseres1offthechchannel.Atthatpoint,bothresponsesres1andres2arecompletedandready.

Ifthereareanyremainingvaluesinthechchannelattheendofthegoroutinerun,theywillbepassedalongtothenextstepinthesequence.So,topassoutmessage(s)fromthefinalgoroutine,put(..)themintoch.Asshown,toavoidtheblockingofthosefinalput(..)s,weswitchchintobufferingmodebysettingitsbuffer_sizeto2(default:0).

Note:Seemanymoreexamplesofusingasynquence-flavoredCSPhere(https://gist.github.com/getify/e0d04f1f5aa24b1947ae).

Promisesandgeneratorsprovidethefoundationalbuildingblocksuponwhichwecanbuildmuchmoresophisticatedand

Review

https://gist.github.com/getify/e0d04f1f5aa24b1947ae

capableasynchrony.

asynquencehasutilitiesforimplementingiterablesequences,reactivesequences(aka"Observables"),concurrentcoroutines,andevenCSPgoroutines.

Thosepatterns,combinedwiththecontinuation-callbackandPromisecapabilities,givesasynquenceapowerfulmixofdifferentasynchronousfunctionalities,allintegratedinonecleanasyncflowcontrolabstraction:thesequence.

Ihavemanypeopletothankformakingthisbooktitleandtheoverallserieshappen.

First,ImustthankmywifeChristenSimpson,andmytwokidsEthanandEmily,forputtingupwithDadalwayspeckingawayatthecomputer.Evenwhennotwritingbooks,myobsessionwithJavaScriptgluesmyeyestothescreenfarmorethanitshould.ThattimeIborrowfrommyfamilyisthereasonthesebookscansodeeplyandcompletelyexplainJavaScripttoyou,thereader.Iowemyfamilyeverything.

I'dliketothankmyeditorsatO'Reilly,namelySimonSt.LaurentandBrianMacDonald,aswellastherestoftheeditorialandmarketingstaff.Theyarefantastictoworkwith,andhavebeenespeciallyaccommodatingduringthisexperimentinto"opensource"bookwriting,editing,andproduction.

Thankyoutothemanyfolkswhohaveparticipatedinmakingthisbookseriesbetterbyprovidingeditorialsuggestionsandcorrections,includingShelleyPowers,TimFerro,EvanBorden,ForrestL.Norvell,JenniferDavis,JesseHarlin,KrisKowal,RickWaldron,JordanHarband,BenjaminGruenbaum,VyacheslavEgorov,DavidNolen,andmanyothers.AbigthankyoutoJakeArchibaldforwritingtheForewordforthistitle.

Thankyoutothecountlessfolksinthecommunity,includingmembersoftheTC39committee,whohavesharedsomuchknowledgewiththerestofus,andespeciallytoleratedmyincessantquestionsandexplorationswithpatienceanddetail.John-DavidDalton,Juriy"kangax"Zaytsev,MathiasBynens,AxelRauschmayer,NicholasZakas,AngusCroll,ReginaldBraithwaite,DaveHerman,BrendanEich,AllenWirfs-Brock,BradleyMeck,DomenicDenicola,DavidWalsh,TimDisney,PetervanderZee,AndreaGiammarchi,KitCambridge,EricElliott,andsomanyothers,Ican'tevenscratchthesurface.

TheYouDon'tKnowJSbookserieswasbornonKickstarter,soIalsowishtothankallmy(nearly)500generousbackers,withoutwhomthisbookseriescouldnothavehappened:

JanSzpila,nokiko,MuraliKrishnamoorthy,RyanJoy,CraigPatchett,pdqtrader,DaleFukami,rayhatfield,R0drigoPerez[Mx],DanPetitt,JackFranklin,AndrewBerry,BrianGrinstead,RobSutherland,SergiMeseguer,PhillipGourley,MarkWatson,JeffCarouth,AlfredoSumaran,MartinSachse,MarcioBarrios,Dan,AimelyneM,MattSullivan,DelnattePierre-Antoine,JakeSmith,EugenTudorancea,Iris,DavidTrinh,simonstl,RayDaly,UrosGruber,JustinMyers,ShaiZonis,Mom&Dad,DevinClark,DennisPalmer,BrianPanahiJohnson,JoshMarshall,Marshall,DennisKerr,MattSteele,ErikSlagter,Sacah,JustinRainbow,ChristianNilsson,Delapouite,D.Pereira,NicolasHoizey,GeorgeV.Reilly,DanReeves,BrunoLaturner,ChadJennings,ShaneKing,JeremiahLeeCohick,od3n,StanYamane,MarkoVucinic,JimB,StephenCollins,ÆgirÞorsteinsson,EricPederson,Owain,NathanSmith,Jeanetteurphy,AlexandreELISÉ,ChrisPeterson,RikWatson,LukeMatthews,JustinLowery,MortenNielsen,VernonKesner,ChetanShenoy,PaulTregoing,MarcGrabanski,DionAlmaer,AndrewSullivan,KeithElsass,TomBurke,BrianAshenfelter,DavidStuart,KarlSwedberg,Graeme,BrandonHays,JohnChristopher,Gior,manojreddy,ChadSmith,JaredHarbour,MinoruTODA,ChrisWigley,DanielMee,Mike,Handyface,AlexJahraus,CarlFurrow,RobFoulkrod,MaxShishkin,LeighPennyJr.,RobertFerguson,MikevanHoenselaar,HasseSchougaard,rajanvenkataguru,JeffAdams,TraeRobbins,RolfLangenhuijzen,JorgeAntunes,AlexKoloskov,HughGreenish,TimJones,JoseOchoa,MichaelBrennan-White,NagaHarishMuvva,BarkócziDávid,KittHodsden,PaulMcGraw,SaschaGoldhofer,AndrewMetcalf,MarkusKrogh,MichaelMathews,MattJared,Juanfran,GeorgieKirschner,KennyLee,TedZhang,AmitPahwa,InbalSinai,DanRaine,SchabseLaks,MichaelTervoort,AlexandreAbreu,AlanJosephWilliams,NicolasD,CindyWong,RegBraithwaite,LocalPCGuy,JonFriskics,ChrisMerriman,JohnPena,JacobKatz,SueLockwood,MagnusJohansson,JeremyCrapsey,GrzegorzPawłowski,niconuzzaci,ChristineWilks,HansBergren,charlesmontgomery,Arielבר-לבבFogel,IvanKolev,DanielCampos,HughWood,ChristianBradford,FrédéricHarper,IonuţDanPopa,JeffTrimble,RupertWood,TreyCarrico,PanchoLopez,Joëlkuijten,TomAMarra,JeffJewiss,JacobRios,PaoloDiStefano,SoledadPenades,ChrisGerber,AndreyDolganov,WilMooreIII,ThomasMartineau,Kareem,BenThouret,UdiNir,MorganLaupies,jorycarson-burson,NathanLSmith,EricDamonWalters,DerryLozano-Hoyland,GeoffreyWiseman,mkeehner,KatieK,ScottMacFarlane,BrianLaShomb,AdrienMas,christopherross,IanLittman,DanAtkinson,ElliotJobe,NickDozier,PeterWooley,John


AppendixC:Acknowledgments

Hoover,dan,MartinA.Jackson,HéctorFernandoHurtado,andyennamorato,PaulSeltmann,MelissaGore,Dave

Pollard,JackSmith,PhilipDaSilva,GuyIsraeli,@megalithic,DamianCrawford,FelixGliesche,AprilCarterGrant,Heidi,jimtierney,AndreaGiammarchi,NicoVignola,DonJones,ChrisHartjes,AlexHowes,johngibbon,DavidJ.Groom,BBox,Yu'Dilys'Sun,NateSteiner,BrandonSatrom,BrianWyant,WesleyHales,IanPouncey,TimothyKevinOxley,GeorgeTerezakis,sanjayraj,JordanHarband,MarkoMcLion,WolfgangKaufmann,PascalPeuckert,DaveNugent,MarkusLiebelt,WellingGuzman,NickCooley,DanielMesquita,RobertSyvarth,ChrisCoyier,RémyBach,AdamDougal,AlistairDuggin,DavidLoidolt,EdRicher,BrianChenault,GoldFireStudios,CarlesAndrés,CarlosCabo,YuyaSaito,robertoricardo,BarnettKlane,MikeMoore,KevinMarx,JustinLove,JoeTaylor,PaulDijou,MichaelKohler,RobCassie,MikeTierney,CodyLeroyLindley,tofuji,ShimonSchwartz,Raymond,LucDeBrouwer,DavidHayes,RhysBrett-Bowen,Dmitry,AzizKhoury,Dean,ScottTolinski-LevelUp,ClementBoirie,DjordjeLukic,AntonKotenko,RafaelCorral,PhilipHurwitz,JonathanPidgeon,JasonCampbell,JosephC.,SwiftOne,JanHohner,DerickBailey,getify,DanielCousineau,ChrisCharlton,EricTurner,DavidTurner,JoëlGaleran,DharmaVagabond,adam,DirkvanBergen,dave♥♫★furf,VedranZakanj,RyanMcAllen,NataliePatriceTucker,EricJ.Bivona,AdamSpooner,AaronCavano,KellyPacker,EricJ,MartinDrenovac,Emilis,MichaelPelikan,ScottF.Walter,JoshFreeman,BrandonHudgeons,vijaychennupati,BillGlennon,RobinR.,TroyForster,otaku_coder,Brad,Scott,FrederickOstrander,AdamBrill,SebFlippence,MichaelAnderson,Jacob,AdamRandlett,Standard,JoshuaClanton,SebastianKouba,ChrisDeck,SwordFire,HannesPapenberg,RichardWoeber,hnzz,RobCrowther,JedidiahBroadbent,SergeyChernyshev,Jay-ArJamon,BenCombee,lucianobonachela,MarkTomlinson,KitCambridge,MichaelMelgares,JacobAdams,AdrianBruinhout,BevWieber,ScottPuleo,ThomasHerzog,AprilLeone,DanielMizieliński,KeesvanGinkel,JonAbrams,ErwinHeiser,AviLaviad,Davidnewell,Jean-FrancoisTurcot,NikoRoberts,ErikDana,CharlesNeill,AaronHolmes,GrzegorzZiółkowski,NathanYoungman,Timothy,JacobMather,MichaelAllan,MohitSeth,RyanEwing,BenjaminVanTreese,MarceloSantos,DenisWolf,PhilKeys,ChrisYung,TimoTijhof,MartinLekvall,Agendine,GregWhitworth,HelenHumphrey,DougalCampbell,JohannesHarth,BrunoGirin,BrianHough,DarrenNewton,CraigMcPheat,OlivierTille,DennisRoethig,MathiasBynens,BrendanStromberger,sundeep,JohnMeyer,RonMale,JohnFCrostonIII,gigante,CarlBergenhem,B.J.May,RebekahTyler,TedFoxberry,JordanReese,TerrySuitor,afeliz,TomKiefer,DarraghDuffy,KevinVanderbeken,AndyPearson,SimonMacDonald,AbidDin,ChrisJoel,TomasTheunissen,DavidDick,PaulGrock,BrandonWood,JohnWeis,dgrebb,NickJenkins,ChuckLane,JohnnyMegahan,marzsman,TatuTamminen,GeoffreyKnauth,AlexanderTarmolov,JeremyTymes,ChadAuld,SeanParmelee,RobStaenke,DanBender,Yannickderwa,JoshuaJones,GeertPlaisier,TomLeZotte,ChristenSimpson,StefanBruvik,JustinFalcone,CarlosSantana,MichaelWeiss,PabloVilloslada,PeterdeHaan,DimitrisIliopoulos,seyDoggy,AdamJordens,NoahKantrowitz,AmolM,MatthewWinnard,DirkGinader,PhinamBui,DavidRapson,AndrewBaxter,FlorianBougel,MichaelGeorge,AlbanEscalier,DanielSellers,SashaRudan,JohnGreen,RobertKowalski,DavidI.Teixeira(@ditma,CharlesCarpenter,JustinYost,SamS,DenisCiccale,KevinSheurs,YannickCroissant,PauFracés,StephenMcGowan,ShawnSearcy,ChrisRuppel,KevinLamping,JessicaCampbell,ChristopherSchmitt,Sablons,JonathanReisdorf,BunniGek,TeddyHuff,MichaelMullany,MichaelFürstenberg,CarlHenderson,RickYoesting,ScottNichols,HernánCiudad,AndrewMaier,MikeStapp,JesseShawl,SérgioLopes,jsulak,ShawnPrice,JoelClermont,ChrisRidmann,SeanTimm,JasonFinch,AidenMontgomery,ElijahManor,DerekGathright,JesseHarlin,DillonCurry,CourtneyMyers,DiegoCadenas,ArnedeBree,JoãoPauloDubas,JamesTaylor,PhilippKraeutli,MihaiPăun,SamGharegozlou,joshjs,MattMurchison,EricWindham,TimoBehrmann,AndrewHall,joshuaprice,ThéophileVillard

Thisbookseriesisbeingproducedinanopensourcefashion,includingeditingandproduction.WeoweGitHubadebtofgratitudeformakingthatsortofthingpossibleforthecommunity!

ThankyouagaintoallthecountlessfolksIdidn'tnamebutwhoInonethelessowethanks.Maythisbookseriesbe"owned"byallofusandservetocontributetoincreasingawarenessandunderstandingoftheJavaScriptlanguage,tothebenefitofallcurrentandfuturecommunitycontributors.

Documents

You Don't Know JS: Async & Performance