Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
1.10
1.11
1.12
1.13
TableofContentsForeword
Introduction
1.EctoisnotyourORM
2.Schemalessqueries
3.Schemalesschangesets
4.Dynamicqueries
5.Multitenancywithqueryprefixes
6.Aggregatesandsubqueries
7.Improvedassociationsandfactories
8.Manytomanyandcasting
9.Manytomanyandupserts
10.ComposabletransactionswithEcto.Multi
11.ConcurrenttestswiththeSQLSandbox
2
ForewordInJanuary2017,wewillcelebrate5yearssincewedecidedtoinvestinElixir.Backin2012,JoséValim,ourco-founderandpartner,presentedustheideaofaprogramminglanguagethatwouldbeexpressive,embraceproductivityinitstooling,andleveragetheErlangVMtonotonlytackletheproblemsinwritingconcurrentsoftwarebutalsotobuildfault-tolerantanddistributedsystems.
Elixircontinued,insomesense,tobeariskyprojectformonths.Wewerecertainlyexcitedaboutspreadingfunctional,concurrentanddistributedprogrammingconceptstomoreandmoredevelopers,hopingitwouldleadtoapositiveimpactonthesoftwaredevelopmentindustry,butdevelopingalanguageisalong-termeffortthatmayneverbecomeconcrete.
Duringthesummerof2013,othercompaniesanddevelopersstartedtoshowinterestonElixir.Weheardaboutcompaniesusingitinproduction,moredevelopersbegantocontributeandcreatetheirownprojects,differentpublisherswerewritingbooksonthelanguage,andsoon.SucheventsgaveustheconfidencetoinvestmoreinElixirandbringthelanguagetoversion1.0.
OnceElixir1.0waslaunchedinSeptember2014,weturnedourfocustothewebplatform.WetidiedupPlug,thebuildingblockforwritingwebapplicationsinElixir.WealsofocusedintensivelyonEcto,bringingittoversion1.0togetherwiththeEctoteam,andthenworkedalongsideChrisMcCordandteamtogetthefirstmajorPhoenixreleaseout.Duringthistimewealsostartedothercommunitycentricinitiatives,suchasElixirRadar,andbeganourfirstcommercialElixirprojects.
Today,boththecommunityandouropensourceprojectsareshowingsteadyandhealthygrowth.Elixirisastablelanguagewithcontinuousimprovementslandedinminorversions.PlugcontinuestobeasolidfoundationforframeworkssuchasPhoenix.Ecto,however,requiredmorethanasmallnudgeintherightdirection.Werealizedthatweneededtoletgoofold,harmfulhabitsandmakeEctolessofanabstractionlayerandmoreofatoolyoucontrolandapplytodifferentproblems.
ThisbookisthefinaleffortbehindEcto2.0.ItshowcasesthenewdirectionwehaveplannedforEcto,thestructuralimprovementsmadebytheEctoteamandmanyofitsnewfeatures.Wehopeyouwillenjoyit.Afterall,itistimetoletgoofpasthabits.
Havefun,
-ThePlataformatecteam
Foreword
3
Foreword
4
CONTACTUS
Foreword
5
IntroductionAuthornote:WelcometotheearlycutofourEcto2.0book.Thisisourlastbetaeditionwithallbookchaptersreadyforreview.Ifyouhavesuggestionsorstumbleduponerrors,[email protected].
Ecto2.0isasubstantialdeparturefromearlierversions.Insteadofthinkingaboutmodels,Ecto2.0aimstoprovidedevelopersawiderangeofdata-centrictools.Therefore,inordertouseEcto2.0effectively,wemustlearnhowtowieldthosetoolsproperly.That'sthegoalofthisbook.
Thisbook,however,isnotanintroductiontoEcto.IfyouhaveneverusedEctobefore,werecommendyoutogetstartedwithEcto'sdocumentationandlearnmoreaboutrepositories,queries,schemasandchangesets.Weassumethereaderisfamiliarwiththesebuildingblocksandhowtheyrelatetoeachother.
ThefirstchaptersofthebookwillcoverthebiggestconceptualchangesinEcto2.0.Wewilltalkaboutrelationalmappersin"EctoisnotyourORM"andthenexploreSchemalessQueriesandtherelationshipbetweenSchemasandChangesets.
Afterwewilltakeadeeperlookintoqueries,discussinghowEcto2.0makesiteasiertobuilddynamicqueries,howtotargetdifferentdatabasesviaqueryprefixes,aswellasthenewaggregateandsubqueryfeatures.
Thenwewillgobacktoschemasanddiscusstheschema-relatedenhancementsthatarepartofEcto2.0,suchastheimprovedassociationsupport,many_to_manyassociationsandEcto's2.1upsertsupport.
Finally,wewillexplorebrandnewtopics,likethenewEctoSQLSandbox,thatallowsdeveloperstoruntestsagainstthedatabaseconcurrently,aswellasEcto.Multi,whichmakesworkingwithtransactionssimplerthanever.
Introduction
6
EctoisnotyourORMDependingonyourperspective,thisisaratherboldorobviousstatementtostartthisbook.Afterall,Elixirisnotanobject-orientedlanguage,soEctocan'tbeanObject-relationalMapper.However,thisstatementisslightlymorenuancedthanitlooksandthereareimportantlessonstobelearnedhere.
OisforObjectsAtitscore,objectscouplestateandbehaviourtogether.Inthesameuserobject,youcanhavedata,liketheuser.name,aswellasbehaviour,likeconfirmingaparticularuseraccountviauser.confirm().Whilesomelanguagesenforcedifferentsyntaxesbetweenaccessingdata(user.namewithoutparentheses)andbehaviour(user.confirm()withparentheses),otherlanguagesfollowtheUniformAccessPrincipleinwhichanobjectshouldnotmakeadistinctionbetweenthetwosyntaxes.EiffelandRubyarelanguagesthatfollowsuchprinciple.
Elixirfailsthe"couplingofstateandbehaviour"test.InElixir,weworkwithdifferentdatastructuressuchastuples,lists,mapsandothers.Behaviourcannotbeattachedtodatastructures.Behaviourisalwaysaddedtomodulesviafunctions.
Whenthereisaneedtoworkwithstructureddata,Elixirprovidesstructs.Structsdefineasetoffields.Astructwillbereferencedbythenameofthemodulewhereitisdefined:
defmoduleUserdo
defstruct[:name,:email]
end
user=%User{name:"JohnDoe",email:"[email protected]"}
Onceauserstructiscreated,wecanaccessitsemailviauser.email.However,structsareonlydata.Itisimpossibletoinvokeuser.confirm()onaparticularstructinawayitwillexecutecoderelatedtoe-mailconfirmation.
Althoughwecannotattachbehaviourtostructs,itispossibletoaddfunctionstothesamemodulethatdefinesthestruct:
1.EctoisnotyourORM
7
defmoduleUserdo
defstruct[:name,:email]
defconfirm(user)do
#Confirmtheuseremail
end
end
Evenwiththedefinitionabove,itisimpossibleinElixirtoconfirmagivenuserbycallinguser.confirm().Instead,theUserprefixisrequiredandtheuserstructmustbeexplicitlygivenasargument,asinUser.confirm(user).Attheendoftheday,thereisnostructuralcouplingbetweentheuserstructandthefunctionsintheUsermodule.HenceElixirdoesnothavemethods,ithasfunctions.
Withouthavingobjects,Ectocertainlycan'tbeanORM.However,ifweletgooftheletter"O"forasecond,canEctostillbearelationalmapper?
RelationalmappersAnObject-RelationalMapperisatechniqueforconvertingdatabetweenincompatibletypesystems,commonlydatabases,toobjects,andback.
Similarly,EctoprovidesschemasthatmapsanydatasourceintoanElixirstruct.Whenappliedtoyourdatabase,Ectoschemasarerelationalmappers.Therefore,whileEctoisnotarelationalmapper,itcontainsarelationalmapperaspartofthemanydifferenttoolsitoffers.
Forexample,theschemabelowtiesthefieldsname,email,inserted_atandupdated_attofieldssimilarlynamedintheuserstable:
defmoduleMyApp.Userdo
useEcto.Schema
schema"users"do
field:name
field:email
timestamps()
end
end
Theappealbehindschemasisthatyoudefinetheshapeofthedataonceandyoucanusethisshapetoretrievedatafromthedatabaseaswellascoordinatechangeshappeningonthedata:
1.EctoisnotyourORM
8
MyApp.User
|>MyApp.Repo.get!(13)
|>Ecto.Changeset.cast([name:"newname"],[:name,:email])
|>MyApp.Repo.update!
Byrelyingontheschemainformation,Ectoknowshowtoreadandwritedatawithoutextrainputfromthedeveloper.Insmallapplications,thiscouplingbetweenthedataanditsrepresentationisdesired.However,whenusedwrongly,itleadstocomplexcodebasesandsubparsolutions.
ItisimportanttounderstandtherelationshipbetweenEctoandrelationalmappersbecausesaying"EctoisnotyourORM"doesnotautomaticallysaveEctoschemasfromsomeofthedownsidesmanydevelopersassociateORMswith.
HerearesomeexamplesofissuesoftenassociatedwithORMsthatEctodevelopersmayrunintowhenusingschemas:
ProjectsusingEctomayend-upwith"GodSchemas",commonlyreferredas"GodModels","FatModels"or"CanonicalModels"insomelanguagesandframeworks.Suchschemascouldcontainhundredsoffields,oftenreflectingbaddecisionsdoneatthedatalayer.Insteadofprovidingonesingleschemawithfieldsthatspanmultipleconcerns,itisbettertobreaktheschemaacrossmultiplecontexts.Forexample,insteadofasingleMyApp.Userschemawithdozensoffields,considerbreakingitintoMyApp.Accounts.User,MyApp.Purchases.Userandsoon.Eachstructwithfieldsexclusivetoitsenclosingcontext
Developersmayexcessivelyrelyonschemaswhensometimesthebestwaytoretrievedatafromthedatabaseisintoregulardatastructures(likemapsandtuples)andnotpre-definedshapesofdatalikestructs.Forexample,whendoingsearches,generatingreportsandothers,thereisnoreasontorelyorreturnschemasfromsuchqueries,asitoftenreliesondatacomingfrommultipletableswithdifferentrequirements
Developersmaytrytousethesameschemaforoperationsthatmaybequitedifferentstructurally.Manyapplicationswouldboltfeaturessuchasregistration,accountlogin,intoasingleUserschema,whilehandlingeachoperationindividually,possiblyusingdifferentschemas,wouldleadtosimplerandclearersolutions
Inthenexttwochapters,wewanttobreakthose"badpractices"apartbyexploringhowtouseEctowithnoormultipleschemaspercontext.Bylearninghowtoinsert,delete,manipulateandvalidatedatawithandwithoutschemas,wehopedeveloperswillfeelcomfortablewithbuildingcomplexapplicationswithoutrelyingonone-size-fits-allschemas.
1.EctoisnotyourORM
9
1.EctoisnotyourORM
10
SchemalessqueriesMostqueriesinEctoarewrittenusingschemas.Forexample,toretrieveallpostsinadatabase,onemaywrite:
MyApp.Repo.all(Post)
Intheconstructabove,Ectoknowsallfieldsandtheirtypesintheschema,rewritingthequeryaboveto:
MyApp.Repo.all(frompinPost,select:%Post{title:p.title,body:p.body,...})
Interestingly,backinEcto'searlydays,therewasnosuchthingasschemas.Queriescouldonlybewrittendirectlyagainstadatabasetablebypassingthetablenameasastring:
MyApp.Repo.all(frompin"posts",select:{p.title,p.body})
Whenwritingschemalessqueries,theselectexpressionmustbeexplicitlywrittenwithallthedesiredfields.
WhiletheabovesyntaxmadeitintoEcto1.0,bythetimeEcto1.0waslaunched,mostofthedevelopmentfocusinEctohadchangedtowardsschemas.Thismeanswhiledeveloperswereabletoreaddatawithoutschemas,theywereoftentooverbose.Notonlythat,ifyouwantedtoinsertentriestoyourdatabasewithoutschemas,youwereoutofluck.
Ecto2.0levelsupthegamebyaddingmanyimprovementstoschemalessqueries,notonlybyimprovingthesyntaxforreadingandupdatingdata,butalsobyallowingalldatabaseoperationstobeexpressedwithoutaschema.
insert_allOneofthefunctionsaddedtoEcto2.0isEcto.Repo.insert_all/3.Withinsert_all,developerscaninsertmultipleentriesatonceintoarepository:
MyApp.Repo.insert_all(Post,[[title:"hello",body:"world"],
[title:"another",body:"post"]])
2.Schemalessqueries
11
Althoughinsert_allisjustaregularElixirfunction,itplaysanimportantroleinEcto2.0asitallowsdeveloperstoread,create,updateanddeleteentrieswithoutaschema.insert_allwasthelastpieceofthepuzzle.Let'sseesomeexamples.
Ifyouarewritingareportingview,itmaybecounter-productivetothinkhowyourexistingapplicationschemasrelatetothereportbeinggenerated.Itisoftensimplertowriteaquerythatreturnsonlythedatayouneed,withouttryingtofitthedataintoexistingschemas:
importEcto.Query
defrunning_activities(start_at,end_at)
MyApp.Repo.all(
fromuin"users",
join:ain"activities",
on:a.user_id==u.id,
where:a.start_at>type(^start_at,Ecto.DateTime)and
a.end_at<type(^end_at,Ecto.DateTime),
group_by:a.user_id,
select:%{user_id:a.user_id,interval:a.start_at-a.end_at,count:count(u.id
)}
)
end
Thefunctionabovedoesnotrelyonschemas.Itreturnsonlythedatathatmattersforbuildingthereport.Noticehowweusethetype/2functiontospecifywhatistheexpectedtypeoftheargumentweareinterpolating,benefitingfromthesametypecastingguaranteesaschemawouldgive.
Inserts,updatesanddeletescanalsobedonewithoutschemasviainsert_all,update_allanddelete_allrespectively:
#InsertdataintopostsandreturnitsID
[%{id:id}]=
MyApp.Repo.insert_all"posts",[[title:"hello"]],returning:[:id]
#UsetheIDtotriggerupdates
post=frompin"posts",where:[id:^id]
{1,_}=MyApp.Repo.update_allpost,set:[title:"newtitle"]
#Aswellasfordeletes
{1,_}=MyApp.Repo.delete_allpost
ItisnothardtoseehowtheseoperationsdirectlymaptotheirSQLvariants,keepingthedatabaseatyourfingertipswithouttheneedtointermediatealloperationsthroughschemas.
2.Schemalessqueries
12
SimplerqueriesBesidessupportingschemalessinserts,updatesanddeletesqueries,withvaryingdegreesofcomplexity,Ecto2.0alsomakesregularschemalessqueriesmoreexpressive.
Oneexampleistheabilitytoselectalldesiredfieldswithoutduplication.Inearlyversions,youwouldhavetowriteverboseselectexpressionssuchas:
frompin"posts",select:%{title:p.title,body:p.body}
WithEcto2.0youcansimplypassthedesiredlistoffieldsdirectly:
from"posts",select:[:title,:body]
Thetwoqueriesaboveareequivalent.Whenalistoffieldsisgiven,Ectowillautomaticallyconvertthelistoffieldstoamaporastruct.
SupportforpassingalistoffieldsorkeywordlistshasbeenaddedtoalmostallqueryconstructsinEcto2.0.Forexample,wecanuseanupdatequerytochangethetitleofagivenpostwithoutaschema:
defupdate_title(post,new_title)do
query=from"posts",where:[id:^post.id],update:[set:[title:^new_title]]
MyApp.Repo.update_all(query)
end
Theupdateconstructsupportsfourcommands:
:set-setsthegivencolumntothegivenvalues:inc-incrementsthegivencolumnbythegivenvalue:push-pushes(appends)thegivenvaluetotheendofanarraycolumn:pull-pulls(removes)thegivenvaluefromanarraycolumn
Forexample,wecanincrementacolumnatomicallybyusingthe:inccommand,withorwithoutschemas:
defincrement_page_views(post)do
query=from"posts",where:[id:^post.id],update:[inc:[page_views:1]]
MyApp.Repo.update_all(query)
end
2.Schemalessqueries
13
Byallowingregulardatastructurestobegiventomostqueryoperations,Ecto2.0makesquerieswithandwithoutschemasmoreaccessible.Notonlythat,italsoenablesdeveloperstowritedynamicqueries,wherefields,filters,orderingcannotbespecifiedupfront.Wewillexploresuchwithmoredetailsinupcomingchapters.Fornow,let'scontinueexploringschemasinthecontextofchangesets.
2.Schemalessqueries
14
SchemasandchangesetsInthelastchapterwelearnedhowtoperformalldatabaseoperations,frominsertiontodeletion,withoutusingaschema.Whilewehavebeenexploringtheabilitytowriteconstructswithoutschemas,wehaven'tdiscussedwhatschemasactuallyare.Inthischapter,wewillrectifythat.
Inthischapterwewilltakealookattheroleschemasplaywhenvalidatingandcastingdatathroughchangesets.Aswewillsee,sometimesthebestsolutionisnottocompletelyavoidschemas,butbreakalargeschemaintosmallerones.Maybeoneforreadingdata,anotherforwriting.Maybeoneforyourdatabase,anotherforyourforms.
SchemasaremappersTheEctodocumentationsays:
AnEctoschemaisusedtomapanydatasourceintoanElixirstruct.
WeputemphasisonanybecauseitisacommonmisconceptiontothinkEctoschemasmaponlytoyourdatabasetables.
Forinstance,whenyouwriteawebapplicationusingPhoenixandyouuseEctotoreceiveexternalchangesandapplysuchchangestoyourdatabase,wehavethismapping:
Database<->Ectoschema<->Forms/API
AlthoughthereisasingleEctoschemamappingtobothyourdatabaseandyourAPI,inmanysituationsitisbettertobreakthismappingintwo.Let'sseesomepracticalexamples.
Imagineyouareworkingwithaclientthatwantsthe"SignUp"formtocontainthefields"Firstname","Lastname"alongside"E-mail"andotherinformation.Youknowthereareacoupleproblemswiththisapproach.
Firstofall,noteveryonehasafirstandlastname.Althoughyourclientisdecidedonpresentingbothfields,theyareaUIconcern,andyoudon'twanttheUItodictatetheshapeofyourdata.Furthermore,youknowitwouldbeusefultobreakthe"SignUp"informationacrosstwotables,the"accounts"and"profiles"tables.
Giventherequirementsabove,howwouldweimplementtheSignUpfeatureinthebackend?
3.Schemalesschangesets
15
Oneapproachwouldbetohavetwoschemas,AccountandProfile,withvirtualfieldssuchasfirst_nameandlast_name,anduseassociationsalongsidenestedformstotietheschemastoyourUI.Oneofsuchschemaswouldbe:
defmoduleProfiledo
useEcto.Schema
schema"profiles"do
field:name
field:first_name,:string,virtual:true
field:last_name,:string,virtual:true
...
end
end
ItisnothardtoseehowwearepollutingourProfileschemawithUIrequirementsbyaddingfieldssuchfirst_nameandlast_name.IftheProfileschemaisusedforbothreadingandwritingdata,itmayend-upinanawkwardplacewhereitisnotusefulforany,asitcontainsfieldsthatmapjusttooneortheotheroperation.
Onealternativesolutionistobreakthe"Database<->Ectoschema<->Forms/API"mappingintwoparts.Thefirstwillcastandvalidatetheexternaldatawithitsownstructurewhichyouthentransformandwritetothedatabase.Forsuch,let'sdefineaschemanamedRegistrationthatwilltakecareofcastingandvalidatingtheformdataexclusively,mappingdirectlytotheUIfields:
defmoduleRegistrationdo
useEcto.Schema
embedded_schemado
field:first_name
field:last_name
field:email
end
end
Weusedembedded_schemabecauseitisnotourintenttopersistitanywhere.Withtheschemainhand,wecanuseEctochangesetsandvalidationstoprocessthedata:
3.Schemalesschangesets
16
fields=[:first_name,:last_name,:email]
changeset=
%Registration{}
|>Ecto.Changeset.cast(params["sign_up"],fields)
|>validate_required(...)
|>validate_length(...)
Nowthattheregistrationchangesaremappedandvalidated,wecancheckiftheresultingchangesetisvalidandactaccordingly:
ifchangeset.valid?do
#Getthemodifiedregistrationstructoutofthechangeset
registration=Ecto.Changeset.apply_changes(changeset)
MyApp.Repo.transactionfn->
MyApp.Repo.insert_all"accounts",[Registration.to_account(registration)]
MyApp.Repo.insert_all"profiles",[Registration.to_profile(registration)]
end
{:ok,registration}
else
#AnnotatetheactionwetriedtoperformsotheUIshowserrors
changeset=%{changeset|action::registration}
{:error,changeset}
end
Theto_account/1andto_profile/1functionsinRegistrationwouldreceivetheregistrationstructandsplittheattributesapartaccordingly:
defto_account(registration)do
Map.take(registration,[:email])
end
defto_profile(%{first_name:first,last_name:last})do
%{name:"#{first}#{last}"}
end
Intheexampleabove,bybreakingapartthemappingbetweenthedatabaseandElixirandbetweenElixirandtheUI,ourcodebecomesclearerandourdatastructuressimpler.
NotewehaveusedMyApp.Repo.insert_all/2toadddatatoboth"accounts"and"profiles"tablesdirectly.Wehavechosentobypassschemasaltogether.However,thereisnothingstoppingyoufromalsodefiningbothAccountandProfileschemasandchangingto_account/1andto_profile/1torespectivelyreturn%Account{}and%Profile{}
3.Schemalesschangesets
17
structs.Oncestructsarereturned,theycouldbeinsertedthroughtheusualMyApp.Repo.insert/2operation.Doingsocanbeespeciallyusefulifthereareuniquenessorotherconstraintsthatyouwanttocheckduringinsertion.
SchemalesschangesetsAlthoughwechosetodefineaRegistrationschematouseinthechangeset,Ecto2.0alsoallowsdeveloperstousechangesetswithoutschemas.Wecandynamicallydefinethedataandtheirtypes.Let'srewritetheregistrationchangesetabovetobypassschemas:
data=%{}
types=%{first_name::string,last_name::string,email::string}
changeset=
{data,types}#Thedata+typestupleisequivalentto%Registration{}
|>Ecto.Changeset.cast(params["sign_up"],Map.keys(types))
|>validate_required(...)
|>validate_length(...)
YoucanusethistechniquetovalidateAPIendpoints,searchforms,andothersourcesofdata.Thechoiceofusingschemasdependsmostlyifyouwanttousethesamemappingindifferentplacesorifyoudesirethecompile-timeguaranteesElixirstructsgivesyou.Otherwise,youcanbypassschemasaltogether,beitwhenusingchangesetsorinteractingwiththerepository.
However,themostimportantlessoninthischapterisnotwhentouseornottouseschemas,butratherunderstandwhenabigproblemcanbebrokenintosmallerproblemsthatcanbesolvedindependentlyleadingtoanoverallcleanersolution.Thechoiceofusingschemasornotabovedidn'taffectthesolutionasmuchasthechoiceofbreakingtheregistrationproblemapart.
3.Schemalesschangesets
18
DynamicqueriesEctowasdesignedfromthegrounduptohaveanexpressivequeryAPIthatleveragesElixirsyntaxtoprovidequeriesthatarepre-compiledtobeperformantandsafe.Whenbuildingqueries,wemayusethekeywordssyntax
importEcto.Query
frompinPost,
where:p.author=="José"andp.category=="Elixir",
where:p.published_at>^minimum_date,
order_by:[desc:p.published_at]
orthepipe-basedone
importEcto.Query
Post
|>where([p],p.author=="José"andp.category=="Elixir")
|>where([p],p.published_at>^minimum_date)
|>order_by([p],desc:p.published_at)
Whilemanydeveloperspreferthepipe-basedsyntax,havingtorepeatthebindingpmadeitquiteverbosecomparedtothekeywordone.Furthermore,thecompile-timeaspectofEctoquerieswasatoddswithbuildingqueriesdynamically.Imagineforexampleawebapplicationthatprovidessearchfunctionalityontopofexistingposts.Theusershouldbeabletospecifymultiplecriteria,suchastheauthorname,thepostcategory,publishinginterval,etc.
InEcto1.0,theonlywaytowritesuchfunctionalitywouldbeviaEnum.reduce/3:
4.Dynamicqueries
19
deffilter(params)do
Enum.reduce(params,Post,&filter/2)
end
defpfilter({"author",author},query)do
where(query,[p],p.author==^author)
end
defpfilter({"category",category},query)do
where(query,[p],p.category==^category)
end
defpfilter({"published_at",minimum_date},query)do
where(query,[p],p.published_at>^minimum_date)
end
defpfilter({"order_by","published_at_asc"},query)do
order_by(query,[p],asc:p.published_at)
end
defpfilter({"order_by","published_at_desc"},query)do
order_by(query,[p],desc:p.published_at)
end
defpfilter(_ignore_unknown,query)do
query
end
Whilethecodeaboveworksfine,itcouplestheprocessingoftheparameterswiththequerygeneration.Itisaverboseimplementationthatisalsohardtotestsincetheresultoffilteringandhandlingofparametersarestoreddirectlyinsidethequerystruct.
Apreferredapproachwouldbetoprocesstheparametersintoregulardatastructuresandthenbuildthequeryaslateaspossible.That'sexactlywhatEcto2.0andlaterallowustodo.
FocusingondatastructuresEcto2.0providesasimplerAPIforbothkeywordandpipebasedqueriesbymakingdatastructuresfirst-class.Let'srewritetheoriginalqueriestousedatastructureswhenpossible:
frompinPost,
where:[author:"José",category:"Elixir"],
where:p.published_at>^minimum_date,
order_by:[desc::published_at]
and
4.Dynamicqueries
20
Post
|>where(author:"José",category:"Elixir")
|>where([p],p.published_at>^minimum_date)
|>order_by(desc::published_at)
Noticehowwewereabletoditchthepselectorinmostexpressions.InEcto2.0,allconstructs,fromselectandorder_bytowhereandgroup_by,acceptdatastructuresasinput.Thedatastructurecanbespecifiedatcompile-time,asabove,andalsodynamicallyatruntime,shownbelow:
where=[author:"José",category:"Elixir"]
order_by=[desc::published_at]
Post
|>where(^where)
|>where([p],p.published_at>^minimum_date)
|>order_by(^order_by)
Theadvantageofinterpolatingdatastructuresasaboveisthatwecandecoupletheprocessingofparametersfromthequerygeneration.Howevernotallexpressionscanbeconvertedtothedatastructureshape.Sincewhereconvertsakey-valuetoakey==valuecomparison,order-basedcomparisonssuchasp.published_at>^minimum_datestillneedtobewrittenaspartofthequeryasbefore.
Luckily,theupcomingEcto2.1releasesolvesthisissue.
ThedynamicmacroForcaseswherewecannotrelyondatastructuresbutstilldesiretobuildqueriesdynamically,Ecto2.1includestheEcto.Query.dynamic/2macro.
Inordertounderstandhowthedynamicmacroworkslet'srewritethefilter/1functionfromthebeginningofthischapterusingbothdatastructuresandthedynamicmacro.NotetheexamplebelowrequiresEcto2.1:
4.Dynamicqueries
21
deffilter(params)do
Post
|>order_by(^filter_order_by(params["order_by"]))
|>where(^filter_where(params))
|>where(^filter_published_at(params["published_at"]))
end
deffilter_order_by("published_at_desc"),do:[desc::published_at]
deffilter_order_by("published_at"),do:[asc::published_at]
deffilter_order_by(_),do:[]
deffilter_where(params)do
forkey<-[:author,:category],
value=params[Atom.to_string(params)],
do:{key,value}
end
deffilter_published_at(date)whenis_binary(date),
do:dynamic([p],p.published_at>^date)
deffilter_published_at(_date),
do:true
Thedynamicmacroallowsustobuilddynamicexpressionsthatarelaterinterpolatedintothequery.dynamicexpressionscanalsobeinterpolatedintodynamicexpressions,allowingdeveloperstobuildcomplexexpressionsdynamicallywithouthassle.
Becausewewereabletobreakourproblemintosmallerfunctionsthatreceiveregulardatastructures,wecanuseallthetoolsavailableinElixirtoworkwithdata.Forhandlingtheorder_byparameter,itmaybebesttosimplypatternmatchontheorder_byparameter.Forbuildingthewhereclause,wecantraversethelistofknownkeysandconvertthemtotheformatexpectedbyEcto.Forcomplexconditions,weusethedynamicmacro.
Testingalsobecomessimpleraswecantesteachfunctioninisolation,evenwhenusingdynamicqueries:
test"filterpublishedatbasedonthegivendate"do
assertinspect(filter_published_at("2010-04-17"))==
"dynamic([p],p.published_at>^\"2010-04-17\")"
assertinspect(filter_published_at(nil))==
"true"
end
WhileattheendofthedaysomedevelopersmayfeelmorecomfortablewithusingtheEnum.reduce/3approachthewholeway,Ecto2.0andlatergivesustheoptiontochoosewhichapproachworksbest.
ThankstoMichałMuskałaforsuggestionsandfeedbackonthischapter.
4.Dynamicqueries
22
4.Dynamicqueries
23
MultitenancywithqueryprefixesEcto2.0introducestheabilitytorunqueriesindifferentprefixesusingasinglepoolofdatabaseconnections.FordatabasesenginessuchasPostgres,Ecto'sprefixmapstoPostgres'DDLschemas.ForMySQL,eachprefixisadifferentdatabaseonitsown.
Queryprefixesmaybeusefulindifferentscenarios.Forexample,multitenantappsrunningonPostgreswoulddefinemultipleprefixes,usuallyoneperclient,underasingledatabase.Theideaisthatprefixeswillprovidedataisolationbetweenthedifferentusersoftheapplication,guaranteeingeithergloballyoratthedatalevelthatqueriesandcommandsactonaspecificprefix.
Prefixesmayalsobeusefulonhigh-trafficapplicationswheredataispartitionedupfront.Forexample,agamingplatformmaybreakgamedataintoisolatedpartitions,eachgivenbyadifferentprefix.Apartitionforagivenplayeriseitherchosenatrandomforeachgameplayorcalculatedbasedontheplayerinformation.
Whilequeryprefixesweredesignedwiththetwoscenariosaboveinmind,theymayalsobeusedinothercircumstances,whichwewillexplorethroughoutthischapter.AlltheexamplesbelowassumeyouareusingPostgres.Otherdatabasesenginesmayrequireslightlydifferentsolutions.
GlobalprefixesAsastartingpoint,let'sstartwithasimplescenario:yourapplicationmustconnecttoaparticularprefixwhenrunninginproduction.Thismaybeduetoinfrastructureconditions,databaseadministrationrulesorothers.
Let'sdefinearepositoryandaschematogetstarted:
5.Multitenancywithqueryprefixes
24
#lib/repo.ex
defmoduleMyApp.Repodo
useEcto.Repo,otp_app::my_app
end
#lib/sample.ex
defmoduleMyApp.Sampledo
useEcto.Schema
schema"samples"do
field:name
timestamps
end
end
Nowlet'sconfiguretherepository:
#config/config.exs
config:my_app,MyApp.Repo,
adapter:Ecto.Adapters.Postgres,
username:"postgres",
password:"postgres",
database:"demo",
hostname:"localhost",
pool_size:10
Anddefineamigration:
#priv/repo/migrations/20160101000000_create_sample.exs
defmoduleMyApp.Repo.Migrations.CreateSampledo
useEcto.Migration
defchangedo
createtable(:samples)do
add:name,:string
timestamps()
end
end
end
Nowlet'screatethedatabase,migrateitandthenstartanIExsession:
5.Multitenancywithqueryprefixes
25
$mixecto.create
$mixecto.migrate
$iex-Smix
InteractiveElixir(1.4.0-dev)-pressCtrl+Ctoexit(typeh()ENTERforhelp)
iex(1)>MyApp.Repo.allMyApp.Sample
[]
Wehaven'tdoneanythingunusualsofar.Wecreatedourdatabaseinstance,madeituptodatebyrunningmigrationsandthensuccessfullymadeaqueryagainstthe"samples"table,whichreturnedanemptylist.
Bydefault,connectionstoPostgres'databasesrunonthe"public"prefix.Whenwerunmigrationsandqueries,theyareallrunningagainstthe"public"prefix.Howeverimagineyourapplicationhasarequirementtorunaparticularprefixinproduction,let'scallit"global_prefix".
LuckilyPostgresallowsustochangetheprefixourdatabaseconnectionsrunonbysettingtheschemasearchpath.Thebestmomenttochangetheschemasearchpathisrightafterwesetupthedatabaseconnection,ensuringallofourquerieswillrunonthatparticularprefix,throughouttheconnectionlife-cycle.
Todoso,let'schangeourdatabaseconfigurationin"config/config.exs"andspecifyan:after_connectoption.:after_connectexpectsatuplewithmodule,functionandargumentsitwillinvokewiththeconnectionprocessassoonasadatabaseconnectionisestablished:
config:my_app,MyApp.Repo,
adapter:Ecto.Adapters.Postgres,
username:"postgres",
password:"postgres",
database:"demo_dev",
hostname:"localhost",
pool_size:10,
after_connect:{Postgrex,:query!,["SETsearch_pathTOglobal_prefix",[]]}
Nowlet'strytorunthesamequeryasbefore:
$iex-Smix
InteractiveElixir(1.4.0-dev)-pressCtrl+Ctoexit(typeh()ENTERforhelp)
iex(1)>MyApp.Repo.allMyApp.Sample
**(Postgrex.Error)ERROR(undefined_table):relation"samples"doesnotexist
Ourpreviouslysuccessfulquerynowfailsbecausethereisnotable"samples"underthenewprefix.Let'strytofixthatbyrunningmigrations:
5.Multitenancywithqueryprefixes
26
$mixecto.migrate
**(Postgrex.Error)ERROR(invalid_schema_name):noschemahasbeenselectedtocreate
in
Oops.Nowmigrationsaysthereisnosuchschemaname.That'sbecausePostgresautomaticallycreatesthe"public"prefixeverytimewecreateanewdatabase.Ifwewanttouseadifferentprefix,wemustexplicitlycreateitonthedatabasewearerunningon:
$psql-ddemo_dev-c"CREATESCHEMAglobal_prefix"
Nowwearereadytomigrateandrunourqueries:
$mixecto.migrate
$iex-Smix
InteractiveElixir(1.4.0-dev)-pressCtrl+Ctoexit(typeh()ENTERforhelp)
iex(1)>MyApp.Repo.allMyApp.Sample
[]
Dataindifferentprefixesareisolated.Writingtothe"samples"tableinoneprefixcannotbeaccessedbytheotherunlesswechangetheprefixintheconnectionorusingtheEctoconvenienceswewilldiscussbelow.
Per-queryandper-structprefixesWhilestillconfiguredtoconnecttothe"global_prefix"on:after_connect,let'srunsomequeries:
iex(1)MyApp.Repo.allMyApp.Sample
[]
iex(2)MyApp.Repo.insert%MyApp.Sample{name:"mary"}
{:ok,%MyApp.Sample{...}}
iex(3)MyApp.Repo.allMyApp.Sample
[%MyApp.Sample{...}]
Nowwhathappensiftrytorunthesamplequeryonthe"public"prefix?Todoso,let'sbuildaquerystructandsettheprefixfieldmanually:
iex(4)>query=Ecto.Queryable.to_queryMyApp.Sample
#Ecto.Query<fromsinMyApp.Sample>
iex(5)>MyApp.Repo.all%{query|prefix:"public"}
[]
5.Multitenancywithqueryprefixes
27
Noticehowwewereabletochangetheprefixthequeryrunson.Backinthedefault"public"prefix,thereisnodata!
Ecto2.1alsosupportsthe:prefixoptiononallrelevantrepositoryoperations:
iex(6)>MyApp.Repo.allMyApp.Sample
[%MyApp.Sample{...}]
iex(7)>MyApp.Repo.allMyApp.Sample,prefix:"public"
[]
OneinterestingaspectofprefixesinEctoisthattheprefixinformationiscarriedalongeachstruct:
iex(8)[sample]=MyApp.Repo.allMyApp.Sample
[%MyApp.Sample{}]
iex(9)>Ecto.get_meta(sample,:prefix)
nil
Theexampleabovereturnednil,whichmeansnoprefixwasspecifiedbyEcto,andthereforethedatabaseconnectiondefaultwillbeused.Inthiscase,"global_prefix"willbeusedbecauseofthe:after_connectcallbackweaddedatthebeginningofthischapter.
Sincetheprefixdataiscarriedinthestruct,wecanusesuchtocopydatafromoneprefixtotheother.Let'scopythesampleabovefromthe"global_prefix"tothe"public"one:
iex(10)>public_sample=Ecto.put_meta(sample,prefix:"public")
%MyApp.Sample{}
iex(11)>MyApp.Repo.insertpublic_sample
{:ok,%MyApp.Sample{}}
iex(12)>MyApp.Repo.allMyApp.Sample,prefix:"public"
[%MyApp.Sample{}]
Nowwehavedatainsertedinbothprefixes.
Prefixesinqueriesandstructsalwayscascade.Forexample,ifyourunMyApp.Repo.preload(sample,[:some_association]),theassociationwillbequeriedforandloadedinthesameprefixasthesamplestruct.IfsamplehasassociationsandyoucallMyApp.Repo.insert(sample)orMyApp.Repo.update(sample),theassociateddatawillalsobeinserted/updatedinthesameprefixassample.That'sbydesigntofacilitateworkingwithgroupsofdatainthesameprefix,andespeciallybecausedataindifferentprefixesmustbekeptisolated.
Migrationprefixes
5.Multitenancywithqueryprefixes
28
SofarwehaveexploredhowtosetaglobalprefixusingPostgres'andhowtosettheprefixatthequeryorstructlevel.Whentheglobalprefixisset,italsochangestheprefixmigrationsrunon.Howeveritisalsopossibletosettheprefixthroughthecommandlineorpertableinthemigrationitself.
Forexample,imagineyouareagamingcompanywherethegameisbrokenin128partitions,named"prefix_1","prefix_2","prefix_3"upto"prefix_128".Now,wheneveryouneedtomigratedata,youneedtomigratedataonalldifferent128prefixes.Therearetwowaysofachievethat.
Thefirstmechanismistoinvokemixecto.migratemultipletimes,onceperprefix,passingthe--prefixoption:
$mixecto.migrate--prefix"prefix_1"
$mixecto.migrate--prefix"prefix_2"
$mixecto.migrate--prefix"prefix_3"
...
$mixecto.migrate--prefix"prefix_128"
Theotherapproachisbychangingeachdesiredmigrationtorunacrossmultipleprefixes.Forexample:
defmoduleMyApp.Repo.Migrations.CreateSampledo
useEcto.Migration
defchangedo
fori<-1..128do
prefix="prefix_#{i}"
createtable(:samples,prefix:prefix)do
add:name,:string
timestamps()
end
#Executethecommandsonthecurrentprefix
#beforemovingontothenextprefix
flush()
end
end
end
SchemaprefixesFinally,Ecto2.1addstheabilitytosetaparticularschematorunonaspecificprefix.Imagineyouarebuildingamulti-tenantapplication.Eachclientdatabelongstoaparticularprefix,suchas"client_foo","client_bar"andsoforth.Yetyourapplicationmaystillrelyona
5.Multitenancywithqueryprefixes
29
setoftablesthatissharedacrossallclients.OneofsuchtablesmaybeexactlythetablethatmapstheClientIDtoitsdatabaseprefix.Let'sassumewewanttostorethisdatainaprefixnamed"main":
defmoduleMyApp.Mappingdo
useEcto.Schema
@schema_prefix"main"
schema"mappings"do
field:client_id,:integer
field:db_prefix
timestamps
end
end
NowrunningMyApp.Repo.allMyApp.Mappingwillbydefaultrunonthe"main"prefix,regardlessofthevalueconfiguredgloballyonthe:after_connectcallback.Similarwillhappentoinsert,update,andsimilaroperations,unlessthe:prefixisexplicitlychangedviaEcto.put_meta/2orbypassingthe:prefixoptiontotherepositoryoperation.
Keepinmind,however,thatqueriesrunonasingleprefix.Forexample,ifMyApp.Mappingonprefix"main"dependsonaschemanamedMyApp.Otheronprefix"another",aquerystartingwithMyApp.Mappingwillalwaysrunonthe"main"prefix.Bydesignitisnotpossibletoperformqueryjoinsacrossprefixes.Ifdatabelongstodifferentprefixes,itisbesttonotcouplethemstructurallynorviaqueries,inordertokeepdataindifferentprefixesisolated.
SummingupEcto2.0providesmanyconveniencesforworkingwithqueryingprefixes.ThoseconvenienceshavebeenfurtherimprovedinEcto2.1,allowingdeveloperstoconfigureprefixwithdifferentlevelofgranularity:
globalprefixes>schemaprefix>query/structprefixes
Thisallowsdeveloperstotackledifferentscenarios,fromproductionrequirementstomulti-tenantapplications.Ourjourneyonexploringthenewqueryconstructsisalmostover.Thenextandlastquerychapterisonaggregatesandsubqueries.
5.Multitenancywithqueryprefixes
30
AggregatesandsubqueriesThelastfeatureswewilldiscussregardingEctoqueriesareaggregatesandsubqueries.Aswewilllearn,thoseareintrinsicallyrelated.
AggregatesEcto2.0includesaconveniencefunctioninrepositoriestocalculateaggregates.
Forexample,tofindtheaveragenumberofvisitsacrossallposts:
MyApp.Repo.aggregate(MyApp.Post,:avg,:visits)
#=>#Decimal<1743>
Behindthescenes,thequeryabovetranslatesto:
MyApp.Repo.one(frompinMyApp.Post,select:avg(p.visits))
Theaggregate/3functionsupportsanyoftheaggregatefunctionslistedintheEctoQueryAPI.
Atfirst,itlooksliketheimplementationofaggregate/3isquitestraight-forward.YoucouldevenstarttowonderwhyitwasaddedtoEctointhefirstplace.However,complexitiesstarttoariseonqueriesthatrelyonlimit,offsetordistinctclauses.
Imaginethatinsteadofcalculatingtheaverageofallposts,youwanttheaverageofonlythetop10.Youmaytrytoitasfollows:
MyApp.Repo.one(frompinMyApp.Post,
order_by:[desc::visits],
limit:10,
select:avg(p.visits))
#=>#Decimal<1743>
Oops.Thequeryabovereturnedthesamevalueasthequeriesbefore.Theoptionlimit:10hasnoeffectheresinceitislimitingtheaggregatedresultandquerieswithaggregatesreturnonlyasinglerowanyway.Inordertoretrievethecorrectresult,wewouldneedtofirstfindthetop10postsandonlythenaggregate.That'sexactlywhataggregate/3does:
6.Aggregatesandsubqueries
31
query=fromMyApp.Post,order_by:[desc::visits],limit:10
MyApp.Repo.aggregate(query,:avg,:visits)#=>#Decimal<4682>
Whenlimit,offsetordistinctisspecifiedinthequery,aggregate/3automaticallywrapsthegivenqueryinasubquery.Thequeryexecutedbyaggregate/3abovewouldbeequivalentto:
query=fromMyApp.Post,order_by:[desc::visits],limit:10
MyApp.Repo.one(fromqinsubquery(query),select:avg(q.visits))
Let'stakeacloserlookatsubqueries.
SubqueriesIntheprevioussectionwehavealreadylearnedsomequeriesthatwouldbehardtoexpresswithoutsupportforsubqueries.That'soneofmanyexamplesthatledtosubquerysupportbeingaddedtoEcto.
SubqueriesinEctoarecreatedbycallingEcto.Query.subquery/1.ThefunctionreceivesanydatastructurethatcanbeconvertedtoaqueryviatheEcto.Queryableprotocolandreturnsasubqueryconstruct(whichisalsoqueryable).
InEcto2.0,itisallowedforasubquerytoselectawholetable(p)orafield(p.field).Allfieldsselectedinasubquerycanbeaccessedfromtheparentquery.Let'srevisittheaggregatequerywesawintheprevioussection:
query=fromMyApp.Post,order_by:[desc::visits],limit:10
MyApp.Repo.one(fromqinsubquery(query),select:avg(q.visits))
Becausethequerydoesnotspecifya:selectclause,itwillreturnselect:pwherepiscontrolledbyMyApp.Postschema.SincethequerywillreturnallfieldsinMyApp.Post,whenweconvertittoasubquery,allofthefieldsfromMyApp.Postwillbeavailableontheparentquery,suchasq.visits.Infact,Ectowillkeeptheschemapropertiesacrossqueries.Forexample,ifyouwriteq.field_that_does_not_exist,yourEctoquerywon'tcompile.
Ecto2.1furtherimprovessubqueriesbyallowinganElixirmaptobereturnedfromasubquery,makingthemapfieldsdirectlyavailabletotheparentquery.
Let'sseeonelastexample.Imagineyoumanagealibrary(anactuallibraryintherealworld)andthereisatablethatlogseverytimethelibrarylendsabook.The"lendings"tableusesauto-incrementingindexesandcanbebackedbythefollowingschema:
6.Aggregatesandsubqueries
32
defmoduleLibrary.Lendingdo
useEcto.Schema
schema"lendings"do
belongs_to:book,MyApp.Book#definesbook_id
belongs_to:visitor,MyApp.Visitor#definesvisitor_id
end
end
Nowconsiderwewanttoretrievethenameofeverybookalongsidethenameofthelastpersonthelibraryhaslentitto.Todoso,weneedtofindthelastlendingIDofeverybook,andthenjoinonthebookandvisitortables.Withsubqueries,that'sstraight-forward:
last_lendings=
fromlinMyApp.Lending,
group_by:l.book_id,
select:%{book_id:l.book_id,last_lending_id:max(l.id)}
fromlinLending,
join:lastinsubquery(last_lendings),
on:last.last_lending_id==l.id,
join:binassoc(l,:book),
join:vinassoc(l,:visitor),
select:{b.name,v.name}
SubqueriesareanimportantimprovementtoEctowhichmakesitpossibletoexpressqueriesthatwerenotpossiblebefore.Ontopofthat,wewereabletoaddfeaturessuchasaggregates,whichprovideusefulfunctionalitywhileshieldingtheuserfromcornercases.
6.Aggregatesandsubqueries
33
ImprovedassociationsandfactoriesEcto2.0largelyimprovedhowassociationswork.Tounderstandwhyandhow,let'stalkaboutEcto'soriginaldesigngoals.
EctofirststartedasaSummerofCodeprojectfromEricMeadows-Jönsson,todaypartoftheElixirteamandcreatorofHex,withJoséValim,creatorofElixir,asmentor.Thiswasbackin2013andElixirwasstillatversion0.9!
Similartomanyprojectsatthetime,oneofthegoalsbehindEctowastovalidateElixiritselfasaprogramminglanguage.Oneofthequestionsitaimedtoanswerwas:"canElixirbeusedtocreateadatabasewrapperthatisperformantandsecure?".Bybecomingastable,performantandsecurefoundation,Ectowouldbeabletoaddsyntaxsugar,conveniencesanddynamismlateron-whiletheoppositedirectionwouldhavebeenexceptionallyhardintheexperienceoftheEctoteam.
Ecto1.0becamethissecureandperformantfoundation.Asaconsequence,itfeltrigidinmanyaspectsandthosewerefrequentlyreportedaslimitationsbythecommunity.
Ecto2.0improvesonthemistakesmadebyEcto1.0andthenbuildsontopofitsfoundationbyaddingtheflexibilitiesusershavelongedfor.Wehaveexploredmanyoftheminthefirstchaptersofthisbook:schemalessqueries,schemalesschangesets,dynamicqueriesandmore.Inthenextthreechapters,wewillexploretheenhancementsdonetoschemasandassociations.
Inthisparticularchapter,wewilllearnhowEctoiscapableofinsertingcomplexdatastructureswithouttheneedtousechangesetsandhowtousethisfeaturetomanagecomplexdata,incasessuchasyourapplicationtestsuite,withoutaneedtorelyonexternalprojects.
LesschangesetsAtthesametimeEcto2.0bringsmanyfeaturestochangesets,itmakeschangesetslessnecessarythroughoutEctoAPIs.Forexample,inEcto1.0,Ecto.Repo.insert/2requiredchangesets.Thismeansthat,inordertoinsertanyentrytothedatabase,suchasapost,wehadtowrapitinachangesetfirst:
%Post{title:"helloworld"}
|>Ecto.Changeset.change
|>Repo.insert!()
7.Improvedassociationsandfactories
34
ThisreflectedthroughoutEcto1.0APIs.Ifyouwantedtocreateapostwithsomecomments,youhadtowrapeachcommentinachangesetandthenputitinthepostchangeset:
comment1=%Comment{body:"excellentarticle"}|>Ecto.Changeset.change
comment2=%Comment{body:"Ilearnedsomethingnew"}|>Ecto.Changeset.change
%Post{title:"helloworld"}
|>Ecto.Changeset.put_assoc(:comments,[comment1,comment2])
|>Repo.insert!()
Furthermore,whenhandlingassociations,Ecto1.0forcedyoutoalwayswritetheparentchangesetfirstandthenthechildren.Sotheexampleabovewhereinsertapost(theparent)withmultiplecomments(children)workedbutthefollowingexamplewouldnot:
post=%Post{title:"helloworld"}|>Ecto.Changeset.change()
%Comment{body:"excellentarticle"}
|>Ecto.Changeset.put_assoc(:post,[post])
|>Repo.insert!()
Ecto2.0goesawaywithallofthoselimitations.YoucannowpassstructstotherepositoryandchangesetsandEctowilltakecareofbuildingthechangesetsforyoubehindthescenes.InEcto2.0,apostwithcommentscanbeinserteddirectlyasfollows:
Repo.insert!%Post{
title:"helloworld",
comments:[
%Comment{body:"excellentarticle"},
%Comment{body:"Ilearnedsomethingnew"}
]
}
Youarealsoabletoinsertandupdateassociationsfromanydirection,beitfromparenttochildorchildtoparent:
Repo.insert!%Comment{
body:"excellentarticle",
post:%Post{title:"helloworld"}
}
Thisfeatureisnotonlyusefulwhenwritingourapplicationsbutalsowhentestingthem,aswewillseenext.
7.Improvedassociationsandfactories
35
TestfactoriesManyprojectsdependonexternallibrariestobuildtheirtestdata.Someofthoselibrariesarecalledfactoriesbecausetheyprovideconveniencefunctionsforbuildingdifferentgroupsofdata.However,givenEcto2.0isabletomanagecomplexdatatrees,wecanimplementsuchfunctionalitywithoutrelyingonthird-partyprojects.
Togetstarted,let'screateafileat"test/support/factory.ex"withthefollowingcontents:
defmoduleMyApp.Factorydo
aliasMyApp.Repo
#Factories
defbuild(:post)do
%MyApp.Post{title:"helloworld"}
end
defbuild(:comment)do
%MyApp.Comment{body:"goodpost"}
end
defbuild(:post_with_comments)do
%MyApp.Post{
title:"hellowithcomments",
comments:[
build(:comment,body:"first"),
build(:comment,body:"second")
]
}
end
defbuild(:user)do
%MyApp.User{
email:"hello#{System.unique_integer()}",
username:"hello#{System.unique_integer()}"
}
end
#ConvenienceAPI
defbuild(factory_name,attributes)do
factory_name|>build()|>struct(attributes)
end
definsert!(factory_name,attributes\\[])do
Repo.insert!build(factory_name,attributes)
end
end
7.Improvedassociationsandfactories
36
Ourfactorymoduledefinesfour"factories"asdifferentclausestothebuildfunction::post,:comment,:post_with_commentsand:user.Eachclausedefinestructswiththefieldsthatrequiredbythedatabase.Incertaincases,thegeneratedstructalsoneedtogenerateuniquefields,suchastheuser'semailandusername.WedidsobycallingElixir'sSystem.unique_integer()-youcouldcallSystem.unique_integer([:positive])ifyouneedastrictlypositivenumber.
Attheend,wedefinedtwofunctions,build/2andinsert!/2,whichareconveniencesforbuildingstructswithspecificattributesorforinsertingdatadirectlyintherepository.
That'sliterallyallthatisnecessaryforbuildingourfactories.Wearenowreadytousefactoriesinourtests.First,openupyour"mix.exs"fileandlet'smakesurethe"test/support/factory.ex"fileiscompiled:
defprojectdo
[...,
elixirc_paths:elixirc_paths(Mix.env),
...]
end
defpelixirc_paths(:test),do:["lib","test/support"]
defpelixirc_paths(_),do:["lib"]
Nowinanyoftheteststhatneedtogeneratedata,wecanimporttheMyApp.Factorymoduleanduseitsfunctions:
importMyApp.Factory
build(:post)
#=>%MyApp.Post{id:nil,title:"helloworld",...}
build(:post,title:"customtitle")
#=>%MyApp.Post{id:nil,title:"customtitle",...}
insert!(:post,title:"customtitle")
#=>%MyApp.Post{id:...,title:"customtitle"}
BybuildingthefunctionalityweneedontopofEctocapabilities,weareabletoextendandimproveourfactoriesonwhateverwaywedesire,withoutbeingconstrainedtothird-partylimitations.
7.Improvedassociationsandfactories
37
ManytomanyandcastingBesidesbelong_to,has_many,has_oneand:throughassociations,Ecto2.0alsoincludesmany_to_many.many_to_manyrelationships,asthenamesays,allowsXtohavemanyassocaitedentriesYandvice-versa.Althoughmany_to_manyassociationscanbewrittenasahas_many:through,usingmany_to_manymayconsiderablysimplifysomeworkflows.
Inthischapter,wewilltalkaboutpolymorphicassociationsandhowmany_to_manycanconsiderablyremoveboilerplatefromcertainapproachescomparedtohas_many:through.
Todolistsv65131Thewebhasseenitsshareoftodolistapplications.Butthatwon'tstopusfromcreatingourown!
Inourcase,thereisoneaspectoftodolistapplicationsweareinterestedon,whichistherelationshipthetodolistwhichhasmanytodoitems.WehaveexploredthisexactscenarioindetailinanarticlewepostedonPlataformatec'sblogaboutnestedassociationsandembeds.Let'srecaptheimportantpoints.
Inordertomodelatodolistapp,weneedtohavetwoschemas,Todo.ListandTodo.Item:
defmoduleMyApp.TodoListdo
useEcto.Schema
schema"todo_lists"do
field:title
has_many:todo_items,MyApp.TodoItem
timestamps()
end
end
defmoduleMyApp.TodoItemdo
useEcto.Schema
schema"todo_items"do
field:description
timestamps()
end
end
8.Manytomanyandcasting
38
OneofthewaystointroduceatodolistwithmultipleitemsintothedatabaseistocoupleourUIrepresentationtoourschemas.That'stheapproachwetookintheblogpostabovewithPhoenix.Roughly:
<%=form_for@todo_list_changeset,todo_list_path(@conn,:create),fnf->%>
<%=text_inputf,:title%>
<%=inputs_forf,:todo_items,fni->%>
...
<%end%>
<%end%>
WhensuchaformissubmittedinPhoenix,itwillsendparameterswiththefollowingshape:
%{"todo_list"=>%{
"title"=>"shippinglist",
"todo_items"=>%{
0=>%{"description"=>"bread"},
1=>%{"description"=>"eggs"},
}
}}
WecouldthenretrievethoseparametersandpassittoanEctochangesetandEctowouldautomaticallyfigureoutwhattodo:
#InMyApp.TodoList
defchangeset(struct,params\\%{})do
struct
|>Ecto.Changeset.cast(params,[:title])
|>Ecto.Changeset.cast_assoc(:todo_list_items,required:true)
end
#AndtheninMyApp.TodoItem
defchangeset(struct,params\\%{})do
struct
|>Ecto.Changeset.cast(params,[:description])
end
BycallingEcto.Changeset.cast_assoc/3,Ectowilllookfora"todo_items"keyinsidetheparametersgivenoncast,andcomparethoseparameterswiththeitemsstoredinthetodoliststructs.Ectowillautomaticallygenerateinstructionstoinsert,updateordeletetodoitemssuchthat:
ifatodoitemsentasparameterhasanIDanditmatchesanexistingassociatedtodoitem,weconsiderthattodoitemshouldbeupdatedifatodoitemsentasparameterdoesnothaveanID(noramatchingID),weconsider
8.Manytomanyandcasting
39
thattodoitemshouldbeinsertedifatodoitemiscurrenetlyassociatedbutitsIDwasnotsentasparameter,weconsiderthetodoitemisbeingreplacedandweactaccordingtothe:on_replacecallback.Bydefault:on_replacewillraisesoyouchooseabehaviourbetweenreplacing,deleting,ignoringornilifyingtheassociation
Theadvantageofusingcast_assoc/3isthatEctoisabletodoallofthehardworkofkeepingtheentriesassociated,aslongaswepassthedataexactlyintheformatthatEctoexpects.However,aswelearnedinthefirstthreechaptersofthisbook,suchapproachisnotalwayspreferrableandinmanysituationsitisbettertodesignourassociationsdifferentlyordecoupleourUIsfromourdatabaserepresentation.
PolymorphictodoitemsToshowanexampleofwhereusingcast_assoc/3isjusttoocomplicatedtobeworthit,let'simagineyouwantyour"todoitems"tobepolymorphic.Forexample,youwanttobeabletoaddtodoitemsnotonlyto"todolists"buttomanyotherpartsofyourapplication,suchasprojects,dates,younameit.
Firstofall,itisimportanttorememberEctodoesnotprovidethesametypeofpolymorphicassociationsknowninframeworkssuchasRailsandLaravel.Insuchframeworks,apolymorphicassociationusestwocolumns,theparent_idandparent_type.Forexample,onetodoitemwouldhaveparent_idof1withparent_typeof"TodoList"whileanotherwouldhaveparent_idof1withparent_typeof"Project".
Theissuewiththedesignaboveisthatitbreaksdatabasereferences.Thedatabaseisnolongercapableofguaranteeingtheitemyouassociatetoexistsorwillcontinuetoexistinthefuture.Thisleadstoaninconsistentdatabasewhichend-uppushingworkaroundstoyourapplication.
Thedesignaboveisalsoextremelyinefficient.Inthepastwehaveworkedwithalargeclientonremovingsuchpolymorphicreferencesbecausefrequentpolymorphicqueriesweregrindingthedatabasetoahaltevenafteraddingindexesandoptimizingthedatabase.
Luckily,thedocumentationforthebelongs_tomacroincludesexamplesonhowtodesignsaneandperformantassociations.Oneofthoseapproachesconsistsinusingmanyjointables.Besidesthe"todo_lists"and"projects"tablesandthe"todo_items"table,wewouldcreate"todo_list_items"and"project_items"toassociatetodoitemstotodolistsandtodoitemstoprojectsrespectively.Intermsofmigrations,wearelookingatthefollowing:
8.Manytomanyandcasting
40
createtable("todo_lists")do
add:title
timestamps()
end
createtable("projects")do
add:name
timestamps()
end
createtable("todo_items")do
add:description
timestamps()
end
createtable("todo_lists_items")do
add:todo_item_id,references(:todo_items)
add:todo_list_id,references(:todo_lists)
timestamps()
end
createtable("projects_items")do
add:todo_item_id,references(:todo_items)
add:project_id,references(:projects)
timestamps()
end
Firstlet'sseehowimplementthisfunctionalityusingahas_many:throughandthenusemany_to_manytoremovealotoftheboilerplatewewereforcedtointroduce.
Polymorphismwithhas_many:throughGivenwewantourtodoitemstobepolymorphic,wecannolongerassociateatodolisttotodoitemsdirectly.InsteadwewillcreateanintermediateschematotieMyApp.TodoListandMyApp.TodoItemtogether.
8.Manytomanyandcasting
41
defmoduleMyApp.TodoListdo
useEcto.Schema
schema"todo_lists"do
field:title
has_many:todo_list_items,MyApp.TodoListItem
has_many:todo_items,through:[:todo_list_items,:todo_item]
timestamps()
end
end
defmoduleMyApp.TodoListItemdo
useEcto.Schema
schema"todo_list_items"do
belongs_to:todo_list,MyApp.TodoList
belongs_to:todo_item,MyApp.TodoItem
timestamps()
end
end
defmoduleMyApp.TodoItemdo
useEcto.Schema
schema"todo_items"do
field:description
timestamps()
end
end
AlthoughweintroducedMyApp.TodoListItemasanintermediateschema,has_many:throughallowsustoaccessalltodoitemsforanytodolisttransparently:
todo_lists|>Repo.preload(:todo_items)
Thetroubleisthat:throughassociationsareread-onlysinceEctodoesnothaveenoughinformationtofillintheintermediateschema.Thismeansthat,ifwestillwanttousecast_assoctoinsertatodolistwithmanytodoitemsdirectlyfromtheUI,wewouldneedtofirstcast_assoc(:todo_list_items)fromTodoListandthencallcast_assoc(:todo_item)fromtheTodoListItemschema:
8.Manytomanyandcasting
42
#InMyApp.TodoList
defchangeset(struct,params\\%{})do
struct
|>Ecto.Changeset.cast(params,[:title])
|>Ecto.Changeset.cast_assoc(:todo_list_items,required:true)
end
#AndthenintheMyApp.TodoListItem
defchangeset(struct,params\\%{})do
struct
|>Ecto.Changeset.cast_assoc(:todo_item,required:true)
end
#AndtheninMyApp.TodoItem
defchangeset(struct,params\\%{})do
struct
|>Ecto.Changeset.cast(params,[:description])
end
Tofurthercomplicatethings,remembercast_assocexpectsaparticularshapeofdatathatreflectsyourassociations.Inthiscase,becauseoftheintermediateschema,thedatasentthroughyourformsinPhoenixwouldhavetolookasfollows:
%{"todo_list"=>%{
"title"=>"shippinglist",
"todo_list_items"=>%{
0=>%{"todo_item"=>%{description"=>"bread"}},
1=>%{"todo_item"=>%{description"=>"eggs"}},
}
}}
Tomakemattersworse,youwouldhavetoduplicatethislogicforeveryintermediateschema,bydefiningMyApp.TodoListItemfortodolists,MyApp.ProjectItemforprojects,etc.
Luckily,many_to_manyallowsustoremoveallofthisboilerplate.
Polymorphismwithmany_to_manyInaway,theideabehindmany_to_manyassociationsisthatitallowsustoassociatetwoschemasviaanintermediateschemaswhileautomaticallytakingcareofalldetailsabouttheintermediateschema.Let'srewritetheschemasabovetousemany_to_many:
8.Manytomanyandcasting
43
defmoduleMyApp.TodoListdo
useEcto.Schema
schema"todo_lists"do
field:title
many_to_many:todo_items,join_through:MyApp.TodoListItem
timestamps()
end
end
defmoduleMyApp.TodoListItemdo
useEcto.Schema
schema"todo_list_items"do
belongs_to:todo_list,MyApp.TodoList
belongs_to:todo_item,MyApp.TodoItem
timestamps()
end
end
defmoduleMyApp.TodoItemdo
useEcto.Schema
schema"todo_items"do
field:description
timestamps()
end
end
NoticeMyApp.TodoListnolongerneedstodefineahas_manyassociationpointingtotheMyApp.TodoListItemschemaandinsteadwecanjustassociateto:todo_itemsusingmany_to_many.
Differentlyfromhas_many:through,many_to_manyassociationsarealsowriteable.Thismeanswecansenddatathroughourformsexactlyaswedidatthebeginningofthischapter:
%{"todo_list"=>%{
"title"=>"shippinglist",
"todo_items"=>%{
0=>%{"description"=>"bread"},
1=>%{"description"=>"eggs"},
}
}}
Andwenolongerneedtodefineachangesetfunctionintheintermediateschema:
8.Manytomanyandcasting
44
#InMyApp.TodoList
defchangeset(struct,params\\%{})do
struct
|>Ecto.Changeset.cast(params,[:title])
|>Ecto.Changeset.cast_assoc(:todo_items,required:true)
end
#AndtheninMyApp.TodoItem
defchangeset(struct,params\\%{})do
struct
|>Ecto.Changeset.cast(params,[:description])
end
Inotherwords,wecanuseexactlythesamecodewehadinthe"todolistshas_manytodoitems"case.Soevenwhenexternalconstraintsrequireustouseajointable,many_to_manyassociationscanautomaticallymanagethemforus.Everythingyouknowaboutassociationswilljustworkwithmany_to_manyassociations,includingtheimprovementswediscussedinthepreviouschapter.
Finally,eventhoughwehavespecifiedaschemaasthe:join_throughoptioninmany_to_many,many_to_manycanalsoworkwithoutintermediateschemasaltogetherbysimplygivingitatablename:
defmoduleMyApp.TodoListdo
useEcto.Schema
schema"todo_lists"do
field:title
many_to_many:todo_items,join_through:"todo_list_items"
timestamps()
end
end
Inthiscase,youcancompletelyremovetheMyApp.TodoListItemschemafromyourapplicationandthecodeabovewillstillwork.Theonlydifferenceisthatwhenusingtables,anyautogeneratedvaluethatisfilledbyEctoschema,suchastimestamps,won'tbefilled(aswenolongerhaveaschema).Tosolvethis,youcaneitherdropthosefieldsfromyourmigrationsorsetadefaultatthedatabaselevel.
SummaryInthischapterweusedmany_to_manyassociationstodrasticallyimproveapolymorphicassociationdesignthatreliedonhas_many:through.Ourgoalwastoallow"todo_items"toassociatetodifferententitiesinourcodebase,suchas"todo_lists"and"projects".Wehave
8.Manytomanyandcasting
45
donethisbycreatingintermediatetablesandbyusingmany_to_manyassociationstoautomaticallymanagethosejointables.
Attheend,ourschemasmaylooklike:
defmoduleMyApp.TodoListdo
useEcto.Schema
schema"todo_lists"do
field:title
many_to_many:todo_items,join_through:"todo_list_items"
timestamps()
end
defchangeset(struct,params\\%{})do
struct
|>Ecto.Changeset.cast(params,[:title])
|>Ecto.Changeset.cast_assoc(:todo_items,required:true)
end
end
defmoduleMyApp.Projectdo
useEcto.Schema
schema"todo_lists"do
field:name
many_to_many:todo_items,join_through:"project_items"
timestamps()
end
defchangeset(struct,params\\%{})do
struct
|>Ecto.Changeset.cast(params,[:name])
|>Ecto.Changeset.cast_assoc(:todo_items,required:true)
end
end
defmoduleMyApp.TodoItemdo
useEcto.Schema
schema"todo_items"do
field:description
timestamps()
end
defchangeset(struct,params\\%{})do
struct
|>Ecto.Changeset.cast(params,[:description])
end
end
8.Manytomanyandcasting
46
Andthedatabasemigration:
createtable("todo_lists")do
add:title
timestamps()
end
createtable("projects")do
add:name
timestamps()
end
createtable("todo_items")do
add:description
timestamps()
end
#Primarykeyandtimestampsarenotrequiredifusingmany_to_manywithoutschemas
createtable("todo_lists_items",primary_key:false)do
add:todo_item_id,references(:todo_items)
add:todo_list_id,references(:todo_lists)
#timestamps()
end
#Primarykeyandtimestampsarenotrequiredifusingmany_to_manywithoutschemas
createtable("projects_items",primary_key:false)do
add:todo_item_id,references(:todo_items)
add:project_id,references(:projects)
#timestamps()
end
Overallourcodelooksstructurallythesameashas_manywould,althoughatthedatabaselevelourrelationshipsareexpressedwithjointables.
Whileinthischapterwechangedourcodetocopewiththeparameterformatrequiredbycast_assoc,inthenextchapterwewilldropcast_assocaltogetheranduseput_assocwhichbringsmoreflexibilitieswhenworkingwithassociations.
8.Manytomanyandcasting
47
ManytomanyandupsertsInthepreviouschapterwehavelearnedaboutmany_to_manyassociationsandhowtomapexternaldatatoassociatedentrieswiththehelpofEcto.Changeset.cast_assoc/3.Whileinthepreviouschapterwewereabletofollowtherulesimposedbycast_assoc/3,doingsoisnotalwayspossiblenordesired.
Inthischapter,wearegoingtolookatEcto.Changeset.put_assoc/4incontrasttocast_assoc/3andexploresomeexamples.WewillalsopeekattheupsertfeaturescominginEcto2.1.
put_assocvscast_assocImaginewearebuildinganapplicationthathasblogpostsandsuchpostsmayhavemanytags.Notonlythat,agiventagmayalsobelongtomanyposts.Thisisaclassicscenariowherewewouldusemany_to_manyassociations.Ourmigrationswouldlooklike:
createtable(:posts)do
add:title
add:body
timestamps()
end
createtable(:tags)do
add:name
timestamps()
end
createunique_index(:tags,[:name])
createtable(:posts_tags,primary_key:false)do
add:post_id,references(:posts)
add:tag_id,references(:tags)
end
Noteweaddedauniqueindextothetagnamebecausewedon'twanttohaveduplicatedtagsinourdatabase.Itisimportanttoaddanindexatthedatabaselevelinsteadofusingavalidationsincethereisalwaysachancetwotagswiththesamenamewouldbevalidatedandinsertedsimultaneously,passingthevalidationandleadingtoduplicatedentries.
9.Manytomanyandupserts
48
Nowlet'salsoimaginewewanttheuserinputsuchtagsasalistofwordssplitbycomma,suchas:"elixir,erlang,ecto".Oncethisdataisreceivedintheserver,wewillbreakitapartintomultipletagsandassociatethemtothepost,creatinganytagthatdoesnotyetexistinthedatabase.
Whiletheconstraintsabovesoundreasonable,that'sexactlywhatputusintroublewithcast_assoc/3.Rememberthecast_assoc/3changesetfunctionwasdesignedtoreceiveexternalparametersandcomparethemwiththeassociateddatainourstructs.Todosocorrectly,Ectorequirestagstobesentasalistofmaps.Howeverhereweexpecttagstobesentinastringseparatedbycomma.
Furthermorecast_assoc/3reliesontheprimarykeyfieldforeachtagsentinordertodecideifitshouldbeinserted,updatedordeleted.Again,becausetheuserissimplypassingastring,wedon'thavetheIDinformationathand.
Whenwecan'tcopewithcast_assoc/3,itistimetouseput_assoc/4.Input_assoc/4,wegiveEctostructsorchangesetsinsteadofparameters,givingustheabilitytomanipulatethedataaswewant.Let'sdefinetheschemaandthechangesetfunctionforapostwhichmayreceivetagsasastring:
9.Manytomanyandupserts
49
defmoduleMyApp.Postdo
useEcto.Schema
schema"posts"do
add:title
add:body
many_to_many:tags,MyApp.Tag,join_through:"posts_tags"
timestamps()
end
defchangeset(struct,params\\%{})do
struct
|>Ecto.Changeset.cast(struct,[:title,:body])
|>Ecto.Changeset.put_assoc(:tags,parse_tags(params))
end
defpparse_tags(params)do
(params["tags"]||"")
|>String.split(",")
|>Enum.map(&String.trim/1)
|>Enum.reject(&&1=="")
|>Enum.map(&get_or_insert_tag/1)
end
defpget_or_insert_tag(name)do
Repo.get_by(MyApp.Tag,name:name)||
Repo.insert!(MyApp.Tag,%Tag{name:name})
end
end
Inthechangesetfunctionabove,wemovedallthehandlingoftagstoaseparatefunction,calledparse_tags/1,whichchecksfortheparameter,breaksitsentriesapartviaString.split/2,thenremovesanyleftoverwhitespacewithString.trim/1,rejectsanyemptystringandfinallychecksifthetagexistsinthedatabaseornot,creatingoneincasenoneexists.
Theparse_tags/1functionisgoingtoreturnalistofMyApp.Tagstructswhicharethenpassedtoput_assoc/3.Bycallingput_assoc/3,wearetellingEctothoseshouldbethetagsassociatedtothepostfromnowon.Incaseaprevioustagwasassociatedtothepostandnotgiveninput_assoc/3,Ectowillalsotakecareofremovingtheassociationbetweenthepostandtheremovedtagfromthedatabase.
Andthat'sallweneedtousemany_to_manyassociationswithput_assoc/3.put_assoc/3workswithhas_many,belongs_toandallothersassociationtypes.However,ourcodeisnotyetreadyforproduction.Let'sseewhy.
9.Manytomanyandupserts
50
ConstraintsandraceconditionsRememberweaddedauniqueindextothetag:namecolumnwhencreatingthetagstable.Wedidsotoprotectusfromhavingduplicatetagsinthedatabase.
Byaddingtheuniqueindexandthenusingget_bywithainsert!togetorinsertatag,weintroducedapotentialerrorinourapplication.Iftwopostsaresubmittedatthesametimewithasimilartag,thereisachancewewillcheckifthetagexistsatthesametime,leadingbothsubmissionstobelievethereisnosuchtaginthedatabase.Whenthathappens,onlyoneofthesubmissionswillsucceedwhiletheotheronewillfail.That'saracecondition:yourcodewillerrorfromtimetotime,onlywhencertainconditionsaremet.Andthoseconditionsaretimesensitive.
Manydevelopershaveatendencytothinksucherrorswon'thappeninpracticeor,iftheyhappened,theywouldbeirrelevant.Butinpracticetheyoftenleadtoveryfrustratinguserexperiences.Ihaveheardafirst-handexamplecomingfromamobilegamecompany.Inthegame,aplayerisabletoplayquestsandoneveryquestyouhavetochooseaguestcharacterfromanotherplayeroutofashortlisttogoonthequestwithyou.Attheendofthequest,youhavetheoptiontoaddtheguestcharacterasafriend.
Originallythewholeguestlistwasrandombut,astimepassed,playersstartedtocomplainsometimesoldaccounts,ofteninactive,werebeingshownintheguestsoptionslist.Toimprovethesituation,thegamedevelopersstartedtosorttheguestlistbymostrecentlyactive.Thismeansthat,ifyouhavejustplayedrecently,thereisahigherchanceofyoutobeonsomeoneguestlists.
However,whentheydidsuchchange,manyerrorsstartedtoshowupandusersweresuddenlyfuriousinthegameforum.That'sbecausewhentheysortedplayersbyactivity,assoontwoplayersloggedin,theircharacterswouldlikelyappearoneachothersguestlist.Ifthoseplayerspickedeachotherscharacters,thefirsttoaddtheotherasfriendattheendofaquestwouldbeabletosucceedbutanerrorwouldappearwhenthesecondplayertriedtoaddthatcharacterasafriendsincetherelationshipalreadyexistedinthedatabase!Notonlythat,alltheprogressdoneinthequestwouldbelost,becausetheserverwasunabletoproperlypersistthequestresultstothedatabase.Understandably,playersstartedtofilecomplaints.
Longstoryshort:wemustaddresstheracecondition.
LuckilyEctogivesusamechanismtohandleconstrainterrorsfromthedatabase.
Checkingforconstrainterrors
9.Manytomanyandupserts
51
Sinceourget_or_insert_tag(name)functionfailswhenatagalreadyexistsinthedatabase,weneedtohandlesuchscenariosaccordingly.Let'srewriteittakingraceconditionsintoaccountinmind:
defpget_or_insert_tag(name)do
%Tag{}
|>Ecto.Changeset.change(name:name)
|>Ecto.Changeset.unique_constraint(:name)
|>Repo.insert
|>casedo
{:ok,tag}->tag
{:error,_}->Repo.get_by!(MyApp.Tag,name:name)
end
end
Insteadofinsertingthetagdirectly,weknowbuildachangeset,whichallowsustousetheunique_constraintannotation.NowiftheRepo.insertoperationfailsbecausetheuniqueindexfor:nameisviolated,Ectowon'traise,butreturnan{:error,changeset}tuple.Therefore,iftheRepo.insertsucceeds,itisbecausethetagwassaved,otherwisethetagalreadyexists,whichwethenfetchwithRepo.get_by!.
Whilethemechanismabovefixestheracecondition,itisaquiteexpensiveone:weneedtoperformtwoqueriesforeverytagthatalreadyexistsinthedatabase:the(failed)insertandthentherepositorylookup.Giventhat'sthemostcommonscenario,wemaywanttorewriteittothefollowing:
defpget_or_insert_tag(name)do
Repo.get_by(MyApp.Tag,name:name)||maybe_insert_tag(name)
end
defpmaybe_insert_tag(name)do
%Tag{}
|>Ecto.Changeset.change(name:name)
|>Ecto.Changeset.unique_constraint(:name)
|>Repo.insert
|>casedo
{:ok,tag}->tag
{:error,_}->Repo.get_by!(MyApp.Tag,name:name)
end
end
Theaboveperforms1queryforeverytagthatalreadyexists,2queriesforeverynewtagandpossibly3queriesinthecaseofraceconditions.Whiletheabovewouldperformslightlybetteronaverage,Ecto2.1hasabetteroptioninstock.
9.Manytomanyandupserts
52
UpsertsEcto2.1supportstheso-called"upsert"commandwhichisanabbreviationfor"updateorinsert".Theideaisthatwetrytoinsertarecordandincaseitconflictswithanexistingentry,forexampleduetoauniqueindex,wecanchoosehowwewantthedatabasetoactbyeitherraisinganerror(thedefaultbehaviour),ignoringtheinsert(noerror)orbyupdatingtheconflictingdatabaseentries.
"upsert"inEcto2.1isdonewiththe:on_conflictoption.Let'srewriteget_or_insert_tag(name)oncemorebutthistimeusingthe:on_conflictoption.Alsorememberthat"upsert"isanewfeatureinPostgreSQL9.5,somakesureyouareuptodate:
defpget_or_insert_tag(name)do
Repo.insert!(%MyApp.Tag{name:name},on_conflict::nothing)
end
Andthat'sit!Wetrytoinsertatagwiththegivennameandifsuchtagalreadyexists,wetellEctotonoterrorandreturnthetagwehavepassedasargumentasifitwaspersisted.Whiletheaboveiscertainlyastepupfromallsolutionssofar,itstillperformsonequerypertag.If10tagsaresent,wewillperform10queries.Canwefurtherimproveit?
Upsertsandinsert_allEcto2.1didnotonlyaddthe:on_conflictoptiontoRepo.insert/2butalsototheRepo.insert_all/3functionintroducedinEcto2.0.Thismeanswecanbuildonequerythatattemptstoinsertallmissingtagsandthenanotherquerythatfetchesallofthematonce.Let'sseehowourPostschemawilllooklikeafterthosechanges:
9.Manytomanyandupserts
53
defmoduleMyApp.Postdo
useEcto.Schema
#Schemaisthesame
schema"posts"do
add:title
add:body
many_to_many:tags,MyApp.Tag,join_through:"posts_tags"
timestamps()
end
#Changesetisthesame
defchangeset(struct,params\\%{})do
struct
|>Ecto.Changeset.cast(struct,[:title,:body])
|>Ecto.Changeset.put_assoc(:tags,parse_tags(params))
end
#Parsetagshasslightlychanged
defpparse_tags(params)do
(params["tags"]||"")
|>String.split(",")
|>Enum.map(&String.trim/1)
|>Enum.reject(&&1=="")
|>insert_and_get_all()
end
defpinsert_and_get_all([])do
[]
end
defpinsert_and_get_all(names)do
maps=Enum.map(names,&%{name:&1})
Repo.insert_allMyApp.Tag,names,on_conflict::nothing
Repo.allfromtinMyApp.Tag,where:t.namein^names
end
end
Insteadofattemptingtogetandinserteachtagindividually,thecodeaboveworkonalltagsatonce,firstbybuildingalistofmapswhichisgiventoinsert_allandthenbylookingupalltagswiththeexistingnames.Therefore,regardlessofhowmanytagsaresent,wewillperformonly2queries(unlessnotagissent,inwhichwereturnanemptylistbackpromptly).ThissolutionisonlypossibleinEcto2.1thankstothe:on_conflictoption,whichguaranteesinsert_allwon'tfailincaseagiventagnamealreadyexists.
Finally,keepinmindthatwehaven'tusedtransactionsinanyoftheexamplessofar.Suchdecisionwasdeliberate.Sincegettingorinsertingtagsisanidempotentoperation,i.e.wecanrepeatitmanytimesanditwillalwaysgiveusthesameresultback.Therefore,evenifwefailtointroducetheposttothedatabaseduetoavalidationerror,theuserwillbefreeto
9.Manytomanyandupserts
54
resubmittheformandwewilljustattempttogetorinsertthesametagsonceagain.Thedownsideofthisapproachisthattagswillbecreatedevenifcreatingthepostfails,whichmeanssometagsmaynothavepostsassociatedtothem.Incasethat'snotdesired,thewholeoperationcouldbewrappedinatransactionormodeledwiththeEcto.Multiabstractionwewilldiscussinthenextchapter.
9.Manytomanyandupserts
55
ComposabletransactionswithEcto.MultiEctoreliesondatabasetransactionswhenmultipleoperationsmustbeperformedatomically.TransactionscanbeperformedviatheRepo.transactionfunction:
Repo.transaction(fn->
mary|>Ecto.Changeset.change(balance:mary.balance-10)|>Repo.update!
john|>Ecto.Changeset.change(balance:john.balance+10)|>Repo.update!
end)
Whenweexpectbothoperationstosucceed,asabove,transactionsarequitestraight-forward.However,transactionsgetmorecomplicatediftheoperationscanfail:
Repo.transaction(fn->
casemary|>Ecto.Changeset.change(balance:mary.balance-10)|>Repo.updatedo
{:ok,mary}->
casejohn|>Ecto.Changeset.change(balance:john.balance+10)|>Repo.updatedo
{:ok,john}->
{mary,john}
{:error,changeset}->
Repo.rollback({:john,changeset})
end
{:error,changeset}->
Repo.rollback({:mary,changeset})
end
end)
Ontheotherhand,transactionsinEctocanbenestedarbitrarily.Forexample,imaginethetransactionaboveismovedintoitsownfunction,definedastransfer_money(mary,john,10),andbesidestransferringmoneywealsowanttologthetransfer:
Repo.transaction(fn->
casetransfer_money(mary,john,10)do
{:ok,{mary,john}}->
Repo.insert!(%Transfer{from:mary.id,to:john.id,amount:10})
{:error,{who,changeset}}->
Repo.rollback({who,changeset})
end
end)
Thesnippetaboverunsinatransactionandthencallstransfer_money/3thatalsorunsinatransaction.ThisworksbecauseEctoconvertsanynestedtransactionintosavepointsautomatically.Incaseaninnertransactionfails,itrollsbacktoitsspecificsavepoint.
10.ComposabletransactionswithEcto.Multi
56
Whilenestingtransactionscanbeimprovethecodereadabilitybybreakingalargetransactionsintomultiplesmallertransactions,thereisstillalotofboilerplateinvolvedinhandlingthesuccessandfailurescenarios.Furthermore,compositionisquitelimited,asalloperationsmuststillbeperformedinsidetransactionblocks.
Amoredeclarativeapproachwouldbetodefinealloperationswewanttoperforminatransactiondecoupledfromthetransactionexecution.Thiswaywewouldbeabletocomposetransactionsoperationswithoutworryingaboutitsexecutioncontextorabouteachindividualsuccess/failurescenario.That'sexactlywhatEcto.Multiallowsustobuild.
ComposingwithdatastructuresLet'srewritethesnippetsaboveusingEcto.Multi.Thefirstsnippetthattransfersmoneybetweenmaryandjohncanrewrittento:
Ecto.Multi.new
|>Ecto.Multi.update(:mary,Ecto.Changeset.change(mary,balance:mary.balance-10))
|>Ecto.Multi.update(:john,Ecto.Changeset.change(john,balance:john.balance+10))
Ecto.Multiisadatastructurethatallowsustodefinewhichoperationsmustbeperformedtogether,withoutworryingaboutwhereandhowitwillbeexecuted.Ecto.MultimirrorsmostoftheEcto.RepoAPI,withthedifferenceeachoperationmustbeexplicitlynamed.Intheexampleabove,wehavedefinedtwoupdateoperations,named:maryand:john.Aswewillseelater,thenamesareimportantwhenhandlingsuccessandfailureresults.
SinceEcto.Multiisjustadatastructure,wecanpassitasargumenttootherfunctions,aswellasreturnit.Assumingthemultiaboveismovedintoitsownfunction,definedastransfer_money(mary,john,10),wecannowaddanoperationthatlogsthetransferasfollows:
transfer_money(mary,john,10)
|>Ecto.Multi.insert(:transfer,%Transfer{from:mary.id,to:john.id,amount:10})
Thisisconsiderablysimplerthanthenestedtransactionapproachedwehaveseenearlier!Oncealloperationsaredefinedinthemulti,wecanfinallycallRepo.transaction,thistimepassingthemulti:
10.ComposabletransactionswithEcto.Multi
57
transfer_money(mary,john,10)
|>Ecto.Multi.insert(:transfer,%Transfer{from:mary.id,to:john.id,amount:10})
|>Repo.transaction()
|>casedo
{:ok,%{mary:mary,john:john,transfer:transfer}}->
#Handlesuccesscase
{:error,name,value,rolled_back_changes}->
#Handlefailurecase
end
Ifalloperationsinthemultisucceed,itreturns{:ok,map}wherethemapcontainsthenameofalloperationsaskeysandtheirsuccessvalue.Ifanyoperationinthemultifails,thetransactionisrolledbackandRepo.transactionreturns{:error,name,value,rolled_back_changes},wherenameisthenameofthefailedoperation,valueisthefailurevalueandrolled_back_changesisamapofthepreviouslysuccessfulmultioperationsthathavebeenrolledbackduetothefailure.
Inotherwords,Ecto.Multitakescareofalltheflowcontrolboilerplatewhiledecouplingthetransactiondefinitionfromitsexecution,allowingustocomposeoperationstrivially.
TestingAnotherupsideofusingEcto.Multiisthatwecanlatertraversealloperationsstoredinthemultiandusethisinformationtowritetests.Forexample,wecouldtestthemultireturnedbytransfer_money/3asfollows:
test"transfersfrommarytojohn"do
multi=transfer_money(mary,john,10)
assert[{:mary,{:update,mary_changeset,_}},
{:john,{:update,john_changeset,_}}]=Ecto.Multi.to_list(multi)
assertmary_changeset.changes.balance==mary.balance-10
assertjohn_changeset.changes.balance==mary.balance-10
end
DependentvaluesBesidesoperationssuchasinsert,updateanddelete,Ecto.Multialsoprovidesfunctionsforhandlingmorecomplexscenarios.Forexample,prependandappendcanbeusedtomergemultistogether.Andmoregenerally,theEcto.Multi.run/3canbeusedtodefineanyoperationthatdependsontheresultsofapreviousmultioperation.
10.ComposabletransactionswithEcto.Multi
58
Let'sstudyamorepracticalexamplebyrevisitingtheproblemdefinedinthepreviouschapter.Backthen,wewantedtomodifyapostwhilepossiblygivingitalistoftagsasastringseparatedbycommas.Attheendofthechapter,webuiltasolutionthatwouldinsertanymissingtagandthenfetchallofthemusingonlytwoqueries:
defmoduleMyApp.Postdo
useEcto.Schema
#Schemaisthesame
schema"posts"do
add:title
add:body
many_to_many:tags,MyApp.Tag,join_through:"posts_tags"
timestamps()
end
#Changesetisthesame
defchangeset(struct,params\\%{})do
struct
|>Ecto.Changeset.cast(struct,[:title,:body])
|>Ecto.Changeset.put_assoc(:tags,parse_tags(params))
end
#Parsetagshasslightlychanged
defpparse_tags(params)do
(params["tags"]||"")
|>String.split(",")
|>Enum.map(&String.trim/1)
|>Enum.reject(&&1=="")
|>insert_and_get_all()
end
defpinsert_and_get_all([])do
[]
end
defpinsert_and_get_all(names)do
maps=Enum.map(names,&%{name:&1})
Repo.insert_allMyApp.Tag,names,on_conflict::nothing
Repo.allfromtinMyApp.Tag,where:t.namein^names
end
end
Whileinsert_and_get_all/1isidempotent,allowingustorunitmultipletimesandgetthesameresultback,itdoesnotruninsideatransaction,soanyfailurewhileattemptingtomodifytheparentpoststructwouldend-upcreatingtagsthathasnopostsassociatedtothem.
Let'sfixtheproblemabovebyintroducingusingEcto.Multi.Let'sstartbysplittingthelogicintobothPostandTagmodulesandkeepingitfreefromside-effects:
10.ComposabletransactionswithEcto.Multi
59
defmoduleMyApp.Postdo
useEcto.Schema
schema"posts"do
add:title
add:body
many_to_many:tags,MyApp.Tag,join_through:"posts_tags"
timestamps()
end
defchangeset(struct,tags,params)do
struct
|>Ecto.Changeset.cast(struct,[:title,:body])
|>Ecto.Changeset.put_assoc(:tags,tags)
end
end
defmoduleMyApp.Tagdo
useEcto.Schema
schema"tags"do
add:name
timestamps()
end
defparse(tags)do
(tags||"")
|>String.split(",")
|>Enum.map(&String.trim/1)
|>Enum.reject(&&1=="")
end
end
Now,wheneverweneedtointroduceapostwithtags,wecancreateamultithatwrapsalloperationsandtherepositoryaccess:
10.ComposabletransactionswithEcto.Multi
60
definsert_or_update_post_with_tags(post,params)do
Ecto.Multi.new
|>Ecto.Multi.run(:tags,&insert_and_get_all_tags(&1,params))
|>Ecto.Multi.run(:post,&insert_or_update_post(&1,post,params)
|>Repo.transaction()
end
defpinsert_and_get_all_tags(_changes,params)do
caseMyApp.Tag.parse(params["tags"])do
[]->
[]
tags->
maps=Enum.map(names,&%{name:&1})
Repo.insert_all(MyApp.Tag,names,on_conflict::nothing)
Repo.all(fromtinMyApp.Tag,where:t.namein^names)
end
end
defpinsert_or_update_post(%{tags:tags},post,params)do
Repo.insert_or_updateMyApp.Post.changeset(post,tags,params)
end
IntheexampleabovewehaveusedEcto.Multi.run/3twice,albeitfortwodifferentreasons.
1. InEcto.Multi.run(:tags,...),weusedrun/3becauseweneedtoperformbothinsert_allandalloperations,andwhilethemultiexposesEcto.Multi.insert_all/4,itdoesnotyetexposeaEcto.Multi.all/3.WheneverweneedtoperformarepositoryoperationthatisnotsupposedbyEcto.Multi,wecanalwaysfallbacktorun/3
2. InEcto.Multi.run(:post,...),weusedrun/3becauseweneedtoaccessavalueofapreviousmultioperation.Thefirstargumentofthefunctiongiventorun/3isamapwiththeresultsoftheoperationsperformedsofar.Tograbthetagsreturnedinthepreviousstep,wesimplypatternmatchon%{tags:tags}oninsert_or_update_post
Whilerun/3isveryhandywhenitneedtogobeyondthefunctionalitiesprovidednativelybyEcto.Multi,ithasthedownsidethatoperationsdefinedwithEcto.Multi.run/3areopaque,andthereforetheycannotbetestedusingEcto.Multi.to_list/1aswedidintheprevioussection.Still,Ecto.Multiallowsustogreatlyreducecontrolflowlogicandboilerplatewhenworkingwithtransactions.
10.ComposabletransactionswithEcto.Multi
61
ConcurrenttestswiththeSQLSandboxOurlastchapterisaboutoneofthemostimportantfeaturesinEcto2.0:theconcurrentSQLsandbox.GivenElixir'scapabilityofusingallofthemachineresourcesavailable,theabilitytorunteststhattalktothedatabaseconcurrentlygivesdevelopersaloweffortopportunitytospeeduptheirtestsuiteby2x,4x,8xormoretimes,dependingonthenumberofcoresavailable.
WheneveryoustartanEctorepositoryinyoursupervisiontree,suchassupervisor(MyApp.Repo,[]),Ectostartsasupervisorwithaconnectionpool.Theconnectionpoolholdsmultipleopenconnectionstothedatabase.Wheneveryouwanttoperformadatabaseoperation,forexampleinawebrequest,Ectoautomaticallygetsaconnectionfromthepool,performstheoperationyourequested,andthenputstheconnectionbackinthepool.
Thismeansthat,whenwritingtestsusingEcto'sdefaultconnectionpool(andnottheSQLsandbox),eachtimeyourunaquery,youwilllikelygetadifferentconnectionfromthepool.Thisisnotgoodfortestssincewewantalloperationsinthesametesttousethesameconnection.
Notonlythat,wealsowantdataisolationbetweenthetests.IfIintroducearecordtothe"users"tableintestA,testBshouldnotseethoseentrieswhenqueryingthesame"users"table.
Ecto1.0solvedthefirstproblembysimplyforcingteststohaveonlyoneconnectioninthedatabasepool.Thesecondproblem,regardingdataisolation,wassolvedbynotallowingteststorunconcurrently!Whileitworked,wewereunabletoleverageconcurrency.
ExplicitcheckoutsEcto2.0solvedtheproblemsabovedifferently.Themainideaisthatwewillallowthepooltohavemultipleconnectionsbut,insteadoftheconnectionbeingcheckedoutimplicitlyeverytimewerunaquery,nowtheconnectionmustbeexplicitlycheckedoutbyeverytest.Thiswayweguaranteethateverytimeaconnectionisusedinatest,itisalwaysthesameconnection.
Onceaconnectionisexplicitlycheckedout,thetestnowownsthatparticularconnectionuntilthetestisover.
11.ConcurrenttestswiththeSQLSandbox
62
Let'sstartbysettingupourdatabasetotheuseEcto.Adapters.SQL.Sandboxpool.Youcansetthoseoptionsinyourconfig/config.exs(orpreferablyconfig/test.exs):
config:my_app,Repo,
pool:Ecto.Adapters.SQL.Sandbox
Bydefaultthesandboxpoolstartsin:automaticmode,whichisexactlyhowEctoworkswithouttheSQLsandboxpool.Inotherwords,theSQLsandboxisinitiallydisabled.Thisallowsustosetupthedatabase,forexamplebyrunningmigrationsorinyourtest/test_helper.exs,asusual.
Beforeourtestsstart,weneedtoconvertthepoolto:manualmode,whereeachconnectionmustbeexplicitlycheckedout.Wedosobycallingthemode/2function,typicallyattheendofthetest/test_helper.exsfile:
#Attheendofyourtest_helper.exs
#Setthepoolmodetomanualforexplicitcheckouts
Ecto.Adapters.SQL.Sandbox.mode(MyApp.Repo,:manual)
Ifyousimplyaddthelineaboveandyoudonotchangeyourteststoexplicitlycheckaconnectionoutfromthepool,allofyourtestswillnowfail.Tosolvethis,youcouldexplicitlycheckouttheconnectiononeachtestbut,toavoidrepetition,let'sdefineaExUnit.CaseTemplatethatautomaticallydoessoinsetup:
defmoduleMyApp.RepoCasedo
useExUnit.CaseTemplate
setupdo
#Explicitlygetaconnectionbeforeeachtest
:ok=Ecto.Adapters.SQL.Sandbox.checkout(MyApp.Repo)
end
end
Nowinyourtests,insteadofuseExUnit.Case,youmaywriteuseMyApp.RepoCase,async:true.Byfollowingthestepsabove,wearenowabletohavemultipletestsrunningconcurrently,eachowningaspecificdatabasetransaction.
However,youmaywonder,howdoesEctoguaranteesthatthedatageneratedinonetestdoesnotaffecttheothertests?
Transactions
11.ConcurrenttestswiththeSQLSandbox
63
ThesecondmaininsightbesidesexplicitlycheckoutsintheSQLSandboxistheideaofrunningeachexplicitlycheckedoutconnectioninsideatransactions.EverytimeyourunEcto.Adapters.SQL.Sandbox.checkout(MyApp.Repo)inatest,itdoesnotonlycheckoutaconnectionbutitalsoguaranteesthatconnectionhasopenedatransactiontothedatabase.Thisway,anyinsert,updateordeleteyouperforminyourtestswillbevisibleonlytothattest.
Furthermore,attheendofeverytest,weautomaticallyrollbackthetransaction,effectivelyrevertingallofthedatabasechangesyouhaveperformedinyourtests.Thisguaranteesatestwon'taffecttestsrunningconcurrentlynorteststhatmayrunsubsequently.
Whiletheapproachofusingmultipleconnectionswithtransactionsworksinmanycases,italsoimposessomelimitationsrelatedtohowthedatabaseenginemanagestransactionsandperformsconcurrencycontrol.Forexample,whilebothPostgreSQLandMySQLsupportSQLSandbox,onlyPostgreSQLsupportsconcurrenttestswhilerunningtheSQLSandbox.Therefore,donotuseasync:truewithMySQLasyoumayrunintodeadlocks.
ThereisalsoachanceofrunningintodeadlockswhenrunningtestswithPostgreSQLwhenitcomestosharedresources,suchasdatabaseindexes.ButthosecasesarewelldocumentedintheEcto.Adapters.SQL.Sandbox,undertheFAQsection.
OwnershipWheneveratestexplicitlychecksoutaconnectionfromtheSQLSandboxpool,wesaythetestprocessownstheconnection.Alsorememberthatifatest,oranyotherprocess,doesnotexplicitlycheckaconnectionoutofthepool,thattestwillerrorwithanerrormessagesayingithasnodatabaseconnection.
Let'sseeanexample:
useMyApp.RepoCase,async:true
test"createtwoposts,onesync,anotherasync"do
task=Task.async(fn->
Repo.insert!(%Post{title:"async"})
end)
assert%Post{}=Repo.insert!(%Post{title:"sync"})
assert%Post{}=Task.await(task)
end
Thetestabovewillfailwithanerrorsimilarto:
**(RuntimeError)cannotfindownershipprocessfor#PID<0.35.0>
11.ConcurrenttestswiththeSQLSandbox
64
OncewespawnaTask,thereisnoconnectionassignedtothetaskprocess,causingittofail.
Whilemosttimeswewantdifferentprocessestohavetheirowndatabaseconnection,sometimesatestmayneedtointeractwithmultipleprocesses,allusingthesameconnectionsotheyallbelongtothesametransaction.
Thesandboxmoduleprovidestwowaysofdoingso,viaallowancesorbyrunninginsharedmode.
Allowances
Ifaprocessexplicitlyownsaconnection,thatprocessmayalsoallowotherprocessestousethatconnection.Effectivelyallowingmultipleprocessestocollaborateoverthesameconnectionatthesametime.Let'sgiveitatry:
test"createtwoposts,onesync,anotherasync"do
parent=self()
task=Task.async(fn->
Ecto.Adapters.SQL.Sandbox.allow(Repo,parent,self())
Repo.insert!(%Post{title:"async"})
end)
assert%Post{}=Repo.insert!(%Post{title:"sync"})
assert%Post{}=Task.await(task)
end
Andthat'sit!Bycallingallow/3,weareexplicitlyassigningtheparent'sconnection(i.e.thetestprocess'connection)tothetask.
Becauseallowancesuseanexplicitmechanism,theiradvantageisthatyoucanstillrunyourtestsinasyncmode.Thedownsideisthatyouneedtoexplicitlycontrolandalloweverysingleprocess,whichisnotalwayspossible.Insuchcases,youmayresorttosharedmode.
Sharedmode
Sharedmodeallowsaprocesstoshareitsconnectionwithanyotherprocessautomatically,withoutrelyingonexplicitallowances.
Let'schangetheexampleabovetousesharedmode:
11.ConcurrenttestswiththeSQLSandbox
65
test"createtwoposts,onesync,anotherasync"do
#Settingthesharedmodemustbedoneonlyaftercheckout
Ecto.Adapters.SQL.Sandbox.mode(Repo,{:shared,self()})
task=Task.async(fn->
Repo.insert!(%Post{title:"async"})
end)
assert%Post{}=Repo.insert!(%Post{title:"sync"})
assert%Post{}=Task.await(task)
end
Bycallingmode({:shared,self()}),anyprocessthatneedstotalktothedatabasewillnowusethesameconnectionastheonecheckedoutbythetestprocess.
Theadvantageofsharedmodeisthatbycallingasinglefunction,youwillensureallupcomingprocessesandoperationswillusethatsharedconnection,withoutaneedtoexplicitlyallowthem.Thedownsideisthattestscannolongerrunconcurrentlyinsharedmode.
SummingupInthischapterwehavelearnedaboutthepowerfulconcurrentSQLsandboxandhowitleveragestransactionsandanownershipmechanismwithexplicitcheckoutsthatallowsteststorunconcurrentlyevenwhentheyneedtocommunicatetothedatabase.
Wehavealsodiscussedtwomechanismforsharingownerships:
Usingallowances-requiresexplicitallowancesviaallow/3.Testsmayrunconcurrently.
Usingsharedmode-doesnotrequireexplicitallowances.Testscannotrunconcurrently.
WhilethroughoutthebookwecoveredhowEctoisacollectionoftoolsforworkingonyourdomain,thelastchaptersalsoshowedEctoprovidestoolstobetterinteracttothedatabase,suchasEcto.MultiwhichleveragesthefunctionalpropertiesbehindElixir,aswellastheSQLSandboxwhichexploitstheconcurrencypowerbehindtheErlangVM.
Wehopeyouhavelearnedalotthroughoutthisjourneyandthatyouarereadytowriteclean,performantandmaintainableapplications.
11.ConcurrenttestswiththeSQLSandbox
66
11.ConcurrenttestswiththeSQLSandbox
67
CONTACTUS
11.ConcurrenttestswiththeSQLSandbox
68