agentzh's Nginx Tutorials (ver 2016.07.21) - OpenResty · agentzh's Nginx Tutorials (version 2016.07.21) Table of Contents Foreword Writing Plan for the Tutorials Nginx Variables

agentzh'sNginxTutorials(version2016.07.21)TableofContents

ForewordWritingPlanfortheTutorialsNginxVariables(01)NginxVariables(02)NginxVariables(03)NginxVariables(04)NginxVariables(05)NginxVariables(06)NginxVariables(07)NginxVariables(08)NginxDirectiveExecutionOrder(01)NginxDirectiveExecutionOrder(02)NginxDirectiveExecutionOrder(03)NginxDirectiveExecutionOrder(04)NginxDirectiveExecutionOrder(05)NginxDirectiveExecutionOrder(06)NginxDirectiveExecutionOrder(07)NginxDirectiveExecutionOrder(08)NginxDirectiveExecutionOrder(09)NginxDirectiveExecutionOrder(10)

ForewordI'vebeendoingalotofworkintheNginxworldoverthelastfewyearsandI'vealsobeenthinkingaboutwritingaseriesoftutorial-likearticlestoexplaintomorepeoplewhatI'vedoneandwhatI'velearnedinthisarea.NowIhavefinallydecidedtopostserialarticlestotheSinaBloghttp://blog.sina.com.cn/openrestyinChinese.Everyarticlewillroughlycoverasingletopicandwillbeinarathercasualstyle.ButatsomepointinthefutureImayrestructurethearticlesandtheirstyleinordertoturnthemintoa"real"book.

Thearticlesaredividedintoseries.Forexample,thefirstseriesis"NginxVariables".EachseriescanbethoughtofasmappingtoachapterintheNginxbookthatImaypublishinthefuture.

ThearticlesareintendedforNginxusersofallexperiencelevels,includinguserswithextensiveApacheandLighttpdexperiencewhomayhaveneverusedNginxbefore.

TheexamplesinthearticlesareatleastcompatiblewithNginx0.8.54.DonottrytheexampleswitholderversionsofNginx.ThelateststableversionofNginxasofthiswritingis1.7.9.

AlloftheNginxmodulesreferencedinthearticlesareproduction-ready.IwillnotbecoveringanyNginxcoremodulesthatareeitherexperimentalorbuggy.Additionally,Iwillbemakingextensiveuseof3rd-partyNginxmodulesintheexamples.Ifit'sinconvenientforyoutodownloadandinstalltheindividualmodulesoneatatimethenIhighlyrecommendthatyoudownloadandinstallthengx_openrestysoftwarebundlethatImaintain.

http://openresty.org/

Allofthemodulesreferencedinthearticles,includingthecoreNginxmodulesthatarenew(butstable),areincludedintheOpenRestybundle.

AprinciplethatIwillbetryingtoadheretoistousesmallconciseexamplestoexplainandvalidatetheconceptsandbehaviorsbeingdescribed.Myhopeisthatitwillhelpthereadertodevelopthegoodhabitofnotacceptingothers'viewpointsorstatementsatfacevaluewithouttestingthemfirst.ThisapproachmayhavesomethingtodowithmyQAbackground.Infact,Ikeeptweakingandcorrectingthearticlesbasedontheresultsofrunningtheexampleswhilewriting.

Theexamplesinthearticlesfallintooneoftwocategories,goodandproblematic.ThepurposeoftheproblematicexamplesistohighlightpotentialpitfallsandotherareaswhereNginxoritsmodulesbehaveinwaysthatreadersmaynotexpect.Problematicexamplesareeasytoidentifybecauseeachlineoftextintheexamplewillbeprefixedwithaquestionmark,i.e.,"?".Hereisanexample:

?server{

?listen8080;

?

?location/bad{

?echo$foo;

?}

?}

Donotreproducethesearticleswithoutexplicitpermissionsfromus.Copyrightreserved.

Iencouragereaderstosendfeedback([email protected]),especiallyconstructivecriticism.

ThesourceforallthearticlesisonGitHub:

http://github.com/agentzh/nginx-tutorials/

Thesourcefilesareundertheen/directory.IamusingalittlemarkuplanguagethatisamixtureofWikiandPODtowritethesearticles.Theyarethe.tutfiles.Youarewelcometocreateforksand/orprovidepatches.

Thee-booksfilesthataresuitableforcellphones,Kindle,iPad/iPhone,SonyReaders,andotherdevicescanbedownloadedfromhere:

http://openresty.org/#eBooks

SpecialthanksgotoKaiWu(kai10k)whokindlytranslatesthesearticlestoEnglish.

agentzhathomeintheFuzhoucity

October30,2011

http://blog.sina.com.cn/openresty

http://openresty.org/

http://github.com/agentzh/nginx-tutorials/

http://openresty.org/#eBooks

WritingPlanfortheTutorialsHereliststhetutorialseriesthathavealreadybeenpublishedortobepublished.

GettingStartedwithNginxHowNginxMatchesURIsNginxVariablesNginxDirectiveExecutionOrderNginx'sifisEvilNginxSubrequestsNginxStaticFileServicesNginxLogServicesApplicationGatewaysbasedonNginxReverse-ProxiesbasedonNginxNginxandMemcachedNginxandRedisNginxandMySQLNginxandPostgreSQLApplicationcachingBasedonNginxSecurityandAccessControlinNginxWebServicesBasedonNginxAJAXApplicationsDrivenbyNginxPerformanceTestingforNginxanditsApplicationsStrengthoftheNginxCommunity

TheseriesnamescanroughlycorrespondtothechapternamesinmyfinalNginxbook,buttheyareunlikelytostayexactlythesame.Theactualseriesnamesmaychangeandtherelativeorderoftheseriesmaychangeaswell.

Thelistabovewillbeconstantlyupdatedtoalwaysreflectthelatestplan.

Variablesarevaluecontainers

NginxVariables(01)VariablesasValueContainers

Nginx'sconfigurationfilesuseamicroprogramminglanguage.Manyreal-worldNginxconfigurationfilesareessentiallysmallprograms.Thislanguage'sdesignisheavilyinfluencedbyPerlandBourneShellasfarasIcansee,despitethefactthatitmightnotbeTuring-Completeanditisdeclarativeinmanyplaces.ThisisadistinguishingfeatureofNginx,ascomparedtootherwebserverslikeApacheorLighttpd.Beingaprogramminglanguage,"variables"arethusanaturalpartofit(exceptionsdoexist,ofcourse,asinpurefunctionallanguageslikeHaskell).

VariablesarejustcontainersholdingvariousvaluesinimperativelanguageslikePerl,BourneShell,andC/C++.And"values"canbenumberslike3.14,stringslikehelloworld,orevencomplicatedthingslikereferencestoarraysorhashtablesinthoselanguages.FortheNginxconfigurationlanguage,however,variablescanholdonlyonetypeofvalues,thatis,strings(thereisaninterestingexception:the3rd-partymodulengx_array_varextendsNginxvariablestoholdarrays,butitisimplementedbyencodingaCpointerasabinarystringvaluebehindthescene).

http://wiki.nginx.org/HttpArrayVarModule

VariableSyntaxandInterpolation

Let'ssayournginx.confconfigurationfilehasthefollowingline:

set$a"helloworld";

Weassignavaluetothevariable$aviathesetconfigurationdirectivecomingfromthestandardngx_rewritemodule.Inparticular,weassignthestringvaluehelloworldto$a.

WecanseethattheNginxvariablenametakesadollarsign($)infrontofit.Thisisrequiredbythelanguagesyntax:wheneverwewanttoreferenceanNginxvariableintheconfigurationfile,wemustadda$prefix.ThislooksveryfamiliartothosePerlandPHPprogrammers.

SuchvariableprefixmodifiersmaydiscomfortsomeJavaandC#programmers,thisnotationdoeshaveanobviousadvantagethough,thatis,variablescanbeembeddeddirectlyintoastringliteral:

set$ahello;

set$b"$a,$a";

HereweusethevalueoftheexistingNginxvariable$atoconstructthevalueforthevariable$b.Soafterthesetwodirectivescompleteexecution,thevalueof$aishello,and$bishello,hello.Thistechniqueiscalled"variableinterpolation"inthePerlworld,whichmakesad-hocstringconcatenationoperatorsnolongerthatnecessary.Let'susethesametermfortheNginxworldfromnowon.

Let'sseeanothercompleteexample:

server{

listen8080;

location/test{

set$foohello;

echo"foo:$foo";

}

}

Thisexampleomitsthehttpdirectiveandeventsconfigurationblocksintheouter-mostscopeforbrevity.Torequestthis/testinterfaceviacurl,anHTTPclientutility,onthecommandline,weget

$curl'http://localhost:8080/test'

foo:hello

Hereweusetheechodirectiveofthe3rdpartymodulengx_echotoprintoutthevalueofthe$foovariableastheHTTPresponse.

Apparentlytheargumentsoftheechodirectivedoessupport"variableinterpolation",butwecannottakeitforgrantedforotherdirectives.Becausenotalltheconfigurationdirectivessupport"variableinterpolation"anditisinfactuptotheimplementationofthedirectiveinthatmodule.Alwayslookupthedocumentationtobesure.

Escaping"$"

We'vealreadylearnedthatthe$characterisspecialanditservesasthevariablenameprefix,butnowconsiderthatwewanttooutputaliteral$characterviatheechodirective.Thefollowingnaiveexampledoesnotworkatall:

?:nginx

?location/t{

?echo"$";

?}

Wewillgetthefollowingerrormessagewhileloadingthisconfiguration:

[emerg]invalidvariablenamein...

ObviouslyNginxtriestoparse$"asavariablename.Isthereawaytoescape$inthestringliteral?Theansweris"no"(itisstillthecaseinthelatestNginxstablerelease1.2.7)andIhavebeenhopingthatwecouldwritesomethinglike$$toobtainaliteral$.

Luckily,workaroundsdoexistandhereisoneproposedbyMaximDounin:firstweassigntoavariablealiteralstringcontainingadollarsigncharacterviaaconfigurationdirectivethatdoesnotsupport"variableinterpolation"(rememberthatnotallthedirectivessupport"variableinterpolation"?),andthenreferencethisvariablelaterwheneverweneedadollarsign.Hereissuchanexampletodemonstratetheidea:

geo$dollar{

default"$";

}

server{

listen8080;

location/test{

echo"Thisisadollarsign:$dollar";

}

}

Let'stestitout:


Thisisadollarsign:$

http://wiki.nginx.org/HttpRewriteModule#set

http://wiki.nginx.org/HttpRewriteModule

http://wiki.nginx.org/HttpEchoModule#echo

http://wiki.nginx.org/HttpEchoModule



Herewemakeuseofthegeodirectiveofthestandardmodulengx_geotoinitializethe$dollarvariablewiththestring"$",thereaftervariable$dollarcanbeusedinplacesthatrequireadollarsign.Thisworksbecausethegeodirectivedoesnotsupport"variableinterpolation"atall.However,thengx_geomoduleisoriginallydesignedtosetaNginxvariabletodifferentvaluesaccordingtotheremoteclientaddress,andinthisexample,wejustabuseittoinitializethe$dollarvariablewiththestring"$"unconditionally.

DisambiguatingVariableNames

Thereisaspecialcasefor"variableinterpolation",thatis,whenthevariablenameisfolloweddirectlybycharactersallowedinvariablenames(likeletters,digits,andunderscores).Insuchcases,wecanuseaspecialnotationtodisambiguatethevariablenamefromthesubsequentliteralcharacters,forinstance,

server{

listen8080;

location/test{

set$first"hello";

echo"${first}world";

}

}

Herethevariable$firstisconcatenatedwiththeliteralstringworld.Ifitwerewrittendirectlyas"$firstworld",Nginx's"variableinterpolation"engine(alsoknownasthe"scriptengine")wouldtrytoaccessthevariable$firstworldinsteadof$first.Toresolvetheambiguityhere,curlybracesmustbeusedaroundthevariablename(excludingthe$prefix),asin${first}.Let'stestthissample:

$curl'http://localhost:8080/test

helloworld

http://wiki.nginx.org/HttpGeoModule#geo

http://wiki.nginx.org/HttpGeoModule



VariableDeclarationandCreation

InlanguageslikeC/C++,variablesmustbedeclared(orcreated)beforetheycanbeusedsothatthecompilercanallocatestorageandperformtypecheckingatcompile-time.Similarly,NginxcreatesalltheNginxvariableswhileloadingtheconfigurationfile(orinotherwords,at"configurationtime"),thereforeNginxvariablesarealsorequiredtobedeclaredsomehow.

FortunatelythesetdirectiveandthegeodirectivementionedabovedohavethesideeffectofdeclaringorcreatingNginxvariablesthattheywillassignvaluestolaterat"requesttime".Ifwedonotdeclareavariablethiswayanduseitdirectlyin,say,theechodirective,wewillgetanerror.Forexample,

?server{

?listen8080;

?

?location/bad{

?echo$foo;

?}

?}

Herewedonotdeclarethe$foovariableandaccessitsvaluedirectlyinecho.Nginxwilljustrefuseloadingthisconfiguration:

[emerg]unknown"foo"variable

Yes,wecannotevenstarttheserver!

Nginxvariablecreationandassignmenthappenatcompletelydifferentphasesalongthetime-line.VariablecreationonlyoccurswhenNginxloadsitsconfiguration.Ontheotherhand,variableassignmentoccurswhenrequestsareactuallybeingserved.ThisalsomeansthatwecannevercreatenewNginxvariablesat"requesttime".





VariableScope

OnceanNginxvariableiscreated,itisvisibletotheentireconfiguration,evenacrossdifferentvirtualserverconfigurationblocks,regardlessoftheplacesitisdeclaredat.Hereisanexample:

server{

listen8080;

location/foo{

echo"foo=[$foo]";

}

location/bar{

set$foo32;

echo"foo=[$foo]";

}

}

Herethevariable$fooiscreatedbythesetdirectivewithinlocation/bar,andthisvariableisvisibletotheentireconfiguration,thereforewecanreferenceitinlocation/foowithoutworries.Belowistheresultoftestingthesetwointerfacesviathecurltool.

$curl'http://localhost:8080/foo'

foo=[]

$curl'http://localhost:8080/bar'

foo=[32]


foo=[]

Wecanseethattheassignmentoperationisonlyperformedinrequeststhataccesslocation/bar,sincethecorrespondingsetdirectiveisonlyusedinthatlocation.Whenrequestingthe/foointerface,wealwaysgetanemptyvalueforthe$foovariablebecausethatiswhatwegetwhenaccessinganuninitializedvariable.

AnotherimportantcharacteristicthatwecanobservefromthisexampleisthateventhoughthescopeofNginxvariablesistheentireconfiguration,eachrequestdoeshaveitsownversionofallthosevariables'containers.Requestsdonotinterferewitheachothereveniftheyarereferencingavariablewiththesamename.ThisisverymuchlikelocalvariablesinC/C++functionbodies.EachinvocationoftheC/C++functiondoesuseitsownversionofthoselocalvariables(onthestack).

Forinstance,inthissample,werequest/barandthevariable$foogetsthevalue32,whichdoesnotaffectthevalueof$fooinsubsequentrequeststo/foo(itisstilluninitialized!),becausetheycorrespondtodifferentvaluecontainers.

OnecommonmistakeforNginxnewcomersistoregardNginxvariablesassomethingsharedamongalltherequests.EventhoughthescopeofNginxvariablenamesgoacrossconfigurationblocksat"configurationtime",itsvaluecontainer'sscopenevergoesbeyondrequestboundariesat"requesttime".Essentiallyherewedohavetwodifferentkindsofscopehere.



NginxVariables(02)VariableLifetime&InternalRedirection

WealreadyknowthatNginxvariablesareboundtoeachrequesthandledbyNginx,forthisreasontheyhaveexactlythesamelifetimeasthecorrespondingrequest.

Thereisanothercommonmisunderstandingherethough:somenewcomerstendtoassumethatthelifetimeofNginxvariablesisboundtothelocationconfigurationblock.Let'sconsiderthefollowingcounterexample:

server{

listen8080;

location/foo{

set$ahello;

echo_exec/bar;

}

location/bar{

echo"a=[$a]";

}

}

Hereinlocation/fooweusetheecho_execdirective(providedbythe3rd-partymodulengx_echo)toinitiatean"internalredirection"tolocation/bar.The"internalredirection"isanoperationthatmakesNginxjumpfromonelocationtoanotherwhileprocessingarequest.This"jumping"happenscompletelywithintheserveritself.Thisisdifferentfromthose"externalredirections"basedontheHTTP301and302responsesbecausethelatteriscollaboratedexternally,bytheHTTPclients.Also,incaseof"externalredirections",theendusercouldusuallyobservethechangeoftheURLinherwebbrowser'saddressbarwhilethisisnotthecaseforinternalones."Internalredirections"areverysimilartotheexeccommandinBourneShell;itisa"onewaytrip"andneverreturns.AnothersimilarexampleisthegotostatementintheClanguage.

Beingan"internalredirection",therequestaftertheredirectionremainstheoriginalone.Itisjustthecurrentlocationthatischanged,sowearestillusingtheoriginalcopyoftheNginxvariablecontainers.Backtoourexample,thewholeprocesslookslikethis:Nginxfirstassignstothe$avariablethestringvaluehelloviathesetdirectiveinlocation/foo,andthenitissuesaninternalredirectionviatheecho_execdirective,thusleavinglocation/fooandenteringlocation/bar,andfinallyitoutputsthevalueof$a.Becausethevaluecontainerof$aremainsuntouched,wecanexpecttheresponseoutputtobehello.Thetestresultconfirmsthis:

$curllocalhost:8080/foo

a=[hello]

Butwhenaccessing/bardirectlyfromtheclientside,wewillgetanemptyvalueforthe$avariable,sincethisvariablereliesonlocation/footogetinitialized.

Itcanbeobservedthatduringarequest'slifetime,thecopyofNginxvariablecontainersdoesnotchangeatallevenwhenNginxgoesacrossdifferentlocationconfigurationblocks.Herewealsoencountertheconceptof"internalredirections"forthefirsttimeandit'sworthmentioningthattherewritedirectiveofthengx_rewritemodulecanalsobeusedtoinitiate"internalredirections".Forinstance,wecanrewritetheexampleabovewiththerewritedirectiveasfollows:

server{

listen8080;

location/foo{

set$ahello;

rewrite^/bar;

}

location/bar{

echo"a=[$a]";

}

}

It'sfunctionallyequivalenttoecho_exec.Wewilldiscusstherewritedirectiveinmoredepthinlaterchapters,likeinitiating"externalredirections"like301and302.

Toconclude,thelifetimeofNginxvariablecontainersisindeedboundtotherequestbeingprocessed,andisirrelevanttolocation.

http://wiki.nginx.org/HttpEchoModule#echo_exec




http://wiki.nginx.org/HttpRewriteModule#rewrite





NginxBuilt-inVariables

TheNginxvariableswehaveseensofarareall(implicitly)createdbydirectiveslikeset.Weusuallycallsuchvariables"user-definedvaraibles",orsimply"uservariables".ThereisalsoanotherkindofNginxvariablesthatarepre-definedbyeithertheNginxcoreorNginxmodules.Let'scallthiskindofvariables"built-invariables".

$uri&$request_uri

OnecommonuseofNginxbuilt-invariablesistoretrievevarioustypesofinformationaboutthecurrentrequestorresponse.Forinstance,thebuilt-invariable$uriprovidedbyngx_http_coreisusedtofetchthe(decoded)URIofthecurrentrequest,excludinganyquerystringarguments.Anotherexampleisthe$request_urivariableprovidedbythesamemodule,whichisusedtofetchtheraw,non-decodedformoftheURI,includinganyquerystring.Let'slookatthefollowingexample.

location/test{

echo"uri=$uri";

echo"request_uri=$request_uri";

}

Weomittheserverconfigurationblockhereforbrevity.Justasallthosesamplesabove,westilllistentothe8080localport.Inthisexample,weoutputboththe$uriand$request_uriintotheresponsebody.Belowistheresultoftestingthis/testinterfacewithdifferentrequests:


uri=/test

request_uri=/test

$curl'http://localhost:8080/test?a=3&b=4'

uri=/test

request_uri=/test?a=3&b=4

$curl'http://localhost:8080/test/hello%20world?a=3&b=4'

uri=/test/helloworld

request_uri=/test/hello%20world?a=3&b=4

VariableswithInfiniteNames

Thereisanotherverycommonbuilt-invariablethatdoesnothaveafixedvariablename.Instead,Ithasinfinitevariations.Thatis,allthosevariableswhosenameshavetheprefixarg_,like$arg_fooand$arg_bar.Let'sjustcallitthe$arg_XXX"variablegroup".Forexample,the$arg_namevariableisevaluatedtothevalueofthenameURIargumentforthecurrentrequest.Also,theURIargument'svalueobtainedhereisnotdecodedyet,potentiallycontainingthe%XXsequences.Let'scheckoutacompleteexample:

location/test{

echo"name:$arg_name";

echo"class:$arg_class";

}

ThenwetestthisinterfacewithvariousdifferentURIargumentcombinations:


name:

class:

$curl'http://localhost:8080/test?name=Tom&class=3'

name:Tom

class:3

$curl'http://localhost:8080/test?name=hello%20world&class=9'

name:hello%20world

class:9

Infact,$arg_namedoesnotonlymatchthenameargumentname,butalsoNAMEorevenName.Thatis,thelettercasedoesnotmatterhere:

$curl'http://localhost:8080/test?NAME=Marry'

name:Marry

class:

$curl'http://localhost:8080/test?Name=Jimmy'

name:Jimmy

class:

Behindthescene,NginxjustconvertstheURIargumentnamesintothepurelower-caseformbeforematchingagainstthenamespecifiedby$arg_XXX.

Ifyouwanttodecodethespecialsequenceslike%20intheURIargumentvalues,thenyoucouldusetheset_unescape_uridirectiveprovidedbythe3rd-partymodulengx_set_misc.

location/test{

set_unescape_uri$name$arg_name;

set_unescape_uri$class$arg_class;

echo"name:$name";

echo"class:$class";

}

Let'scheckouttheactualeffect:

http://wiki.nginx.org/HttpRewiteModule#set

http://wiki.nginx.org/HttpCoreModule#.24uri

http://nginx.org/en/docs/http/ngx_http_core_module.html

http://wiki.nginx.org/HttpCoreModule#.24request_uri



http://wiki.nginx.org/HttpCoreModule#.24arg_PARAMETER


http://wiki.nginx.org/HttpSetMiscModule#set_unescape_uri

http://wiki.nginx.org/HttpSetMiscModule

$curl'http://localhost:8080/test?name=hello%20world&class=9'

name:helloworld

class:9

Thespacehasindeedbeendecoded!

Anotherthingthatwecanobservefromthisexampleisthattheset_unescape_uridirectivecanalsoimplicitlycreateNginxuser-definedvariables,justlikethesetdirective.Wewilldiscussthengx_set_miscmoduleinmoredetailinfuturechapters.

Thistypeofvariableslike$arg_XXXpossessesinfinitenumberofpossiblenames,sotheydonotcorrespondtoanyvaluecontainers.Furthermore,suchvariablesarehandledinaveryspecificwaywithintheNginxcore.Itisthusnotpossiblefor3rd-partymodulestointroducesuchmagicalbuilt-invariablesoftheirown.

TheNginxcoreoffersalotofsuchbuilt-invariablesinadditionto$arg_XXX,likethe$cookie_XXXvariablegroupforfetchingHTTPcookievalues,the$http_XXXvariablegroupforfetchingrequestheaders,aswellasthe$sent_http_XXXvariablegroupforretrievingresponseheaders.Wewillnotgointothedetailsforeachofthemhere.Interestedreaderscanrefertotheofficialdocumentationforthengx_http_coremodule.

Read-onlyBuilt-inVariables

Alltheuser-definedvariablesarewritable.Actuallythewaythatwedeclareorcreatesuchvariablessofaristouseaconfiguredirective,likeset,thatperformsvalueassignmentatrequesttime.Butitisnotnecessarilythecaseforbuilt-invariables.

Mostofthebuilt-invariablesareeffectivelyread-only,likethe$uriand$request_urivariablesthatwejustintroducedearlier.Assignmentstosuchread-onlyvariablesmustalwaysbeavoided.Otherwiseitwillleadtounexpectedconsequences,forexample,

?location/bad{

?set$uri/blah;

?echo$uri;

?}

ThisproblematicconfigurationjusttriggersaconfusingerrormessagewhenNginxisstarted:

[emerg]theduplicate"uri"variablein...

Attemptsofwritingtosomeotherread-onlybuilt-invariableslike$arg_XXXwilljustleadtoservercrashesinsomeparticularNginxversions.






http://wiki.nginx.org/HttpCoreModule#.24cookie_COOKIE

http://wiki.nginx.org/HttpCoreModule#.24http_HEADER

http://wiki.nginx.org/HttpCoreModule#.24sent_http_HEADER






NginxVariables(03)WritableBuilt-inVariable$args

Somebuilt-invariablesarewritableaswell.Forinstance,whenreadingthebuilt-invariable$args,wegettheURLquerystringofthecurrentrequest,butwhenwritingtoit,weareeffectivelymodifyingthequerystring.Hereissuchanexample:

location/test{

set$orig_args$args;

set$args"a=3&b=4";

echo"originalargs:$orig_args";

echo"args:$args";

}

HerewefirstsavetheoriginalURLquerystringintoourownvariable$orig_args,thenmodifythecurrentquerystringbyoverridingthe$argsvariable,andfinallyoutputthevariables$orig_argsand$args,respectively,withtheechodirective.Let'stestitlikethis:


originalargs:

args:a=3&b=4

$curl'http://localhost:8080/test?a=0&b=1&c=2'

originalargs:a=0&b=1&c=2

args:a=3&b=4

Inthefirsttest,wedidnotprovideanyURLquerystring,hencetheemptyoutputforthe$orig_argsvariable.Andinbothtests,thecurrentquerystringwasforciblyoverriddentothenewvaluea=3&b=4,regardlessofthepresenceofaquerystringintheoriginalrequest.

Itshouldbenotedthatthe$argsvariableherenolongerownsavaluecontainerasuservariables,justlike$arg_XXX.Whenreading$args,Nginxwillexecuteaspecialpieceofcode,fetchingdatafromaparticularplacewheretheNginxcorestorestheURLquerystringforthecurrentrequest.Ontheotherhand,whenweoverwrite$args,Nginxwillexecuteanotherspecialpieceofcode,storingnewvalueintothesameplaceinthecore.OtherpartsofNginxalsoreadthesameplacewheneverthequerystringisneeded,soourmodificationto$argswillimmediatelyaffectalltheotherparts'functionalitylateron.Let'sseeanexampleforthis:

location/test{

set$orig_a$arg_a;

set$args"a=5";

echo"originala:$orig_a";

echo"a:$arg_a";

}

Herewefirstsavethevalueofthebuilt-invaraible$arg_a,thevalueoftheoriginalrequest'sURLargumenta,intoouruservariable$orig_a,thenchangetheURLquerystringtoa=5byassigningthenewvaluetothebuilt-invariable$args,andfinallyoutputthevariables$orig_aand$arg_a,respectively.Becausemodificationsto$argseffectivelychangetheURLquerystringofthecurrentrequestforthewholeserver,thevalueofthebuilt-invariable$arg_XXXshouldalsochangeaccordingly.Thetestresultverifiesthis:

$curl'http://localhost:8080/test?a=3'

originala:3

a:5

Wecanseethattheinitialvalueof$arg_ais3sincetheURLquerystringoftheoriginalrequestisa=3.Butthefinalvalueof$arg_aautomaticallybecomes5afterwemodify$argswiththevaluea=5.

Belowisanotherexampletodemonstratethatassignmentsto$argsalsoaffecttheHTTPproxymodulengx_proxy.

server{

listen8080;

location/test{

set$args"foo=1&bar=2";

proxy_passhttp://127.0.0.1:8081/args;

}

}

server{

listen8081;

location/args{

echo"args:$args";

}

}

Twovirtualserversaredefinedhereinthehttpconfigurationblock(omittedforbrevity).

Thefirstvirtualserverislisteningatthelocalport8080.Its/testlocationfirstupdatesthecurrentURLquerystringtothevaluefoo=1&bar=2bywritingto$args,thensetsupanHTTPreverseproxyviatheproxy_passdirectiveofthengx_proxymodule,targetingtheHTTPservice/argsonthelocalport8081.Bydefaultthengx_proxymoduleautomaticallyforwardsthecurrentURLquerystringtotheremoteHTTPservice.

The"remoteHTTPservice"onthelocalport8081isprovidedbythesecondvirtualserverdefinedbyourselves,whereweoutputthecurrentURLquerystringviatheechodirectiveinlocation/args.Bydoingthis,wecaninvestigatetheactualURLquerystringforwardedbythengx_proxymodulefromthefirstvirtualserver.

http://wiki.nginx.org/HttpCoreModule#.24args













http://wiki.nginx.org/HttpProxyModule


http://wiki.nginx.org/HttpProxyModule#proxy_pass





Let'saccessthe/testinterfaceexposedbythefirstvirtualserver.

$curl'http://localhost:8080/test?blah=7'

args:foo=1&bar=2

WecanseethattheURLquerystringisfirstrewrittentofoo=1&bar=2eventhoughtheoriginalrequesttakesthevalueblah=7,thenitisforwardedtothe/argsinterfaceofthesecondvirtualserverviatheproxy_passdirective,andfinallyitsvalueisoutputtotheclient.

Tosummarize,theassignmentto$argsalsosuccessfullyinfluencesthebehaviorofthengx_proxymodule.




Variable"GetHandlers"and"SetHandlers"

Wehavealreadylearnedinprevioussectionsthatwhenreadingthebuilt-invariable$args,Nginxexecutesaspecialpieceofcodetoobtainavalueon-the-flyandwhenwritingtothisvariable,Nginxexecutesanotherspecialpieceofcodetopropagatethechange.InNginx'sterminology,thespecialcodeexecutedforreadingthevariableiscalled"gethandler"andthecodeforwritingtothevariableiscalled"sethandler".DifferentNginxmodulesusuallypreparedifferent"gethandlers"and"sethandlers"fortheirownvariables,whicheffectivelyputmagicintothesevariables'behavior.

Suchtechniquesarenotuncommoninthecomputingworld.Forexample,inobject-orientedprogramming(OOP),theclassdesignerusuallydoesnotexposethemembervariableoftheclassdirectlytotheuserprogrammer,butinsteadprovidestwomethodsforreadingfromandwritingtothemembervariable,respectively.Suchclassmethodsareoftencalled"accessors".BelowisanexampleintheC++programminglanguage:

#include<string>

usingnamespacestd;

classPerson{

public:

conststringget_name(){

returnm_name;

}

voidset_name(conststringname){

m_name=name;

}

private:

stringm_name;

};

InthisC++classPerson,weprovidetwopublicmethods,get_nameandset_name,toserveasthe"accessors"fortheprivatemembervariablem_name.

Thebenefitsofsuchdesignareobvious.Theclassdesignercanexecutearbitrarycodeinthe"accessors",toimplementanyextrabusinesslogicorusefulsideeffects,likeautomaticallyupdatingothermembervariablesdependingonthecurrentmember,orupdatingthecorrespondingfieldinadatabaseassociatedwiththecurrentobject.Forthelattercase,itispossiblethatthemembervariabledoesnotexistatall,orthatthemembervariablejustservesasadatacachetomitigatethepressureontheback-enddatabase.

Correspondingtotheconceptof"accessors"inOOP,Nginxvariablesalsosupportbindingcustom"gethandlers"and"sethandlers".Additionally,notallNginxvariablesownacontainertoholdvalues.Somevariableswithoutacontainerjustbehavelikeamagicalcontainerbymeansofitsfancy"gethandler"and"sethandler".Infact,whenavariableisbeingcreatedat"configuretime",thecreatingNginxmodulemustmakeadecisiononwhethertoallocateavaluecontainerforitandwhethertoattachacustom"gethandler"and/ora"sethandler"toit.

Thosevariablesowningavaluecontainerarecalled"indexedvariables"inNginx'sterminology.Otherwise,theyaresaidtobenotindexed.

Wealreadyknowthatthe"variablegroups"like$arg_XXXdiscussedinearliersectionsdonothaveavaluecontainerandthusarenotindexed.Whenreading$arg_XXX,itisits"gethandler"atwork,thatis,its"gethandler"scansthecurrentURLquerystringon-the-fly,extractingthevalueofthespecifiedURLargument.Manybeginnersmisunderstandtheway$arg_XXXisimplemented;theyassumethatNginxwillparsealltheURLargumentsinadvanceandpreparethevaluesforallthosenon-empty$arg_XXXvariablesbeforetheyareactuallyread.Thisisnottrue,however.NginxnevertriestoparsealltheURLargumentsbeforehand,butratherscansthewholeURLquerystringforaparticularargumentina"gethandler"everytimethatargumentisrequestedbyreadingthecorresponding$arg_XXXvariable.Similarly,whenreadingthebuilt-invariable$cookie_XXX,its"gethandler"justscanstheCookierequestheadersforthecookienamespecified.








NginxVariables(04)ValueContainersforCaching&ngx_map

SomeNginxvariableschoosetousetheirvaluecontainersasadatacachewhenthe"gethandler"isconfigured.Inthissetting,the"gethandler"isrunonlyonce,i.e.,atthefirsttimethevariableisread,whichreducesoverheadwhenthevariableisreadmultipletimesduringitslifetime.Let'sseeanexampleforthis.

map$args$foo{

default0;

debug1;

}

server{

listen8080;

location/test{

set$orig_foo$foo;

set$argsdebug;

echo"originalfoo:$orig_foo";

echo"foo:$foo";

}

}

Hereweusethemapdirectivefromthestandardmodulengx_mapforthefirsttime,whichdeservessomeintroduction.Thewordmapheremeansmappingorcorrespondence.Forexample,functionsinMathsareakindof"mapping".AndNginx'smapdirectiveisusedtodefinea"mapping"relationshipbetweentwoNginxvariables,orinotherwords,"functionrelationship".Backtothisexample,weusethemapdirectivetodefinethe"mapping"relationshipbetweenuservariable$fooandbuilt-invariable$args.WhenusingtheMathfunctionnotation,y=f(x),our$argsvariableiseffectivelythe"independentvariable",x,while$fooisthe"dependentvariable",y.Thatis,thevalueof$foodependsonthevalueof$args,orrather,wemapthevalueof$argsontothe$foovariable(insomeway).

Nowlet'slookattheexactmappingruledefinedbythemapdirectiveinthisexample.

map$args$foo{

default0;

debug1;

}

Thefirstlinewithinthecurlybracesisaspecialrulecondition,thatis,thisconditionholdsifandonlyifotherconditionsallfail.Whenthis"default"conditionholds,the"dependentvariable"$fooisassignedbythevalue0.Thesecondlinewithinthecurlybracesmeansthatthe"dependentvariable"$fooisassignedbythevalue1ifthe"independentvariable"$argsmatchesthestringvaluedebug.Combiningthesetwolines,weobtainthefollowingcompletemappingrule:ifthevalueof$argsisdebug,variable$foogetsthevalue1;otherwise$foogetsthevalue0.Soessentially,thisisaconditionalassignmenttothevariable$foo.

Nowthatweunderstandwhatthemapdirectivedoes,let'slookatthedefinitionoflocation/test.Wefirstsavethevalueof$foointoanotheruservariable$orig_foo,thenoverwritethevalueof$argstodebug,andfinallyoutputthevaluesof$orig_fooand$foo,respectively.

Intuitively,afterweoverwritethevalueof$argstodebug,thevalueof$fooshouldautomaticallygetadjustedto1accordingtothemappingruledefinedearlier,regardlessoftheoriginalvalueof$foo.Butthetestresultsuggeststheotherwayaround.


originalfoo:0

foo:0

Thefirstoutputlineindicatesthatthevalueof$orig_foois0,whichisexactlywhatweexpected:theoriginalrequestdoesnottakeaURLquerystring,sotheinitialvalueof$argsisempty,leadingtothe0initialvalueof$foo,accordingtothe"default"conditioninourmappingrule.

Butsurprisingly,thesecondoutputlineindicatesthatthefinalvalueof$fooisstill0,evenafterweoverwrite$argstothevaluedebug.Thisapparentlyviolatesourmappingrulebecausewhen$argstakesthevaluedebug,thevalueof$fooshouldreallybe1.Sowhatishappeninghere?

Actuallythereasonisprettysimple:whenthefirsttimevariable$fooisread,itsvaluecomputedbyngx_map's"gethandler"iscachedinitsvaluecontainer.WealreadylearnedearlierthatNginxmodulesmaychoosetousethevaluecontainerofthevariablecreatedbythemselvesasadatacacheforits"gethandler".Obviously,thengx_mapmoduleconsidersthemappingcomputationbetweenvariablesexpensiveenoughandcachestheresultautomatically,sothatthenexttimethesamevariableisreadwithinthelifetimeofthecurrentrequest,Nginxcanjustreturnthecachedresultwithoutinvokingthe"gethandler"again.

Toverifythisfurther,wecantryspecifyingtheURLquerystringasdebugintheoriginalrequest.

$curl'http://localhost:8080/test?debug'

originalfoo:1

foo:1

Itcanbeseenthatthevalueof$orig_foobecomes1,complyingwithourmappingrule.Andsubsequentreadingsof$fooalwaysyieldthesamecachedresult,1,regardlessofthenewvalueof$argslateron.

Themapdirectiveisactuallyauniqueexample,becauseitnotonlyregistersa"gethandler"fortheuservariable,butalsoallowstheusertodefinethecomputingruleinthe"gethandler"directlyintheNginxconfigurationfile.Ofcourse,therulethatcanbedefinedhereislimitedtosimplemappingrelationswithanothervariable.Meanwhile,itmustbemadeclearthatnotallthevariablesusinga"gethandler"willcachetheresult.Forinstance,wehavealreadyseenearlierthatthe$arg_XXXvariabledoesnotuseitsvaluecontaineratall.

Similartothengx_mapmodule,thestandardmodulengx_geothatweencounteredearlieralsoenablesvaluecachingforthevariablescreatedbyitsgeodirective.

ASideNoteforUseContextsofDirectives

http://wiki.nginx.org/HttpMapModule#map

http://wiki.nginx.org/HttpMapModule






















Inthepreviousexample,weshouldalsonotethatthemapdirectiveisputoutsidetheserverconfigurationblock,thatis,itisdefineddirectlywithintheoutermosthttpconfigurationblock.Somereadersmaybecuriousaboutthissetting,sinceweonlyuseitinlocation/testafterall.Ifwetryputtingthemapstatementwithinthelocationblock,however,wewillgetthefollowingerrorwhilestartingNginx:

[emerg]"map"directiveisnotallowedherein...

Soitisexplicitlyprohibited.Infact,itisonlyallowedtousethemapdirectiveinthehttpblock.Everyconfiguredirectivedoeshaveapre-definedsetofusecontextsintheconfigurationfile.Whenindoubt,alwaysrefertothecorrespondingdocumentationfortheexactusecontextsofaparticulardirective.




LazyEvaluationofVariableValues

ManyNginxfreshmenwouldworrythattheuseofthemapdirectivewithintheglobalscope(i.e.,thehttpblock)willleadtounnecessaryvariablevaluecomputationandassignmentforallthelocationsinallthevirtualserversevenifonlyonelocationblockactuallyusesit.Fortunately,thisisnotwhatishappeninghere.Wehavealreadylearnedhowthemapdirectiveworks.Itisthe"gethandler"(registeredbythengx_mapmodule)thatperformsthevaluecomputationandrelatedassignment.Andthe"gethandler"willnotrunatallunlessthecorrespondinguservariableisactuallybeingread.Therefore,forthoserequeststhatneveraccessthatvariable,therecannotbeany(useless)computationinvolved.

Thetechniquethatpostponesthevaluecomputationofftothepointwherethevalueisactuallyneedediscalled"lazyevaluation"inthecomputingworld.Programminglanguagesnativelyoffering"lazyevaluation"isnotverycommonthough.ThemostfamousexampleistheHaskellprogramminglanguage,wherelazyevaluationisthedefaultsemantics.Incontrastwith"lazyevaluation",itismuchmorecommontosee"eagerevaluation".Weareluckytoseeexamplesoflazyevaluationhereinthengx_mapmodule,butthe"eagerevaluation"semanticsisalsomuchmorecommonintheNginxworld.Considerthefollowingsetstatementthatcannotbesimpler:

set$b"$a,$a";

Whenrunningthesetdirective,Nginxeagerlycomputesandassignsthenewvalueforthevariable$bwithoutpostponingtothepointwhen$bisactuallyreadlateron.Similarly,theset_unescape_uridirectivealsoevaluateseagerly.








NginxVariables(05)VariablesinSubrequests

ADetourtoSubrequests

Wehaveseenearlierthatthelifetimeofvariablecontainersisboundtotherequest,butIownyouaformaldefinitionof"requests"there.Youmighthaveassumedthatthe"requests"inthatcontextarejustthoseHTTPrequestsinitiatedfromtheclientside.Infact,therearetwokindsof"requests"intheNginxworld.Oneiscalled"mainrequests",andtheotheriscalled"subrequests".

MainrequestsarethoseinitiatedexternallybyHTTPclients.Alltheexamplesthatwehaveseensofarinvolvemainrequestsonly,includingthosedoing"internalredirections"viatheecho_execorrewritedirective.

WhereassubrequestsareaspecialkindofrequestsinitiatedfromwithintheNginxcore.ButpleasedonotconfusesubrequestswiththoseHTTPrequestscreatedbythengx_proxymodules!SubrequestsmaylookverymuchlikeanHTTPrequestinappearance,theirimplementation,however,hasnothingtodowithneithertheHTTPprotocolnoranykindofsocketcommunication.Asubrequestisanabstractinvocationfordecomposingthetaskofthemainrequestintosmaller"internalrequests"thatcanbeservedindependentlybymultipledifferentlocationblocks,eitherinseriesorinparallel."Subrequests"canalsoberecursive:anysubrequestcaninitiatemoresub-subrequests,targetingotherlocationblocksoreventhecurrentlocationitself.AccordingtoNginx'sterminology,ifrequestAinitiatesasubrequestB,thenAiscalledthe"parentrequest"ofB.ItisworthmentioningthattheApachewebserveralsohastheconceptofsubrequestsforlong,soreaderscomingfromthatworldshouldbenostrangertothis.

Let'scheckoutanexampleusingsubrequests:

location/main{

echo_location/foo;

echo_location/bar;

}

location/foo{

echofoo;

}

location/bar{

echobar;

}

Hereinlocation/main,weusetheecho_locationdirectivefromthengx_echomoduletoinitiatetwoGET-typedsubrequeststargeting/fooand/bar,respectively.Thesubrequestsinitiatedbyecho_locationarealwaysrunningsequentiallyaccordingtotheirliteralorderintheconfigurationfile.Therefore,thesecond/barrequestwillnotbefireduntilthefirst/foorequestcompletesprocessing.Theresponsebodyofthesetwosubrequestsgetconcatenatedtogetheraccordingtotheirrunningorder,toformthefinalresponsebodyoftheirparentrequest(for/main):

$curl'http://localhost:8080/main'

foo

bar

Itshouldbenotedthatthecommunicationoflocationblocksviasubrequestsislimitedwithinthesameserverblock(i.e.,thesamevirtualserverconfiguration),sowhentheNginxcoreprocessesasubrequest,itjustcallsafewCfunctionsbehindthescene,withoutdoinganykindofnetworkorUNIXdomainsocketcommunication.Forthisreason,subrequestsareextremelyefficient.

IndependentVariableContainersinSubrequests

BacktoourearlierdiscussionforthelifetimeofNginxvariablecontainers,nowwecanstillstatethatthelifetimeisboundtothecurrentrequest,andeveryrequestdoeshaveitsowncopyofallthevariablecontainers.Itisjustthatthe"request"herecanbeeitheramainrequest,orasubrequest.Variableswiththesamenamebetweenaparentrequestandasubrequestwillgenerallynotinterferewitheachother.Let'sdoasmallexperimenttoconfirmthis:

location/main{

set$varmain;

echo_location/foo;

echo_location/bar;

echo"main:$var";

}

location/foo{

set$varfoo;

echo"foo:$var";

}

location/bar{

set$varbar;

echo"bar:$var";

}

Inthissample,weassigndifferentvaluestothevariable$varinthreelocationblocks,/main,/foo,and/bar,andoutputthevalueof$varinalltheselocations.Inparticular,weintentionallyoutputthevalueof$varinlocation/mainaftercallingthetwosubrequests,soifvaluechangesof$varinthesubrequestscanaffecttheirparentrequest,weshouldseeanewvalueoutputinlocation/main.Theresultofrequesting/mainisasfollows:


foo:foo




http://wiki.nginx.org/HttpEchoModule#echo_location



bar:bar

main:main

Apparently,theassignmentstovariable$varinthosetwosubrequestsdonotaffectthemainrequest/mainatall.Thissuccessfullyverifiesthatboththemainrequestanditssubrequestsdoowndifferentcopiesofvariablecontainers.

SharedVariableContainersamongRequests

Unfortunately,subrequestsinitiatedbycertainNginxmodulesdosharevariablecontainerswiththeirparentrequests,likethoseinitiatedbythe3rd-partymodulengx_auth_request.Belowissuchanexample:

location/main{

set$varmain;

auth_request/sub;

echo"main:$var";

}

location/sub{

set$varsub;

echo"sub:$var";

}

Hereinlocation/main,wefirstassigntheinitialvaluemaintovariable$var,thenfireasubrequestto/subviatheauth_requestdirectivefromthengx_auth_requestmodule,andfinallyoutputthevalueof$var.Notethatinlocation/subweintentionallyoverwritethevalueof$vartosub.Whenaccessing/main,weget


main:sub

Obviously,thevaluechangeof$varinthesubrequestto/subdoesaffectthemainrequestto/main.Thusthevariablecontainerof$varisindeedsharedbetweenthemainrequestandthesubrequestcreatedbythengx_auth_requestmodule.

Forthepreviousexample,somereadersmightask:"whydoesn'ttheresponsebodyofthesubrequestappearinthefinaloutput?"Theanswerissimple:itisjustbecausetheauth_requestdirectivediscardstheresponsebodyofthesubrequestitmanages,andonlycheckstheresponsestatuscodeofthesubrequest.Whenthestatuscodelooksgood,like200,auth_requestwilljustallowNginxcontinueprocessingthemainrequest;otherwiseitwillimmediatelyabortthemainrequestbyreturninga403errorpage,forexample.Inourexample,thesubrequestto/subjustreturna200responseimplicitlycreatedbytheechodirectiveinlocation/sub.

Eventhoughsharingvariablecontainersamongthemainrequestandallitssubrequestscouldmakebidirectionaldataexchangeeasier,itcouldalsoleadtounexpectedsubtleissuesthatarehardtodebuginreal-worldconfigurations.Becauseusersoftenforgetthatavariablewiththesamenameisactuallyusedinsomedeeplyembeddedsubrequestandjustuseitforsomethingelseinthemainrequest,thisvariablecouldgetunexpectedlymodifiedduringprocessing.Suchbadsideeffectsmakemany3rd-partymoduleslikengx_echo,ngx_luaandngx_srcachechoosetodisablethevariablesharingbehaviorforsubrequestsbydefault.

http://mdounin.ru/hg/ngx_http_auth_request_module/





http://wiki.nginx.org/HttpLuaModule

http://wiki.nginx.org/HttpSRCacheModule

NginxVariables(06)Built-inVariablesinSubrequests

TherearesomesubtletiesinvolvedinusingNginxbuilt-invariablesinthecontextofasubrequest.Wewilldiscussthedetailsinthissection.

Built-inVariablesSensitivetotheSubrequestContext

Wealreadyknowthatmostbuilt-invariablesarenotsimplevaluecontainers.Theybehavedifferentlythanuservariablesbyregistering"gethandlers"and/or"sethandlers".Evenwhentheydoownavaluecontainer,theyusuallyjustusethecontainerasaresultcachefortheir"gethandlers".The$argsvariablewediscussedearlier,forexample,justusesits"gethandler"toreturntheURLquerystringforthecurrentrequest.Thecurrentrequestherecanalsobeasubrequest,sowhenreading$argsinasubrequest,its"gethandler"shouldnaturallyreturnthequerystringforthesubrequest.Let'sseesuchanexample:

location/main{

echo"mainargs:$args";

echo_location/sub"a=1&b=2";

}

location/sub{

echo"subargs:$args";

}

Hereinthe/maininterface,wefirstechooutthevalueof$argsforthecurrentrequest,andthenuseecho_locationtoinitiateasubrequestto/sub.Itshouldbenotedthatherewegiveasecondargumenttotheecho_locationdirective,tospecifytheURLquerystringforthesubrequestbeingfired(thefirstargumentistheURIforthesubrequest,aswealreadyknow).Finally,wedefinethe/subinterfaceandprintoutthevalueof$argsinthere.Queryingthe/maininterfacegives

$curl'http://localhost:8080/main?c=3'

mainargs:c=3

subargs:a=1&b=2

Itisclearthatwhen$argsisreadinthemainrequest(to/main),itsvalueistheURLquerystringofthemainrequest;whereaswheninthesubrequest(to/foo),itisthequerystringofthesubrequest,a=1&b=2.Thisbehaviorindeedmatchesourintuition.

Justlike$args,whenthebuilt-invariable$uriisusedinasubrequest,its"gethandler"alsoreturnsthe(decoded)URIofthecurrentsubrequest:

location/main{

echo"mainuri:$uri";

echo_location/sub;

}

location/sub{

echo"suburi:$uri";

}

Belowistheresultofquerying/main:


mainuri:/main

suburi:/sub

Theoutputiswhatwewouldexpect.

Built-inVariablesforMainRequestsOnly

Unfortunately,notallbuilt-invariablesaresensitivetothecontextofsubrequests.Severalbuilt-invariablesalwaysactonthemainrequestevenwhentheyareusedinasubrequest.Thebuilt-invariable$request_methodissuchanexception.

Whenever$request_methodisread,wealwaysgettherequestmethodname(suchasGETandPOST)forthemainrequest,nomatterwhetherthecurrentrequestisasubrequestornot.Let'stestitout:

location/main{

echo"mainmethod:$request_method";

echo_location/sub;

}

location/sub{

echo"submethod:$request_method";

}

Inthisexample,the/mainand/subinterfacesbothoutputthevalueof$request_method.Meanwhile,weinitiateaGETsubrequestto/subviatheecho_locationdirectivein/main.Nowlet'sdoaPOSTrequestto/main:

$curl--datahello'http://localhost:8080/main'

mainmethod:POST

submethod:POST

Hereweusethe--dataoptionofthecurlutilitytospecifyourPOSTrequestbody,alsothisoptionmakescurlusethePOSTmethodfortherequest.Thetestresultturnsoutaswepredicted:thevariable$request_methodisevaluatedtothemainrequest'smethodname,POST,despiteitsuseinaGETsubrequest.

Somereadersmightchallengeourconclusionherebypointingoutthatwedidnotruleoutthepossibilitythatthevalueof$request_methodgotcachedatitsfirstreadinginthemainrequestandwhatwewereseeinginthesubrequestwasactuallythecachedvaluethatwasevaluatedearlierinthemainrequest.Thisconcernis










http://wiki.nginx.org/HttpCoreModule#.24request_method






unnecessary,however,becausewehavealsolearnedthatthevariablecontainerrequiredbydatacaching(ifany)isalwaysboundtothecurrentrequest,alsothesubrequestsinitiatedbythengx_echomodulealwaysdisablevariablecontainersharingwiththeirparentrequests.Backtothepreviousexample,evenifthebuilt-invariable$request_methodinthemainrequestusedthevaluecontainerasthedatacache(actuallyitdoesnot),itcannotaffectthesubrequestbyanymeans.

Tofurtheraddresstheconcernofthesereaders,let'sslightlymodifythepreviousexamplebyputtingtheechostatementfor$request_methodin/mainaftertheecho_locationdirectivethatrunsthesubrequest:

location/main{

echo_location/sub;

echo"mainmethod:$request_method";

}

location/sub{

echo"submethod:$request_method";

}

Let'stestitagain:


submethod:POST

mainmethod:POST

Nochangeintheoutputcanbeobserved,exceptthatthetwooutputlinesreversedtheorder(sinceweexchangetheorderofthosetwongx_echomodule'sdirectives).

Consequently,wecannotobtainthemethodnameofasubrequestbyreadingthe$request_methodvariable.Thisisacommonpitfallforfreshmenwhendealingwithmethodnamesofsubrequests.Toovercomethislimitation,weneedtoturntothebuilt-invariable$echo_request_methodprovidedbythengx_echomodule:

location/main{

echo"mainmethod:$echo_request_method";

echo_location/sub;

}

location/sub{

echo"submethod:$echo_request_method";

}

Wearefinallygettingwhatwewant:


mainmethod:POST

submethod:GET

Nowwithinthesubrequest,wegetitsownmethodname,GET,asexpected,andthemainrequestmethodremainsPOST.

Similarto$request_method,thebuilt-invariable$request_urialsoalwaysreturnsthe(non-decoded)URLforthemainrequest.Thisismoreunderstandable,however,becausesubrequestsareessentiallyfakedrequestsinsideNginx,whichdonotreallytakeanon-decodedrawURL.

VariableContainerSharingandValueCachingTogether

Intheprevioussection,someofthereaderswereworriedaboutthecasethatvariablecontainersharinginsubrequestsandvaluecachingforvariable's"gethandlers"wereworkingtogether.Ifitwereindeedthecase,thenitwouldbeanightmarebecauseitwouldbereallyreallyhardtopredictwhatisgoingonbyjustlookingattheconfigurationfile.Inprevioussections,wealreadylearnedthatthesubrequestsinitiatedbythengx_auth_requestmodulearesharingthesamevariablecontainerswiththeirparents,sowecanmaliciouslyconstructsuchahorribleexample:

map$uri$tag{

default0;

/main1;

/sub2;

}

server{

listen8080;

location/main{

auth_request/sub;

echo"maintag:$tag";

}

location/sub{

echo"subtag:$tag";

}

}

Hereweuseouroldfriend,themapdirective,tomapthevalueofthebuilt-invariable$uritoouruservariable$tag.When$uritakesthevalue/main,thevalue1isassignedto$tag;when$uritakesthevalue/sub,thevalue2isassignedinsteadto$tag;underalltheotherconditions,0isassigned.Next,in/main,wefirstinitiateasubrequestto/subbyusingtheauth_requestdirective,andthenoutputthevalueof$tag.Andwithin/sub,wedirectlyoutputthevalueof$tag.Guesswhatwewillgetwhenweaccess/main?


maintag:2

Ouch!Didn'twemapthevalue/mainto1?Whytheactualoutputfor/mainisthevalue,2,for/sub?Whatisgoingonhere?








http://wiki.nginx.org/HttpEchoModule#.24echo_request_method









Actuallyitworkedlikethis:our$tagvariablewasfirstreadinthesubrequestto/sub,andthe"gethandler"registeredbymapcomputedthevalue2for$taginthatcontext(because$uriwas/subinthesubrequest)andthevalue2gotcachedinthevaluecontainerof$tagfromthenon.Becausetheparentrequestsharedthesamecontainerasthesubrequestcreatedbyauth_request,whentheparentrequestread$taglater(afterthesubrequestwasfinished),thecachedvalue2wasdirectlyreturned!Suchresultscanindeedbeverysurprisingatfirstglance.

Fromthisexample,wecanconcludeagainthatitcanhardlybeagoodideatoenablevariablecontainersharinginsubrequests.



NginxVariables(07)SpecialValue"Invalid"and"NotFound"

WehavementionedthatthevaluesofNginxvariablescanonlybeofonesingletype,thatis,thestringtype,butvariablescouldalsohavenomeaningfulvaluesatall.Variableswithoutanymeaningfulvaluesstilltakeaspecialvaluethough.Therearetwopossiblespecialvalues:"invalid"and"notfound".

Forexample,whenauservariable$fooiscreatedbutnotassignedyet,$footakesthespecialvalueof"invalid".AndwhenthecurrentURLquerystringdoesnothavetheXXXargumentatall,thebuilt-invariable$arg_XXXtakesthespecialvalueof"notfound".

Both"invalid"and"notfound"arespecialvalues,completelydifferentfromanemptystringvalue("").Thisisverysimilartothosedistinctspecialvaluesinsomedynamicprograminglanguages,likeundefinPerl,nilinLua,andnullinJavaScript.

Wehaveseenearlierthatanuninitializedvariableisevaluatedtoanemptystringwhenusedinaninterpolatedstring,itsrealvalue,however,isnotanemptystringatall.Itisthe"gethandler"registeredbythesetdirectivethatautomaticallyconvertsthe"invalid"specialvalueintoanemptystring.Toverifythis,let'sreturntotheexamplewehavediscussedbefore:

location/foo{

echo"foo=[$foo]";

}

location/bar{

set$foo32;

echo"foo=[$foo]";

}

Whenaccessing/foo,theuservariable$fooisuninitializedwhenusedintheinterpolatedstringfortheechodirective.Theoutputshowsthatthevariableisevaluatedtoanemptystring:


foo=[]

Fromtheoutput,theuninitialized$foovariablebehavesjustliketakinganemptystringvalue.Butcarefulreadersshouldhavealreadynoticedthat,fortherequestabove,thereisawarningintheNginxerrorlogfile(whichislogs/error.logbydefault):

[warn]5765#0:*1usinguninitialized"foo"variable,...

Whoonearthgeneratesthiswarning?Theansweristhe"gethandler"of$foo,registeredbythesetdirective.When$fooisread,Nginxfirstchecksthevalueinitscontainerbutseesthe"invalid"specialvalue,thenNginxdecidestocontinuerunning$foo's"gethandler",whichfirstprintsthewarning(asshownabove)andthenreturnsanemptystringvalue,whichthereaftergetscachedin$foo'svaluecontainer.

Carefulreadersshouldhaveidentifiedthatthisprocessforuservariablesisexactlythesameasthemechanismwediscussedearlierforbuilt-invariablesinvolving"gethandlers"andresultcachinginvaluecontainers.Yes,itisthesamemechanisminaction.Itisalsoworthnotingthatonlythe"invalid"specialvaluewilltriggerthe"gethandler"invocationintheNginxcorewhile"notfound"willnot.

Thewarningmessageaboveusuallyindicatesatypointhevariablenameormisuseofuninitializedvariables,notnecessarilyinthecontextofaninterpolatedstring.Becauseoftheexistenceofvaluecachinginthevariablecontainer,thiswarningwillnotgetprintedmultipletimesinthelifetimeofthecurrentrequest.Also,thengx_rewritemoduleprovidestheuninitialized_variable_warndirectivefordisablingthiswarningaltogether.

TestingSpecialValuesofNginxVariablesinLua

Aswehavejustmentioned,thebuilt-invariable$arg_XXXtakesthespecialvalue"notfound"whentheURLargumentXXXdoesnotexist,butunfortunately,itisnoteasytodistinguishitfromtheemptystringvaluedirectlyintheNginxconfigurationfile,forexample:

location/test{

echo"name:[$arg_name]";

}

HereweintentionallyomittheURLargumentnameinourrequest:


name:[]

Wecanseethatwearestillgettinganemptystringvalue,becausethistimeitistheNginx"scriptengine"thatautomaticallyconvertsthe"notfound"specialvaluetoanemptystringwhenperformingvariableinterpolation.

Thenhowcanwetestthespecialvalue"notfound"?Orinotherwords,howcanwedistinguishitfromnormalemptystringvalues?Obviously,inthefollowingexample,theURLargumentnamedoestakeanordinaryvalue,whichisatrueemptystring:

$curl'http://localhost:8080/test?name='

name:[]

Butwecannotreallydifferentiatethisfromtheearliercasethatdoesnotmentionthenameargumentatall.

Luckily,wecaneasilyachievethisinLuabymeansofthe3rd-partymodulengx_lua.Pleaselookatthefollowingexample:

location/test{

content_by_lua'

ifngx.var.arg_name==nilthen

ngx.say("name:missing")

else






http://wiki.nginx.org/HttpRewriteModule#uninitialized_variable_warn



ngx.say("name:[",ngx.var.arg_name,"]")

end

';

}

Thisexampleisveryclosetothepreviousoneintermsoffunctionality.Weusethecontent_by_luadirectivefromthengx_luamoduletoembedasmallpieceofourownLuacodetotestagainstthespecialvalueoftheNginxvariable$arg_name.When$arg_nametakesaspecialvalue(either"notfound"or"invalid"),wewillgetthefollowingoutputwhenrequesting/foo:


name:missing

Thisisourfirsttimemeetingthengx_luamodule,whichdeservesabriefintroduction.ThismoduleembedstheLualanguageinterpreter(orLuaJIT'sJust-in-Timecompiler)intotheNginxcore,toallowNginxusersdirectlyruntheirownLuaprogramsinsidetheserver.TheusercanchoosetoinsertherLuacodeintodifferentrunningphasesoftheserver,tofulfilldifferentrequirements.SuchLuacodeareeitherspecifieddirectlyasliteralstringsintheNginxconfigurationfile,orresideinexternal.luasourcefiles(orLuabinarybytecodefiles)whosepathsarespecifiedintheNginxconfiguration.

Backtoourexample,wecannotdirectlywritesomethinglike$arg_nameinourLuacode.Instead,wereferenceNginxvariablesinLuabymeansofthengx.varAPIprovidedbythengx_luamodule.Forexample,toreferencetheNginxvariable$VARIABLEinLua,wejustwritengx.var.VARIABLE.WhentheNginxvariable$arg_nametakesthespecialvalue"notfound"(or"invalid"),ngx.var.arg_nameisevaluatedtothenilvalueintheLuaworld.ItshouldalsobenotingthatweusetheLuafunctionngx.saytoprintouttheresponsebodycontents,whichisfunctionallyequivalenttotheechodirectivewearealreadyveryfamiliarwith.

IfweprovideanameURIargumentthattakesanemptyvalueintherequest,theoutputisnowverydifferent:

$curl'http://localhost:8080/test?name='

name:[]

Inthistest,thevalueoftheNginxvariable$arg_nameisatrueemptystring,neither"notfound"nor"invalid".SoinLua,theexpressionngx.var.arg_nameevaluatestotheLuaemptystring(""),clearlydistinguishedfromtheLuanilvalueintheprevioustest.

Thisdifferentiationisimportantincertainapplicationscenarios.Forinstance,somewebserviceshavetodecidewhethertouseacolumnvaluetofilterthedatasetbycheckingtheexistenceofthecorrespondingURIargument.Fortheseserives,whenthenameURIargumentisabsent,thewholedatasetarejustreturned;whenthenameargumenttakesanemptyvalue,however,onlythoserecordsthattakeanemptyvaluearereturned.

Itisworthmentioningafewlimitationsinthestandard$arg_XXXvariable.Considerusingthefollowingrequesttotest/testinourpreviousexampleusingLua:

$curl'http://localhost:8080/test?name'

name:missing

Nowthe$arg_namevariablestillreadsthe"notfound"specialvalue,whichisapparentlycounter-intuitive.Additionally,whenmultipleURIargumentswiththesamenamearespecifiedintherequest,$arg_XXXjustreturnsthefirstvalueoftheargument,discardingothervaluessilently:

$curl'http://localhost:8080/test?name=Tom&name=Jim&name=Bob'

name:[Tom]

Tosolvetheseproblems,wecanusetheLuafunctionngx.req.get_uri_argsprovidedbythengx_luamoduleinstead.

http://wiki.nginx.org/HttpLuaModule#content_by_lua




http://wiki.nginx.org/HttpLuuaModule#ngx.var.VARIABLE

http://wiki.nginx.org/HttpLuaModule#ngx.say




http://wiki.nginx.org/HttpLuaModule#ngx.req.get_uri_args


NginxVariables(08)In(02)wementionedthatanothercategoryofbuiltinvariables$cookie_XXXarelike$arg_XXX.SimilarlywhenthereexistnocookienamedXXX,itscorrespondingNginxvariable$cookie_XXXhasnon-value"notfound".

location/test{

content_by_lua'

ifngx.var.cookie_user==nilthen

ngx.say("cookieuser:missing")

else

ngx.say("cookieuser:[",ngx.var.cookie_user,"]")

end

';

}

Thecurlutilityoffersthe--cookiename=valueoption,whichdesignatesname=valueasacookieofitsrequest(byaddingtheCookieheader).Let'stestafewcasescontainingcookies.

$curl--cookieuser=agentzh'http://localhost:8080/test'

cookieuser:[agentzh]

$curl--cookieuser='http://localhost:8080/test'

cookieuser:[]


cookieuser:missing

Asexpected,whencookieuserdoesnotexist,Luavariablengx.var.cookie_userisnil.Sowehavesuccessfullydistinguishedthecasewithemptystringandthecasewithnon-value.

Aniceadd-onwithmodulengx_luaiswhenluareferencesanundeclaredvariableofNginx,thevariableisnilandNginxwillnotabortsitloadingasbefore.

location/test{

content_by_lua'

ngx.say("$blah=",ngx.var.blah)

';

}

Uservariable$blahisneverdeclaredintheNginxconfigurationnginx.conf,butitisreferencedasngx.var.blahinLuacode.Nginxcanbestartedstill,becausewhenNginxloadsitsconfiguration,Luacodeisonlycompiledbutnotexecuted,SoNginxhasnoideaavariable$blahisreferenced.Whenluacommandisexecutedinruntimebycommandcontent_by_lua,theluavariableisevaluatedasnil.Modulengx_luaanditscommandngx.saywillconvertLuanilintostring"nil"beforeitisprinted,sotheoutputwillbe:

curl'http://localhost:8080/test'

$blah=nil

Thisisindeedwhatwewant.

Weshouldhavenoticedalso,whencommandcontent_by_luaincludes$blahinitsparameter,itisneverevaluatedas"variableinterpolation"does(otherwiseNginxwillbecomplainingvariable$blahisnotdeclared).Thisisbecausecommandcontent_by_luadoesnotreallysupport"variableinterpolation".Aswehavesaidearlierin(01),Nginxcommanddoesnotnecessarilysupport"variableinterpolation"anditisentirelyuptothemoduleimplementation.

It'sactuallydifficulttoreturnan"invalid"non-value.Aswelearntin(07),variableswhicharedeclaredbutnotinitializedbysethasnon-value"invalid".However,assoonasthevariableisdevalued,the"gethandler"isexecutedandanemptystringiscomputedandcached,soeventuallyemptystringisreturned,notthe"invalid"non-value.Followingluacodecanprovethis:

location/foo{

content_by_lua'

ifngx.var.foo==nilthen

ngx.say("$fooisnil")

else

ngx.say("$foo=[",ngx.var.foo,"]")

end

';

}

location/bar{

set$foo32;

echo"foo=[$foo]";

}

Byrequestingtolocation/foowehave:


$foo=[]

Aswecantell,whenLuareferencesuninitializedNginxvariable$foo,itobtainsemptystring.

Lastnottheleast,weshouldhavepointedout,althoughNginxvariablecanhaveonlystringsasvalidvalue.The3rdpartymodulengx_array_varcansupportarraylikeoperationsforNginxvariable.Hereisanexample:

location/test{












array_split","$arg_namesto=$array;

array_map"[$array_it]"$array;

array_join""$arrayto=$res;

echo$res;

}

Modulengx_array_varprovidescommandsarray_split,array_mapandarray_join.Thesemanticsisprettyclosetothebuiltinfunctionssplit,mapandjoininPerl(otherlanguagessupportsimilarfunctionalitiestoo).Nowlet'scheckwhathappenswhenlocation/testisrequested:

$curl'http://localhost:8080/test?names=Tom,Jim,Bob'

[Tom][Jim][Bob]

Clearlymodulengx_array_varmakeiteasiertohandleinputswithvariablelength,suchastheURLparametername,whichcomposesofmultiplecommadelimitednames.Stillwemustemphasize,modulengx_luaisamuchbetterchoicetoexecutethiskindofcomplicatedtasks,usuallyitismoreflexibleandmaintainable.

TillnowthetutorialcoverstheNginxvariable.Intheprocesswehavebeendiscussingmanybuiltinand3rdpartyNginxmodules,thesemoduleshelpusbetterunderstandfeaturesandinternalsofNginxvariablebycomposingvariousminiconstructs.Lateronthetutorialwillbecoveringmoredetailsofthosemodules.

Withtheseexamples,weshouldunderstandthatNginxvariableplaysakeyroleintheNginxminilanguage:variablesarethewaysandmeansNginxcommunicateinternally,theycontainalltheneededinformation(includingtherequestinformation)andtheyarethecornerstoneelementswhichbridgeeveryotherNginxmodules.Nginxvariablesareeverywhereinthecomingtutorials,understandthemisabsolutelynecessary.

Inthecomingtutorial"NginxDirectiveExecutionOrder",wewillbediscussingindetailtheNginxexecutionorderingandthephaseseveryrequesttraverses.It'sindispensabletounderstandthemsincefortheNginxminilanguage,theorderingofwritingcanbedramaticallydifferentfromtheorderingofexecutinginthetimeline.ItusuallyconfusesmanyNginxusers.




Nginxdirectiveexecutionorder(01)WhentherearemultipleNginxmodulecommandsinalocationdirective,theexecutionordercanbedifferentfromwhatyouexpect.BusyNginxuserswhoattempttoconfigureNginxby"trialanderror"maybeveryconfusedbythisbehavior.Thisseriesistouncoverthemysteriesandhelpyoubetterunderstandtheexecutionorderingbehindthescenes.

Westartwithaconfusedexample:

?location/test{

?set$a32;

?echo$a;

?

?set$a56;

?echo$a;

?}

Clearly,we'dexpecttooutput32,followedby56.Becausevariable$ahasbeenresetaftercommandecho"isexecuted".Really?therealityis:

$curl'http://localhost:8080/test

56

56

Wow,statementset$a56musthavehadbeenexecutedbeforethefirstecho$acommand,butwhy?IsitaNginxbug?

No,thisisnotanNginxbug.WhenNginxhandleseveryrequest,theexecutionfollowsafewpredefinedphases.

Therecanbealtogether11phaseswhenNginxhandlesarequest,let'sstartwiththreemostcommonones:rewrite,accessandcontent(Theotherphaseswillbeaddressedlater.)

UsuallyanNginxmoduleanditscommandsregistertheirexecutioninonlyoneofthosephases.Forexamplecommandsetrunsinphaserewrite,andcommandechorunsinphasecontent.Sincephaserewriteoccursbeforephasecontentforeveryrequestprocessing,itscommandsareexecutedearlieraswell.Therefore,commandsetalwaysgetsexecutedbeforecommandechowithinonelocationdirective,regardlessoftheirstatementorderingintheconfiguration.

Backtoourexample:

set$a32;

echo$a;

set$a56;

echo$a;

Theactualexecutionorderingis:

set$a32;

set$a56;

echo$a;

echo$a;

It'sclearnow,twocommandssetareexecutedinphaserewrite,twocommandsechoareexecutedafterwardsinphasecontent.Commandsindifferentphasescannotbeexecutedbackandforth.

Toprovethis,wecanenableNginx's"debuglog".

IfyouhavenotworkedwithNginx"debuglog"before,hereisabriefintroduction.The"debuglog"isdisabledbydefaultbecauseperformanceisdegradedwhenitisenabled.Toenable"debuglog"youmustreconfigureandrecompileNginx,andsetthe--with-debugoptionforthepackage's./configurescript.WhenbuildingunderLinuxorMacOSXfromsource:

tarxvfnginx-1.0.10.tar.gz

cdnginx-1.0.10/

./configure--with-debug

make

sudomakeinstall

Incasethepackagengx_openrestyisused.Theoption--with-debugcanbeusedwithits./configurescriptaswell.

AfterwerebuildtheNginxdebugbinarywith--with-debugoption,westillneedtoexplicitlyusethedebugloglevel(it'sthelowestlevel)forcommanderror_log,inNginxconfiguration:

error_loglogs/error.logdebug;

debug,thesecondparameterofcommanderror_logiscrucial.Itsfirstparameteriserrorlog'sfilepath,logs/error.log.Certainlywecanuseanotherfilepathbutdorememberthelocationbecauseweneedtocheckitscontentrightaway.

Nowlet'srestartNginx(Attention,it'snotenoughtoreloadNginx.Itneedstobekilledandrestartedbecausewe'veupdatedtheNginxbinary).Thenwecansendtherequestagain:


56

56

It'stimetocheckNginx'serrorlog,whichisbecomingalotmoreverbose(morethan700linesfortherequestinmysetup).Solet'sapplythegrepcommandtofilterwhatwewouldbeinterested:








http://openresty.org

http://wiki.nginx.org/CoreModule#error_log

http://wiki.nginx.org/CoreModule#error_log

grep-E'http(outputfilter|script(set|value))'logs/error.log

It'sapproximatelylikebelow(forclearness,I'veeditedthegrepoutputandremoveitstimestampetc):

[debug]5363#0:*1httpscriptvalue:"32"

[debug]5363#0:*1httpscriptset$a



[debug]5363#0:*1httpoutputfilter"/test?"



Itbarelymakesanysenses,doesit?Soletmeinterpret.Commandsetdumpstwolinesofdebuginfowhichstartwithhttpscript,thefirstlinetellsthevaluewhichcommandsethaspossessed,andthesecondlinebeingthevariablenameitwillbegivento,sofortheleadingfilteredlog:



Thesetwolinesaregeneratedbythisstatement:

set$a32;

Andforthefollowingfilteredlog:



Theyaregeneratedbythisstatement:

set$a56;

Besides,wheneverNginxoutputsitsresponse,its"outputfilter"willbeexecuted,ourfavoritecommandechoisnoexception.AssoonasNginx's"outputfilter"isexecuted,itgeneratesdebugloglikebelow:


Ofcoursethedebuglogmightnothave"/test?",sincethispartcorrespondstotheactualrequestURI.Byputtingeverythingtogether,wecanfinallyconcludethosetwocommandssetareindeedexecutedbeforetheothertwocommandsecho.

Consideratereadersmusthavenoticedthattherearethreelinesofhttpoutputfilterdebuglogbutwewerehavingonlytwooutputcommandsecho.Infact,onlythefirsttwodebuglogsaregeneratedbythetwoechostatements.Thelastdebuglogisaddedbymodulengx_echobecauseitneedstoflagtheendofoutput.TheflagoperationitselfcausesNginx's"outputfilter"tobeexecutedagain.Manymodulesincludingngx_proxyhassimilarbehavior,whentheyneedtogiveoutputdata.

Allright,therearenosurpriseswiththoseduplicated56outputs.Wearenotgivenachancetoexecuteechoinfrontofthesecondsetcommand.Luckily,wecanstillachievethiswithafewtechniques:

location/test{

set$a32;

set$saved_a$a;

set$a56;

echo$saved_a;

echo$a;

}

Nowwehavewhatwehavewanted:


32

56

Withthehelpofanotheruservariable$saved_a,thevalueof$aissavedbeforeitisoverwritten.Becareful,theexecutionorderofmultiplesetcommandsareensuredtobeliketheirorderofwritingbymodule.Similarly,modulengx_echoensuresmultipleechocommandsgetexecutedinthesameorderoftheirwriting.

IfwerecallexamplesinNginxVariables,thistechniquehasbeenappliedextensively.ItbypassestheexecutionorderingdifficultiesintroducedbyNginxphasedprocessing.

Youmightneedtoask:"howwouldIknowthephaseaNginxcommandbelongsto?"Indeed,theanswerisRTFD.(SurelyadvanceddeveloperscanexaminetheCsourcecodedirectly).Manymodulemarksexplicitlyitsapplicablephaseinthemodule'sdocumentation,suchascommandechowritesbelowinitsdocumentation:

phase:content

Itsaysthecommandisexecutedinphasecontent.Ifyouencountersamodulewhichmissestheapplicablephaseinthedocument,youcanwritetoitsauthorsrightawayandaskforit.However,weshallbereminded,noteverycommandhasanapplicablephase.ExamplesarecommandgeointroducedinNginxVariables(01)andcommandmapintroducedinNginxVariables(04).Thesecommands,whohavenoexplicitapplicablephase,aredeclarativeandunrelatedtotheconceptionofexecutionordering.IgorSysoev,theauthorofNginx,hasmadethestatementsafewtimespublicly,thatNginxminilanguageinitsconfigurationis"declarative"not"procedural".


















Nginxdirectiveexecutionorder(02)We'vejustlearnt,allsetcommandswithinlocationareexecutedinrewritephase.Infact,almostallcommandsimplementedbymodulerewriteareexecutedinrewritephaseunderthespecificcontext.CommadrewriteintroducedinNginxVariables(02)isoneofthem.However,weshallpointoutthatwhenthesecommandsarefoundinserverdirective,theywillbeexecutedinanearlierphasewe'venotaddressed:theserverrewritephase.

Commandset_unescape_uri,introducedinNginxVariables(02)isalsoexecutedinrewritephase.Actually,commandsimplementedbymodulengx_set_misccanmixwithcommandsimplementedbymodulengx_rewriteandtheexecutionorderingisensured.Let'scheckanexample:

location/test{

set$a"hello%20world";

set_unescape_uri$b$a;

set$c"$b!";

echo$c;

}

Bysendingarequestaccordinglywehave:


helloworld!

Apparently,theset_unescape_uricommandanditsneighboringsetcommandsareallexecutedintheorderoftheirwriting.

Tofurtherdemonstrateourassertion,wecheckagainNginx"debuglog"(incaseit'sunclearforyouhowtocheck"debuglog",pleasereferencestepsfoundin(01)).

grep-E'httpscript(value|copy|set)'logs/error.log

Thedebuglogsarefilteredas:

[debug]11167#0:*1httpscriptvalue:"hello%20world"


[debug]11167#0:*1httpscriptvalue(postfilter):"helloworld"

[debug]11167#0:*1httpscriptset$b

[debug]11167#0:*1httpscriptcopy:"!"

[debug]11167#0:*1httpscriptset$c

Theleadingtwolines:

[debug]11167#0:*1httpscriptvalue:"hello%20world"


Theycorrespondtothecommand


Thefollowingtwolines:

[debug]11167#0:*1httpscriptvalue(postfilter):"helloworld"

[debug]11167#0:*1httpscriptset$b

Theyaregeneratedbycommand


Thereareminordifferencesinthefirstline,ifwecomparetothelogsgeneratedbycommandset:the"(postfilter)"addition.Intheendoftheline,URLdecodinghassuccessfullyexecutedaswewish."hello%20world"isdecodedas"helloworld".

Thelasttwolinesofdebuglog:

[debug]11167#0:*1httpscriptcopy:"!"

[debug]11167#0:*1httpscriptset$c

Theyaregeneratedbythelastsetcommand

set$c"$b!";

Asyoumighthavenoticed,since"variableinterpolation"isevaluatedwhenvariable$cisdeclaredandinitialized,thedebuglogstartswithhttpscriptcopy.Intheendofthelogitisthestringconstant"!"tobeconcatenated.

Withtheloginformation,it'sfairlyeasytotellthecommandexecutionordering:



set$c"$b!";

Itisaperfectmatchtothestatementsordering.

Justlikethecommandsimplementedinmodulengx_set_misc,commandset_by_luaimplementedin3rdpartymodulengx_lua,canmixwithcommandsofmodulengx_rewriteaswell.AsintroducedinNginxVariables(07),commandset_by_luasupportscomputationwithgivenLuacode,andassignsthecomputedresulttoaNginxvariable.Ascommandsetdoes,commandset_by_luadeclaresNginxvariablebeforeinitializationifthevariabledoesnotexist.

Let'scheckamixedexamplewhichcomprisescommandset_by_luaandset:











http://wiki.nginx.org/HttpLuaModule#set_by_lua








location/test{

set$a32;

set$b56;

set_by_lua$c"returnngx.var.a+ngx.var.b";

set$equation"$a+$b=$c";

echo$equation;

}

Variable$aand$bareinitializedwithnumericalvalue32and56respectively,thencommandset_by_luaisusedtogetherwithgivenLuacodetocomputethesumof$aand$b.Variable$cisinitializedwiththecomputedvalue.Finally,variables$a,$band$careconcatenatedby"variableinterpolation"andassignstheresulttovariable$equation,whichisprintedbycommandecho.

Weshallpayattentiontoafewpointsintheexample:FirstlyNginxvariable$VARIABLEisreferencedasngx.var.VARIABLEinLuacode.Secondly,sinceNginxvariablesarestrings,thevalueofvariablengx.var.aandngx.var.bareactuallystrings"32"and"56",howevertheyareautomaticallyconvertedtonumericalvaluesbyLuaintheadditionoperation.ThirdlyLuacodereturnstoNginxvariable$cthecomputedsumvaluebystatementreturn.FinallywhenLuacodereturns,itactuallyconvertsthenumericalvaluebacktostring.(becausestringistheonlyvalidvalueforNginxvariable)

Theactualoutputmeetsourexpectation:


32+56=88

Thisinfactassertsthatcommandset_by_luacanmixwithcommandsimplementedbymodulengx_rewrite,suchasset.

Manyother3rdpartymodulessupportthemixwithmodulengx_rewriteaswell.Theexamplesincludemodulengx_array_var,discussedinNginxVariables(08)andmodulengx_encrypted_session,whichencryptssessions.Thelatterwillbestudiedindetailshortly.

Sincebuiltinmodulengx_rewriteisvirtuallyindispensable,it'sagreatadvantageforthe3rdpartymodulehasthecaliberofbeingmixedwith.Truthis,allofthose3rdpartymoduleshaveadoptedaspecialtechnique,whichallowsthe"injection"oftheirexecutionintocommandsofmodulerewrite(withthehelpofa3rdpartymodulengx_devel_kitdevelopedbyMarcusClyne).Fortherestregular3rdpartymodules,whichalsoregistertheirexecutioninphaserewrite,theircommandsareexecutedseparatelyfrommodulengx_rewriteinruntime.Infact,it'shardlyaccuratetotellthecommandsexecutionorderinginbetweendifferentmodules(strictlyspeakingtheyareusuallyexecutedintheorderofloading,butexceptiondoesexist).Forexamplebothmodules,AandBregistertheircommandstobeexecutedinphaserewrite,thenitiseitherthecaseinwhichcommandsofAareexecutedfollowedbyBortheothercompletewayaround.Unlessitisexplicitlydocumented,wecannotrelyontheuncertainorderinginourconfigurations.



http://wiki.nginx.org/HttpLuaModule#ngx.var.VARIABLE






http://wiki.nginx.org/HttpEncryptedSessionModule


https://github.com/simpl/ngx_devel_kit


Nginxdirectiveexecutionorder(03)Asdiscussedearlier,unlessspecialtechniquesareutilizedasmodulengx_set_miscdoes,amodulecannotmixitscommandswithngx_rewrite,andexpectsthecorrectexecutionorder.Evenifthecommandsareregisteredintherewritephaseaswell.Wecandemonstratewithsomeexamples.

3rdpartymodulengx_headers_moreprovidesafewcommands,whichdealwiththecurrentrequestheaderandresponseheader.Oneofthemismore_set_input_header.Thecommandcanmodifyagivenrequestheaderinrewritephase(oraddthespecificheaderifit'snotavailableincurrentrequest).Asdescribedinitsdocumentation,thecommandalwaysexecutesintheendofrewritephase:

phase:rewritetail

Beingtersethough,rewritetailmeanstheendofphaserewrite.

Sinceitexecutesintheendofphaserewrite,theimplicationisitsexecutionisalwaysafterthecommandsimplementedinmodulengx_rewrite.Evenifitiswrittenattheverybeginning:

?location/test{

?set$valuedog;

?more_set_input_headers"X-Species:$value";

?set$valuecat;

?

?echo"X-Species:$http_x_species";

?}

AsbrieflyintroducedinNginxVariables(02),Builtinvariable$http_XXXhastheheaderXXXforthecurrentrequest.Wemustbecarefulthough,variable<$http_XXX>matchestothenormalizedrequestheader,i.e.itlowercasescapitallettersandturnsminus-intounderscore_fortherequestheadernames.Thereforevariable$http_x_speciescansuccessfullycatchestherequestheaderX-Species,whichisdeclaredbycommandmore_set_input_header.

Becauseofthestatementordering,wemighthavemistakenlyconcludedheaderX-Specieshasthevaluedogwhen/testisrequested.Buttheactualresultisdifferent:


X-Species:cat

Clearly,statementset$valuecatisexecutedearlierthanmore_set_input_headers,althoughitiswrittenafterwards.

Thisexampletellsusthatcommandsofdifferentmodulesareexecutedindependentlyfromeachother,eveniftheyareallregisteredinthesameprocessingphase.(unlessitisimplementedasmodulengx_set_misc,whosecommandsarespecificallytunedwithmodulengx_rewrite).Inotherwords,everyprocessingphaseisfurtherdividedintosub-phasesbyNginxmodules.

Similartomore_set_input_headers,commandrewrite_by_luaprovidedby3rdpartymodulengx_luaexecuteintheendofrewritephaseaswell.Wecanverifythis:

?location/test{

?set$a1;

?rewrite_by_lua"ngx.var.a=ngx.var.a+1";

?set$a56;

?

?echo$a;

?}

ByusingLuacodespecifiedbycommandrewrite_by_luaNginxvariable$aisincrementedby1.Wemighthaveexpectedtheresultbe56ifwearelookingatthewritingsequence.Theactualresultis57becausecommandisalwaysexecutedafterallthesetstatements.


57

Admittedlycommandrewrite_by_luahasdifferentbehaviorthancommandset_by_lua,whichisdiscussedin(02).

Outofsheercuriosity,weshallaskimmediatelythatwhatwouldbeexecutionorderinginbetweenmore_set_input_headersandrewrite_by_lua,sincetheybothrideonrewritetail?Theansweris:undefined.Wemustavoidaconfigurationwhichreliesontheirexecutionorders.

Nginxphaserewriteisaratherearlyprocessingphase.Usuallycommandsregisteredinthisphaseexecutevariousrewritetasksontherequest(forexamplerewritetheURLortheURLparameters),thecommandsmightalsodeclareandinitializeNginxvariableswhichareneededinthesubsequenthandling.Certainly,onecannotforbidotherstocomplicatethemselvesbycheckingtherequestbody,orvisitadatabaseetc.Afterall,commandlikerewrite_by_luaoffersthecalibertostuffinanypotentiallymindtwistedLuacode.

Afterphaserewrite,Nginxhasanotherphasecalledaccess.Thecommandsprovidedby3rdpartymodulengx_auth_request,whichisdiscussedinNginxVariables(05),executeinphaseaccess.CommandsregisteredinaccessphasemostlycarryoutACLfunctionalities,suchasguardinguserclearance,checkinguserorigins,examiningsourceIPvalidityetc.

Forexamplecommandallowanddenyprovidedbybuiltinmodulengx_accesscancontrolwhichIPaddresseshavetheprivilegestovisit,orwhichIPaddressesarerejected:

location/hello{

allow127.0.0.1;

denyall;

echo"helloworld";

}

Location/helloallowsvisitfromlocalhost(IPaddress127.0.0.1)andrejectrequestsfromallotherIPaddresses(returnshttperror403)Therulesdefinedbyngx_accesscommandsareassertedinthewritingsequence.Onceoneruleismatched,theassertionstopsandalltherestallowordenycommandsareignored.Ifno



http://wiki.nginx.org/HttpHeadersMoreModule

http://wiki.nginx.org/HttpHeadersMoreModule#more_set_input_header

http://wiki.nginx.org/HttpCoreModule#.24http_HEADER

http://wiki.nginx.org/HttpHeadersMoreModule#more_set_input_header

http://wiki.nginx.org/HttpHeadersMoreModule#more_set_input_headers




http://wiki.nginx.org/HttpLuaModule#rewrite_by_lua










http://wiki.nginx.org/HttpAccessModule#allow

http://wiki.nginx.org/HttpAccessModule#deny

http://wiki.nginx.org/HttpAccessModule




ruleismatched,handlingcontinuesinthefollowingstatements.Ifthematchedruleisdeny,handingisabortedanderror403isreturnedimmediately.Inourexample,requestissuedfromlocalhostmatchestotheruleallow127.0.0.1andhandingcontinuestotheotherstatements,howeverrequestissuedfromeveryotherIPaddresseswillmatchruledenyallhandlingisthereforeabortedanderror403isreturned.

Wecangiveitatest,bysendingrequestfromlocalhost:

$curl'http://localhost:8080/hello'

helloworld

Ifrequestissentfromanothermachine(supposeNginxrunsonIP192.168.1.101)wehave:

$curl'http://192.168.1.101:8080/hello'

<html>

<head><title>403Forbidden</title></head>

<bodybgcolor="white">

<center><h1>403Forbidden</h1></center>

<hr><center>nginx</center>

</body>

</html>

Bytheway,modulengx_accesssupportsthe"CIDRnotation"todesignateasub-network.Forexample169.200.179.4/24representsthesub-networkwhichhastheroutingprefix169.200.179.0(orsubnetmask255.255.255.0)

Becausecommandsofmodulengx_accessexecuteinaccessphase,andphaseaccessisbehindrewritephase.Soforthosecommandswehavebeendiscussing,regardlessofthewritingordertheyalwaysexecuteinrewritephase,whichisearlierthanallowordeny.Keepthisinmind,weshalltryourbesttokeepthewritingandexecutionorderconsistent.






Nginxdirectiveexecutionorder(04)Modulengx_luaimplementsanothercommandaccess_by_lua.Thecommandallowsluacodetobeexecutedintheendofaccessphase,whichmeansitalwaysexecutesafterallowanddenyeventheybelongtothesamephase.Inmanycases,weexaminetherequest'ssourceIPaddresswithngx_access,andusecommandaccess_by_luatoexecutemorecomplicatedverificationswithLua.Forexamplebyqueryingadatabaseorotherbackendservices,thecurrentuser'sidentityandprivilegesareexamined.

Wecancheckasimpleexample,whichusescommandaccess_by_luatoimplementtheIPfilteringfunctionalityofmodulengx_access

location/hello{

access_by_lua'

ifngx.var.remote_addr=="127.0.0.1"then

return

end

ngx.exit(403)

';

echo"helloworld";

}

Nginx'sbuiltinvariable$remote_addrisreferencedinLuatogettheclient'sIPaddress.ThenLuastatementifisusedtodetermineiftheaddressequals127.0.0.1.Luareturnsifitequals,Nginxthuscontinuesthesubsequenthandling(includingthecontentphasewherecommandechoappliesto).Ifitisnotthelocalhostaddress,currenthandlingisabortedbyusingngx_luamodule'sLuafunctionngx.exitClientgetsahttperror403.

Theexampleisequivalenttotheotherexampleusingngx_accessmoduleintermsoffunctionality,whichwasdiscussedin(03):

location/hello{

allow127.0.0.1;

denyall;

echo"helloworld";

}

Howeverweshallpointout,performancewisethetwostillhavedifferences.Modulengx_accessperformsbetterbecauseitisspecificallyimplementedasaNginxmoduleinC.

Wecanmeasuretheperformancedifferencesofthetwo.Afterall,performanceiswhatweareafterbyusingNginx.Ontheotherhand,it'sabsolutelynecessarytobeequippedwithmeasuringtechniques,becauseonlyactualdatadistinguishesamateursandprofessionals.Infact,bothngx_luaandngx_accessperformprettygoodforIPfiltering.Tominimizemeasuringerrorswecouldmeasuredirectlytheelapsedtimeofaccessphase.Traditionally,thismeanshackingNginxsourcecodewithtimingcodeandstatisticalcode,orrecompileNginxbinarysothatitcanbemonitoredbyspecificprofilingtoolslikeGNUgprof.

Wearelucky,becausecurrentreleasesofSolaris,MacOSXorFreeBSDofferasystemutilitydtrace,whichallowsmicromonitoringofuserprocessintermsofperformance(andfunctionalityaswell).Thetoolsparesusfromhackingsourcecodeorrecompilationwithprofiling.Let'sdemonstratethemeasuringscenarioontheMacBookAirbecausedtraceisavailablesinceMacOSX10.5

First,opentheTerminalapplicationofMacOSX,changetoyourpreferablepathandcreateafilenamedasnginx-access-time.d,editthefilewithfollowingcontent:

#!/usr/bin/envdtrace-s

pid$1::ngx_http_handler:entry

{

elapsed=0;

}

pid$1::ngx_http_core_access_phase:entry

{

begin=timestamp;

}

pid$1::ngx_http_core_access_phase:return

/begin>0/

{

elapsed+=timestamp-begin;

begin=0;

}

pid$1::ngx_http_finalize_request:return

/elapsed>0/

{

@elapsed=avg(elapsed);

elapsed=0;

}

Savethefileandmakeitexecutable.

$chmod+x./nginx-access-time.d

The.dfileactuallycontainscodewritteninDlanguageofferedbyutilitydtrace(attention,theDlanguageisnottheotherDlanguage,whichisadvocatedbyWalterBrightforabetterC++).SofarwecannotreallyexplainindetailthecodebecauseitrequiresathoroughunderstandingofNginxinternals.Anywayweshallbeclearof


http://wiki.nginx.org/HttpLuaModule#access_by_lua







http://wiki.nginx.org/HttpCoreModule#.24remote_addr



http://wiki.nginx.org/HttpLuaModule#ngx.exit





thecode'spurpose:measurerequestsbeinghandledbyspecificNginxworkerprocessandcalculatetheaveragetimeelapsedinaccessphase.

NowwecangettheDscriptrunning.Thescripttakesacommandlineparameter,whichistheprocessid(pid)ofNginxworker.SinceNginxsupportsmultipleworkerprocessesandtherequestscanberandomlyhandledbyanyoneofthem,we'dliketoconfigureNginxinitsconfigurationnginx.confsothatonlyoneworkerisrequested.

worker_processes1;

AfterNginxbinaryisrestarted,theworkerprocessidcanbeobtainedbycommandps.

$psax|grepnginx|grepworker|grep-vgrep

Typicallywehave:

10975??S0:34.28nginx:workerprocess

10975ismyNginxworkerpid.Incaseyouhavemultiplelines,youmusthavestartedmultipleNginxserverinstancesorthecurrentNginxserverhasstartedmultipleworkerprocesses.

Thenasroot,scriptnginx-access-time.disexecutedwiththeworkerpid

$sudo./nginx-access-time.d10975

WeshallhaveoneoutputmessageifeverythinggoesOK.

dtrace:script'./nginx-access-time.d'matched4probes

ThemessagesaysourDscripthassuccessfullydeployed4probesonthetargetprocess.Thenthescriptisreadytotraceprocess10975constantly.

Let'sopenanotherTerminal,andsendmultiplerequestswithcurltoourmonitoredprocess


helloworld


helloworld

BacktoourTerminalwhereDscriptisrunning,presskeysCtrl-Ctointerruptit.Whenthescriptbailsoutitprintsonconsolethestatisticalresult.Forexamplemyconsolehasfollowingresult:

$sudo./nginx-access-time.d10975

dtrace:script'./nginx-access-time.d'matched4probes

^C

19219

Thefinal19219istheaveragetimeelapsedinaccessphaseinnanoseconds(1second=1000x1000x1000nanoseconds)

Donewiththesteps.Wecanrunthenginx-access-time.dscripttocalculateaverageelapsedtimeinphaseaccessforthreedifferentNginxsetupsrespectively.TheyareIPfilteringwithmodulengx_access,IPfilteringwithcommandaccess_by_lua,andfinallynofilteringforaccessphase.Thelastresulthelpseliminatethesideeffectcausedbyprobesorother"systematicerrors".Besides,wecanusetrafficloadertoolssuchasabtosendshalfamillionrequeststominimize"randomerrors",asbelow:

$ab-k-c1-n100000'http://127.0.0.1:8080/hello'

ThereforethestatisticalresultofDscriptisascloseaspossibletothe"actual"time.

IntheMacOSX,atypicalrunhasfollowingresults:

ngx_access18146

access_by_lua35011

nofiltering15887

Weminusthelastvaluefromtheformertwo:

ngx_access2259

access_by_lua19124

Well,modulengx_accessoutperformscommandaccess_by_luabyamagnitude,aswemighthaveexpected.Stilltheabsolutedifferenceistiny.FortheIntelCore2Due1.86GHzCPUofmine,thereisonlyafewmicroseconds.

Infacttheaccess_by_luaexamplecanbefurtheroptimizedusingbuiltinvariable$binary_remote_addr.ThisvariablehastheIPaddressinbinaryformwhereasvariable$remote_addrhastheaddressinalongerstringformat.ShorteraddresscanbecomparedquickerwhenLuaexecutesitsstringoperations.

Becareful,if"debuglog"isenabledasintroducedin(01)thecomputedelapsedtimewillincreasedramatically,because"debuglog"hasahugeoverhead.






http://wiki.nginx.org/HttpCoreModule#.24binary_remote_addr


Nginxdirectiveexecutionorder(05)contentisbyallmeansthemostsignificantphaseinNginx'srequesthandling,becausecommandsrunninginthephasehavetheresponsibilitytogenerate"content"andoutputHTTPresponse.Becauseofitsimportance,Nginxhasarichsetofcommandsrunninginit.Thecommandsincludeecho,echo_exec,proxy_pass,echo_location,content_by_lua,whichwerediscussedinNginxVariables(02),NginxVariables(03),NginxVariables(05)andNginxVariables(07)respectively.

contentisaphasewhichrunslaterthanrewriteandaccess.Thereforeitscommandsalwaysexecuteintheendwhentheyareusedtogetherwithcommandsofrewriteandaccess.

location/test{

#rewritephase

set$age1;

rewrite_by_lua"ngx.var.age=ngx.var.age+1";

#accessphase

deny10.32.168.49;

access_by_lua"ngx.var.age=ngx.var.age*3";

#contentphase

echo"age=$age";

}

Thisisaperfectexample,inwhichcommandsareexecutedinanexactsequenceastheyarewritten.Thetestingresultmatchestoourexpectationstoo.


age=6

Infact,thecommands'writingordercanbecompletelyshuffledanditwon'thaveanyimpacttotheirexecutionsequence.Commandset,whichisimplementedbymodulengx_rewrite,executesinrewritephase.Commandrewrite_by_luafrommodulengx_luaexecutesintheendofrewritephase.Commanddenyfrommodulengx_accessexecutesinaccessphase.Commandaccess_by_luafrommodulengx_luaexecutesintheendofaccessphase.Finally,ourfavoritecommandecho,implementedbymodulengx_echo,executesincontentphase.

TheexamplealsodemonstratesthecollaboratinginbetweencommandsrunningoneachdifferentNginxphase.Intheprocess,Nginxvariableisthedatacarrierinterconnectingcommandsandmodules.Theexecutionorderofthesecommandsislargelydecidedbythephaseeachappliesto.

Asmatteroffact,multiplecommandsfromdifferentmodulescouldcoexistinphaserewriteandaccess.Astheexampleshows,commandsetandcommandrewrite_by_luabothbelongtophaserewrite.Commanddenyandcommandaccess_by_luabothbelongtophaseaccess.Howeveritisnotthesamestoryforphasecontent.

Mostmodules,whentheyimplementcommandsforphasecontent,theyareactuallyinserting"contenthandler"forthecurrentlocationdirective,howevertherecanbeoneandonlyone"contenthandler"foralocation.Soonlyonemodulecouldbeattherestwhenmultiplemodulesarecontendingtherole.Considerfollowingproblematicexample:

?location/test{

?echohello;

?content_by_lua'ngx.say("world")';

?}

Commandechofrommodulengx_echoandcommandcontent_by_luafrommodulengx_luabothexecuteinphasecontent.Butonlyoneofthemcouldsuccessfullybecome"contenthandler":


world

Ourtestindicates,thatthewinneriscontent_by_luaalthoughitiswrittenafterwards,andcommandechoneverreallyhasachancetorun.Wecannotbeassuredwhichmodulewinsinthecircumstance.Forexample,modulengx_echowinsandtheoutputbecomeshelloifweswapthecontent_by_luaandechostatements.Soweshallavoidtousemultiplecommandsforphasecontent,ifthecommandsareimplementedbydifferentmodules.

Theexamplecanbemodifiedbyreplacingcommandcontent_by_luawithcommandechoandwewillgetwhatweneed:

location/test{

echohello;

echoworld;

}

Againtestproves:


hello

world

Wecanusemultipleechocommands,thereisnoproblemwiththisbecausetheyallbelongtomodulengx_echo.Modulengx_echoregulatestheexecutionorderingofthem.Becarefulthough,noteverymodulesupportsthecommandsbeingexecutedmultipletimeswithinonelocation.Commandcontent_by_luaforaninstance,canbeusedonlyonce,sofollowingexampleisincorrect:

?location/test{

?content_by_lua'ngx.say("hello")';

?content_by_lua'ngx.say("world")';

?}

Nginxdumpserrorfortheconfiguration:



































[emerg]"content_by_lua"directiveisduplicate...

Thecorrectwayofdoingitis:

location/test{

content_by_lua'ngx.say("hello")ngx.say("world")';

}

Insteadofusingtwicethecontent_by_luacommandinlocation,theapproachistocallfunctionngx.saytwiceintheLuacode,whichisexecutedbycommandcontent_by_lua

Similarly,commandproxy_passfrommodulengx_proxycannotcoexistwithcommandechowithinonelocationbecausetheybothexecuteincontentphase.ManyNginxnewbiesmakefollowingmistake:

?location/test{

?echo"before...";

?proxy_passhttp://127.0.0.1:8080/foo;

?echo"after...";

?}

?

?location/foo{

?echo"contentstobeproxied";

?}

Theexampletriestooutputstrings"before..."and"after..."withcommandechobeforeandaftermodulengx_proxyreturnsitscontent.Howeveronlyonemodulecouldexecuteincontent.Thetestindicatesmodulengx_proxywinsandcommandechofrommodulengx_echoneverruns


contentstobeproxied

Toimplementwhattheexamplehadwantedto,weshallusetwoothercommandsprovidedbymodulengx_echo,echo_before_bodyandecho_after_body:

location/test{

echo_before_body"before...";

proxy_passhttp://127.0.0.1:8080/foo;

echo_after_body"after...";

}

location/foo{

echo"contentstobeproxied";

}

Testtellswemakeit:


before...

contentstobeproxied

after...

Thereasoncommandsecho_before_bodyandecho_after_bodycouldcoexistwithothermodulesincontentphase,istheyarenot"contenthandler"but"outputfilter"ofNginx.Backin(01)whenweexaminethe"debuglog"generatedbycommandecho,we'velearntNginxcallsits"outputfilter"wheneverNginxoutputsdata.Sothatmodulengx_echotakestheadvantageofittomodifycontentgeneratedbymodulengx_proxy(byaddingsurroundingcontent).Weshallpointoutthough,"outputfilter"isnotoneofthose11phasesmentionedin(01)(manyphasescouldtrigger"outputfilter"whentheyoutputdata).Stillit'sperfectlyallrighttodocumentcommandsecho_before_bodyandecho_after_bodyasfollowing:

phase:outputfilter

Itmeansthecommandexecutesin"outputfilter".













http://wiki.nginx.org/HttpEchoModule#echo_before_body

http://wiki.nginx.org/HttpEchoModule#echo_after_body








Nginxdirectiveexecutionorder(06)We'velearntin(05)thatwhenacommandexecutesincontentphaseforaspecificlocation,itusuallymeansitsNginxmoduleregistersa"contenthandler"forthelocation.However,whathappensifnomoduleregistersitscommandas"contenthandler"forphasecontent?Whowillbetakingthegloryofgeneratecontentandoutputresponses?Theansweristhestaticresourcemodule,whichmapstherequestURItothefilesystem.Staticresourcemoduleonlycomesintoplaywhenthereisnone"contenthandler",otherwiseithandsoffthedutyto"contenthandler".

TypicallyNginxhasthreestaticresourcemodulesforthecontentphase(unlessoneormoreofthosemodulesaredisabledexplicitly,orsomeotherconflictingmodulesareenabledwhenNginxisbuilt)Thethreemodules,intheorderoftheirexecutionorder,arengx_indexmodule,ngx_autoindexmoduleandngx_staticmodule.Let'sdiscussthemonebyone.

Modulengx_indexandngx_autoindexonlyapplytothoserequestURI,whichendswith/.FortheotherrequestURIwhichdoesnotendwith/,bothmodulesignorethemandletthefollowingcontentphasemodulehandle.Modulengx_statichowever,hasanexactoppositestrategy.ItignorestherequestURIwhichendswith/andhandlestherest.

Modulengx_indexmainlylooksforaspecifichomepagefile,suchasindex.htmlorindex.htminthefilesystem.Forexample:

location/{

root/var/www/;

indexindex.htmindex.html;

}

Whenaddress/isrequested,Nginxlooksforfileindex.htmandindex.html(inthisorder)inapathinthefilesystem.Thepathisspecifiedbycommandroot.Iffileindex.htmexists,Nginxjumpsinternallytolocationindex.htm;ifitdoesnotexistandfileindex.htmlexists,Nginxjumpsinternallytolocationindex.html.Iffileindex.htmldoesnotexisteither,andhandlingistransferredtotheothermodulewhichexecutesitcommandsinphasecontent.

WehavelearntinNginxVariables(02),commandsecho_execandrewritecantrigger"internalredirects"aswell.ThejumpmodifiestherequestURI,andlooksforthecorrespondinglocationdirectiveforsubsequenthandling.Intheprocess,phasesrewrite,accessandcontentarereiteratedforthelocation.The"internalredirect"isdifferentfromthe"externalredirect"definedbyHTTPresponsecode302and301,clientbrowserwon'tupdateitsURIaddresses.Thereforeassoonasinternaljumpoccurswhenmodulengx_indexfindsthefilesspecifiedbycommandindex,theneteffectislikeclientwouldhavebeenrequestingthefile'sURIattheverybeginning.

Wecancheckfollowingexampletowitnessthe"internalredirect"triggeredbymodulengx_index,whenitfindstheneededfile.

location/{

root/var/www/;

indexindex.html;

}

location/index.html{

set$a32;

echo"a=$a";

}

Weneedtocreateanemptyfileindex.htmlunderthepath/var/www/,andmakesurethefileisreadablefortheNginxworkerprocess.Thenwecouldsendrequestto/:

$curl'http://localhost:8080/'

a=32

Whathappened?Whytheoutputisnotthecontentoffileindex.html(whichshallbeempty)?FirstlyNginxusesdirectivelocation/tohandleoriginalGET/request,thenmodulengx_indexexecutesincontentphase,anditfindsfileindex.htmlunderpath/var/www/.Atthismoment,ittriggersan"internalredirect"tolocation/index.html.

Sofarsogood.Butherecomesthesurprises!WhenNginxlooksforlocationdirectivewhichmatchesto/index.html,location/index.htmlhasahigherprioritythanlocation/.ThisisbecauseNginxuses"longestmatchedsubstring"semanticstomatchlocationdirectivestorequestURI'sprefix.Whendirectiveischosen,phasesrewrite,accessandcontentarereiterated,andeventuallyitoutputsa=32.

Whatifweremovefile/var/www/index.htmlintheexample,andrequestto/again?Theansweriserror403Forbidden.Why?Whenmodulengx_indexcannotfindthefilespecifiedbycommandindex(index.htmlinhere),ittransfersthehandlingtothefollowingmodulewhichexecutesincontent.Butnoneofthosefollowingmodulescanfulfilltherequest,Nginxbailsoutanddumpsuserror.MeanwhileitlogstheerrorinNginxerrorlog:

[error]28789#0:*1directoryindexof"/var/www/"isforbidden

Themeaningofdirectoryindexistogenerate"indexes".Usuallythisimpliestogenerateawebpage,whichlistseveryfileandsubdirectoriesunderpath/var/www/.Ifweusemodulengx_autoindexrightafterngx_index,itcangeneratesuchapagejustlikewhatweneed.Nowlet'smodifytheexamplealittlebit:

location/{

root/var/www/;

indexindex.html;

autoindexon;

}

When/isrequestedagainmeanwhilefile/var/www/index.htmliskeptmissing.Anicehtmlpageisgenerated:

$curl'http://localhost:8080/'

<html>

<head><title>Indexof/</title></head>


<h1>Indexof/</h1><hr><pre><ahref="../">../</a>

<ahref="cgi-bin/">cgi-bin/</a>08-Mar-201019:36-

http://wiki.nginx.org/HttpIndexModule

http://wiki.nginx.org/HttpAutoindexModule




http://wiki.nginx.org/HttpCoreModule#root




http://wiki.nginx.org/HttpIndexModule#index




http://wiki.nginx.org/HttpIndexModule#index



<ahref="error/">error/</a>08-Mar-201019:36-

<ahref="htdocs/">htdocs/</a>05-Apr-201003:55-

<ahref="icons/">icons/</a>08-Mar-201019:36-

</pre><hr></body>

</html>

Thepageshowsthereareafewsubdirectoriesundermy/var/www/.Theyarecgi-bin/,error/,htdocs/andicons/.Theoutputmightbedifferentifyouhavetriedbyyourself.

Again,iffile/var/www/index.hmtldoesexist,modulengx_indexwilltrigger"internalredirect",andmodulengx_autoindexwillnothaveachancetoexecute,youmaytestityourselftoo.

The"goalkeeper"moduleexecutedinphasecontentisngx_static.whichisalsousedintensively.Themoduleservesthestaticfiles,includingthestaticresourcesofawebsite,suchasstatic.htmlfiles,static.cssfiles,static.jsfilesandstaticimagefilesetc.Althoughngx_indexcouldtriggeran"internalredirect"tothespecifiedhomepage,buttheactualoutputtask(takesthefilecontentasresponse,andmarksthecorrespondingresponseheaders)iscarriedoutbymodulengx_static.



Nginxdirectiveexecutionorder(07)Let'scheckanexampleinwhichmodulengx_staticservesdiskfiles,withfollowingconfigurationsnippet:

location/{

root/var/www/;

}

Meanwhiletwofilesarecreatedunder/var/www/.Onefileisnamedindex.htmlanditscontentcontainsonelineoftextthisismyhome.Anotherfileisnamedhello.htmlanditscontentcontainsonelineoftexthelloworld.Againbeawareofthefiles'privilegesandmakesuretheyarereadablebyNginxworkerprocess.

Nowwesendrequeststothefiles'correspondingURI:

$curl'http://localhost:8080/index.html'

thisismyhome

$curl'http://localhost:8080/hello.html'

helloworld

Aswecansee,thecreatedfilecontentsaresentasoutputs.

Wecanexaminewhatishappeninghere:location/doesnothaveanycommandtoexecuteinphasecontent,thereforenomodulehasregistereda"contenthandler"inthelocation.Thehandlingthusfallstothethreestaticresourcemoduleswhicharethelastresortsofphasecontent.Theformertwomodulesngx_indexandngx_autoindexnoticesthattherequestURIdoesnotendwith/sotheyhandoffimmediatelytomodulengx_static,whichrunsintheend.Accordingtothe"documentroot"specifiedbycommandroot,modulengx_staticmapstherequestURIs/index.htmland/hello.htmltodiskfiles/var/www/index.htmland/var/www/hello.htmlrespectively.Asbothfilescanbefound,theircontentareoutputtedasresponse,meanwhileresponseheaderContent-Type,Content-LengthandLast-Modifiedareaccordinglyindicated.

Toverifymodulengx_statichasexecuted,wecouldenablethe"debuglog"introducedin(01).Againwesendrequestto/index.htmlandNginxerrorlogwillcontainfollowingdebuginformation:

[debug]3033#0:*1httpstaticfd:8

Thislineisgeneratedbymodulengx_static.Itsmeaningis"outputtingstaticresourcewhosefilehandleis8".Ofcoursethenumericalfilehandlechangeseverytime,andthelineisonlyatypicaloutputinmysetup.Tobereminded,builtinmodulengx_gzip_staticcouldgeneratethesamedebuginfoaswell,bydefaultitisnotenabledthough,whichwillbediscussedlater.

Commandrootonlydeclaresa"documentroot",itdoesnotenablesthengx_staticmodule.Themoduleisasmatteroffact,alwaysenabledalready,butitmightnothavethechancetoexecute.Thisisentirelyuptotheothermodules,whichexecuteearlierincontentphase.Modulengx_staticexecuteonlywhenallofthemhave"gaveup".Toprovethis,checkfollowingblanklocationdefinition:

location/{

}

Becausethereisnorootcommand,Nginxcomputesadefault"documentroot"whenthelocationisrequested.Thedefaultshallbethehtml/subdirectoryunder"configureprefix".Forexamplesupposeour"configureprefix"is/foo/bar/,thedefault"documentroot"is/foo/bar/html/.

Sowhodecides"configureprefix"?ActuallyittheNginxrootdirectorywhenitisinstalled(orthevalueof--prefixoptionofscript./configurewhenNginxisbuilt).IfNginxisinstalledinto/usr/local/nginx/,"configureprefix"is/usr/local/nginx/anddefault"documentroot"istherefore/usr/local/nginx/html/.Certainlyacommandlineoption--prefixcanbegivenwhenNginxisstarted,tochangethe"configureprefix"(sothatwecaneasilytestmultiplesetups).SupposeNginxisstartedasfollowing:

nginx-p/home/agentzh/test/

Forthisserver,its"configureprefix"becomes/home/agentzh/test/andits"documentroot"becomes/home/agentzh/test/html/.The"configureprefix"notonlydetermines"documentroot",itactuallydeterminesthewaymanyrelationalpathresolutestoabsolutepathinNginxconfiguration.Wewillencountermanyexampleswhichreference"configureprefix".

Infactthereisasimplewayoftellingcurrent"documentroot",whichistorequestanon-existedfile,Suchas:

$curl'http://localhost:8080/blah-blah.txt'

<html>

<head><title>404NotFound</title></head>


<center><h1>404NotFound</h1></center>


</body>

</html>

Naturally,the404errorpageisreturned.AgainwhenwecheckNginxerrorlog,weshallhavefollowingerrormessage:

[error]9364#0:*1open()"/home/agentzh/test/html/blah-blah.txt"failed(2:Nosuchfileordirectory)

Theerrormessageisprintedbymodulengx_static,sinceitcannotfindafileblah-blah.txtinitscorrespondingpath.Andbecausetheerrormessagecontainstheabsolutepath,whichngx_staticattemptstoopenwith,it'squiteobviousthatcurrent"documentroot"is/home/agentzh/test/html/.

Manynewbiesmighttakeitforgrantedthaterror404iscausedwhentheneededlocationdoesnotexist.Theformerexampletellsus,404errorcouldbereturnedeveniftheneededlocationisconfiguredandmatched.Thisisbecauseerror404meansthenon-existenceofanabstract"resource",notthespecificlocation.

Anotherfrequentmistakeismissingthecommandforphasecontent,whentheyactuallydon'texpectthedefaultstaticmodulestocomeintoplay,forexample:




http://wiki.nginx.org/HttpGzipStaticModule



location/auth{

access_by_lua'

--alotofLuacodeomittedhere...

';

}

Apparently,onlycommandsforphaseaccessaregivenfor/auth,whichisaccess_by_lua.Andithasnocommandsforphasecontent.Sowhen/authisrequested,theLuacodespecifiedinaccessphasewillexecute,thenthestaticresourcewillbeservedinphasecontentbymodulengx_static.Sinceitactuallylooksforthefile/authonthedisknormallyitdumpsa404errorunlessweareluckilyandfile/authiscreatedonthecorrespondingpath.Sothethumbofrule,whenerror404isencounteredundernostaticresourcecircumstances,weshallfirstcheckifthelocationhasproperlyconfigureditscommandsforphasecontent,thecommandscanbecontent_by_lua,echoandproxy_passetc.Infact,Nginxerrorlogerror.logcouldonlygiveveryconfusingmessageforthecase.Astheonesbelow,whichisfoundfortheaboveexample:

[error]9364#0:*1open()"/home/agentzh/test/html/auth"failed(2:Nosuchfileordirectory)





Nginxdirectiveexecutionorder(08)Sofarwehaveaddressedindetailrewrite,accessandcontent,whicharealsothemostfrequentlyencounteredphasesinNginxrequestprocessing.WehavelearntmanyNginxmodulesandtheircommandsthatexecuteinthosephases,andit'scleartousthatthecommands'executionorderisdirectlydecidedbythephasetheyarerunningin.UnderstandingthephaseisourkeynoteforcorrectconfigurationwhichorchestratesvariousNginxmodules.Thereforelet'scovertherestphaseswe'venotmet.

Asmentionedin(01),altogethertherecanbe11phaseswhenNginxhandlesarequest.Intheirexecutionorderthephasesarepost-read,server-rewrite,find-config,rewrite,post-rewrite,preaccess,access,post-access,try-files,content,andfinallylog.

Phasepost-readistheveryfirst,commandsregisteredinthisphaseexecuterightafterNginxhasprocessedtherequestheaders.Similartophaserewritewe'velearntearlier,post-readsupportshooksbyNginxmodules.Built-inmodulengx_realipisanexample,ithooksitshandlerinpost-readphase,andforcefullyrewritetherequest'soriginaladdressasthevalueofaspecificrequestheader.Thefollowingcaseillustratesngx_realipmoduleanditscommandsset_real_ip_from,real_ip_header.

server{

listen8080;

set_real_ip_from127.0.0.1;

real_ip_headerX-My-IP;

location/test{

set$addr$remote_addr;

echo"from:$addr";

}

}

TheconfigurationtellsNginxtoforcefullyrewritetheoriginaladdressofeveryrequestcomingfrom127.0.0.1tobethevalueoftherequestheaderX-My-IP.Meanwhileitusesthebuilt-invariable$remote_addrtooutputtherequest'soriginaladdress,sothatweknowiftherewriteissuccessful.

Firstwesendarequestto/testfromlocalhost:

$curl-H'X-My-IP:1.2.3.4'localhost:8080/test

from:1.2.3.4

Thetestutilizes-Hoptionprovidedbycurl,theoptionincorporatesanextraHTTPheaderX-My-IP:1.2.3.4intherequest.Aswecantell,variable$remote_addrhasbecome1.2.3.4inrewritephase,thevaluecomesfromtherequestheaderX-My-IP.SowhendoesNginxrewritetherequest'soriginaladdress?yesit'sinthepost-readphase.Sincephaserewriteisfarbehindphasepost-read,whencommandsetreadsvariable$remote_addr,itsvaluehasalreadybeenrewritteninpost-readphase.

Ifhowever,therequestsentfromlocalhostto/testdoesnothaveaX-My-IPheaderortheheadervalueisaninvalidIPaddress,Nginxwillnotmodifytheoriginaladdress.Forexample:

$curllocalhost:8080/test

from:127.0.0.1

$curl-H'X-My-IP:abc'localhost:8080/test

from:127.0.0.1

Ifarequestissentfromanothermachineto/test,itoriginaladdresswon'tbeoverwrittenbyNginxeither,evenifithasaperfectX-My-IPheader.Itisbecauseourpreviouscasemarksexplicitlywithcommandset_real_ip_from,thattherewritingonlyoccursfortherequestscomingfrom127.0.0.1.ThisfilteringmechanismprotectNginxfrommaliciousrequestssentbyuntrustedsources.Asyoumighthaveexpected,commandset_real_ip_fromcandesignateaIPsubnet(byusingCIDRnotationintroducedearlierin(03)).Besides,commandset_real_ip_fromcanbeusedmultipletimessothatwecansetupmultipletrustedsources,belowisanexample:


set_real_ip_from127.0.0.0/24;

Youmightbeasking,what'sthebenefitmodulengx_realipbringstous?Whywouldwerewritearequest'soriginaladdress?Theansweris:whentherequesthascomethroughoneormoreHTTPproxies,themodulebecomesveryhandy.Whenarequestisforwardedbyaproxy,itsoriginaladdresswillbecometheproxyserver'sIPaddress,consequentlyNginxandtheservicesrunningonitwillnolongerhavetheactualsource.However,wecouldletproxyserverrecordtheoriginaladdressinaspecificheader(suchasX-My-IP)andrecoveritinNginx,sothatitssubsequentprocessing(andtheservicesrunningonNginx)willtaketherequestasifitcomesrightfromitsoriginaladdressandtheproxiesinbetweenaretransparent.Forthisexactpurpose,modulengx_realipneedshookhandlersinthefirstphase,thepost-readphase,sotherewritingoccursasearlyaspossible.

Behindpost-readistheserver-rewritephase.Webrieflymentionedin(02),whenmodulengx_rewriteanditscommandsareconfiguredinserverdirective,theybasicallyexecuteinserver-rewritephase.Wehaveanexamplebelow:

server{

listen8080;

location/test{

set$b"$a,world";

echo$b;

}

set$ahello;

}

Attentiontheset$ahellostatementisputinserverdirective,soitrunsinserver-rewritephase,whichrunsearlierthanrewritephase.Thereforestatementset$b"$a,world'"inlocationdirectiveisexecutedafterwardsanditobtainsthecorrect$avalue:

http://wiki.nginx.org/HttpRealIpModule


http://wiki.nginx.org/HttpRealIpModule#set_real_ip_from

http://wiki.nginx.org/HttpRealIpModule#real_ip_header












hello,world

Sincephaseserver-rewriteexecuteslaterthanpost-readphase,commandsetinserverdirectivealwaysrunslaterthanmodulengx_realip,whichrewritestherequest'soriginaladdress,example:

server{

listen8080;



real_ip_headerX-Real-IP;

location/test{

echo"from:$addr";

}

}

Sendrequestto/testwehave:

$curl-H'X-Real-IP:1.2.3.4'localhost:8080/test

from:1.2.3.4

Again,commandsetiswritteninfrontofcommandsofngx_realip,itsactualexecutionisonlyafterwards.Sowhencommandsetassignsvariable$addrinserver-rewritephase,thevariable$remote_addrhasbeenoverwritten.







Nginxdirectiveexecutionorder(09)Rightafterserver-rewriteisthephasefind-config.ThisphasedoesnotallowNginxmodulestoregistertheirhandlers,insteaditisaphasewhenNginxcorematchesthecurrentrequesttothelocationdirectives.Itmeansarequestisnotcateredbyanylocationdirectiveuntilitreachesfind-config.Apparently,forphaseslikepost-readandserver-rewrite,theeffectivecommandsarethosewhichgetspecifiedonlyinserverdirectivesandtheirouterdirectives,becausethetwophasesareexecutedearlierthanfind-config.Thisexplainsthatcommandsofmodulengx_rewriteareexecutedinphaseserver-rewriteonlyiftheyarewrittenwithinseverdirective.Similarly,theformerexamplesconfigurethecommandsofmodulengx_realipinserverdirectivetomakesurethehandlersregisteredinpost-readphasecouldfunctioncorrectly.

AssoonasNginxmatchesalocationdirectiveinthefind-configphase,itprintsadebuglogintheerrorlogfile.Let'scheckfollowingexample:

location/hello{

echo"helloworld";

}

IfNginxenablesthe"debuglog",adebuglogcanbecapturedinfileerror.logwheneverinterface/helloisrequested.

$grep'usingconfig'logs/error.log

[debug]84579#0:*1usingconfiguration"/hello"

Forthepurposeofconvenience,thelog'stimestamphasbeenomitted.

Afterphasefind-config,itisouroldbuddyrewrite.SinceNginxalreadymatchestherequesttoaspecificlocationdirective,startingfromthisphase,commandswrittenwithinlocationdirectivesarebecomingeffective.Asillustratedearlier,commandsofmodulengx_rewriteareexecutedinrewritephasewhentheyarewritteninlocationdirectives.Likewise,commandsofmodulengx_set_miscandmodulengx_lua(set_by_luaandrewrite_by_lua)arealsoexecutedinphaserewrite.

Afterrewrite,itisthepost-rewritephase.Justlikefind-config,thisphasedoesnotallowNginxmodulestoregistertheirhandlerseither,insteaditcarriesouttheneeded"internalredirects"byNginxcore(ifthishasbeenrequestedinrewritephase).Wehaveaddressedthe"internaljump"conceptin(02),anddemonstratedhowtoissuethe"internalredirect"withcommandecho_execorcommandrewrite.However,let'sfocusoncommandrewriteforthemomentsincecommandecho_execisexecutedincontentphaseandbecomesirrelevanttopost-rewrite,theformerdrawsgreaterinterestbecauseitexecutesinrewritephase.Backtoourexamplein(02):

server{

listen8080;

location/foo{

set$ahello;

rewrite^/bar;

}

location/bar{

echo"a=[$a]";

}

}

Thecommandrewritefoundindirectivelocation/foo,rewritestheURIofcurrentrequestas/barunconditionally,meanwhile,itissuesan"internalredirect"andexecutioncontinuesfromlocation/bar.Whatultimatelyintriguesus,isthemagicalbitsandpiecesof"internalredirect"mechanism,"internalredirect"effectivelyrewindsourprocessingofcurrentrequestbacktothefind-configphase,sothatthelocationdirectivescanbematchedagaintotherequestURI,whichusuallyhasbeenrewritten.Justlikeourexample,whoseURIisrewrittenas/barbycommandrewrite,thelocation/bardirectiveismatchedandexecutionrepeatstherewritephasethereafter.

Itmightnotbeobvious,thattheactualactofrewindingtofind-configdoesnotoccurinrewritephase,insteaditoccursinthefollowingpost-rewritephase.Commandrewriteintheformerexample,simplyrequestsNginxtoissuean"internalredirect"initspost-rewritephase.ThisdesignisusuallyquestionedbyNginxbeginnersandtheytendtocomeupwithanideatoexecutethe"internaljump"directlybycommandrewrite.Theanswerhowever,isfairlysimple.ThedesignallowsURIberewrittenmultipletimesinthelocationdirective,whichismatchedattheverybeginning.Suchas:

location/foo{

rewrite^/bar;

rewrite^/baz;

echofoo;

}

location/bar{

echobar;

}

location/baz{

echobaz;

}

TherequestURIhasbeenrewrittentwiceinlocation/foodirective:firstlyitbecomes/bar,secondlyitbecomes/baz.Astheneteffectofbothrewritestatements,"internalredirect"occursonlyonceinpost-rewritephase.Ifitwouldhaveexecutedthe"internalredirect"atthefirstURIrewrite,thesecondwouldhavenochancetobeexecutedsinceprocessingwouldhaveleftcurrentlocationdirective.Toprovethiswesendarequestto/foo:


baz

Itcanbeassertedfromtheoutput,theactualjumpisfrom/footo/baz.WecouldfurtherprovethisbyenablingNginx"debuglog"andinterrogatethedebuglog

















generatedinfind-configphaseforthematched:


[debug]89449#0:*1usingconfiguration"/foo"

[debug]89449#0:*1usingconfiguration"/baz"

Clearly,forthespecificrequest,Nginxonlymatchestwolocationdirectives:/fooand/baz,and"internaljump"occursonlyonce.

Quiteobviously,ifcommandngx_rewrite/rewriteisusedtorewritetherequestURIinserverdirective,therewon'tbeany"internalredirects",thisisbecausetheURIrewriteishappeninginserver-rewritephase,whichgetsexecutedearlierthanfind-configphasethatmatchesinbetweenthelocationdirectives.Wecanchecktheexamplebelow:

server{

listen8080;

rewrite^/foo/bar;

location/foo{

echofoo;

}

location/bar{

echobar;

}

}

Intheexample,everyrequestwhoseURIstartswith/foogetsitsURIrewrittenas/bar.Therewritingoccursinserver-rewritephase,andtherequesthasneverbeenmatchedtoanylocationdirective.OnlyafterwardsNginxexecutesthematchesinfind-configphase.Soifwesendarequestto/foo,location/foonevergetsmatchedbecausewhenthematchoccursinfind-configphase,therequestURIhasbeenrewrittenas/bar.Solocation/baristheoneandtheonlyonematcheddirective.Actualoutputillustratesthis:


bar

Againlet'scheckNginx"debuglog":


[debug]92693#0:*1usingconfiguration"/bar"

Aswecantell,Nginxaltogetherfinishesoncethelocationmatch,andthereisno"internalredirect".

Nginxdirectiveexecutionorder(10)Afterpost-rewrite,itisthepreaccessphase.Justasitsnameimplies,thephaseiscalledpreaccesssimplybecauseitisexecutedrightbeforeaccessphase.

Built-inmodulengx_limit_reqandngx_limit_zoneareexecutedinthisphase.Theformerlimitsthenumberofrequestsperhour/minute,andthelatterlimitsthenumberofsimultaneousrequests.Wewillbediscussingthemmorethoroughlyafterwards.

Actually,built-inmodulengx_realipregistersitshandlerinpreaccessaswell.Youmightneedtoaskthen:"whydoitagain?Diditregisteritshandlersinpost-readphasealready".Beforetheanswerisuncoveredlet'sstudyfollowingexample:

server{

listen8080;

location/test{



echo"from:$remote_addr";

}

}

Comparingtotheearlierexample,themajordifferenceisthatcommandsofmodulengx_realiparewritteninaspecificlocationdirective.Aswehavelearntbefore,Nginxmatchesitslocationdirectivesinfind-configphase,whichisfarbehindpost-read,hencetherequesthasnothingtodowithcommandswritteninanylocationdirectiveinpost-readphase.Backtoourexample,itisexactlythecasewherecommandsarewritteninalocationdirectiveandmodulengx_realipwon'tcarryoutanyrewriteoftheremoteaddress,becauseitisnotinstructedassuchinpost-readphase.

Whatifwedoneedtherewrite?Tohelpresolvetheissue,modulengx_realipregistersitshandlersinpreaccessagain,sothatitisgiventhechancetoexecuteinalocationdirective.Nowtheexamplerunsaswewould'veexpected:


from:1.2.3.4

Bereallycarefulthough,modulengx_realipcouldeasilybemisused,asourfollowingexampleillustrates:

server{

listen8080;

location/test{




echo"from:$addr";

}

}

Intheexample,weintroducesavariable$addr,towhichthevalueof$remote_addrissavedinrewritephase.Thevariableisthenusedintheoutput.Slowdownrighthereandyoumighthavenoticedtheissue,phaserewriteoccursearlierthanpreaccess,sovariableassignmentactuallyhappensbeforemodulengx_realiphasthechancetorewritetheremoteaddressinpreaccessphase.Theoutputprovesourobservation:


from:127.0.0.1

Theoutputgivestheactualremoteaddress(nottherewrittenone)AgainNginx"debuglog"helpsassertittoo:

$grep-E'httpscript(var|set)|realip'logs/error.log

[debug]32488#0:*1httpscriptvar:"127.0.0.1"

[debug]32488#0:*1httpscriptset$addr

[debug]32488#0:*1realip:"1.2.3.4"

[debug]32488#0:*1realip:0100007FFFFFFFFF0100007F


Amongthelogs,thefirstlinewrites:


Thelogisgeneratedwhenvariable$remote_addrisfetchedbycommandset,string"127.0.0.1"isthefetchedvalue.

Thesecondlinewrites:

[debug]32488#0:*1httpscriptset$addr

ItindicatesNginxassignsvaluetovariable$addr.

Forthefollowingtwolines:

[debug]32488#0:*1realip:"1.2.3.4"

[debug]32488#0:*1realip:0100007FFFFFFFFF0100007F

Theyaregeneratedwhenmodulengx_realiprewritestheremoteaddressinpreaccessphase.Aswecantell,thenewaddressbecomes1.2.3.4asexpectedbutithappensonlyafterthevariableassignmentandthat'salreadytoolate.

http://wiki.nginx.org/HttpLimitReqModule

http://wiki.nginx.org/HttpLimitZoneModule











Nowthelastline:


Itisgeneratedwhencommandechooutputsvariable$addr,clearlythevalueistheoriginalremoteaddress,nottherewrittenone.

Somepeoplemightcomeupwithasolutionimmediately:"whatifmodulengx_realipregistersitshandlersinrewritephaseinstead,notinpreacccessphase?"Thesolutionhoweveris,notnecessarilycorrect.Thisisbecausemodulengx_rewriteregistersitshandlersinrewritephasetoo,andwehavelearntin(02)thattheexecutionorder,underthecircumstances,cannotbeguaranteed,sothereisagoodchancethatmodulengx_realipstillexecutesitscommandsaftercommandset.

Alwayswehavethebackupoption:insteadofpreaccess,tryusengx_realipmoduleinserverdirective,itbypassesthebothersomesituationsencounteredabove.

Afterphasepreaccess,itisanotheroldfriend,theaccessphase.Aswe'velearnt,built-inmodulengx_access,3rdpartymodulengx_auth_requestand3rdpartymodulengx_lua(access_by_lua)havetheircommandsexecutedinthisphase.

Afterphaseaccess,itisthepost-accessphase.Againasthenameimplies,wecaneasilyspotthatthephaseisexecutedrightafteraccessphase.Similartopost-rewrite,thephasedoesnotallowNginxmoduletoregistertheirhandlers,insteaditrunsafewtasksbyNginxcore,amongthem,primarilyisthesatisfyfunctionality,providedbymodulengx_http_core.

WhenmultipleNginxmoduleexecutetheircommandsinaccessphase,commandsatisfycontrolstheirrelationshipsinbetween.Forexample,bothmoduleAandmoduleBregistertheiraccesscontrolhandlersinaccessphase,wemayhavetwoworkingmodes,oneistoletaccesswhenbothAandBpasstheircontrol,theotheristoletaccesswheneitherAorBpasstheircontrol.Thefirstoneiscalledallmode("AND"relation),thesecondoneiscalledanymode("OR"relation)Bydefault,Nginxusesallmode,belowisanexample:

location/test{

satisfyall;

denyall;

access_by_lua'ngx.exit(ngx.OK)';

echosomethingimportant;

}

Under/testdirective,bothngx_accessandngx_luaareused,sowehavetwomodulesmonitoringaccessinaccessphase.Specifically,statementdenyalltellsmodulengx_accesstorejectsallaccess,whereasstatementaccess_by_lua'ngx.exit(ngx.OK)'allowsallaccess.Whenallmodeisusedwithcommandsatisfy,itmeanstoletaccessonlyifeverymoduleallowsaccess.Sincemodulengx_accessalwaysrejectsinourcase,therequestisrejected:


<html>

<head><title>403Forbidden</title></head>


<center><h1>403Forbidden</h1></center>


</body>

</html>

CarefulreadersmightfindfollowingerrorlogintheNginxerrorlogfile:

[error]6549#0:*1accessforbiddenbyrule

Ifhowever,wechangethesatisfyallstatementtosatisfyany.

location/test{

satisfyany;

denyall;

access_by_lua'ngx.exit(ngx.OK)';


}

Theoutcomeiscompletelydifferent:


somethingimportant

Therequestisallowedtoaccess.Becauseoverallaccessisallowedwheneveronemodulepassesthecontrolinanymode.Inourexample,modulengx_luaanditscommandaccess_by_luaalwaysallowtheaccess.

Certainly,ifeverymodulerejectstheaccessinthesatisfyanycircumstances,therequestwillberejected:

location/test{

satisfyany;

denyall;

access_by_lua'ngx.exit(ngx.HTTP_FORBIDDEN)';


}

Nowrequestto/testwillencounter403Forbiddenerrorpage.Intheprocess,the"OR"relationofaccesscontrolofeachaccessmodule,isimplementedinpost-access.











http://wiki.nginx.org/HttpCoreModule#satisfy










Pleasenotethatthisexamplerequiresatleastngx_lua0.5.0rc19orlater;earlierversionscannotworkwiththesatisfyanystatement.


Documents

agentzh's Nginx Tutorials (ver 2016.07.21) - OpenResty · agentzh's Nginx Tutorials (version 2016.07.21) Table of Contents Foreword Writing Plan for the Tutorials Nginx Variables