Solving anything in VCL

Solving anything in VCLAndrew Betts, Financial Times

Who is this guy?1.Helped build the original HTML5

web app for the FT2.Created our Origami component

system3.Ran FT Labs for 3 years4.Now working with Nikkei to rebuild

nikkei.com5.Also W3C Technical Architecture

Group6.Live in Tokyo, Japan

2

Pic of me.

Nikkei1.Largest business

newspaper in Japan2.Globally better known

for the Nikkei 225 stock index

3.Around 3 million readers

4

5

6

7

8

Coding on the edge

9

Benefits of edge code10

1.Smarter routing2.Faster authentication3.Bandwidth management4.Higher cache hit ratio

Edge side includes11

<esi:include src="http://example.com/1.html" alt="http://bak.example.com/2.html" onerror="continue"/>

index.html

my-news.html

Cache-control: max-age=86400

Cache-control: private

Server

The VCL way1.Request and response bodies are opaque2.Everything happens in metadata3.Very restricted: No loops or variables4.Extensible: some useful Fastly extensions include geo-ip and

crypto5.Incredibly powerful when used creatively

12

SOA RoutingSend requests to multiple microservice backends

This is great if...You have a microservice

architectureMany backends, one domainYou add/remove services

regularly

1

SOA Routing in VCL14

Front page

Article page

Timeline

Content API

Choose a backend based on a path

match of the request URL

/article/123

SOA Routing in VCL15

[ { name, paths, host, useSsl, }, …]

{{#each backends}} backend {{name}} { .port = "{{p}}"; .host = "{{h}}"; }{{/each}}

let vclContent = vclTemplate(data);

fs.writeFileSync( vclFilePath, vclContent, 'UTF-8');

services.json

Defines all the backends and paths that they control.

routing.vcl.handlebars

VCL template with Handlebars placeholders for backends & routing

build.js

Task script to merge service data into VCL template

SOA Routing: key tools and techniques●Choose a backend:set req.backend = {{backendName}};

●Match a route pattern:if (req.url ~ "{{pattern}}")

●Remember to set a Host header:set req.http.Host = "{{backendhost}}";

●Upload to Fastly using FT Fastly tools○ https://github.com/Financial-Times/fastly-tools

16

service-registry.json17

[ { "name": "front-page", "paths": [ "/(?qs)", "/.resources/front/(**)(?qs)" ], "hosts": [ "my-backend.ap-northeast-1.elasticbeanstalk.com" ] }, { "name": "article-page", ... }]

Common regex patterns simplified into shortcuts

routing.vcl.handlebars18

{{#each backends}}backend {{name}} { .port = "{{port}}"; .host = "{{host}}"; .ssl = {{use_ssl}}; .probe = { .request = "GET / HTTP/1.1" "Host: {{host}}" "Connection: close"; }}{{/each}}

sub vcl_recv { {{#each routes}} if (req.url ~ "{{pattern}}") { set req.backend = {{backend}}; {{#if target}} set req.url = regsub(req.url, "{{pattern}}", "{{target}}"); {{/if}}

{{!-- Fastly doesn't support the host_header property in backend definitions --}} set req.http.Host = "{{backendhost}}"; } {{/each}} return(lookup);}

build.js19

const vclTemplate = handlebars.compile(fs.readFileSync('routing.vcl.handlebars'), 'UTF-8'));const services = require('services.json');

// ... transform `services` into `viewData`

let vclContent = vclTemplate(viewData);fs.writeFileSync(vclFilePath, vclContent, 'UTF-8');

UA TargetingReturn user-agent specific responses without destroying your cache hit ratio

This is great if...You have a response that is

tailored to different device types

There are a virtually infinite number of User-Agent values

2

21

Polyfill screenshot

UA Targeting22

/normalizeUA

/polyfill.js?ua=ie/11

/polyfill.js

Add the normalised User-Agent to the URL and restart the original request

Add a Vary: User-Agent header to the response before sending it back to the browser

We call this a preflight request

UA targeting: key tools and techniques●Remember something using request headers:set req.http.tmpOrigURL = req.url;

●Change the URL of the backend request:set req.url = "/api/normalizeUA?ua=" req.http.User-Agent;

●Reconstruct original URL adding a backend response header:set req.url = req.http.tmpOrigURL "?ua=" resp.http.NormUA;

●Restart to send the request back to vcl_recv:restart;

23

ua-targeting.vcl24

sub vcl_recv { if (req.url ~ "^/v2/polyfill\." && req.url !~ "[\?\&]ua=") { set req.http.X-Orig-URL = req.url; set req.url = "/v2/normalizeUa?ua=" urlencode(req.http.User-Agent); }}

sub vcl_deliver { if (req.url ~ "^/v\d/normalizeUa" && resp.status == 200 && req.http.X-Orig-URL) { set req.http.Fastly-force-Shield = "1"; if (req.http.X-Orig-URL ~ "\?") { set req.url = req.http.X-Orig-URL "&ua=" resp.http.UA; } else {

set req.url = req.http.X-Orig-URL "?ua=" resp.http.UA; } restart; } else if (req.url ~ "^/v\d/polyfill\..*[\?\&]ua=" && req.http.X-Orig-URL && req.http.X-Orig-URL !~ "[\?\&]ua=") { add resp.http.Vary = "User-Agent"; } return(deliver);}

AuthenticationImplement integration with your federated identity system entirely in VCL

This is great if...You have a federated login

system using a protocol like OAuth

You want to annotate requests with a simple verified authentication state

3

Magic circa 200126

<?phpecho $_SERVER['PHP_AUTH_USER'];?>

http://intranet/my/example/app

New magic circa 201627

app.get('/', (req, res) => { res.end(req.get('Nikkei-UserID'));});

Authentication28

/article/123

Decode+verifyauth cookie!

Nikkei-UserID: andrew.bettsNikkei-UserRank: premium

Vary: Nikkei-UserRank

Article

Cookie: Auth=a139fm24...

Cache-control: private

Authentication: key tools and techniques●Get a cookie by name: req.http.Cookie:MySiteAuth●Base64 normalisation:digest.base64url_decode(), digest.base64_decode

●Extract the parts of a JSON Web Token (JWT):regsub({{cookie}}, "(^[^\.]+)\.[^\.]+\.[^\.]+$", "\1");

●Check JWT signature: digest.hmac_sha256_base64()●Set trusted headers for backend use:req.http.Nikkei-UserID = regsub({{jwt}}, {{pattern}}, "\1");

29

authentication.vcl30

if (req.http.Cookie:NikkeiAuth) { set req.http.tmpHeader = regsub(req.http.Cookie:NikkeiAuth, "(^[^\.]+)\.[^\.]+\.[^\.]+$", "\1"); set req.http.tmpPayload = regsub(req.http.Cookie:NikkeiAuth, "^[^\.]+\.([^\.]+)\.[^\.]+$", "\1");

set req.http.tmpRequestSig = digest.base64url_decode( regsub(req.http.Cookie:NikkeiAuth, "^[^\.]+\.[^\.]+\.([^\.]+)$", "\1") );

set req.http.tmpCorrectSig = digest.base64_decode( digest.hmac_sha256_base64("{{jwt_secret}}", req.http.tmpHeader "." req.http.tmpPayload) );

if (req.http.tmpRequestSig != req.http.tmpCorrectSig) { error 754 "/login; NikkeiAuth=deleted; expires=Thu, 01 Jan 1970 00:00:00 GMT"; }

... continues ...

authentication.vcl (cont)31

set req.http.tmpPayload = digest.base64_decode(req.http.tmpPayload);

set req.http.Nikkei-UserID = regsub(req.http.tmpPayload, {"^.*?"sub"\s*:\s*"(\w+)".*?$"}, "\1"); set req.http.Nikkei-Rank = regsub(req.http.tmpPayload, {"^.*?"ds_rank"\s*:\s*"(\w+)".*?$"}, "\1");

unset req.http.base64_header; unset req.http.base64_payload; unset req.http.signature; unset req.http.valid_signature; unset req.http.payload;

} else {

set req.http.Nikkei-UserID = "anonymous"; set req.http.Nikkei-Rank = "anonymous";}

Feature flagsDark deployments and easy A/B testing without reducing front end perf or cache efficiency

This is great if...You want to serve different

versions of your site to different users

Test new features internally on prod before releasing them to the world

4

33

34

Now you see it...

Feature flags parts35

●A flags registry - a JSON file will be fine○ Include all possible values of each flag and what percentage of the audience it

applies to

○ Publish it statically - S3 is good for that

●A flag toggler tool○ Reads the JSON, renders a table, writes an override cookie with chosen values

●An API○ Reads the JSON, responds to requests by calculating a user's position number

on a 0-100 line and matches them with appropriate flag values

●VCL○ Merges flag data into requests

Feature flags36

Flags API

Article

Merge the flags response with the override cookie, set as HTTP header, restart original request...

Decode+verifyauth cookie!

/article/123

Cookie: Flgs-Override= Foo=10;

/api/flags?userid=6453

Flgs: highlights=true; Foo=42;

Flgs: highlights=true; Foo=42; Foo=10

Vary: Flgs

ExpressJS flags middleware37

app.get('/', (req, res) => { if (req.flags.has('highlights')) { // Enable highlights feature }});

HTTP/1.1 200 OKVary: Nikkei-Flags...

Middleware provides convenient interface to flags header

Invoking the middleware on a request automatically applies a Vary header to the response

Dynamic backendsOverride backend rules at runtime without updating your VCL

This is great if...You have a bug you can't

reproduce without the request going through the CDN

You want to test a local dev version of a service with live integrations

5

Dynamic backends39

Developer laptop

Dynamic backend proxy(node-http-proxy)

Check forwarded IP is whitelisted or auth header is also present

GET /article/123Backend-Override: article -> fc57848a.ngrok.io

Detect override header, if path

would normally be routed to article,

change it to override proxy

instead.

ngrok

fc57848a.ngrok.i

o

Normal production backends

Dynamic backends: key tools and techniques●Extract backend to override:

set req.http.tmpORBackend = regsub(req.http.Backend-Override, "\s*\-\>.*$", "");

●Check whether current backend matchesif (req.http.tmpORBackend == req.http.tmpCurrentBackend) {

●Use node-http-proxy for the proxy app○ Remember res.setHeader('Vary', 'Backend-Override');○ I use {xfwd: false, changeOrigin: true, hostRewrite: true}

40

Debug headersCollect request lifecycle information in a single HTTP response header

This is great if...You find it hard to understand

what path the request is taking through your VCL

You have restarts in your VCL and need to see all the individual backend requests, not just the last one

6

42

The VCL flow

43

The VCL flow

44

The VCL flow

Debug journey45

vcl_recv { set req.http.tmpLog = if (req.restarts == 0, "", req.http.tmpLog ";"); # ... routing ... set req.http.tmpLog = req.http.tmpLog " {{backend}}:" req.url;}vcl_fetch { set req.http.tmpLog = req.http.tmpLog " fetch"; ... }vcl_hit { set req.http.tmpLog = req.http.tmpLog " hit"; ... }vcl_miss { set req.http.tmpLog = req.http.tmpLog " miss"; ... }vcl_pass { set req.http.tmpLog = req.http.tmpLog " pass"; ... }vcl_deliver { set resp.http.CDN-Process-Log = req.http.tmpLog;}

Debug journey46

CDN-Process-Log: apigw:/flags/v1/rnikkei/allocate?output=diff&segid=foo&rank=X HIT (hits=2 ttl=1.204/5.000 age=4 swr=300.000 sie=604800.000); rnikkei_front_0:/ MISS (hits=0 ttl=1.000/1.000 age=0 swr=300.000 sie=86400.000)

RUM++Resource Timing API + data Fastly exposes in VCL. And no backend.

This is great if...You want to track down hotspots

of slow response timesYou'd like to understand how

successfully end users are being matched to their nearest PoPs

7

Resource timing on front end48

var rec = window.performance.getEntriesByType("resource").find(rec => rec.name.indexOf('[URL]') !== -1)

;

(new Image()).src = '/sendBeacon'+'?dns='+(rec.domainLookupEnd-rec.domainLookupStart)+'&connect='+(rec.connectEnd-rec.connectStart)+'&req='+(rec.responseStart-rec.requestStart)+'&resp='+(rec.responseEnd-rec.responseStart)

;

Add CDN data in VCL & respond with synthetic

49

sub vcl_recv { if (req.url ~ "^/sendBeacon") { error 204 "No content"; }}

RUM++50

/sendBeacon?foo=42&...

No backend request!

204 No Content

Write logs in 1 minute batches to Amazon S3

Use an 'error' response to return a 204!

51

Custom format string for log entries

Crunch the data52

Beyond ASCIIUse these encoding tips to embed non-ASCII content in your VCL file.

This is great if...Your users don't speak English,

but you can only write ASCII in VCL files

8

Everyone does UTF-8 now, right?54

synthetic {"Responsive Nikkeiアルファプログラムのメンバーの皆様、アルファバージョンのサイトにアクセスできない場合、 [email protected] までその旨連絡ください。 "};

55

Quick conversion56

"string" .split('') .map( char => char.codePointAt(0) < 128 ? char : "&#"+char.codePointAt(0)+";" ) .join('');

"Fixed"57

synthetic {"Responsive Nikkeiアルファプログラムのメンバーの皆様、アルファバージョンのサイトにアクセスできない場合、[email protected] までその旨連絡ください。"};

"Fixed"58

synthetic digest.base64decode({"IlJlc3BvbnNpdmUgTmlra2Vp44Ki44Or44OV44Kh44OX44Ot44Kw44Op44Og44Gu44Oh44Oz44OQ44O844Gu55qG5qeY44CB44Ki44Or44OV44Kh44OQ44O844K444On44Oz44Gu44K144Kk44OI44Gr44Ki44Kv44K744K544Gn44GN44Gq44GE5aC05ZCI44CBcm5mZWVkYmFja0BuZXgubmlra2VpLmNvLmpwIOOBvuOBp+OBneOBruaXqOmAo+e1oeOBj+OBoOOBleOBhOOAgiI="});

59

Here's what you always wanted! Hugs,

I have 68 backends

60

Varnish transaction ID!!

Varnishlog to the rescue

A way to submit a varnish transaction ID to the API, and get all varnishlog events relating to that transaction, including related (backend) transactions

61

> fastly log 1467852934

17 SessionOpen c 66.249.72.22 47013 :8017 ReqStart c 66.249.72.22 47013 146785293417 RxRequest c GET17 RxURL c /articles/12317 RxProtocol c HTTP/1.117 RxHeader c Host: www.example.com ...

Thanks for listening

62

Andrew [email protected]@triblondon

Get the slidesbit.ly/ft-fastly-altitude-2016

Technology

Solving anything in VCL