Varnish bof

VarnishBOF!

What's this talk about?

• A basic intro to Varnish and reverse proxies• Some cool ESI stuff• How to setup, implement + configure

Varnish/ESI in Drupal• Teh Awesomeness of speed

Why Varnish?

Drupal performs…just…

How much faster?

20 reqs/sec?

200 reqs/sec?

Drupal

Varnish

When personalisation goes wrong

Dries logs in

Drupal creates a personalised page

Logged in as dries

The page gets cached

Varnish

When webchick hits the site

The page is fetched from the cache

Varnish

The personalisation is incorrect

Logged in as dries

WTF?

WTF?

No-cache headers can prevent this

Varnish

But now it's slow :-(

EdgeSideIncludes

ESI: Tales from the past

• The idea originated in SSI• SSI, but moved to the edge• ESI is a W3C standard• ESI is a bunch of xml tags/attributes.• Supported by:– Varnish (partial support – no ESI variables)– Akamai– Some Nginx support– ???

Dries is visiting the site

The page is requested

Varnishhttp://example.com/node/1

The page includes an ESI tag

<esi src= "http://example.com/esi/user/0" />

The proxy sends a second request

Varnish

http://example.com/esi/user/0

http://example.com/node/1

The ESI URL sends a HTML snip

<div id="userinfo"> <a title="View &amp; edit your user profile">Logged in as dries</a> <a href="/logout">Logout</a></div>

The ESI code is merged with the page

Proxy forwards the page on

Varnish

http://example.com/esi/user/0

http://example.com/node/1

This doesn’t sound quicker

The Importance of Tests

• It's important to measure the results(I'll come to measuring tools later!)

• The ESI snippets can be cached too

Cached ESIs

1st Page = 2 Requests

Varnishhttp://example.com/node/1

http://example.com/esi/user/0

2nd Page = 1 request

Varnishhttp://example.com/node/2

1 request for each new page

• 10 pages = 11 requests• Each page is cached• The ESI tag is cached

The next visitor gets the page for free

VarnishPage fetched from cache

http://example.com/node/1

1st page = 1 request (for the ESI)

Varnish

http://example.com/esi/user/0

Page fetched from cachehttp://example.com/node/1

Following pages all served from cache

Varnish

1 request for each new page

• 10 pages = 11 requests• The ESI tag is cached• Each page is cached• Subsequent requests have 1 request for the

ESI tag• 20 pages = 12 requests (10 pages + 2 ESI tags)

Fast enough yet?

Warm the cache

A modern infrastructure

Varnish HA Proxy

TehIntertubes

Service webheadadmin/cron/emails/??

Spidering the site

Varnish HA Proxy

TehIntertubes

Service webheadadmin/cron/emails/??

Link-checker /spider app.

Dries is back for more

Each user generates 1 request

Varnish

http://example.com/esi/user/0

Page fetched from cachehttp://example.com/node/1

5 users, 40 pages

• 5 backend requests!

Theory Over

Basics Varnish

+Drupal

Installing Varnish

• sudo apt-get install varnishsudo yum install varnish

• Edit /etc/default/varnish• Edit /etc/varnish/default.vcl• Usual config:– Varnish listens on port 80– Apache listens on port 8080 (or any available port)– SSL is more complex – sometimes Pound is used

for SSL decryption

VCLs are awesome!

• Varnish configuration is controlled by VCL: Varnish Configuration Language.

• Documented athttp://www.varnish-cache.org/trac/wiki/VCL

• Drupal VCL examples can be found in:– Mercury (Amazon AMI)– ESI module

• You can write C code in VCLs. Oh my.

Keeping the cache current

Varnish

Varnish admin port

Varnish Management Port

• The Varnish admin port uses a plain-text protocol to command Varnish

• Test by:telnet localhost 6093

• List commands by typing help• Purge cached pages: url.purge <regexp>

E.g. url.purge .*• The Varnish module provides an API.

Use with caution!

The default behaviour of the Varnish module is to

clear the entire cachewhen you edit a node.

Enter…the Expires module

• Came from the Boost module• Clears specific cache pages – not the entire

cache• Supports path-alias URLs as well as node/1

paths.• Provides hook_expire_cache_alter to add URLs

(e.g. nodes displayed in a page-view)• Supports other cache modules, not just Varnish.

I can haz ESI?

• The ESI module provides block integration

Configuring ESI

• Download the module• Configure the blocks to serve via ESI, and their

TTL• Modify the Varnish VCL (the ESI module comes

with example VCLs to use• Coming soon:– D7– Panels integration

Benchmarking

• Some tools:– J-meter– ab (apache-benchmark)– Seige

• Approaches:– Hammer your URL(s)

ab –n 100 –c5 http://example.com/– Replay your log files

Pitfalls await

• VCLs are complex• Per-user cache rules are

even scarier!• Varnish may be slower• Risk exposing private

data

Alternatives

• Boost module– No need to install a new service on the server– Static file cache

• Advanced cache module• Authcache module• Other proxies:– Squid– Nginx– Akamai?

Summary

How hard?

Teh Awesomeness

Varnish

Varnish module

Expire module

ESI module

Thanks!

Marcus Deglose: marcus@deglos.comt: @manarthirc: manarth

Picture creditsThanks to these CC-licenced flickr users• Mac Book Pro - http://www.flickr.com/photos/wicho/• F1 car - http://www.flickr.com/photos/w3i_yu/• Banger car - http://www.flickr.com/photos/photoplod• Dries - http://www.flickr.com/photos/boris• Dries+laptop - http://www.flickr.com/photos/azugaldia• Webchick - http://www.flickr.com/photos/mortendk • Webchick - http://www.flickr.com/photos/puregin• Fire - http://www.flickr.com/photos/benwatts• Hourglass - http://www.flickr.com/photos/opendemocracy• Stopwatch -

http://www.flickr.com/photos/purplemattfish