Running with Scissors

RTE.ie’s path to caching in Varnish

Varnish Summit London 10th April 2014

John Moylan - http://8t8.eu

RTÉ Digital

PLATFORMS

Started working with RTÉ.ie in 1999

The internet was a very different place

In the beginning there was squid

Squid V. Varnish

Lessons learned

Then there was Varnish

Ram doesn’t come in 64MB sticks anymore.

Then there was Varnish

Bandwidth isn’t counted in Mb/s anymore

● 2001 10Mb’s● 2014 40Gb/s

excluding CDN

409,600% Increase

The great thing about mod_rewrite is it gives you all the configurability and flexibility of Sendmail. The downside to mod_rewrite is that it gives you all the configurability and flexibility of Sendmail.''

-- Brian Behlendorf Apache Group

With Great Power Comes Responsibility

The great thing about VCL is it gives you all the configurability and flexibility of Sendmail. The downside to VCL is that it gives you all the configurability and flexibility of Sendmail.''

-- John MoylanRTÉ Digital

With Great Power Comes Responsibility

Scope

#don't cache ajax requests

if (req.http.X-Requested-With == "XMLHttpRequest") { return (pass); }

Whoops! Scope

Scope

# don't cache authenticated sessions if (req.http.Cookie && req.http.Cookie ~ "(wordpress_|PHPSESSID)") { return(pass); }

Whoops Scope

Lessons learned

1. Scope special conditions properly2. GET’s and HEADS no longer work

for checking cache issues

Third party web app developers:Please don’t break the webGET’s should not have side effects(and while I’m on my soapbox, please use proper status codes)

Old School Monitoring

RRDToolRPS, Bandwidth, pagefaults, Hit Ratio, FD’s, CPU, MemoryGrep’ing, cut’ing, sort’ing origin server logs

Plus GET’s are normally GET’s. Cache is dumb

Varnish Monitoring

VCSVACVarnishlogVarnishstatNewrelicPyVarnish

Varnish Monitoring

PyVarnishhttps://github.com/redsnapper8t8/pyvarnish

Thanks for listening

Questions?