SPA2015: Hooman Beheshti – The Future of CDNs

Evolution of the CDN

Hooman Beheshti, VP Technology

SPA2015

CDN

• Globally distributed network of servers

• Built primarily with caching engines (squid, nginx, varnish, etc)

• Footprints vary

CDN benefits

• Performance• Latency decrease due to footprint

• Origin offload

• Much much more…

Deployment

• Originally simple:• Serve root web page (HTML) direct from origin• Point static assets on the page to a domain owned by the CDN

• Everything controlled through DNS

Types of content

CacheableUncacheable

Static

Dynamic

Cache-Control: max-age=31536000

Dynamic Content

Transport and delivery optimization

Transport and delivery optimization

Application delivery platform

Things have started changing…

Applications have became smarter

Caching has gotten more complicated


Static

Dynamic


StaticDynamic

Static = Cacheable

All Else = Dynamic = Uncacheable

Static = Cacheable

All Else = Dynamic = Uncacheable

Types of content

CacheableUncacheableStatic

Dynamic

EventDriven

Static for unpredictable periods of time

Event Driven Content:

Three types of content

• Static• Infrequent changes, can cache for a long time• Cache-‐control headers are often good enough• Examples: images, javascript, css, etc

• Dynamic• Can’t cache at all – must go to origin• DSA and transport optimization is key for performance• Examples: logins, credit card transactions, Ajax, etc

• Event-‐driven:• Static, but unpredictably• Cache-‐control headers are not good enough on their own• Examples: news stories, wiki pages, user comments, sports scores, etc









Long-‐tail Content(cacheable but seldom fetched)

Usage patterns have changed

CDN problems

• Services were black boxes• Lots of professional services• Caching efficiency has decreased• Not a lot of visibility• No real-‐time feedback• No real-‐time interfaces

• APIs• Not enough real-‐time control

• Removing content from the CDN• Configuration changes

CDNs have to evolve…

Caching

Caching event-‐driven content

• Caching predictably static content is relatively straight forward• Headers• Configuration

• Caching unpredictably static content is really difficult• We don’t know lifetime in advance• Trigger-‐based invalidation

Cache-Control: max-age=600

Uncaching

Purging

Purging content from a CDN

• Event-‐driven content can be cached if the CDN allows instantaneous programmatic purging• Cache normally• Purge when change trigger happens• Rinse and repeat!

• Slow purge times unacceptable in this case

• We need instant purging• Deterministic and predictable

Example:CMS + Purge

WordPress: Then

CDN Node

WordPress: Then

CDN Node

WordPress: Then

CDN Node

WordPress: Then

CDN Node

WordPress: Then

CDN Node

Cache

WordPress: Now

CDN Node

WordPress: Now

CDN Node

HTTP/1.1 200 OKContent-Type: text/htmlContent-Length: 55,666Cache-Control: Totally Long Time!

WordPress: Now

CDN Node


WordPress: Now

CDN Node

WordPress: Now

CDN Node

WordPress: Now

CDN NodePURGE

WordPress: Now

CDN NodePURGE

WordPress: Now

CDN Node

(Has to be instantaneous!)

PURGE

WordPress: Now

CDN Node


WordPress: Now

CDN Node

Long tail content and

cache hit ratio

Origin

Origin

Cache

TCP

Origin

Cache

HTTP

Origin

Cache

Origin

Cache

HTTP

Origin

Cache

HTTP

Origin

Cache

HTTP

Origin

Cache

HOT COLD

Origin

Cache

cache “hit”

Popular Medium Tail (1hr) Long tail (6hr)


Connect (median)Popular 14msec1hr Tail 15msec

6hr Tail 16msec



6hr Tail 16msec



6hr Tail 16msec



6hr Tail 16msec


Connect (median) Wait (median)Popular 14msec 19msec1hr Tail 15msec 26msec

6hr Tail 16msec 32msec


Connect (median) Wait (median)Popular 14msec 19msec1hr Tail 15msec 26msec

6hr Tail 16msec 32msec 6,400+ measurements

77,000+ measurements

38,000+ measurements


Isn’t this better?

Object storage

Caching

• Cache more

• Serve more from the edge

• Control caching and uncaching, instantaneously

Control

Programmability

• Granular invalidation API

• Configuration API

• Logic at the edge

• All configuration interaction must be instantaneous

Control at the edge

• Moving application logic to the edge

• Example: VCL (Varnish Configuration Language)• Script-‐like configuration for functionality at the edge

Control at the edge

• Moving application logic to the edge

• Example: VCL (Varnish Configuration Language)• Script-‐like configuration for functionality at the edge

• Not exclusive to varnish• Any mechanism offered by the CDN to allow logic to be executed at the edge is good

Logic at the edge

• Generate content at the edge• HTTP header manipulation• Origin selection• Caching rules• Geo-‐IP rules• Forcing SSL• Serving stale content • Etc, etc, etc

Applications without origins?

Control

• Programmatic interfaces

• Real-‐time interaction with the CDN

• Running logic at the edge

Visibility

Real-‐time analytics

• Statistics API• Network stats• HTTP stats (status codes, etc)• Caching stats (hits, misses, errors, etc)• Everything has to be real-‐time• Historic data is also a must

Logging

• Daily or hourly logs are not good enough

• Logs in real-‐time

• Log streaming• To any logging destination endpoint• Syslog, S3, FTP, etc.

Visibility

• We need to see what’s going on

• Real-‐time stats• Stats API

• Real-‐time logs

Example:Beacon Termination at the Edge

Then

CDN Node

Origin

Log Analysis

http://collector.site.com/beacon.img?a=1&b=2&c=3

Then

CDN Node

Origin

Log Analysis


Then

CDN Node

Origin

Log Analysis

HTTP/1.1 200 OKExpires: Wed, 19 Apr 2000 11:43:00 GMTCache-Control: no-cache, no-storeLast-Modified: Wed, 21 Jan 2004 19:51:30 GMTContent-Type: image/gifDate: Fri, 20 Jun 2014 12:22:20 GMTServer: ApacheContent-Length: 35


Then

CDN Node

Origin

Log Analysis



Then

CDN Node

Origin

Log Analysis



Now

CDN Node

Originhttp://collector.site.com/beacon.img?a=1&b=2&c=3

Now

CDN Node

Origin



Now (204)

CDN Node

Origin

HTTP/1.1 204 No ContentDate: Sat, 21 Jun 2014 23:21:12 GMTServer: Awesome ServerContent-Length: 0


Now

CDN Node

Origin

Syslog / S3 / FTP/etc

Streamed W3C Log


Summary: CDN is an extension of the app

• Flexible caching…• …and uncaching• All tail sizes should perform comparably!• Control over functionality at the edge• Real time interfaces for programmability• Logic at the edge• Real-‐time visibility• Real-‐time analytics

The future

• Security (more now than future!)

• Even more at the edge• More logic• Other parts of applications• More delivery features

Thank [email protected]