Choosing A Proxy Server - Apachecon 2014

Choosing A Proxy ServerApacheCon 2014

Bryan CallATS Committer / Yahoo

About Me• Yahoo! Employee

– WebRing, GeoCities, Personals, Tiger Team, Platform Architect, Edge Team, Research, ATS and HTTP

(HTTP/2 and TLS at IETF)• Working on Traffic Server for 7 years

– Since 2007• Part of the team that open sourced it in 2009• ATS Committer

Overview• Types of Proxies• Features• Architecture• Cache Architecture• Performance• Pros and Cons

How are you going to use a proxy server?

Reverse Proxy

Reverse Proxy• Proxy in front of your own web servers• Caching?• Geographic location?• Connection handling?• SSL termination?• SPDY support?• Adding business logic?

Forward Proxy

Intercepting Proxy

Forward / Intercepting Proxy• Proxy in front of the Internet• Configure clients to use proxy?• Caching?• SSL - CONNECT?• SSL - termination?

Choices

Plenty of Proxy Servers

PerlBal

Plenty of Proxy Servers

Features And Options

FeaturesATS NGiNX Squid Varnish Apache httpd

mod_proxy

Reverse Proxy Y Y Y Y YForward Proxy Y N Y N YTransp. Proxy Y N Y N YPlugin APIs Y Y partial Y YCache Y Y Y Y YESI Y N Y partial NICP Y N Y N NSSL Y Y Y N YSPDY Y* Y N N partial

* 5.0.0 (May 2014)

SSL Features

Source: https://istlsfastyet.com/ - Ilya Grigorik

What type of proxy do you need?• Of our candidates, only three fully supports all

proxy modes

HTTP/1.1 Compliance

HTTP/1.1 Compliance• Accept-Encoding - gzip• Vary• Age• If-None-Match

How things can go wrong: Vary$ curl -D - -o /dev/null -s --compress http://10.118.73.168/HTTP/1.1 200 OKServer: nginx/1.3.9Date: Wed, 12 Dec 2012 18:00:48 GMTContent-Type: text/html; charset=utf-8Content-Length: 8051Connection: keep-aliveCache-Control: public, max-age=900Last-Modified: Wed, 12 Dec 2012 17:52:42 +0000Expires: Sun, 19 Nov 1978 05:00:00 GMTVary: Cookie,Accept-EncodingContent-Encoding: gzip

How things can go wrong: Vary$ curl -D - -o /dev/null -s http://10.118.73.168/HTTP/1.1 200 OKServer: nginx/1.3.9Date: Wed, 12 Dec 2012 18:00:57 GMTContent-Type: text/html; charset=utf-8Content-Length: 8051Connection: keep-aliveCache-Control: public, max-age=900Last-Modified: Wed, 12 Dec 2012 17:52:42 +0000Expires: Sun, 19 Nov 1978 05:00:00 GMTVary: Cookie,Accept-EncodingContent-Encoding: gzip

EPIC FAIL!

Note: no gzip request

CoAdvisor HTTP protocol quality tests for reverse proxies

ATS 3.3.1

Nginx 1.3.9

Squid 3.2.5

Varnish 3.0.3

0 100 200 300 400 500 600

Failures Violations Success

49%

81%

51%

68%

CoAdvisor HTTP protocol quality tests for reverse proxies

ATS 3.3.1

Nginx 1.3.9

Squid 3.2.5

Varnish 3.0.3

0 100 200 300 400 500 600

Failures Violations Success

25%

6%

27%

15%

Architecture

Architecture And Process Models• Multithreading• Events• Process• Fibers

– Co-operative multitasking, getcontext/setcontext

Threads

Threads• Pros

– Easy to share memory– Lightweight context switching

• Cons– Easy to (accidently) share memory

• Overwriting another threads memory– Locking

• Deadlocks, race conditions, starvation

Event Processing

Problems with Event Processing• Doesn’t work well with

blocking APIs– open(), locking

• It doesn’t scale on SMP by itself

Process Model And ArchitectureATS NGiNX Squid Varnish Apache httpd

mod_proxy

Threads X X XEvents X X X partial XProcesses X X X

Caching Architecture

Cache• Mainly two types

– File system– Database like

• In memory index– Bytes per object

• Minimize disk seeks and system calls

CacheATS NGiNX Squid Varnish Apache httpd

mod_cache

File system X X Xmmap XRaw disk/direct IO X XRam cache X XMemory index X X X*Persistent cache X X X X

Performance Testing

ATS Configurationetc/trafficserver/remap.config:

map / http://origin.example.cometc/trafficserver/records.config:

CONFIG proxy.config.http.server_ports STRING 80CONFIG proxy.config.accept_threads INT 3

NGiNX Configurationworker_processes 24;access_log logs/access.log main;

proxy_cache_path /mnt/nginx_cache levels=1:2 keys_zone=my-cache:8m max_size=16384m inactive=600m;proxy_temp_path /mnt/nginx_temp;

server { set $ae ""; if ($http_accept_encoding ~* gzip) { set $ae "gzip"; }

location / { proxy_pass http://origin.example.com; proxy_cache my-cache; proxy_set_header If-None-Match ""; proxy_set_header If-Modified-Since ""; proxy_set_header Accept-Encoding $ae; proxy_cache_key $uri$is_args$args$ae; }

location ~ /purge_it(/.*) { proxy_cache_purge example.com $1$is_args$args$myae }

Squid Configurationhttp_access allow allhttp_port 80 accelworkers 24

cache_mem 4096 MBmemory_cache_shared oncache_dir rock /usr/local/squid/cache 1000 max-size=32768cache_peer origin.example.com parent 80 0 no-query originserver

Varnish Configurationbackend default { .host = ”origin.example.com”; .port = "80";}

Varnish Configuration (Cont)

sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=4000sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=2000 -p thread_pool_add_delay=2 -p thread_pool_min=200sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=2000 -p thread_pool_add_delay=2 -p thread_pool_min=1000 -p session_linger=0sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=2000 -p thread_pool_add_delay=2 -p thread_pool_min=1000 -p session_linger=10

Apache httpd ConfigurationLoadModule cache_module modules/mod_cache.soLoadModule cache_disk_module modules/mod_cache_disk.soLoadModule proxy_module modules/mod_proxy.soLoadModule proxy_http_module modules/mod_proxy_http.soInclude conf/extra/httpd-mpm.confProxyPass / http://origin.example.com/

<IfModule mod_cache_disk.c> CacheRoot /usr/local/apache2/cache CacheEnable disk / CacheDirLevels 5 CacheDirLength 3</IfModule>

MaxKeepAliveRequests 10000

Benchmark 1• 1,000 clients• 8KB response• 100% cache hit• Keep-alive on• 100K rps rate limited

• Squid used the most CPU and the worst median latency

• 95th percentile latency with NiGNX, Squid and httpd

ATS NGiNX Squid Varnish httpd0

500

1000

1500

2000

2500

RPS / CPU Usage


20000

40000

60000

80000

100000

120000

Requests Per Second


1012141618

Latency

Median

95th

Benchmark 2• 1,000 clients• 8KB response• 100% cache hit• Keep-alive off

• Squid used the most CPU again

• NGiNX had latency issues• ATS most throughput


500

1000

1500

2000

2500

RPS / CPU Usage


5000

10000

15000

20000

25000

30000

Requests Per Second


5

10

15

20

25

30

35

40

Latency

Median

95th

ATS• Pros

– Scales well automatically, little config needed– Best cache implementation

• Cons– Too many config files– Too many options in the default config files

NGiNX• Pros

– Lots of plugins– FastCGI support

• Cons– HTTP/1.1 compliance– Latency issues around accepting new connections– Rebuild server for new plugins

Squid• Pros

– Best HTTP/1.1 compliance• Cons

– Memory index for cache using 10x that of ATS– Least efficient with CPU– Worst median latency for keep-alive benchmarks

Varnish• Pros

– VCL (Varnish Configuration Language)• Can do a lot without writing plugins

• Cons– Thread per connection– mmap for cache

• Persistence is experimental– No SSL or SPDY support

Apache httpd• Pros

– Lots of plugins– Most used http server– Best 95th percentile latency for non-keep-alive

• Cons– SPDY Support

Why ATS?• Scales well

– CPU Usage, auto config• Cache scales well

– Efficient memory index, minimizes seeks• Apache Community• Plugin support

– Easy to port existing plugins over

References• ATS - http://trafficserver.apache.org/• NGiNX - http://nginx.org/• Squid - http://www.squid-cache.org/• Varnish - https://www.varnish-cache.org/• Apache httpd - http://httpd.apache.org/