Building Faster Websites: Optimizing the Server

Jonathan Klein

jklein@wayfair.com

@jonathanklein

November 17, 2011

Agenda

Who Am I?

Why Server Side Performance Matters

Measuring Performance

Speeding up the Server

Improving the Database

Load Testing

3

Who is this Guy?

• Senior Software Engineer/Performance Guy at Wayfair

• Organizer - Boston Web Performance Meetup Group

• Organizer - #WPOChat (monthly Twitter chat)

• Wayfair Stats:1. ~1400 requests/sec for static content

2. ~400 requests/sec for dynamic content

3. ~10 million unique visitors per month

4. On a typical Monday we serve 75,000,000 static files

4

Wayfair Stack Changes

• Windows Server FreeBSD

• IIS Lighttpd

• Classic ASP PHP

• But we are still on MSSQL (more on that later)

• http://engineering.wayfair.com

Why Server Side Performance?

7

Doesn’t 80% happen on the client?

• Yes, but not if your server is slow• Users staring at a blank page• No upper limit• HUGE spikes

• Plus, bots crawl fast sites more

8

Time to first byte

Time To First Byte (TTFB) Matters

How Do We Measure It?

10

Server Side Monitoring

Lots of Options:• Paid:

1. Coradiant

2. dynaTrace

3. Correlsense• http://www.real-user-monitoring.com/ - Free Version

• Free:1. StatsD/Graphite

2. Access Logs

3. Nagios

4. Ganglia

5. Hosted WebPagetest

6. Selenium/dynaTrace Ajax Edition

11

Or Something Simple…

<?php$start = microtime(true);

…script content…

$end = microtime(true);

do_stuff('Description', $end - $start);?>

12

do_stuff()?

• do_stuff() can do one of:1. Log to a database (not ideal)

2. Write to a text file (eww)

3. Make a StatsD call over UDP (good) -http://codeascraft.etsy.com/2011/02/15/measure-anything-measure-everything/

4. Choose your own adventure

14

Averages can be misleading

Better to look at percentiles

What Does a Scalability Problem Look Like?

19

This:

20

Or maybe this!

21

Much better!

So How Do We Do It?

23

What can you do?

• Start with the Database

• Run a database trace 1. Filter: Queries > 50ms

2. Filter: Reads > 1000

3. Start with the worst ones and optimize

24

What does “Optimize” mean?

• Look at execution plan1. Remove calls to remote servers

25

Database Optimizations

• Reduce Joins• Select * from Select Foo, Bar, Baz from…• Minimize/consolidate subqueries• Add indexes where needed

1.We added one index: Procedure dropped from 3.5 sec & 4500 reads to .06 sec and 130 reads!

• Be careful with where clauses (don’t calculate stuff in them)

26

Example

SELECT id, name, salary FROM employee WHERE salary < 25000;

NOT

SELECT id, name, salary FROM employee WHERE salary + 10000 < 35000;

27

Example

SELECT id, name, salary FROM employee WHERE salary < 25000;

NOT

SELECT id, name, salary FROM employee WHERE salary + 10000 < 35000;

28

Not many hard and fast rules…

• Try something, look at the plan

• Try something else…look at how the plan changes

• This is your friend:

The Fastest DB Query is the One You Don’t Make

30

Memcached

• Caching layer between database and webserver

• Can hold PHP objects and arrays

31

Memcached

$m = new Memcached();$m->pconnect(‘1.2.3.4', 11211);

$m->set(‘foo’, $bar, 600);

$baz = $m->get(‘foo’);

32

Install APC

• APC – Alternative PHP Cache

1. Opcode Cache

2. User Cache

3. Awesome!

33

Opcode Cache

34

User Cache

<?php$bar = 'BAR';apc_store('foo', $bar);var_dump(apc_fetch('foo'));?>

Outputs…

string(3) "BAR"

PHP Code Optimizations

36

PHP Optimizations

• Set the max loop value before the loop:

$max = count($rows);for ($i = 0; $i < $max; $i++) {

echo $i;}

• require_once() is slow• Minimize use of define()• Yes, single quotes are slightly faster than double

quotes, but…

37

PHP Optimizations have limited value

• Could go on and on…

1.http://www.wmtips.com/php/tips-optimizing-php-code.htm

• Minimal returns

• Find the hotspots in your application and fix them

38

Example…

• Homepage takes 5 seconds to load

• Optimize PHP…

1. Reduce PHP execution time by 50%!

• But wait! PHP execution was only taking 100ms

1. Saves you 50ms in load time

2. 1% of total page load

39

If you really need to make your code faster…

• HipHop for PHP

• Built by Facebook and Open Sourced

• Compiles PHP into C++

• Currently supports PHP 5.2

• http://developers.facebook.com/blog/post/358/• https://github.com/facebook/hiphop-php

Webserver Considerations

41

Do it right

• Pick the right one

1. Lighttpd/Nginx instead of Apache

2. Designed to solve the C10K problem

• Lighttpd Used By:

1. Youtube

2. Wikipedia

3. Meebo

42

Benefits of Lighttpd/Nginx

• Event driven model:

“Unlike traditional servers, Nginx doesn't rely on threads to handle requests. Instead it uses a much more scalable event-driven (asynchronous) architecture. This architecture uses small, but more importantly, predictable amounts of memory under load.”- http://wiki.nginx.org/

• FastCGI + spawn-fcgi

1. PHP Process Management

2. Many child processes – scale out application tier.

43

If you MUST use Apache

• mod_deflate

1.Gzips content for faster transfer times

• mod_pagespeed

1.Automatic performance improvements

• KeepAlives on

1.Server won’t create a new connection for every resource

Load Testing

45

JMeter

• Open Source

• Generate load via a GUI or command line

• Can watch req/s peak out

• Easy to use (just make sure you set the correct path to Java on your computer).

46

JMeter

48

JMeter

49

JMeter

• Watch your User Agent

• Be careful hitting sites you don’t own

• You might tap out your local box/network before killing the server

50

Conclusion

• Know and watch your site

• More monitoring is better

• Understand what is behind the scenes – look at the full stack

• Load test BEFORE things go to production – figure out when they will fall over

51

Questions?

We’re Hiring!www.wayfair.com/careers

Get In Touch:www.meetup.com/Web-Performance-Boston/

jklein@wayfair.com@jonathanklein