You have NO Idea what your users want

And how to learn

Evan Solomon | Automattic@evansolomon | evan@automattic.com

if( time() % 2 ) $title = ‘Split testing’;else $title = ‘A/B testing’;setcookie( ‘title’, $title );echo $title;

A/B testing

1.Try different things

2.Measure their effects

3.Profit!

Lots of good examples at blog.performable.com

What can it do?

If you have enough data you can tell how much better, but you need quite a bit

Determine how likely one version is to be better than another.

It makes you smart(er)!

Why do it?

Also turns out to be good for getting clients

1.Talk to your customers

2.Listen

3.Profit!

Step 3 is always profit

Start basic

Dashboard

Check out geckoboard.com

We use Optimizely because:1.Real time results (great for debugging)

2.Tests are written entirely in jQuery

3.Easy to target using WordPress logic

Running tests

Version A

Version B

B wins and covers the spread

Signups increase ~25%

What happens?

Handling data

Terms you should get used to:

“Confidence interval”“Significance test”

Check out isvalid.org to calculate these statsI made isvalid for myself so it’s a little rough, but very useful

isvalid.org

• Question assumptions• Unintended (or not) biases• Test liberally, it’s nearly free• Many (most) tests “fail”

Keep in mind

I’m working on a .org plugin

Based on the one we made for WordPress.com

Call for help

@evansolomon