You're testing what!

YOU’RE TESTING WHAT?! FOUR TYPES OF TESTS YOU SHOULD BE RUNNING

MATT LACEY Head of Optimisation

@matt_lacey12

27TH MARCH 2014

CAMP DIGITAL MANCHESTER

@matt_lacey12 #CampDigital

Head of Optimisation at PRWD

One of the UK’s most respected CRO agencies

Lead Optimisation Programmes for The North Face, Student Room & Sliderobes

Experienced in Web Analytics, User Researcher & A/B Testing



What should we be testing?

Pricing & Price Framing

Proposition Testing

Optimising for Engagement

Optimising for Conversion

TALK OVERVIEW


WHAT SHOULD YOU BE TESTING?


WHAT’S WRONG WITH TESTING BUTTON COLOURS

Disclosure: We test buttons. Size, Prominence, Copy, Positioning and yes, Colour.

But:

There are likely to be constraints that limit the number of tests that you can run based on design & developer resource

In most situations we find that there are tests that can be run that will deliver more interesting results and more value


SECTION TITLE

THE CRO PROCESS


SECTION TITLE

WHAT DOES IT INVOLVE?

Analytics

User Research

Cross Device User Testing

Hypothesis Development

Concept Development

Psychology & Persuasion techniques

Usability Best Practice

Testing Strategy and Planning

Test Setup & QA

Test Results Analysis


What is the data telling me about visitor behaviour?

• Top landing pages • Pages with high traffic • Key leak or conversion points

What are users telling me about their experiences? What can we observe about their behaviour?

• Usability errors • Influential proposition messaging

What does the business know about customer struggles or business challenges?

• Customer service insights • Merchandising team

HOW TO CHOOSE WHAT TO TEST


Triage • Just Do It – JDI • Single Feature Test • Batch Testing • Radical Redesign Testing • Larger Redesign projects

Triangulation • Multiple research methods confirm or strengthen the same issues

Potential Impact vs Effort Required • Likelihood to impact key metrics

vs

• Design effort • Development effort • Development schedule

HOW TO PRIORITISE YOUR TESTS


1. PRICE & PRICE FRAMING TESTS


Through moderating or observing hundreds of user testing sessions we have often asked:

WHY TEST PRICING?

“If you could buy the same product from two different websites, what factors would be important in helping you choose which to buy from?”

User consistently respond with:

1. Price 2. Delivery (price and timescales)


1.  Pricing should be based on value, not cost

2.  Prices should be tangible

3.  Prices should be comparable

4.  To change your prices, you must reframe

5.  Price differentiation is the key enabler of profit

6.  Pricing communication shapes the customers perception of value

7.  You must be prepared to lose some sales.

Source: Leigh Caldwell:The Psychology of Price: How to use price to increase demand,

profit and customer satisfaction

SEVEN PRINCIPLES OF PRICING


Three Principles of Anchoring

1.  The perception of price is often based on the context.

2.  Decision making and the power of suggestion

3.  Avoiding extremes

Source: Price Intelligentlyhttp://blog.priceintelligently.com/blog/bid/181199/Price-Anchoring-to-Optimize-Your-Pricing-Strategy

TECHN IQUE:

PRICE ANCHORING


Four Options:

1. Control

2. Variation A – (SAVE £22)

3. Variation B – (SAVE 38%)

4. Variation C – remove ‘(cancel anytime)’ from monthly

WHICH TEST WON?

12.5% overall increase in

subscriptions for Variation A (SAVE £22)


1.  Control - Annual £39.99 (Save £44)

2. Variation – £3.33 per month (£39.99 annually)

WHICH TEST WON?

77% overall increase in annual

subscriptions for the Variation


Consider the range of price options available and test

• This could apply to product or service costs, delivery costs, etc

Run tests to make price more transparent

• What do people get for their money • Test messaging that explains what people are paying for

Control comparisons

• Test using competitor prices for comparisons

PRICE & PRICE FRAMING TAKEAWAYS


2. PROPOSITION TESTING


Watch clip: http://www.youtube.com/watch?feature=player_embedded&v=ps7bB9-ksMY

“ The second a visitor comes to you they have in mind: “Why should I buy from you versus all your competitors?”

You need to display clear unique value propositions or unique campaign propositions.

Everyone who is successful in online conversion is doing this. If you’re not, you’re wasting your time.

Bryan Eisenberg, Godfather of CRO


We can define this as 'Why should I buy from you'. Typically this is a short statement that is displayed prominently on the site.

Here are a few examples:

• ASOS:

"Discover fashion online”

• Naked Wines:

"We are a customer funded wines business"

UNIQUE VALUE PROPOSITION (UVP)


GLOBAL HEADER UVP


USPs - Unique Selling Proposition USPs are more focused on the shopping experience and what advantages visitors gain from shopping with you.

Examples:

• Free next day delivery over £50 • 365 Day Returns • The Official Online Store • The UK's largest stockist

UNIQUE SELLING PROPOSITIONS (USPS)


PRODUCT PAGE USP


BASKET PAGE USPS


Unique Campaign Propositions - UCP’s are messages based on specific campaigns or promotions. These messages will often be time limited.

Examples:

• Last Christmas Deliveries on 18th Dec • XX% off this weekend only • Free next day delivery all week

UNIQUE CAMPAIGN PROPOSITIONS


UNIQUE CAMPAIGN PROPOSITIONS


THE NORTH FACE USP BAR TEST


BEFORE

AFTER

4.35% increase in CR This equated to a six-figure increase in revenue.


GET REVISING SIGN-UP TABLE TEST


GET REVISING SIGN-UP TABLE

“ Price doesn’t put me off, although the basic package is more than enough for me.

I think the site has already got everything I need.

Moderated User testing participants



GET REVISING SIGN-UP TABLE

185% increase in premium subscriptions

with a small drop in free subscription. This was seen to be a good trade-off.

You have more features held back to encourage people to subscribe at a later date.


Test what features/functionality is given for free and what is paid for

Test how your proposition is displayed/framed on your site

Test offering different thresholds for free delivery

Test introducing urgency to the process

PROPOSITION TESTING TAKEAWAYS


3. RADICAL REDESIGNS OPTIMIS ING FOR ENGAGEMENT


This opens up the top of the conversion funnel.

This is likely to be the point where you have the largest number of potential test participants.

Search for landing or key site pages with high bounce rates or those that rarely lead to conversions

Key Areas to Test

• Homepage • Top landing pages • Brand sites/Brand content

OPTIMISING FOR ENGAGEMENT


RUSH HAIR HOMEPAGE TEST


BEFORE AFTER

10.2% performance

improvement in booking conversion rate


MARKED BY TEACHERS ESSAY DETAIL PAGE


BEFORE AFTER

10% improvement in CR


BEFORE AFTER

6.8% improvement in CR


Let users know that they have landed in the right place

Make sure that pages have a clear focus and visual hierarchy. Users need to know where to begin.

Test both simplifying the experience and adding in more detail

A smaller amount of highly qualified visitors can trump large amounts of un-qualified visitors and convert better

ENGAGEMENT OPTIMISATION TAKEAWAYS


4. RADICAL REDESIGNS OPTIMIS ING FOR CONVERSION


Testing towards the end of your funnel helps to convert users who are closest to converting and tend to have higher intent.

Improvements here opens up the bottom of the conversion funnel.

Simplifying and removing issues that slow momentum is crucial.

Key Areas to Test

• Checkout (login, delivery, payment) • Contact/Sign-up forms

OPTIMISING FOR CONVERSIONS


THE STUDENT ROOM HOMEPAGE TEST


BEFORE AFTER

72% improvement in

Registrations from the homepage


BEFORE AFTER

7.4% CR improvement from

sign-up page


Don’t be afraid to offer visitors an opportunity to convert at an early stage

Use persuasive techniques to pull users through (not push!)

Simplify forms to the bare minimum fields required

Reduce errors. There’s a huge correlation between users getting an error and abandoning the funnel

Provide key supporting proposition messaging throughout the process

Re-enforce positive messages (e.g. security in checkout)

CONVERSION OPTIMISATION TAKEAWAYS


With the right kind of research and preparation the odds are stacked in your favour.

Therefore:

Be data driven. Be bold.

Medium risk, high reward.

FINAL TAKEAWAY


Conversion optimisation case studies

Usability best practice

User research insights

CRO presentation slides

Persuasive design slides

Research & optimisation tools

One final resources link...

bit.ly/CROresources


Are there any questions?

THANKS FOR LISTENING

MATT LACEY Head of Optimisation

@matt_lacey12

Presentations & Public Speaking

You're testing what!