Google website optimser 101

www.ivantage.co.uk

1

Website Optimiser 101: An introduction to landing page testing

Presentation to J P MorganGlobal Equity Derivatives Group

Matt TrimmerPrincipal Consultant & Managing Director

Slides:http://www.slideshare.net/ivantage

Friday, 25 March 2011

http://www.slideshare.net/ivantage

http://www.slideshare.net/ivantage

www.ivantage.co.uk

2

Training Analytics 101: Introduction & User Training Analytics 201: Advanced Analysis & Measurement Analytics 301: Advanced Tracking & Technical Implementations

Website Optimiser 101: An introduction to landing page testing

Adwords 101: Introduction to Paid Search Management Adwords 201: Building Profitable Paid Search Campaigns

SEO Essentials SEO Advanced Link Building and Publicity


www.ivantage.co.uk

3

Timings and housekeeping Start 09:30 Break 11:00-11:15 Lunch 12:30-13:30 Break 14:45-15:00 Wrap-up 16:00

Nearest fire exit Toilets Water, Refreshments & snacks


www.ivantage.co.uk

4

slideshare.net/ivantage


www.ivantage.co.uk

5

Course Overview Session 1 – Goals, conversion points and landing pages

Preparing to test – the questions we need to ask Why should we test? Which landing pages should we test? What should we test? How should we test? How do we analyse the test? How do we make conclusions about the test? What do we do after the test? The three business objectives Website goals Key Performance Indicators (KPIs) KPI Examples Website goals The importance of goals Goal conversion The importance of conversion points


www.ivantage.co.uk

6

Course Overview Session 2 – What to test and testing types

Identifying landing pages to test Identifying conversion points Using “voice of customer”, usability testing and heuristic reviews to identify

what to test Identifying which Website Optimizer testing method to use A/B split testing The advantages and disadvantages of A/B split testing Multivariate Testing The advantages and disadvantages of Multivariate testing


www.ivantage.co.uk

7

Course Overview Session 3 – Conducting and analysing A/B testing

How Google Website Optimizer works for A/B testing A/B test set-up – exercise A/B split test set-up A/B split test set-up – settings A/B split test set-up – duration A/B split test reporting A/B split test reporting – key metrics A/B split test reporting – other metrics A/B split test reporting – inconclusive results After A/B testing?


www.ivantage.co.uk

8

Course Overview Session 4 – Conducting and analyzing Multivariate testing

How Google Website Optimizer works for Multivariate testing Multivariate test set-up – exercise Multivariate test set-up Multivariate test set-up – settings Multivariate test reporting Multivariate test set-up – duration Multivariate test reporting – page sections Multivariate test reporting – key metric Multivariate test reporting – other metrics Multivariate test reporting – inconclusive results After Multivariate testing? Getting help Feedback and wrap-up


www.ivantage.co.uk

9

Introductions First

Me (Matt Trimmer) & ivantage overview In a moment

You Your role Your department/website/website area Your objectives for attending today


www.ivantage.co.uk

10

Me Founder, Principal Consultant and Managing

Director of ivantage Love the Internet, 23 years IT, technical

experience, 14 years Internet One of 15 Google Accredited Seminar Leaders

Globally


11

www.ivantage.co.uk

2

About ivantage

Internet marketing agency Established in 2002 Independent 36rd Ranked UK Agency according to New Media Age

rankings


www.ivantage.co.uk

12

Urchin and Google, March 28th 2005

Urchin on Demand > Google Analytics Urchin Software > Urchin Software from Google


www.ivantage.co.uk

13

Website Optimiser Landing page testing and optimisation tool Beta released in Autumn 2006 Full releases in 2007 Uses Google Analytics’ technology

Adds utmx, utmxx first party cookies to utm a,b,c,z and v Launched as a multivariate tool Later released with simplified A/B split test ability


www.ivantage.co.uk

14

What we do We help grow online businesses by

Generating web site traffic through Paid Search (PPC) Natural Search (SEO) Social media marketing Display advertising E-mail marketing

Analyzing web site traffic using Web Analytics to accurately analyse how visitors find and interact with

websites Converting web site traffic using

Conversion Rate Optimisation to convert visitors into customers or clients


www.ivantage.co.uk

15

How we do it We help grow online businesses by generating, analyzing and

converting web site traffic: Using bright, well-trained people who are passionate about what they do Finding, harnessing and deploying innovative, affordable Internet marketing

technology Planning and executing intelligent campaigns that deliver measurable

results Seeking to continually improve return on client marketing spend Ensuring high standards of service and client care Building strong, enduring client relationships Innovating to add additional value to client relationships Following ethical business and technical practices

Delivered through Professional services Training


www.ivantage.co.uk

16

Our vision

To be the Internet’s leading performance-driven Internet marketing agency


www.ivantage.co.uk

17

Professional services

Traffic generation Search Engine Marketing

Paid search – PPC PfP Manage SMA Manage

Natural Search – SEO SEO Manage SMO Manage

Display Advertising

PfP Manage Email Marketing

Email Manage Traffic analysis

Web analytics implementation Accelerator services

Web analytics interpretation Insight services

Web analytics support

Trouble Shooter services Traffic conversion

Conversion optimisation

Landing Page Optimisation Post-Click Optimisation A/B and Multivariate testing Voice of Customer Analytics


www.ivantage.co.uk

18

Training (Public or private on-site) Google Analytics

Google Analytics: Introduction & User Training Google Analytics: Advanced Technical Implementation

Google Adwords Google AdWords 101: Introduction & Fundamentals Google AdWords 201: Optimizing your Campaign Google AdWords 301 & 302: Advanced Adwords Management and Optimization

Google Website Optimizer Google Website Optimizer: Introduction to landing page optimisation

Urchin Software from Google Urchin Software: Introduction & User Training Urchin Software: Advanced Technical Implementation Urchin Software: Developing Custom Reports

Search Engine Optimisation (SEO) SEO: Strategy and on-page fundamentals SEO: Link building and Social Media Optimisation

Social Media Marketing SMM: Strategy and SMM & SMO fundamentals SMM: Social Media Optimisation


www.ivantage.co.uk

19

Who we do it for – multi-channel retail


www.ivantage.co.uk

20

Who we do it for – business to business


www.ivantage.co.uk

21

Who we do it for – Non-Governmental Organisations and Governmental Organisations


www.ivantage.co.uk

22

Introductions – your turn!

You Your role Your department/website/website area Your objectives for attending today


www.ivantage.co.uk

23

Preparing to test - the questions we need to ask ourselves Why should we test? Which landing pages should we test? What should we test? How should we test? How do we analyse the test? How do we make conclusions about the test? What do we do after the test?


www.ivantage.co.uk

24

Why should we test? To understand why visitors behave the way they do

Influence their behaviour For improved website performance


www.ivantage.co.uk

25

The big three business goals? Raise revenue Lower costs Increase customer satisfaction

Raises revenue Lowers costs


www.ivantage.co.uk

26

Key Performance Indicators (KPIs)? A metric

Miles per Hour (mph)

Defining progress to an objective To reach the legal motorway speed limit of 70 mph

Enabling you to take corrective action Accelerate or brake

Speedometer Web Analytics package


www.ivantage.co.uk

27

Key Performance Indicators (KPIs) & web analytics


www.ivantage.co.uk

28

KPIs for your website should include Outcome focused events

“Goals”


www.ivantage.co.uk

29

Goals Goals should be map back to business objectives

The commercial goals of your website Goals should map to visitor tasks

Visitors visit sites to accomplish tasks Tasks normally have a series of steps

Funnel Measure conversion through the task

When goals match user tasks Harmony ensues!


www.ivantage.co.uk

30

The importance of goals for you Commercially justify your work Focus your work Prioritise your work


www.ivantage.co.uk

31

The importance of goals for you Goals map to business metrics

Enable you to build a business case for Analytics investment Analysis investment Action investment

Goals focus on important paths and entrances Enable you to home-in on problem areas

Reverse Goal Path Conversion to goals by traffic source

Monetize goals Enable you to prioritise areas to analyse and optimise

Give goals a goal value Analyse $ Index

(Goal Value + E-commerce value)/unique page views Page up-to goal contribution Focus on high $ Index pages for improvement


www.ivantage.co.uk

32

Placing a value on a goal and the $index Key concept

Goals can have values or not When E-commerce is used the value of the sales transaction is used in the

$index calculation

$index value of A,B and C = $1


www.ivantage.co.uk

33

You can use Regular Expressions (RegEx) to gain insight into the funnel path:

User exhibits behavior: Page 1 = /index.html Page 2 = /category-footwear.html Page 3 = /category-clothing.html Page 4 = /category-headwear.html Page 5 = /products/kangol-tropic-player-trilby.html Page 6= /cart/basketview.html Page 7= /cart/registration.html Page 8 = /cart/paymentoptions.html Goal = /cart/salesorderconfirmation.html

Need to know conversion rate of categories to products to purchase: Step 1: index\.html Step 2-4: ^/category.*/ Step 5: ^/products.*/ Step 6: ^/cart/basketview.html Step 7: ^/cart/registration.html Step 8 : ^/cart/paymentoptions.html Goal: ^/cart/salesorderconfirmation.html

Goals and funnels


www.ivantage.co.uk

34

Goal and funnel behaviour


www.ivantage.co.uk

35

Test conversion points not the rate!

Assume some constants

If more products were added to basket?

At this point


www.ivantage.co.uk

36

Test pages and conversion points

Test Page

Conversion point page


www.ivantage.co.uk

37

Why should we test? To understand why visitors behave the way they do Influence their behaviour For improved website performance

Specifically Conversions points that lead to Website goals That further business objectives


www.ivantage.co.uk

38

Exercise – Your KPIs and goals What are your KPIs for your web

business? What are the goals for your website?

Are you recording them? Use notepad to map the goals out Can you split the goals into logical

conversion points?


www.ivantage.co.uk

39

Which landing pages to test? Landing pages by highest bounce rate? Landing Page Optimisation

Entrance Sources (Top Sources per Page)? Entrance Keywords ( Top keywords per Page?

All traffic sources Cross segment by landing page Navigation summary Entrance paths Funnel visualisation Reverse goal path


www.ivantage.co.uk

40

Identifying problems with Bounce Rate Bounces

Visits to landing pages that don’t view another “tagged” page/Event Bounce rate

The % of visits to landing pages that don’t view another “tagged” page/Event Bounce Rate is a critical metric for analysis unless

Your site is one page (blog) New Visitors might stay longer on a blog for the first time

Your site has a compelling off-line proposition That makes people view one page and leave/call/visit?

Key concept Finding and removing the cause of bounced visits is high priority


www.ivantage.co.uk

41

Page Efficiency Analysis Report

http://www.kaushik.net/avinash/2010/12/best-downloadable-custom-web-analytics-reports.html




www.ivantage.co.uk

42

Page Efficiency Analysis Report





www.ivantage.co.uk

43

Identifying problems with Bounce Rate Exercises

Visitors > Benchmarking Landing Page by Bounce Rate Traffic Sources > All traffic sources > by Bounce rate

Direct is likely to be your benchmark Any source with a higher bounce rate is cause for concern

Compare organic and cpc You would want keyword media to perform consistently Use Weighted Sort

Traffic Sources > Keywords Use Weighted Sort

Traffic Sources > Adwords > Campaigns Use compared to site average


www.ivantage.co.uk

44

Identifying problems with Exit Rate Exits

Visits exiting the site from a “tagged” page Exits Rate

% Visits exiting the site from a “tagged” page Key concept

Only Landing Pages have a Bounce Rate Not a very useful metric

Try Analysing Top Exit Pages by another Dimension Analysing Pages in a closed process, eg checkout

» Identifying where exits occur may indicate a point for usability testing Exercise

Content > Top Exit Pages» Cross segment by a Dimension» Campaign, Source, Visitor Type


www.ivantage.co.uk

45

Identifying problems with Entrance Paths Exercise

Traffic > Traffic Sources > All Traffic Sources Cross Segment by Landing Page

Failure rate Bounce Rate

Success rate Goal conversion rate E-commerce conversion rate

Key concept How could you progress more visits from the Landing Page to the next page?

A/B Testing Multivariate Testing


www.ivantage.co.uk

46

Visitor Acquisition Efficiency Analysis





www.ivantage.co.uk

47

Visitor Acquisition Efficiency Analysis





www.ivantage.co.uk

48

Which landing pages to test? Landing pages that send significant traffic to Conversions points that lead to Website goals That further business objectives


www.ivantage.co.uk

49

Exercise Can we find highly traffic landing pages?

Can we map out conversion paths?


www.ivantage.co.uk

50

What to test? What’s your gut telling you about your pages? What are your customers telling you about your pages?

4Q - “Voice of Customer” - site level KISS Insights - “Voice of Customer” - page Level Usability testing

Lab usertesting.com whatuserdo.com

Heuristic reviews Site reviewed against a list of expected/excepted criteria


www.ivantage.co.uk

51

What to test? Presentation of offers on Landing pages that send significant traffic to Conversions points that lead to Website goals That further business objectives


www.ivantage.co.uk

52

Exercise Can you define some alternative presentations for your

selected landing pages? Can you split these alternative presentations into Page

Sections? Could these sections have variations Which sections will have the biggest impact on conversion

point?


www.ivantage.co.uk

53

Lunch time


www.ivantage.co.uk

54

How does GWO work?


www.ivantage.co.uk

55

How should we test - two methods Simple A/B split testing

Pros Simple and fast to set-up Great for low traffic volumes (small number of combinations)

Cons May miss the fact that some components on two different pages work

better together Complex multivariate testing

Pros Won’t miss the fact that some components (on your A/B pages) work

better together Highlights best performing Page Sections (areas of the test page) Highlights best performing Combinations (Variations in Page Sections) Provides you with a Control that you would keep testing

Cons May take significant time to produce results (traffic/Combinations)


www.ivantage.co.uk

56

A/B Testing

Control script

Tracking Script

Conversion Point

Control script

Tracking Script

A

B

Conversion Page

Tracking Script

Testing Pages


www.ivantage.co.uk

57

How GWO worksWebsite

A B


www.ivantage.co.uk

58

GWO decides which page to serve

cookies

B


www.ivantage.co.uk

59

GA cookies

UTMb - Visit cookie, lasts for 30 minutes from every page view

UTMa - Unique visitor cookie, lasts for 2 years from set or update on every page view

UTMc - Visit end cookie, not set to expire, does so when browser closes

UTMv - Not always set, requires the _setVar() command

UTMz - Campaign visit tracking cookie, determines traffic sources, lasts for 6 months from set or update on every page view


www.ivantage.co.uk

60

The ___utm.gif request from the ga.js to Google’s servers__utm.gif?utmwv=6.3&utmn=101027544&utmsr=1280x854&utmsc=32-bit&utmul=en&utmje=1&utmjv=1.5&utmfl=7.0 r24&utmdt=My%20Website&utmhn=www.site.com&utmp=/index.html&utmac=UA-51244-4&utmcc=__utma%3D117945243.823564440.1131054947.1131080858.1133834608.4%3B+__utmb%3D117945243%3B+__utmc%3D117945243%3B+__utmz%3D117945243.1131054947.1.1.utmccn%3D%28Spring%20Sale%29%7Cutmcsr%3D%28google%29%7Cutmcmd%3D%28ppc%29% %7Cutmtm%3D%28shoes%29%3B

Screen Resolution: 1280x854

Screen Colors: 32-bit

Browser Language: English(en)

Java Enabled? Yes (1)

Java Version: 1.5

Page Title: My Site

Filename: /index.html

Visits: 4

Campaigns Clicked: 1

Campaign: Google(PPC) Spring Sale (shoes)


www.ivantage.co.uk

61

What are “Fortune” Cookies?• Text files sent to browsers by websites

• 1ST or 3rd party• Visit apple.com

• apple.com issues cookies (1st party)• If banana.com attempts to issue cookies (3rd party)

• Session and persistent• Session

• Last for 30 minutes of inactivity• Persistent

• Lasts for 2 years


www.ivantage.co.uk

62

Browser1st visit

web serverNew Visitor

Browser web serverCookie

Subsequent visits Returning Visitor

Respond with Page B as last time

Respond with Page A or B

Persistent UTMx (3) cookies

cookies

B


www.ivantage.co.uk

63

Weaknesses of cookies?• Machine dependant• As close as we can get to a person• Although modern OSs have multiple user IDs

• User 1, User2, User 3

• Can be cleared• Worst scenario?

• Can be blocked/not accepted• Most E-commerce sites use 1st party cookies to make basket

function• So blocking 1st party cookies degrades your Internet experience

• 3rd party cookies are often blocked


www.ivantage.co.uk

64

A/B split test set-up - exercise Visit your website or

http://www.ivantageaaa.co.uk/GAWO/index.html Navigate to Products (A/B) Clear cookies to see different versions


http://www.ivantageaaa.co.uk/GAWO/index.html


www.ivantage.co.uk

65

A/B split test set-up Original page

Control script Sends 1st party cookies to ensure

new visitors enter test (if applicable) returning visitors see same page

Tracking script Records page display

Variation pages (A, B, C) Control and Tracking scripts

Conversion page Tracking script


www.ivantage.co.uk

66

A/B split test set-up - settings % traffic to see experimental pages Selected % is sent uniformly to variations

Disable losing variations - off or


www.ivantage.co.uk

67

A/B split test set-up - duration Start your test Test never stop until you stop it

Statistical prediction of conversion rate from actual performance Duration

Calculator http://www.google.com/support/websiteoptimizer/bin/

answer.py?answer=61688&hl=en_US Combinations =2 (A,B), 3 (A,B,C) This is where A/B split testing can be quick


http://www.google.com/support/websiteoptimizer/bin/answer.py?answer=61688&hl=en_US




www.ivantage.co.uk

68

A/B split test reporting 2-3 hours delay

Original and variations


www.ivantage.co.uk

69

A/B split test reporting - key metric Estimated conversion rate

Forecasted/predicted conversion rate if rolled out Based on actual observed conversion rate

% conversion = No. conversion page hits/No. test page hits X 100 Calculated with an 80% confidence rate

Will fall within this range 80% of the time


www.ivantage.co.uk

70

A/B split test reporting - other metrics Conv/Visitors

Conversion numbers Observed improvement

Between original and variation Chance to beat original

Likelihood of this beating original if rolled out


www.ivantage.co.uk

71

A/B split test reporting - inconclusive No clear winner

Pointlessness of testing small changes Make a big variation change and test again


www.ivantage.co.uk

72

After A/B testing? Test again! Test next conversion point Use multivariate testing

Bring back discarded (but big) changes


www.ivantage.co.uk

73

Multivariate test set-up - exercise Visit your website or

Clear cookies (Firefox) http://www.ivantageaaa.co.uk/GAWO/index.html Navigate to Products (MV)

Clear cookies to see different versions




www.ivantage.co.uk

74

Multivariate Testing

Control script

Tracking Script

Section Scripts (8)

Testing PageConversion Point

Tracking Script

Conversion Page

Variations (in content)


www.ivantage.co.uk

75

Multivariate Testing

Control script

Tracking Script

Section Scripts (8)

Testing PageConversion Point

Tracking Script

Conversion Page

Variations (in content)

Combination (Recipe)


www.ivantage.co.uk

76

How GWO works

Testing Page


www.ivantage.co.uk

77

How GWO works

cookies

Testing Page


www.ivantage.co.uk

78

Multivariate test set-up Original page

Control script Sends 1st party cookies to ensure

new visitors enter test (if applicable) returning visitors see same page

Tracking script Records page display

Section scripts (up to 8) Conversion page

Tracking script


www.ivantage.co.uk

79

Multivariate test set-up - settings % traffic to see experimental pages? Selected % is sent uniformly to variations

Auto-disable losing variations? - off or


www.ivantage.co.uk

80

Multivariate test reporting 2-3 hours delay Combinations and Page Sections

Original and combinations


www.ivantage.co.uk

81

Multivariate test set-up - duration Start your test Tests never stop until you stop them

Statistical prediction of conversion rate from actual performance Duration

Calculator http://www.google.com/support/websiteoptimizer/bin/

answer.py?answer=61688&hl=en_US Combinations =Page Sections x Variations

Grows large very quickly» Traffic requirements/time become great» >Combinations > Traffic/time needed to reach conclusion

Assumes a 12% conversion rate improvement






www.ivantage.co.uk

82

Multivariate test reporting - page sections First to populate with data


www.ivantage.co.uk

83

Multivariate test reporting - key metric Relevance rating

The Page Sections that had the greatest impact on overall results


www.ivantage.co.uk

84

Multivariate test reporting - other metrics Section Variations detail

The best Combination does not always comprise of the best Variations


www.ivantage.co.uk

85

Multivariate test reporting - key metrics Estimated conversion rate

Forecasted/predicted conversion rate if rolled out Based on actual observed conversion rate

% conversion = No. conversion page hits/No. test page hits X 100 Calculated with an 80% confidence rate

Will fall within this range 80% of the time


www.ivantage.co.uk

86

Multivariate test reporting - other metrics Conv/Visitors (not shown)

Conversion numbers Observed improvement

Between original and variation Chance to beat Original

Likelihood of this beating original if rolled out Chance to beat All

Likelihood of this beating all combinations if rolled out


www.ivantage.co.uk

87

Multivariate test reporting - inconclusive No clear winner

Pointlessness of testing small changes Make a big Variation changes and test again


www.ivantage.co.uk

88

After Multivariate testing? Test again! The next conversion point Test big impact changes Your Control Combination is replaced by out-performing Combinations Always try to better the Control Combination


www.ivantage.co.uk

89

Feedback http://www.ivantage.co.uk/feedback [email protected] Write an honest, considered and fair review to receive for each day attended:


http://www.ivantage.co.uk/feedback

http://www.ivantage.co.uk/feedback

mailto:[email protected]




www.ivantage.co.uk

90

Getting help [email protected] for 60 days




www.ivantage.co.uk

91

The End


Technology

Google website optimser 101