Digital Performance Analyst Technical Lead

29 July 2015

Hello!I’m Christopher AuI am the current Web Analyst at Visit Wales. You can find me online at @KonamiKode

About this presentationPart 2: Using analytics to influence change

How analytics can be used to influence change in a business process or web service.

◇ KPI’s and Conversion Rates.◇ Multivariate Testing.◇ Custom Reporting.◇ User Experience.

This template is free to use under Creative Commons Attribution license. If you use the graphic assets (photos, icons and typographies) provided with this presentation you must keep the Credits slide.

Part 1: Analysis of an IPO service

Show the steps taken to set up and analyse completion rates and drop out points and identify areas of change.

Within Google Analytics, we will look at:

◇ Conversions: How to set up goals.◇ Goal Flow - where are we losing them?◇ Time on page and bounce rates.◇ How can we improve?

Analysis of an IPO serviceLet’s start by looking at the new Register a Design service

1

Conversions: How to track completion rates❖ In Google Analytics, we can set up Goals that allow

us to track how often users complete a specific action.

❖ This is useful for monitoring key performance indicators and revenue generation.

❖ By creating a funnel, we are able to see where the drop out points are and determine how to improve.

Google has made this really easy by creating templates for the most common actions...

❖ You can select from a goal template or select Custom for more complicated tasks.

❖ For the purposes of tracking how many people have registered a design, the Acquisition template ‘Create an account’ is perfect.

❖ This goal template can track various actions, such as:

❖ The length of time on page, if you’re worried some steps may take too long.

❖ The number of pages per session, can be useful for seeing how engaged users are with your service.

❖ An event or destination page such as the final summary of this prototype.

Here’s one I made earlier!

❖ This is the goal I set-up to track this service.

❖ To correctly monitor the drop off points, I’ve tried to be concise when including the steps in the goal funnel.

❖ Due to how the service works, there will be loopbacks to previous steps. Google Analytics will show where this occurs.

“A funnel shows the path your traffic traveled through towards a Goal conversion.

We can view the Goal Flow in Analytics to help us see if users are navigating the content as expected, or if there are problems, such as high drop-off rates or unexpected loops.

Here’s what the Goal Flow looks like:

Whilst this is a top level overview, we can see how users progress along the funnel.

The red bits are the drop off points that we’ll need to look at in more detail.

“

Where are we losing them?From the 46 users that entered the funnel, 20 users completed the action to giving a conversion rate of 43%.

This is a good percentage in the real world but given the small dataset and the context in which the data was gathered, we can expect it to be higher.

However, were test subjects instructed to leave the test at a certain point to give us data for expected drop off points?

Single Owner drop off

❖ Intellectual Property Office have indicated they suspect there is a usability issue where a single owner is applying.

❖ We can see here a drop off at these two steps to reinforce that notion.

❖ We see a 2 user drop off when prompted to select the owner of the design and a further 5 exit when asked for the company contact details.

Why do we lose them at Defer?

❖ Another noticeable drop off point is when we ask the user if they want to defer registering the design.

❖ Having run through the process several times, I can’t see a usability reason for this drop off.

❖ Whilst unfamiliar with the product, perhaps the user is unsure of what to choose and elects to come back at a later time when they can say for definite.

“

No back button?In a 2010 study, Firefox creator Mozilla found that the back button is the most used browser function:“By a landslide the 'Back' button was the most clicked of all navigation buttons which include the Back, Forward, Reload, Stop, and Home buttons. Across Windows, Mac and Linux 93.1 percent of users clicked the button at least once over the course of a five-day period. In total the study reported that users clicked on the back button 66 times over the course of five days.”

Whilst this service does allow the user to change details when they reach summary milestones, there is no option to go back to change details. Physical use of the back button appears to break the service, necessitating clearing the browser cache.

Could this to be to blame for some of the drop offs?

Using analytics to influence changeNow we have our data, how can we use this to improve the service?

2

An effective web analysis process

Data collectionData

becomes information

Develop KPI’s Strategy

Basic counts:❖ Visitors❖ Referers❖ Key words

Ratios lead metrics:❖ Unique

visitors❖ Bounce rate❖ Time on page

Counts/Ratios meet strategy:❖ Conversion

rate❖ Abandon rate❖ Revenue

Goals/objective:❖ Save money❖ Make money❖ Increase

conversions

Web analytics is all about human behaviourBehaviours are...

Something that we can detect and record.

BlueActions or goal driven events.

RedReactive responses to contextual stimuli.

Google Analytics can record all of the above behaviours but it can’t record affective, cognitive or situational aspects. For that, we need to look deeper.

We know the drop off pointsWe can see we lose people at the Defer point and then later when they have to select the Owner of the Design.

Based on the figures so far...

We know the completion rateWe can see how many people reach the end of the task.

We can draw basic conclusions based on this data and what we suspect going on past experience. However, can we say for certain how to change this service for the better?

Multivariate TestingIt’s easy to come to a simple conclusion based on one set of data. But to come to the right conclusion, we need more numbers.

“

Google made $200m by testing 42 shades of blue“In the world of the hippo, you ask the chief designer or the marketing director to pick a blue and that’s the solution. In the world of data you can run experiments to find the right answer…We ran ‘1%’ experiments, showing 1% of users one blue, and another experiment showing 1% another blue. And actually, to make sure we covered all our bases, we ran forty other experiments showing all the shades of blue you could possibly imagine and we saw which shades of blue people liked the most, demonstrated by how much they clicked on them. As a result we learned that a slightly purpler shade of blue was more conducive to clicking than a slightly greener shade of blue, and gee whizz, we made a decision…But the implications of that for us, given the scale of our business, was that we made an extra $200m a year in ad revenue.”

Dan Cobley; Marketing Director, Google UK

Every MVT is unique but it’s most often used for...Call To Actions

Should we change the wording? Are the colours working? Should it be placed elsewhere on page?

Headings or Descriptions

Are we using the right font? Is the tone correct for our audience?

Forms

Is it too long? Are we using the right type of fields? Should we paginate?

Page Layout and Style

Are our column ratios correct? Should we use different fonts or colours?

Images

Are the ones we using inspirational or on brand?

And, of course: £££

Copy

Too much or too little text? How does it affect white space? Does the tone match the message?

How to set up MVT split tests❖ Develop at least two versions of a page.❖ Randomly divide users into groups for each version.❖ Show each group a different version.❖ Track how each group performs.❖ Evaluate each version based on results.❖ Repeat as needed.❖ Go with the winning result.

“

What if we can’t implement MVT or A/B testing?If you really want to influence change, MVT is the best way to go because you can back up your decisions with data. It’s no longer “We should do this because I think…” it’s “We should do this because I know…”

However, we don’t always have that luxury.

A lot of changes based on web analytics without using split testing is based on gutfeelings drawn from experience. We can still look at counts and ratios and draw simple conclusions on how to affect the outcomes.

We just can’t back it up with quantifiable data.

Thanks!Any questions?You can find me at:◇ @KonamiKode◇ xopher.au@gmail.com

CreditsSpecial thanks to all the people who made and released these awesome resources for free:◇ Presentation template by SlidesCarnival◇ Portrait photograph by Simon Ayre