Name of presentation

Session Replay ToolsWhy are these so vital?@OptimiseOrDieWAW London Jan 2016

UX and Analytics (1999)User Centred Iterative Design (2001)Agile, Startups, No budget (2003)Funnel optimisation (2004)Form analytics (2004)Multivariate & A/B testing (2005)Conversion Optimisation (2005)Persuasive Copywriting (2006)Lean UX (2008)Holistic Optimisation (2009)Psychology and Conversion (2010)Velocity & Scaling testing (2011)Optimisation Mentoring / Workshops (2013)Innovation and Transformation (2014)

My Timeline@OptimiseOrDie

Ruuhijarvi

2

@OptimiseOrDie

What are these tools were talking about?

What stuff do they do?

What do optimisers find most useful?

Summary

Sep 2015, Ruuhijarvi

What are these tools were talking about?@OptimiseOrDie

@OptimiseOrDie 1. What are these tools?

Naming these tools is quite hard!

What are the characteristics?

Session recordings (all of them)Attention / Click / Scroll maps (all)Polls, Surveys, Feedback (VOC) (most)Funnel or segmentation tools (some)Form Analytics (some)

Ive been working on this presentation and thinking about it for over two months now. And this is one of the first graphs I wanted to include, because it represents the hype cycle in AB testing.

This is what a lot of companies go through with new technology adoption, so I wanted to show you what the AB testing hype cycle would look like [CLICK]5

@OptimiseOrDie1. Who should you consider?

Helping you find out WHY!

Decibel Insight www.decibelinsight.comHotjar www.hotjar.comInspectlet www.inspectlet.comSessionCamwww.sessioncam.comClicktale www.clicktale.comMouseflow www.mouseflow.comGhostrec www.ghostrec.comYandex WebVisor (free)metrica.yandex.com/promo/webvisor

6

@OptimiseOrDie4. If you have 4 hoursPLUS

Snap interviews (Sales, Customer Services, Tech Support)Run a quick poll or survey (See my tools slides)

Less Bullshit!

7

What stuff do they do?@OptimiseOrDie

@OptimiseOrDie1. Change Growth Trajectory1. Session Recording what?

PROSVideo recordings of people using the siteVital to see a qualitative layer with dataSpot form problems, annoyance, friction, validation issues, errors, repetitive or confused behaviour.Push code live, watch people interacting in near realtime. Fix stuff very lean!

CONSNOT a substitute for form analyticsHard to view thousands of recordingsWill hit site performance!Its always an abstraction of reality*

9

@OptimiseOrDie1. Change Growth Trajectory1. Session Recording tips?

Record only when you need toUse it tactically turn on for the landing page, collect 300 recordings, run them at high speed, crunch in an afternoon

Recording all users is fine, as long as you accept the drag on experiences (especially on mobile)Use the GA DOM timings report to check the hit (ask me)

Ensure that the tool records not just mouse but touch, swipe, zoom layer and the right device groups.

Filtering is vital if you have shite filtering of session recordings, its now a shite tool vomiting data.

JavaScript errors/browser & device problems use these tools to mine for large bugs in your device experiences.

10

@OptimiseOrDie1. Change Growth Trajectory1. Session Recording

11

1. An Opportunity? Modelling!22%65%13%92%6%2%65%34%61%37%2%100%

12

@OptimiseOrDie1. ExamplesUS Underwear Brand x.xM per month, 2 bugsEuropean Travel site x.xM per month, 13 bugsUK Mobile Telco x.xM per month, 3 bugsTravel Aggregator x.xM per month, 5 bugsGambling / Event sitex.xM per month, 11 bugs

Total developer time to fix ALL of these = < 5 days!At a cost of 8000 to find and fix, the ROI was 91,150%In many cases, savings larger than the ENTIRE IT budget

13

Browser Checkswww.crossbrowsertesting.comwww.browserstack.comwww.spoon.netwww.saucelabs.com Mobile & Tabletwww.appthwack.comwww.deviceanywhere.comwww.opendevicelab.com

Article & Infobit.ly/devicetesting

1. Here is our 84 Million testing rig!

14

@OptimiseOrDie2. Heat, Scroll and Click Maps

All these session recording tools also have heat/click/scroll mapsHow far do people read down this page? What do they click on? Where did they interact?

15

@OptimiseOrDie1. Change Growth Trajectory2. Scroll maps what?PROSSee how far people scroll down a pageCan be used with analytics data to analyse and optimise read/scrolling behaviour, particularly on long form pages.Helps spot where you have fold issues with the line landing on white or solid colour space (users dont see cues to scroll)

CONSBiased if people see different grid layoutsNeeds filtering/segments to bring out power

16

@OptimiseOrDie1. Change Growth Trajectory2. Scroll maps tips?Split by device experience category (mobile, tablet, desktop)

Make sure you can split by viewport or resolution (or at least design breakpoint)

If you can integrate your own segmentation, this makes it a really good tool see high spending customers segment, for example.

Useful for optimising the bottlenecks and flow problems sometimes introduced into long page designs.

Allows you to focus optimisation on actually where the eyeball attention is!

17

@OptimiseOrDie1. Change Growth Trajectory2. Click maps what?PROSHelps you see where people click (that is NOT ACTUALLY clickable)See broad brush visual representation of data you can also get from analytics

CONSI find using analytics event tracking to be easier to get what I want. Looks nice though!

18

@OptimiseOrDie1. Change Growth Trajectory2. Click maps tips?Look for non clickable areas with high volumes of clicksSort it out!

19

@OptimiseOrDie1. Change Growth Trajectory2. Heat / Attention maps what?PROSModels where attention is being given on a page, by deducing this from mouse movements.Yes there is a correlation between mouse movement and eye gaze (but not big).

CONSThere is also a correlation with frustrated behaviour (wheeling, meerkatting, dogshaking)Interesting but not as useful as the other two, IMHO

20

@OptimiseOrDie1. Change Growth Trajectory2. Heat / Attention maps tips?Look for non clickable areas with high volumes of clicksSort it out!

21

One of the most useful yet undervalued parts of these tools!Insider tip!If you dont know what to do with this page/process/step/widget/badly designed piece of crap why not ASK SOMEONE!Polls are one of the easiest ways to remove bullshit from your organisation.Surveys often turn into a bunfight over how many useless questions you can add (thats a whole presentation in itself)Snap polls (landing pages, triggers) are really bloody useful.Open ended questions can yield some surprising stuff3. VOC / Polls / Surveys what?

22

Task Gap Survey (above)General site feedbackFunnel Abandonment SurveySite Exit SurveyNPS and Customer satisfactionLanding Page Polls 3. VOC / Polls / Surveys typesBased on todays visit, how would you rate your site experience overall?Were you able to complete the purpose of your visit today?If NO, why were you not able to?If YES, what did you value most about the website?Thank You Page survey

Thank you!Was there one thing that nearly stopped you from purchasing a flight today?

23

If you have integration with Google Analytics (or another package), tick!If it integrates with your AB testing tool, tick!If it has decent segmentation capabilities, tick!

Ask what methods the tool supports (poll overlay, inpage, survey, exit survey, inline questions etc.) Talk to customers whove actually used this part of the tool

Check out my list of qualitative tools at the end of this deck. Try usabilityhub.comFollow @Contentverve 3. VOC / Polls / Surveys tips?

24

@OptimiseOrDie1. Change Growth Trajectory3. Competitive AdvantageWhen hacking the growth of LOVEFiLM (now Amazon)Regular competitor surveys we LOVED this!Survey your OWN customers AND competitorsWe got a quarterly sample of 1000 surveys from each brandNPS data collected as well as Service Metric RatingsAllowed us to know where competitors were weak

Speed of Email response

Question Answered first timeHere is one exampleSpend where it will shift delightSpend where competitors are weakStop spending where it doesnt make a differencePrioritise based on ROI

25

4. Funnel and Journey Optimisation

Some of these tools have very nifty funnel optimisation capabilities.

Too much to cover here, but always weigh up whether you want to add meal to your salt (or salt to your meal).

Id rather have the tool data in Google Analytics, to do my analysis there (using flow reports, ring models, horizontal funnels, step drops)

The data layer collected in your tool may not be as rich as your analytics (or collected in the same way) so decide where you want to integrate!

26

@OptimiseOrDie5. Form AnalyticsWhere do people pause?What fields or form areas cause people to abandon you?What errors make them leave?Youd notice this on a phonecall, so why not the website?

Absolutely vital for maximising yield of your formsPrioritise your effort around where the leaks are, not the whole product!

This is so we can call you in case theres a problem delivering your order.

27

What do optimisers find most useful?@OptimiseOrDie

@OptimiseOrDie 1. What do optimisers find most useful?High fidelity recordings, segmentable, filterableTactical insights before optimisation inspecting small areas, template groups, funnel stepsPolls too few people use VOC for instant, lightweight feedback it cuts stupid effort. Bullshit in the office? Ask a Customer is the cry!Form analytics underutilised and yet rich in moneyValidation and submission errors in forms can be one of the largest drivers of abandonment.Device experiences = Money

Ive been working on this presentation and thinking about it for over two months now. And this is one of the first graphs I wanted to include, because it represents the hype cycle in AB testing.

This is what a lot of companies go through with new technology adoption, so I wanted to show you what the AB testing hype cycle would look like [CLICK]29

Summary@OptimiseOrDie

@OptimiseOrDieSummaryWho cares if your conversion rate is 5%. You need to care about why the 95% dont buy and invest huge effort in understanding - WHY?@OptimiseOrDie

Ive been working on this presentation and thinking about it for over two months now. And this is one of the first graphs I wanted to include, because it represents the hype cycle in AB testing.

This is what a lot of companies go through with new technology adoption, so I wanted to show you what the AB testing hype cycle would look like [CLICK]31

32

WERE ALL WINGING IT

I once explained to my daughter you know, when adults like look really in control and making decisions and appearing not to suffer from indecision? Dont believe it for a minute were just better at winging it cause were older.

And this is the huge hole thats gnawing at the hear of many digital operations. The inability to understand what you can and cant be confident about but nobody wants to admit theyre guessing a lot of the time.34

Guessaholics Anonymous

Hopefully now youve joined Guessaholics Anonymous from today, you can move on to a data driven and more productive future. Enjoy less wastage in your products, greater efficiency in marketing spend and grow faster than you imagined. All it takes is a bit of love of your customers, your business and some solid ground rules.

35

@OptimiseOrDieSummaryThe greatest enemy of knowledge is not ignorance, it is the illusion of knowledge.

Stephen Hawking

Ive been working on this presentation and thinking about it for over two months now. And this is one of the first graphs I wanted to include, because it represents the hype cycle in AB testing.

This is what a lot of companies go through with new technology adoption, so I wanted to show you what the AB testing hype cycle would look like [CLICK]36

@OptimiseOrDieEmail me sullivac@gmail.comSlidesWill be mailed to you!LinkedInlinkd.in/pvrg14MentoringTeams/People - ask!

Som, feedbackCrowdsourcing Tools:Usertesting (P/S)*www.usertesting.comUserlytics (P/S)www.userlytics.comUserzoom (S)www.userzoom.comWhat Users Do (P)www.whatusersdo.comLoop11 (S)www.loop11.comOpen Hallway (S)www.openhallway.comEthnio (For Recruiting)www.ethnio.com

Feedback on Creatives:Usability Hubwww.usabilityhub.comFive second testwww.fivesecondtest.comPidocowww.pidoco.comUsabillawww.usabilla.comConceptsharewww.conceptshare.com

* P = Panel, S=Site

3812. Crowdsource your Research!

38