UX analytics to make a good app awesome€¦ · Today the only real solution available is to recruit UX experts to conduct usability tests or to create a home-made system that would

UX analytics to make a good app awesome

White Paper

Copyright © 2014 UXprobe bvba

Executive summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1 . Building a good app is a challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 . How is UX of apps measured today? . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.1. Web analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.2. Surveys and user feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

3 . What UXprobe delivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

3.1. UX & Usability focused . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

3.2. Tracking task-based user behavior . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.3. Surveys and user feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

4 . A good app is not good enough anymore . . . . . . . . . . . . . . . . . . . . . . . 9

4.1. The rising power of apps - it’s happening now . . . . . . . . . . . . . . . . 9

4.2. The opportunity cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

4.3. Show your customers that you care for them . . . . . . . . . . . . . . . . . . 9

Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

We will be happy to give a live demo! . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Table of contents

UX ANALYTICS TO MAKE A GOOD APP AWESOME UXPROBE

3

UX analytics to make a good app awesome

“Just an app” is not enough, customers demand amazing products

For many businesses, applications have now become more and more important. As the mobile web is growing, they turn out to even be a crucial component in the success of companies.

At the same time, customers’ expectations have grown just as fast. As they get used to better and better technology, they also want great design. Customers do not want “just an app”, they want a great app.

More than that, what they experience when using an app - how suc-cessful they are and how they’re feeling - will influence and define their relation with the service, the product and even the company as a whole. This is why User Experience has emerged as a must-have area of concern. User Experience (UX) is a subtle notion that combines user behaviors, attitudes and emotions. In order to manage and improve the UX of apps, it is important to collect metrics and insights that are specific to that subtle notion.

Today, Product Managers are left with a plethora of tools, like web analytics, surveys, reviews, etc. that explore each of these dimensions separately, without any means to combine or aggregate the results. Moreover these tools were mostly developed to measure websites, not apps. They fail to analyze a much more complex behavior that goes beyond clicks and pages. This is where UXprobe comes in, a software service that combines user behavior tracking and sentiment collection that helps you pin-point the problems and opportunities in your app.


4

1 . Building a good app is a challenge

When building an app, Product Managers face a great chal-lenge: find a clear fit between the business, engineering and users’ objectives. Their main goal is to build a product that solves the right problem with the right solution for the right market. In order to make their company’s app that success-ful product, Product Managers will convince their stakehold-ers that they are going in the right direction. They will give trustworthy directions to the development team and they will follow users’ needs.

To understand users’ needs, Product Managers, most of the time, use their own experience; their “gut feeling” or they go for the “easy enough for Moms” principle. Of course they also perform market research, they have focus group meetings, and do usability tests. All of this is good in the early stages of app development where mostly qualitative data is needed to better understand and improve con-cept ideas until they can be put into a first version of the app. Once you have an app that can be tested or used by a larger group of people, e.g. beta testers, it should be complemented with analytics to understand how this larger group uses the app, what they are expecting and how satis-fied they are.

Product Managers use web analytics services to measure the user experience of their apps, but these web analyt-ics services were actually meant to measure marketing and business goals of web sites. Their metrics are very high-level and do not deliver any actionable insights to improve the design and usability of apps. Web analytics services just

Trying to combine all these tools together is very complicated and time consuming task .

record/count what screens or pages users look at, content or features they select. They don’t track what users do as a whole and certainly not why they do it or how satisfied they are. Product Managers use different kinds of feedback gathering tools for asking their users how they feel about their app. Often they use surveys to learn more about their users’ habits, attitudes and feelings.

Unfortunately these tools only focus on one side of user experience, either on user behavior or on user feelings and satisfaction. Trying to combine all these tools together is very complicated and time consuming task. Sometimes it is even impossible to cross over the data from the different types of services. None of the current systems offers them a single platform that measures behavior, shows feedback and collects answers to surveys.

Today the only real solution available is to recruit UX experts to conduct usability tests or to create a home-made system that would allow app builders to capture their own data. As Jared Spool analyzes it: “We’ve found the teams that get the most out of their analytics are the ones that build their own systems. Instead of relying on the http logs provided by the web server, they capture their own events — ones they believe are the most important activities to study.”1

Wouldn’t it be great to avoid the trouble of building a complete home-made system and have a service that does that for you?


5

2 . How is UX of apps measured today?

Today, when one wants to measure the User Experience of an app, often usability testing is part of the process, espe-cially at a very early stage. As for the available tools out there, the state of art in measuring User Experience is split between different actors. Each of them is dealing with one specific aspect of UX: · Web analytics: Google Analytics, Adobe Analytics,

Webtrends, Mixpanel, etc · Surveys: Surveymonkey, Zoomerang, etc · Feedback: Uservoice, Usabilla, etc

2.1. Web analytics

Web analytics tools are about: · counting page views, clicks · calculating abandon and exit rates · showing conversion rates and funnels

Counting is what web analytics tools do by default: page views, clicks, abandons and exits. In apps, these behavioral metrics are of course needed, but only show you what peo-ple did with an app on a macro level. If you want to improve the user experience of apps you also need to understand the user behavior on a micro level, understanding what they did throughout the whole app. It is not just about measuring user interactions, but you should understand the detailed flow the user experienced while using an app. Only a detailed flow can show when things went wrong and where in the app they went wrong. When a user gets e.g. an

understand the detailed flow the user experienced while using an app

error it is more important to know how the user’s interac-tion resulted in that error than just knowing that the error occurred.

Conversion rates today are very important, people use it as a barometer to see how well the app and hence their busi-ness is doing. Also, product teams use it to check whether the improvements they made to the app have an impact, but, unfortunately, it is not the right metric for validating design improvements. A conversion rate is a ratio, the num-ber of people who reach the conversion goal divided by the number of users who used the app. Ratios are a problem because it consists of the numerator (people who meet goal) and the denominator (number of users) and you don’t have control over which side changes. Changing the design and then looking at the conversion rate will not give you the evidence that your design was improved. It could be that you had more people achieving their goal but it could also be that this did not change and that just less people used the app.

Conversion funnels show you which flows people fol-low while they use your app. You can see where people abandon a flow or even exit an app, but you cannot clearly deduce why they abandoned SEE PIC 1 . Was it because they eventually were not interested in that piece of the app or was it because they could not work their way through the app because of UI or technical problems?

PIC 1 . Conversion funnel, but why did people abandon the flow?


6

2.2. Surveys and user feedback

· measuring satisfaction and users’ feelings while they use an app

· engaging with customers to get ratings and comments · implementing polls e.g to understand which features

are preferred the most and which are not understand-ing users’ expectations

Satisfaction questionnaires are good and necessary, but a rating is not enough. Users should be able to tell you why they liked or did not like your app. They should have the possibility to give you specific insights leading to improve-ments. Communicating with your users should be a 2-way process. Users should be able to tell the product team directly how satisfied they are with the app and what func-tionality and features they need.

Combining user feedback and satisfaction surveys with user behavior, will give a much clearer picture of what your users’ are doing and why they are doing it. With current tools it is almost impossible to bring the different data sources together and deduce the actionable data with which you can improve the design of your app.

3 . What UXprobe Delivers

3.1. UX & Usability focused

UXprobe enables enterprises to understand the user experi-ence and usability of their apps as opposed to the market-ing performance and conversion of their websites. UXprobe is a different kind of metrics.

User Experience (UX) involves a person’s behaviors, atti-tudes, and emotions about using a particular product, system or service. User Experience includes the practical, experiential, affective, meaningful and valuable aspects of human-computer interaction and product ownership (Wikipedia)2.

UXprobe deliberately chooses to deal with the whole User Experience in one single service by offering a combination of behavior tracking and feelings collection.

Usability is the effectiveness, efficiency and satisfaction in which specified users achieve specified goals in particular environments (definition - ISO 9241)3.

The metrics in UXprobe, the way they’re collated, the way they’re displayed, are focused on making it easy to under-stand the usability of your app. · Effectiveness (completion rate / error rate): the accu-

racy and completeness with which specified users can achieve specified goals in particular environments » Can users complete their tasks? » What is the error rate?

UXprobe deliberately chooses to deal with the whole User Experience in one single service


7

· Efficiency (avg. duration): the resources expended in relation to the accuracy and completeness of goals achieved » How much effort do users require to perform a task,

measured in time · Satisfaction (Happiness): the comfort and acceptabil-

ity of the work system to its users and other people affected by its use

In the UXprobe dashboard, the graph “Task (user) success” will not only show you whether or not people could suc-cessfully reach your conversion goal, but it will also show you why SEE PIC 2 . UXprobe allows you to drill down into the user’s individual sessions and really understand what they did and where they had problems. It helps you understand by showing you e.g. the errors that occurred in a particu-lar task, how a user went back and forth between specific screens, etc.

Task success also incorporates user satisfaction collection, so you can ask your users why they left the task allowing you to get better insights about what users think is happen-ing. With the feedback panel users can even directly send feedback about their experience and/or expectations.

PIC 2 . Task success: effectiveness, efficiency and satisfaction

3.2. Tracking task-based user behavior

“Moving to an activity-based logging system opens up a world of opportunity for the analytics” writes Jared Spool4.

UXprobe provides task-based analytics which helps you understand what your users are doing with the app: which tasks they perform, the features they use, the errors they encounter, etc. Instead of just looking at pageviews, the product team can capture the complete activities which they believe are the most important activities to measure. Since it is no longer tied to pages, but to tasks, you can col-lect both the state change and the data that was involved (e.g. values that were filled out, etc.)

Conversion funnels show where people abandon the flow but they don’t show you what happened to the users that made them abandon the task. Because UXprobe is a task-based analytics system you can look at the specific task at hand and visualize why people left. User Interface (UI) and system errors for instance can be made visible from the ses-sion details, something you miss from a conversion funnel.UXprobe’s data tracker captures errors that are displayed together with the corresponding data that generated the error:

· Detecting frequent errors helps to understand which parts of the UI should be redesigned.


8

· Detecting an increase of error messages with a new version of the app might mean that a recent change “broke” the task SEE PIC 3 .

3.3. Surveys and user feedback

UXprobe uses a very pragmatic approach to capture users’ attitudes and emotions. Micro surveys, presented at the right moment can help you understand your users’ satisfac-tion SEE PIC 4 .

It can be more than just a rating… When your users are not satisfied you should ask them why! But at the right moment, when they have finished a task or about to exit the app. In the real world we don’t interrupt others when they are in the middle of a task and so it can be when choosing the right moment to pop up a one question survey.

Communication with your users should be a 2-way process, particularly when things are not going well. That’s why UXprobe also offers a feedback panel that users can use no matter where they are in the app. With this panel users can tell you themselves how they feel about the app at the moment that matters to them.

A very pragmatic approach to capture users’ attitudes and emotions

PIC 4 . Surveys

PIC 3 . Drill into sessions


9

4 . A good app is not good enough anymore

4.1. The rising power of apps – it’s happening now

Given the overwhelming range of apps available across mul-tiple platforms today, looking back to their early days shows just how quickly they have evolved. As recently as three to five years ago, apps were just beginning to carve out a niche in our digital world.

Today, apps are everywhere. For many of us, they’re an essential part of everyday life – every kind of service has an app – and we expect no less. Banks, energy providers, telecommunication companies, car rental firms, hotels & airlines – for enterprises across all sectors, the app is now a vital asset to outperform their competition.

Where will apps be in a few years? They will be essential to businesses and customers alike, a crucial method of com-munication between the two, helping to establish a cus-tomer base in a user-friendly format. Companies will have to compete extremely hard to secure, and maintain, a place on people’s smartphones; without an app, enterprises will not even exist. Investing resources into creating innovative, engaging apps will be a key aspect of running a successful business.

4.2. The opportunity cost

In this digital world which changes rapidly, not missing the train has become a real challenge. If you don’t want to be

If you can improve the design of your app based precisely on the direct input of your users, you will, certainly, make their lives much easier .

among those who run to catch up, you know you have to be vigilant and make the right choices.

If you want to be among those who are leading the industry, you have to have an app that performs really well and that satisfies your customers.

Once you’ve missed your chance, there won’t be any “back” button.

4.3. Show your customers that you care for them

In this fast evolving digital world, keeping your custom-ers happy is one of the great challenges you will face. UXprobe gives you the means to listen very carefully to them. Literally when it comes to feedback and surveys, but also in a more visual way by looking directly at what they do through behavior tracking.

If you can improve the design of your app based precisely on the direct input of your users, you will, certainly, make their lives much easier. They will thank you for that.

If you can identify the most common errors and the succes-sions of events that lead to them, you might find solutions to improve the design of your app. Solutions that will effectively help your users perform better. They will thank you for that.

When feelings are listened to and understood, your users will reward you with trust and advocacy.


10

SummaryFor most enterprises, applications are about to be the mandatory step in their online presence. To achieve that goal, a great User Experience, the one that will allow users to not even think twice when using an app, is a must-have. If users can easily access a product or a service through the related app, they will definitely keep a spot for that app on their smartphone; they will regard it as part of their lives.

User Experience (UX) involves behavior, attitude and emotions; all together. Many techniques and tools have been developed around these notions (usability testing, web analytics, feedback and survey tools), but none of them until now is a combination of the three and none of them is fully focused on apps.

UXprobe aggregates the three notions of behavior, attitude and emo-tions in a all-in-one service. The behavioral metrics are humanized by the collection of feedback and answers to surveys. The emotions are contextualized by the behavioral metrics

UXprobe’s behavioral metrics focus on explaining the usability of apps by analyzing Task based success. This gives a fast and high-level under-standing of how effective and efficient users are within the app.UXprobe’s dashboard provides all the necessary means to understand users.

With these insights, making decisions to improve the design and User Interface, to drive apps in the right direction, to make users happy, will be a clear way forward.

A great User Experience, the one that will allow users to not even think twice when using an app, is a must-have


1 . Jared Spool, Activity Analytics for Web Apps, User Interface Engineering, 2012.

http://www.uie.com/articles/web_app_analytics/

2 . Wikipedia “User Experience”,

http://en.wikipedia.org/wiki/User_experience

3 . ISO Standard.,

http://www.w3.org/2002/Talks/0104-usabilityprocess/slide3-0.html

For a full article about UX and Usability definitions:

https://uxpa.org/resources/definitions-user-experience-and-usability

4 . Jared Spool, Activity Analytics for Web Apps, User Interface Engineering, 2012.

http://www.uie.com/articles/web_app_analytics/

We will be happy to give a live demo!Call us +32 (0)485 69 78 35 Send us an email [email protected]

Check our Website www.uxpro.be Watch our video http://youtu.be/xRYuRvhlWVQFollow us on Twitter @uxprobe Facebook Look for UXprobe Google+ Look for UXprobe

References

Documents

UX analytics to make a good app awesome€¦ · Today the only real solution available is to recruit UX experts to conduct usability tests or to create a home-made system that would