Pulse UX Testing: Bring the user into the user story. A.Witteman&R.vandenOever

Copyright © A. Witteman & R. van den Oever, All Rights Reserved.

Bringing the user into the user storyPulse Testing by Anna Witteman & Robin van den OeverMobile App Europe - Sep 2014


Today

What we will talk about


Bringing the user into the user story

helped us deliver better products

faster.


2002 2007 2014


Requirements

• Requirement 1

• Requirement 2

• Requirement 3

• Requirement N


‘As a user I want to

..................................

so that I can

................................’


• deliver a better product, faster

• strengthen your team

• with hardly extra costs

Giving the user a place in the process, will


Introduction

Who we are and where we are from?


Hello :)

Robin van den OeverProject manager/Agile coach

[email protected]

Anna WittemanHead of UXLab

[email protected]

mailto:[email protected]





• Amsterdam based company

• 100 employees

• over 25 nationalities

• Either a foodie, code king, design nerd, beer brewer or unicorn

IceMobile Amsterdam


the full mobile agency


App icons

the mobile agency for food retail


ProductsSpecialized in Food Retail Focus on (end) Users

One-offsAll round Focus on clients


Problem

Why don’t we test frequently with users?


“It doesn’tfit in our sprint.”

“The resultsare hard to measure.”

“It feels like extra

work.”

“We were just up

to speed.”

“It is too soon to testwith users.”

“We don’t have time to make a prototype.”

“Knowing the user is not

part of my job.”

“How do I knowwhat I will get

out of it?”

“Where do we get

users from?”

“How do we know which test results we should use?”

“We do not have resources to do

user tests.”


Interest:

Plan: How and when are we doing user testing?

Is it really a plan or more an improvisation?

Means:

Interest: What is the motivation for user testing?

Improve the product or prove you are right?

Plan:

Vision: Does decision-makers really have a good UX as a priority?

Or is more a buzz word?

Interest:

Vision

Interest

Plan

Means

Skills

What do we need to be able to start user testing?

Users? Prototype? Software? etc.

Skills: Can we conduct a user test?

Do we know how?


Transfer information effectiveThe team should learn most of the user test, not the UX researcher

Fit the user into the processDo testing so that it is not ‘extra work’

Make results visibleValue should be clear and measurable

We wanted to find a way to:


Our solution

How to create a place where great UX will happen.


Which practice proved its value?

Individuals and interactions

Working software

Customer collaboration

Responding to change

over

over

over

over

processes and tools

comprehensive documentation

contract negotiation

following a plan

Agile Manifesto


Setup

Process

Proof


Setup

Process

Proof


Applying the agile principles to user testing

• Give team members responsibility

• Reduce paperwork to the minimum

• Giving the team little overhead

• Reuse documentation written from an user perspective


• Write a scenario

• Create a prototype

• Recruit users

• Pilot test

• Conduct Tests

• Edit videos

• Analyze video

• Write report

• Present results


Product = Prototype

• Write a scenario

• Create a prototype

• Recruit users

• Pilot test

• Conduct Tests

• Edit videos

• Analyze video

• Write report

• Present results

User test = Report


Our setup



More evaluators = more issues identified

Source: Herttzum, M., Jacobsen, N. (2001). The evaluator’s effect: A chilling fact about usability evaluation methods. International Journal of Human-Computer Interaction, vol. 13, pp.421-443.


Ratio benefits to costs of # evaluators4 = ideal # evaluators

Source: Herttzum, M., Jacobsen, N. (2001). The evaluator’s effect: A chilling fact about usability evaluation methods. International Journal of Human-Computer Interaction, vol. 13, pp.421-443.


Conclusion

• Optimized the user test:

• Give the team responsibility

• No paperwork and no overhead

• Maintain a mutual goal

• Make the team smarter over time

• Team gets motivated


Setup

Process

Proof

Setup

Process

Proof


• Divided total work in small iterations

• Recurring rituals

• Easy to change course if required

• Give room to improve the process



Our agile way of working

Product Backlog

Sprint Backlog

PotentiallyShippableProduct

2-3 weeks

24 hours

Daily stand upDesign

Development

Testing


Source: Nielsen, Jakob, and Landauer, Thomas K.: "A mathematical model of the finding of usability problems," Proceedings of ACM INTERCHI'93 Conference (1993), pp. 206-213.

How many users should do a usability test?


Iterative Testing

•Prioritize: test important part(s) most extensive

•Frequent: reliable results, also test improvements

•Empathize: learn what matters to users

•Optimize: learn to test

Allow responding to change

'I have not failed. I've just found 10000

ways that won't work.’

Sir Thomas Edison


iteration is the heartbeatof agile & usertesting

use it!



Our renewed agile way of working

Product Backlog

Sprint Backlog

PotentiallyShippableProduct

2-3 weeks

24 hours

Daily Scrum Meeting

Design

Development

Testing

User Testing


• User test in every sprint

• User test with 4 users

• Scenario about sprint (& previous sprints)

• No need to react immediately to findings

Pulse Testing: bring the user into the Sprint


Setup

Process

Proof

Setup

Process

Proof


• Always show progress visually

• Describe requirements from a user perspective (as a user... so that..)

• Discuss all work created with each other being responsible as a group



• Issues

• Insights

• Wish list

• Fixed issues (adjusted issues)

Make all results tangible


How to keep track of the findings?

# user that did this task

# user that had this problem

time stamps of this issue

on video

description of issue


How to keep track of the findings?


Follow up and integrate in existing process:

1.The most important issues will become user stories itself

2.These user stories are put in the product backlog

3.Most important issues will surface over time


Start tracking your Sprint contents

• Because you want to understand the impact of user testing

• Identify types of user stories in sprint:

• New Features

• Bugs

• Rework resulting from user testing


Example sprint contents

0

15

30

45

60

1 2 3 4 C 5 6 7 8 C

New Features Bugs Rework resulting from UT


Back to you.

What it comes down to.


• Look at common practices within your company

• Eliminate waste

• Merge iterations

• Visualize progress

Learnings


Every project has its own challenges...

the most important thing is to start, you will make it work for you!


How are we and where are we from?Thank you!Questions? Feedback? Share experience with user testing?

Please contact us: [email protected] & [email protected]

We love to stay in contact!







http://instagram.com/icemobile

https://www.facebook.com/IceMobileAmsterdam

https://twitter.com/IceMobile

http://icemobile.com/blog

https://linkedin.com/company/icemobile

Follow us and like us on:











The mobile agency for food retail
