Pulse UX Testing: Bring the user into the user story. A.Witteman&R.vandenOever

Bringing the user into the user storyPulse Testing by Anna Witteman & Robin van den OeverMobile App Europe - Sep 2014

What we will talk about

Bringing the user into the user story

helped us deliver better products

faster.

2002 2007 2014

Requirements

• Requirement 1

• Requirement 2

• Requirement 3

• Requirement N

‘As a user I want to

..................................

so that I can

................................’

• deliver a better product, faster

• strengthen your team

• with hardly extra costs

Giving the user a place in the process, will

Introduction

Who we are and where we are from?

Hello :)

Robin van den OeverProject manager/Agile coach

robin@icemobile.com

Anna WittemanHead of UXLab

anna@icemobile.com

• Amsterdam based company

• 100 employees

• over 25 nationalities

• Either a foodie, code king, design nerd, beer brewer or unicorn

IceMobile Amsterdam

the full mobile agency

App icons

the mobile agency for food retail

ProductsSpecialized in Food Retail Focus on (end) Users

One-offsAll round Focus on clients

Problem

Why don’t we test frequently with users?

“It doesn’tfit in our sprint.”

“The resultsare hard to measure.”

“It feels like extra

work.”

“We were just up

to speed.”

“It is too soon to testwith users.”

“We don’t have time to make a prototype.”

“Knowing the user is not

part of my job.”

“How do I knowwhat I will get

out of it?”

“Where do we get

users from?”

“How do we know which test results we should use?”

“We do not have resources to do

user tests.”

Interest:

Plan: How and when are we doing user testing?

Is it really a plan or more an improvisation?

Means:

Interest: What is the motivation for user testing?

Improve the product or prove you are right?

Vision: Does decision-makers really have a good UX as a priority?

Or is more a buzz word?

Interest:

Vision

Interest

Skills

What do we need to be able to start user testing?

Users? Prototype? Software? etc.

Skills: Can we conduct a user test?

Do we know how?

Transfer information effectiveThe team should learn most of the user test, not the UX researcher

Fit the user into the processDo testing so that it is not ‘extra work’

Make results visibleValue should be clear and measurable

We wanted to find a way to:

Our solution

How to create a place where great UX will happen.

Which practice proved its value?

Individuals and interactions

Working software

Customer collaboration

Responding to change

processes and tools

comprehensive documentation

contract negotiation

following a plan

Agile Manifesto

Process

Applying the agile principles to user testing

• Give team members responsibility

• Reduce paperwork to the minimum

• Giving the team little overhead

• Reuse documentation written from an user perspective

• Write a scenario

• Create a prototype

• Recruit users

• Pilot test

• Conduct Tests

• Edit videos

• Analyze video

• Write report

• Present results

Product = Prototype

• Write a scenario

• Create a prototype

• Recruit users

• Pilot test

• Conduct Tests

• Edit videos

• Analyze video

• Write report

• Present results

User test = Report

Our setup

More evaluators = more issues identified

Source: Herttzum, M., Jacobsen, N. (2001). The evaluator’s effect: A chilling fact about usability evaluation methods. International Journal of Human-Computer Interaction, vol. 13, pp.421-443.

Ratio benefits to costs of # evaluators4 = ideal # evaluators

Source: Herttzum, M., Jacobsen, N. (2001). The evaluator’s effect: A chilling fact about usability evaluation methods. International Journal of Human-Computer Interaction, vol. 13, pp.421-443.

Conclusion

• Optimized the user test:

• Give the team responsibility

• No paperwork and no overhead

• Maintain a mutual goal

• Make the team smarter over time

• Team gets motivated

Process

• Divided total work in small iterations

• Recurring rituals

• Easy to change course if required

• Give room to improve the process

Our agile way of working

Product Backlog

Sprint Backlog

PotentiallyShippableProduct

2-3 weeks

24 hours

Daily stand upDesign

Development

Testing

Source: Nielsen, Jakob, and Landauer, Thomas K.: "A mathematical model of the finding of usability problems," Proceedings of ACM INTERCHI'93 Conference (1993), pp. 206-213.

How many users should do a usability test?

Iterative Testing

•Prioritize: test important part(s) most extensive

•Frequent: reliable results, also test improvements

•Empathize: learn what matters to users

•Optimize: learn to test

Allow responding to change

'I have not failed. I've just found 10000

ways that won't work.’

Sir Thomas Edison

iteration is the heartbeatof agile & usertesting

use it!

Our renewed agile way of working

Product Backlog

Sprint Backlog

PotentiallyShippableProduct

2-3 weeks

24 hours

Daily Scrum Meeting

Design

Development

Testing

User Testing

• User test in every sprint

• User test with 4 users

• Scenario about sprint (& previous sprints)

• No need to react immediately to findings

Pulse Testing: bring the user into the Sprint

Process

• Always show progress visually

• Describe requirements from a user perspective (as a user... so that..)

• Discuss all work created with each other being responsible as a group

• Issues

• Insights

• Wish list

• Fixed issues (adjusted issues)

Make all results tangible

How to keep track of the findings?

# user that did this task

# user that had this problem

time stamps of this issue

on video

description of issue

How to keep track of the findings?

Follow up and integrate in existing process:

1.The most important issues will become user stories itself

2.These user stories are put in the product backlog

3.Most important issues will surface over time

Start tracking your Sprint contents

• Because you want to understand the impact of user testing

• Identify types of user stories in sprint:

• New Features

• Bugs

• Rework resulting from user testing

Example sprint contents

1 2 3 4 C 5 6 7 8 C

New Features Bugs Rework resulting from UT

Back to you.

What it comes down to.

• Look at common practices within your company

• Eliminate waste

• Merge iterations

• Visualize progress

Learnings

Every project has its own challenges...

the most important thing is to start, you will make it work for you!

How are we and where are we from?Thank you!Questions? Feedback? Share experience with user testing?

Please contact us: anna@icemobile.com & robin@icemobile.com

We love to stay in contact!

http://instagram.com/icemobile

https://www.facebook.com/IceMobileAmsterdam

https://twitter.com/IceMobile

http://icemobile.com/blog

https://linkedin.com/company/icemobile

Follow us and like us on:

The mobile agency for food retail

Pulse UX Testing: Bring the user into the user story. A.Witteman&R.vandenOever

Design

TruePort HP-UX User Guidedownload.perle.com/IOLAN/V3.3/trueport/hpux/hp_ug-6.0.pdf · TruePort HP-UX User Guide, Version 6.0, Part #5500170-10 1 TruePort HP-UX User Guide Chapter

Starting up with UX: User centered Design Process, Usability & UX

Mobile App Europe 2015 Pulse UX Workshop Presentation

UI/UX Workshop - gdiindy.github.io · UI/UX Workshop “User Experience (UX) and User Interface (UI) are some of the most confused and misused terms in our field. A UI without UX

User Experience (UX) Overview

UX Lesson 2: User Research

New User Experience (UX)

User-centred Design and UX

User Experience UX UI Design Agency India | … in User...(UX) and User Interface (UI) design- ing by Techved Consulting — a Ux design company. The three-month online weekend course

User Story as UX Method

UX: User Experience

User Experience (UX) View Tips

GET A JOB WITHIN JUST 90 DAYS · — UX vs. UI Design — UX Case Study — UX Principles — UX Design Process — Design Thinking — User Research — How to Conduct User Interview

Designing the User Experience (UX)

Workshop : User Experience (UX) and User Interface(UI) · Workshop : User Experience (UX) and User Interface(UI) วันจันทร์ที5 พฤษภาคม 2557 เวลา

User Experience (UX) Lessons & Myths

USER EXPERIENCE DESIGNER (UX/UI) UX DESIGN EXPERIENCE

Innovatón: User experience UX - Storytelling

UX – User experience

User experience and Security in the Cloudsalsahpc.indiana.edu/CloudCom2010/slides/PDF/User...User Experience (UX)User Experience (UX) Persons’ perceptions and responses that result