Embed Size (px)
Citation preview
Jan Moons Co-founder UXprobe Co-host UX Antwerp Certified Usability Designer [email protected] @moonsjan be.linkedin.com/janmoons +32 485 69 78 35
Agenda • What is usability? • What is usability testing? • In-person usability testing • Remote usability testing
• Moderated testing • Unmoderated testing
• Non-digital product testing
What is usability?
ISO 9241-11
The extent to which a product can be used by specified users to achieve specified goals with effectiveness,
efficiency and satisfaction in specified context of use
ISO 9241-11
The extent to which a product can be used by specified users to achieve specified goals with effectiveness,
efficiency and satisfaction in specified context of use
My humble definition
Usability is a quality aspect of a product that describes how usable the product is
Problem of unusable products
Product team focuses on machine and underlying system Not on the person who will actually use the system
Design vs. User Model User Model Representa*on a user has in his mind about the object he is interac*ng with Design Model Actual model that is given to the user through the design and interface of the product System Model How the system actually works
Good Design Model • Resembles User Model as
close as possible
• The more it resembles the more user friendly the product
• No correct model needed; too complex for all people to grasp
What is usability testing?
My humble definition • Observe real users
• Test participants perform real tasks while talking aloud
• Register what test participants do and say
• Analyze observed data and identify real problems
• Recommend changes to solve identified problems
Ex. Comparison airlines
Task & Questions Order a return trip plane ticket for 1 person from Brussels Airport to either Rome or Naples. On top of the standard hand luggage also order 1 extra piece of luggage with the lowest weight possible.
Task & Questions After the task, participants were asked to rate their experience on a 5 point scale, ranging from “Extremely easy” to “Extremely difficult” We also asked some follow-up questions based on their reactions during the test.
Airlines / Banks
What is the real task?
General Insights Terminology • Non-straightforward term confused participants (Loops) • Hold-luggage: extra luggage that isn’t hand luggage Loading splash screen • Showing between every step gave participants the impression the website
is very slow and found it distracting
Product options • 2 seats booked because of return flight confused participants Etc.
Full Report
What about mobile?
How many test participants do you need?
R.I.T.E. More rounds of small scale tests
1. 1st round: most significant problems 2. Fix these problems… 3. 2nd round: new problems uncovered 4. Fix these problems… 5. Etc.
How many times do you test during a project?
Many teams only conduct tests as an afterthought, the cost to change is enormous…
“When a test goes out we look at the data immediately and adapt the products quickly. We do this on a daily basis”
hAps://www.facebook.com/notes/facebook-‐engineering/building-‐and-‐tes*ng-‐at-‐facebook/10151004157328920/
Continuous testing & delivery
In-person usability testing
1. Recruitement is geographically depended 2. In clean and pristine lab 3. Time consuming – weeks
Remote usability testing
1. Recruit ‘anyone’ 2. User’s own environment, device and time 3. Results in hours or days
Moderated Remote Usability Testing
What is it? • Like an in-person test – over distance • Qualitative study with small group of participants
Remote moderated i.s.o. in-person? • Like an in-person test – over distance • Participants do not have to travel • Hard to recruit participants: doctors, dentists, etc. • Participants can do test when they have time • You do not have to deal with people that much • You can do a test much faster
Tools? There are no specialized tools (?!) UX professionals use screen sharing / conferencing software • Skype • WebEx • GotoMeeting • Join.me • Etc.
Problems • Not having the necessary rights to install software • Forgotten usernames and passwords • Do not want to install software just to do a test
• Annotations, metrics, etc. in other tools
Observer <-> Participant
Unmoderated Remote Usability Testing
What is remote unmoderated usability testing?
No moderator at the same time as when participant is
performing the test
Remote unmoderated = asynchronous
1. Moderator creates the script for the test • What tasks they want participants to do • What questions they want participants to answer
2. Moderator invites participants through email or social media
3. Participants perform the test • Completes tasks • Answer questions
4. Moderator receives the “results”
DEMO Project crea*on + invite par*cipants
When to use? • When there is very little time to get an answer
• Creation of the script is very quick • Results come in very quickly (1 h – 1 day) • “No” recruiting
• When you want to compare designs: • Previous version (New design) • Compare designs over time (Agile development) • Compare to the competition
How many participants? What is useful for you? • How many can you afford? • How much time do you have to watch the videos?
• Somebody has to watch the videos and annotate
Quantitative testing • Comparing designs & looking for numbers (success rates, duration, survey results) • 100’s or more
Qualitative testing • Mostly looking for insights • Small numbers of participants • Do several rounds of testing
Non-digital product usability testing?
Ex. Product Review
• Use UXprobe for “guided” video review of physical product
• Mobile app camera used to capture experience with physical product
• Recordings & survey data captured by UXprobe system
Ex. Product Review
Ex. Product Review
Ex. Brussels Airport
hAp://deredac*e.be/cm/vrtnieuws/videozone/programmas/journaal/2.46076?video=1.2737955
Daredevils 2016 Offering I invite you to try UXprobe out, leave your business card with me and I will tell you all about our Daredevils 2016 offering.
