An Introduction to Usability Testing.

Lennart Overkamp April 5th, 2017

Fontys Eindhoven.

2010-2013: Bachelor Psychology @UT 2013-2014: Master Human Factors & Engineering Psychology @UT 2014-2016: Post-Master User System Interaction @TU/e 2016-now: Interaction Designer @Mirabeau

Building a platform from the ground up

Fully responsive platform Mobile, iPad & desktop

Booking flow redesign Overview and options for customers

Fully responsive platform Mobile, iPad & desktop

Booking flow redesign Overview and options for customers

Redesign Nuon.nl

Relevance & customisation Personal product selector

High-quality service Quick, accessible & contextual

Inspirational Telling a story through design

Relevance & customisation Personal product selector

High-quality service Quick, accessible & contextual

Inspirational Telling a story through design

Redesign Vrouw.nl

Human2Human strategy Content adapts to customer behaviour

Omni-channel approach Seamlessly from magazine to online

Online look&feel Connection to product portfolio

Human2Human strategy Content adapts to customer behaviour

Omni-channel approach Seamlessly from magazine to online

Online look&feel Connection to product portfolio

Creative Consultants

Interaction Designers

Visual Designers

Front-end Developers

UX

The Red Thread?

An Introduction to Usability Testing.

Lennart Overkamp April 5th, 2017

Fontys Eindhoven.

Three Phases: 1. Preparation 2. Moderation 3. Analysis

Phase #1: Preparation.

“ ”

The primary goal of the usability test is to improve the usability

of the product that is being used.

Source: A Practical Guide to Usability Testing (Dumash & Redish, 1999)

Step #1.1: Define the goal.

Step #1.2: Define the users.

Users = ?

All (prospective) customers

Step #1.2: Define the users.All (prospective) customers

Users = short-stay international students

Step #1.2: Define the users.

Users = short-stay international students (the population)

All (prospective) customers

Step #1.3: Recruit users.

Users = short-stay international students (the population)

Participants (the sample)

All (prospective) customers

Step #1.4: Define the measures.

Performance measures (e.g. time-on-task,

errors)

Preference measures (e.g. desirability,

perceived usability)

Step #1.5: Prepare the procedure.

Which materials?

How?

When?

Where?

How long?

With whom?

Step #1.6: Prepare the script.Scenario (example): For a three-months exchange program in the Netherlands, you are looking for a place to live from February 2017 until the end of April 2017. You have a maximum budget of 500 euros, and would like to have a space of at least 30 square meters.

Tasks (examples):

1. On the website, find a place that fits your needs. 2. Calculate the time it takes from Central Station to this place by public transport.

Step #1.7: Prepare the prototype.(If you have one)

Step #1.8: Prepare the consent form.

Step #1.8: Prepare the consent form.Contents: • Goal • Procedure • Duration • Participant’s rights

EVALUATION VESTIDE.NL Goal The goal of this evaluation is to test the usability of vestide.nl, to improve this website in terms of user-friendliness. Procedure You will carry out tasks with the website, which will be provided by the facilitator. While carrying out the tasks, you are asked to think aloud. After completing the tasks, you will fill in a short questionnaire and the facilitator will ask you some questions. The evaluation will take around 45 minutes. You may stop, take a break, or ask questions at any moment. Recording To facilitate the analysis of the test results, we ask for your permission to record the evaluation. The recording will only be used by the design team to improve the prototype, and will not be shared with anyone outside the team. Participation does not have any risks or consequences for you. • Your answers and details will be processed anonymously. • We judge the quality of the prototype, not your performances. Informed consent By signing this form, you acknowledge to have read and understood the information in this document, and to agree with this information. Name: Date: Signature:

Do you agree with recording? ☐ Yes, video- and sound recording. ☐ Yes, only sound recording. ☐ No, rather not.

EVALUATION VESTIDE.NL Goal The goal of this evaluation is to test the usability of vestide.nl, to improve this website in terms of user-friendliness. Procedure You will carry out tasks with the website, which will be provided by the facilitator. While carrying out the tasks, you are asked to think aloud. After completing the tasks, you will fill in a short questionnaire and the facilitator will ask you some questions. The evaluation will take around 45 minutes. You may stop, take a break, or ask questions at any moment. Recording To facilitate the analysis of the test results, we ask for your permission to record the evaluation. The recording will only be used by the design team to improve the prototype, and will not be shared with anyone outside the team. Participation does not have any risks or consequences for you. • Your answers and details will be processed anonymously. • We judge the quality of the prototype, not your performances. Informed consent By signing this form, you acknowledge to have read and understood the information in this document, and to agree with this information. Name: Date: Signature:

Do you agree with recording? ☐ Yes, video- and sound recording. ☐ Yes, only sound recording. ☐ No, rather not.

Step #1.8: Prepare the consent form.Contents: • Recording explanation • Consequences • Signature • Recording consent

Step #1.9: Test your test.

Phase #2: Moderation.

Step #2.1: Welcome your participant.Coffee?

Step #2.2: Introduce the test.Introduce yourself.

Explain your role.

Explain what is expected of the participant.

Mention the duration of the test.

Provide reassurance: you are not testing the participant.

Explain the purpose of the test.

Step #2.3: Obtain informed consent.Sign two copies!

Step #2.4: Explain the think-aloud protocol.

“ ”

In a thinking aloud test, you ask test participants to use the system while continuously thinking out loud — that is, simply verbalizing their thoughts

as they move through the user interface.

Source: Jakob Nielsen (www.nngroup.com)

Step #2.4: Explain the think-aloud protocol.

Example: https://www.youtube.com/watch?v=g34tOmyKaMM

Step #2.5: Start recording.

Step #2.6: Participant performs tasks.Explain the scenarios & tasks.

Let the participant do the talking!

Remind participant to think aloud, when necessary.

Keep questions and distractions to a minimum.

When a scenario/task is finished, explain the next one.

Step #2.7: Post-questionnaire.(Optional, depending on goal/measures)

Example #2: Microsoft Reaction

Card Method

Measures product desirability.

Example #1: System Usability

Scale (SUS)

Measures perceived usability.

Step #2.8: Post-interview.Ask for a general summary.

Elaborate on interesting observations/comments.

Ask any remaining questions you prepared beforehand.

Step #2.9: Debriefing.Explain (again) the purpose of the test.

Answer any remaining questions from the participant.

Explain (again) what you will do with the results.

Give the participant a reward!

Phase #3: Analysis.

Quantitative data (SUS scores, errors, Reaction Cards, time-on-task)

Qualitative data (comments,

explanations, responses)

Step #3.1: Process your data.

Step #3.2: Prioritise the usability problems.Problem #1

Problem #2

Problem #3

Problem #4

Problem #5

Step #3.3: Communicate your results.

Type #2: Presentation slides

Summarised, easily shared or presented

Type #1: Full report

Detailed, useful as a reference.

Type #3: Infographic

Highly visual, useful as a persuasion tool

#Conclusion.Advantages:

Solid, well-researched Tests many aspects of product

Both qualitative & quantitative

Disadvantages: Costly in terms of time, money

and (human) resources

Recommended resources - Books

Recommended resources - Online

smashingmagazine.com

nngroup.com

Recommended resources - Prototyping & Testing

InVisionSketch

Recommended resources - Analysing

ExcelRStudio

Recommended resources - Communicating results

SketchKeynoteWord

Neuro Usability Test.Bonus:

Aspect #1: Eye Tracking

Reference of users’ focus. Input for heat maps.

Aspect #2: EEG

Captures brain activity to pinpoint affects.

Aspect #3: Neuro Feedback

Users comment on significant EEG findings.

Aspect #4: Questionnaire

Users are interviewed and fill in a questionnaire.

Thank you.