Steps to Conduct a Usability TestDr. Jennifer L. Bowie

What is Usability?

“A function of particular users performing particular tasks in a particular environment” (Smith et al. 68)The “people who use the product can do so quickly and easily to accomplish their own tasks” (Dumas and Redish 4)User-centered design, not “user-friendly”

What is Usability Testing?

An empirical study of a product’s usability by observing actual users do real tasks with the productInvolves:Real usersReal tasksSpecific usability goals/concernsObserving and recording the testingData analysis

Step 1: User Analysis & Profiles

Who are your actual users? You may need to break your users into typical user categories. Consider: Demographics: age, sex, race, education level, cultural

background, socioeconomic status,… Experience level with the product, with products of the

same genre, with required technology,... Other things:

motivation learning style subject matter knowledge location of use physical characteristics people with disabilities or impairments (from color blindness

and learning disabilities to more severe disabilities)

Step 1: User Analysis & ProfilesCon.

Create user profiles:Break users into clear subgroupsProfile/Define the characteristics of each

subgroup

Choose user profiles to test: Ideally users from all major profiles will be

tested If limited testing: Choose profiles based on

highest number of users in that profile or profiles that you think may have the greatest usability issues

Step 2: Decide what to Test

1. Choose an overall purpose Example: How useable is our new website?

2. Determine objectives or what you are testing for. Examples:

Does our search engine provide usable results in the first 5 links returned?

Are search results clear to the users?

3. Choose type of test:1. Performance: Can they do it?2. Understandability: Can they understand it?3. Read-and-locate: Can they find it?

Step 2: Decide what to Test con.

4. Select tasks: Consider tasks with a high chance of user failure (complex tasks,

one-of-a-kind tasks, highly abstract or technical tasks) Consider tasks with a high cost of user failure (tasks that require

support, like help or support calls, to complete; tasks where data could be damaged or lost)

Consider: First impressions (look and feel) First tasks Tasks most performed Critical tasks Specific problem areas New task for the product

5. Select performance objectives (should be individualized for each task)

Time: How long to complete tasks, to find things, to performance procedures

Error/Success: user errors, attempts to do/find something, numbers of times section re-read, if the task was completed successfully

Step 3: Preparing for the Testing1. Choose order of tasks: start easy2. Create written test materials:

Task list for users Written welcome speech/ Intro to be read to user Consent forms Observation forms Pre-task and post task questionnaires & interview questions Other materials

3. Recruit participants & determine “payment”4. Define team member’s roles:

Facilitator/Briefer (necessary): Often only team member to interact with users

Observation recorder/note taker (necessary) Camera operator (optional) Help desk operator (optional) Test administer (optional)

5. Create written test plan6. Practice: conduct walkthroughs of the testing and if possible

pilot test (the pilot test users could even be a team member)7. Prepare test environment (day of test)

Step 4: Conducting the test

1. Greet & Brief participant:a. Read/say welcomeb. Emphasize that you are not testing them, but the product

and that they should act as natural as possiblec. Explain think-aloud protocol (if using)d. Emphasize how user tells you she has completed a taske. Stress that the testing is “anonymous”

2. Be unbiased (especially the Facilitator/Briefer)3. Intervene carefully (avoid as much as

possible)4. Observe and record data5. Debrief user

Step 5: Analyzing the Data

1. Collate data into findings:a. Choose an approach:

Top-down approach: predetermine categories of findings (like navigation, design, terminology) and go through data looking for “hits”

Bottom-up approach: put each observation on a sticky note/note card, sort into categories and label categories

b. Determine time and errors/success Examine findings for each user, user profile, and

task Use analysis techniques such as statistics (even

averages help)

Step 5: Analyzing the Datacon.

1. Analyze data: a. Determine cause of problemsb. Determine scope/severity of problemsc. Make recommendations/changes

2. Report Findings

Good Luck & Have Fun!

Where to find out more:Barnum Usability Testing and ResearchBarker Writing Software Documentation, Chapter 6 “Conducting Usability Tests”Hom “General Concepts of Usability Testing” http://jthom.best.vwh.net/usability/general.htm