APPROACHING USABILITY TESTING AT YOUR LEVEL

SIT ANYWHERE

RICE MAJORSUNIVERSITY OF COLORADO BOULDER

Rice MajorsFaculty Director of Libraries Information TechnologyUniversity of Colorado Boulder

AMICAL Conference – April 2012 – American University of Sharjah

Today’s agenda Why do usability testing? Looking at some existing models &

methodologies How to get started and scale upward Preparing for results

Talk in small groups & we will share

Why would a library do usability testing?

Your website For your website, usability is a necessary

condition for survival If a website is difficult to use, people leave There's no such thing as a user reading a

website manual Paradigmatic shift from publisher-focus

to user-focus Why wouldn’t you want your web

presence to be easy to use?

Your intranet For intranets, usability = employee

productivity Spending 10% of your time on usability

will more than double your (Web) quality metrics Somewhat lower return for software &

physical items

Purchased / subscription products Use the data to validate purchase &

subscription decisions Use the data to help the vendor(s) make

data-driven decisions about interface and experience enhancements

Why I am doing user testing Research goals – generate data to compare

the user experiences of discovery tools (to help libraries and vendors)

Business goals – untangle some known challenges with our web “experience” How do we pass a user back and forth between

our website and our web-based platforms (Encore, Research Pro, various databases)?

How do we guide a user toward different paths of article discovery based on their needs (“I have a citation” vs. “I have a topic”)?

Models & methodologies

Some existing models Lots of studies, more and less formal,

published on the web and in articles University of Minnesota and the usability

lab Explore issues of discovery Assess the effectiveness of Primo (across

many metrics) University of Michigan and the guerilla

testing Compare a single set of design options Validate a single design decision

Common Methodologies Card sorts

Open sort – design (“Sort using your own categories”)

Closed sort – validate (“Sort using the provided categories”)

Task completion – assess or validate Find representative users Ask the users to perform representative tasks Observe what the users do, where they succeed, and

where they have difficulties with the user interface Shut up and let the users do the talking

A “real” task I used encore.colorado.edu Find all recordings the library owns by The

Beatles. Somehow remind yourself to look at these again later. (10 minutes)

If you need to log in, use these provided login credentials, using the Public Patron login prompt: Last Name Testpatron ID Number1234567890 PIN 0wh1ne (first digit is a zero)

Talk in small groups & we will share

What was good/bad about the design of that task?

The good… … the bad Beatles is a non-unique

term in bibliographic data

Task probably involved using facets / refinement features

Involved understanding how formats are represented in results

(User would realize the library has recordings)

“Recordings” FRBR Library may have

licensed content as well

More than one way to complete the task

Hard to assess whether the user would actually be reminded

Getting started…

Starting can be the hardest part Identifying goals can make this seem

daunting You have a robust web presence You have a sense that there are many areas

for improvement

Anything you do is better than not doing anything C.f. shelf-reading

Framing a single question Possible starting point: Ask users to

validate a single decision What are the actual questions you get? Printout(s) of possible webpage(s) –

possibly just mocked up “If you wanted to find an article, where

would you start?” “When does Norlin library close today?”

Talk in small groups & we will share

What question would be your first question?

Starting simple Administrative support for this (more on this

later!) Quiet place for doing the test Mostly time and interest

Designing a test Recruiting participants Conducting the tests Reviewing your data

Laptops have built-in cameras/mics One license of Morae (or equivalent)

Scaling upward Work toward making testing normal Dedicated space?

Ours is shared with other functions (the IT dept. testing new OS bundles; troubleshooting off-campus access)

Proper usability lab is probably out of reach, and that’s ok Does a library need to do eye-tracking? Does a library need to have a note-taker

AND a video recording?

Finding participants

Finding participants I started with posters, Twitter/Facebook

But my real success came when a student forwarded the opportunity to a listserv

There is no substitute for real users Use library employees only for intranet

testing

Finding participants Do you have time for one question to

help us improve the library website? Incentives for longer participation – food

for a little time, gift cards for longer amounts of time? If incentives are not possible, then frame

things as single decisions as much as you can

It doesn’t take many participants Reach consensus (or not) in 6-10

participants An absence of consensus is informative

(A few thoughts of preparation)

What happens once we have results from our testing?

Those darn results Be prepared for results that may surprise

you The problems may not be where you

thought they were Be prepared for inconclusive results

Accepting the results Be prepared for a new power structure

What is your existing power structure? (Design by Committee?)

Emotional decisions and data-driven decisions

“I spent a lot of time on this” and/or “those users are dumb” vs. “the design needs improving”

Craigslist is very easy to use

Talk in small groups & we will share

What are the first three things you will do when you return to your library?

Further reading Dumas, J.S., & Redish, J.C. (1993). A practical

guide to usability testing. Norwood, New Jersey: Ablex Publishing, 1993.

Dumas, J.S., & Loring, B.A. (2008). Moderating usability tests: Principles and practices for interacting. Burlington, Mass.: Morgan Kaufmann Publishers, 2008.

Rubin, J. (1994). Handbook of usability testing: How to plan, design, and conduct effective tests. New York: Wiley, 1994.

THANK YOU!RICE.MAJORS@COLORADO.EDU

Rice MajorsFaculty Director of Libraries Information TechnologyUniversity of Colorado Boulder

AMICAL Conference – April 2012 – American University of Sharjah