YOU ONLY THINK YOU’RE LIKE GOOGLE : COMPARATIVE USER EXPERIENCE OF DISCOVERY PLATFORMS

Rice MajorsFaculty Director of Libraries Information TechnologyUniversity of Colorado, Boulder

AMICAL Conference – April 2012 – American University of Sharjah

Purpose(s) of study Create a “common good” set of data

Vendors lack access/resources for consistent & comparative studies

Many libraries lack access/resources to carry out studies (especially comparative)

Working on a campus, I have relatively easy access to study participants

Pursuing this research justified acquiring usability resources we can use for other purposes

Study design & methodology Task based testing with undergraduate

students Usability software (Morae)

Video capture of actions taken Video/audio capture of “thinking aloud”

Survey instrument Tested against five discovery tools /

next-gens

Partner libraries James Madison University (EBSCO

Discovery Service) University of Colorado Boulder (Encore

Synergy) Vanderbilt University (Primo Central) Arizona State University (Summon) Auraria Library, Denver (WorldCat Local)

Participants Undergraduate students only

Library student employees were excluded

28 participants in total January 26-March 17, 2011 Each participant tested one interface only

(to avoid “learning”)

Participant tasks1. Find three books on a particular topic

toward writing a paper. 2. Find three articles on a particular topic

toward writing a paper – two articles must be peer-reviewed.

3. Find recordings by a particular artist. Remind yourself to look at these again later.

4. The library doesn’t own the book “[title].” Have the library get this book for you.

Survey Instrument 7 Likert items assessing perceptions of ease of

use: I was able to find what I need for these tasks using

this discovery platform If I were doing my own research, I would be able to

find what I needed using this discovery platform [etc] Short answer questions:

What is easy to use about this platform? What is hard to use about this platform? What one change would make the biggest

improvement to this discovery platform?

Perceptions of ease of use

Highly Positive

Positive Neutral Negative Highly Negative

0%10%20%30%40%50%60%70%80%

EDSEncorePrimoSummonWCL

Features appreciated – all Finding different

types of resources & narrowing search (12)

Ability to email books & articles (7)

Found relevant articles/results (2)

Ability to save records for later

Smooth interface to library website

Simple interface Integration with

consortial borrowing Clearly labeling

peer-reviewed articles

Comments specific to each product Summary recommendations for each

product are based on consensus of study participants only

Where there are fewer recommendations, there was simply less consensus on what issues to address

Note that some vendors have meanwhile addressed some of the issues documented here Testing was done January 26-March 17,

2011 Plausible that vendors heard similar

feedback from existing and potential customers

Product-specific feedback EBSCO Discovery

Call it something else

Connect users to ILL / resource-sharing more easily

Make it clear[er] how to use the structured search boxes

Address the need to login just to perform searches

Encore Synergy Simplify / streamline

the interface Add an advanced

search option Primo Central

Make it easier to find options in “additional services”

Connect users to ILL / resource-sharing more easily

Product-specific feedback Summon

Make it easier to find & use email option

Connect users to ILL / resource-sharing more easily

Address misconception of “add results beyond your library collection”

Rely less on link resolver technology

Make it clear[er] when/why options reset

WorldCat Local Make it easier (in

several ways) to use email option

Connect users to ILL / resource-sharing more easily

Make it clear[er] when/why options reset

Improve layout (e.g. so it’s clear[er] when an abstract isn’t available)

Feedback for all products1. Introduce further enrichment options to

allow patrons to evaluate resources (c.f. Amazon)

2. Make it possible/intuitive to add all results to a list & export all results

3. Provide context so that it’s clear what has been searched (& what is not included)

4. Partner (better) with libraries on who does what

5. Make it easy to get help (“ask a librarian” etc.)

Inadvertent findings Jargon & library practices

Do know what “interlibrary loan” is Do not know what “government publications”

or “electronic resources” are, nor the catalog nickname

Musings about “book” qua “book” Want FRBR solutions (maybe)

Participant behavior Will type anything into the search box Will look for expected features

Issues with study design User defined success Inescapable testing of information

literacy Students already have multiple

frameworks for using library catalog software

Library implementation choices Library website design choices

More information & intentions

“Highlights” video footage to demonstrate findings (available on request)

Article forthcoming in Library Trends Follow up studies

Involve students from several campuses, with different article database providers and different discovery tools

Use multiple implementations of each interface

More tasks (with randomization of order)

THANK YOU!RICE.MAJORS@COLORADO.EDU

Rice MajorsFaculty Director of Libraries Information TechnologyUniversity of Colorado, Boulder

AMICAL Conference – April 2012 – American University of Sharjah