User-Generated Content and Social Discovery in the Academic Library Catalogue: Findings From User Research

1

User-Generated Content and Social Discovery in the Academic Library Catalogue:

Findings From User ResearchMartha Whitehead, Queen’s University Library

Steve Toub, BiblioCommons

Access 2008, Hamilton, October 2, 2008

2

Agenda

1 Overview of the problems we want to address2 Observations about research and intriguing ideas

to explore3 User research on user-generated content and

social discovery at an academic library4 Preliminary thoughts on how to ensure quality5 Vision and status of BiblioCommons academic

product at Queens6 Questions

3

The Problem

Discovery

getting answers to questions you don’t know how to ask

finding gems you don’t know exist

4

5

Observation: Learning and Research are Social

Social

considering the judgements

and insights of others

6

User-Generated Content

Narrow sensetags, ratings,

reviews

Broader sensecurated content

7

Existing Implementations: UGC in the narrow sense

8

Existing Implementations: UGC in the broader sense

9


10


11

Existing Implementations: UGC in broader sense

12

What have scholars told us about research?

Ontario Council of University Libraries (OCUL) Scholars Portal User Study, May 2008, with Usability Matters

Primary Objective:Understand the information research processes of experienced researchers in a variety of disciplines

Methodology6 collaborative design sessions with 8-10 participants each3 discipline areas

Arts & HumanitiesSocial SciencesSciences (Natural, Applied, Health, etc.)

13

Discover, gather, create, share

A Multi-Dimensional Framework for Academic Support, June 2006, University of Minnesota Libraries funded by Mellon Foundation*, building upon John Unsworth’s concept of scholarly primitives: “basic functions common to scholarly activity across disciplines, over time, and independent of theoretical orientation.”**

Validated in OCUL study: A number of groups introduced the terms “interaction”, “collaboration” or “conversation” and ultimately decided that these are not discrete steps in the process but, rather, are overarching throughout all “phases”. They were adamant about the importance of this aspect of their information research process.

* http://www.lib.umn.edu/about/mellon/UMN_Multi-dimensional_Framework_Final_Report.pdf**John Unsworth. “Scholarly Primitives: What Methods do Humanities Researchers Have in Common and How Might Our Tools Reflect This?”

Humanities Computing, Formal Methods, Experimental Practice Symposium, Kings College, London, May 13, 2000. http://jefferson.village.virginia.edu/~jmu2m/Kings.5-00/primitives.html

14

Aspects of “typical” research process

Thinking of keyword search termsBrainstorm with colleaguesUse a familiar or seminal article or journalUse books and bibliographies/references

Participants appreciate being able to find sources based on:What others searched forMost accessed resourcesMost cited resourcesMost credible or prestigious sources

In storyboarding the ideal discovery process, they included: Seeing recommendations from “authorities”Classics in the fieldWays to find “surprises”, unanticipated sources

15

Social?

The research process is social, but that doesn’t refer to social tools

Few had heard of Del.icio.usEveryone had heard of Facebook but only one or

two were using it for academic purposes

16

“Social discovery,” not “social networking”

User consultation at Queen’s University, February 2007, with Usability Matters

4 focus groups, 6-8 participants each, undergrads, grads, facultyObjectives

Elicit opinions regarding current state of the library website(s)Determine how the library website can support user needsExplore user expectations around information searching

and browsingDetermine user interest and expectations around library

instruction and guidanceElicit ideas for new website features and services

Personalized website featuresSocial networking features

17

Research on UGC: Attitudes

18

19

20

The main point

Participants expressed their desireto know what trusted colleagues (professors, fellow researchers) think

21

March 2008: Research on the research process

At Queen’s, held in a library conference roomFaculty, graduate students and undergraduates

Recruitment of faculty through emailRecruitment of others through link/survey on website, catalogueHour-long one-on-onesStarting with library website, existing catalogue

Also got reaction to several NGCs

4 faculty; 2 staff researchers (both taught occasionally); 2 graduate students; 5 undergraduates

Librarians2 focus groups of 8-10 librarians each

22

March 2008: Key findings on the research process (1)

Faculty, graduate students, and undergraduatesEmploy post-search limiting very infrequently

Only people who’ve seen facets before noticed facets in NGCsReformulate queries via back button and typing variant terms

insteadDo not reformulate queries by ORing all terms or using

truncation, wildcards

LibrariansWant the power of a comprehensive command-line query syntax

(e.g., fielded searches, Boolean operators, truncation)

23

March 2008: key findings on the research process (2)

Faculty, graduate students, and undergraduatesAvoid hyperlinked subject headings

Result screens that list headings are perceived as confusing and unhelpful

Virtual shelf-browsing tested well

LibrariansValue the bibliographic control provided by subject headings

24


Faculty, graduate students, and undergraduatesDon’t experience pain when manually managing and formatting

citations

LibrariansThink DirectExport to RefWorks is high priority

25


Faculty, graduate students, and undergraduatesWhen frustrated, students tend not to as for help, especially

from librariansEven in the face of atrocious information literacy skillsInformation literacy skills not dependent on age/role

LibrariansWant to help

26

June 2008 one-on-ones devoted to UGC

Recruited May 28-30 via Facebook ad that linked to SurveyMonkey survey

175,662 impressions. 111 clicks = 0.06% click-through rate

36 completed surveys = 32% who viewed survey completed it

Nine one-hour sessions June 4, 5 at BiblioCommonsEight undergraduates (U of T, Queens, Western)

3 completed fourth year2 completed second year1 completed first year

Two graduate students: brother-sister pair who attended the same hour session

27

Solicited feedback using a variety of different media

• Web sites they’ve used to make product choices (e.g., Restaurantica.com, Rotten Tomatoes)

• Their university library web page and OPAC• And a page in University of Michigan’s OPAC directing

their attention to MTagger• On-screen mockups of three different ways

in which they could contribute UGC in the library context

• Paper mockup of a course-related hub page that provided links to various items associated with a course

• List of possible motivations for contributing UGC

28

This round focused more on narrow UGC than broad

Existing application already has strong support for curated listsCompleted initial exploratory R&D on broad UGC in public library productWill need more R&D to tweak list creation flows

Faculty syllabus tool that outputs to course reserves, bookstore, courseware

Convert ad hoc reference interview into a BiblioCommons list

Still early days for narrow UGC in academic library catalogueNot sure why data elements and motivations apply in academic contextNot aware of any published research on this

Concepts and motivations apply equally well

29

Attitudes about UGC

30

Attitudes about UGC

31

Tagging?

What is this?“I’m not entirely sure… I would like to assume… some sort of user feedback…but I don’t why they’d say “tag” … but if you were a student there you’d probably know what it was…”

D_____, graduated 4th year, entering FIS

32

Tagging!

Have you ever heard of tagging?“No.”Have you ever tagged on Facebook?

“Of course. That’s with photos. They have it for text – but no one uses it.”D____, Graduated 4th year, entering FIS

33

34

35

The most important data elements (course context)

Relevance to courseA lot of “Is it going to be on the test?” But some nuances we could explore further:

“How related is this reading is to other readings?” or “How related this reading is to the lecture?”

Clarity [a.k.a. level of difficulty] was second most popular data element

36

Mechanics of data entry process for UGC

Most said they might not fill out more than 1-2 data elements.The “sliders” represented in the mockups tested well.

Moving a “slider” is perceived to be easier than typing. If there are no open-ended comments that involve typing, it may be

possible to ask for more than two data elements using “sliders”. Segments on the “sliders” should be visually apparent; could be 5-or

10-point scale.

Anonymity, i.e., having the ability to choose a username that isn’t personally identifiable, will make contributions more likely.

Most wanted other students to view their comments.Even the person least likely to contribute (when we first saw the concept,

her reaction was, “Why would I do that?”) in the end said she would be willing to share comments with others if she only had to fill in 1-2 things for each item and if her comments were anonymous.

37

38

Course hub mockup

39

Reaction to the course hub mockup

“That would be amazing!”

“That would become my new first place to go to start my searches.”

--L_____, completed 2nd year at Western

40

Likeliest opportunities to contribute

The course reading context seems the likeliest opportunity to contribute.Solicitation in this context only works if the syllabus were online and the student is

looking at the syllabus online rather than a paper copy.When using the syllabus online, if they were looking at what to read for Week 2,

they wouldn’t mind an invitation to comment on the readings for Week 1.They would like the ability to edit their comments later on.

Soliciting contributions from a “recently returned” also well received.Mixed reaction on email solicitations on “recently returned”.

Need to probe further on how to make emails palatable. Several said they didn’t want email at all. One person said that if she’d much prefer email but not on each recently returned item but only if she got a single email once a month.

The “answers” mockup may only be utilized by who would ask questions, a definite minority.

Many fiercely resist the idea of asking others for help, even a TA or professor.

41

Possible motivations to contributeEarn Campus Credits

Chances to win prizes$ off fines, bookstore, foodservices “Printer Points”Charitable contributions

Opportunity to give feedback / Have my say.which online articles, library materials or course readings are useful; which are not

To the libraryTo my professors

Contribute / Give back to my university…the library. Help build a richer, more useful catalogue / database.

Get recommendations, suggestions – for materials I might not have otherwise found

Help others/everyone get to useful resources faster. More time thinking – less time finding

Quid pro Quo: I earn rights to ask others questions, when I answer some myself.

42

Possible motivations to contributeEarn Campus Credits

Chances to win prizes$ off fines, bookstore, foodservices “Printer Points”Charitable contributions

Opportunity to give feedback / Have my say.which online articles, library materials or course readings are useful; which are not

To the libraryTo my professors

Contribute / Give back to my university…the library. Help build a richer, more useful catalogue / database.

Get recommendations, suggestions – for materials I might not have otherwise found

Help others/everyone get to useful resources faster. More time thinking – less time finding

Quid pro Quo: I earn rights to ask others questions, when I answer some myself.

43

The #1 motivation

Helps [others] get to useful resources fasterHelp us be more helpful to youCould do more probing on languageSeveral commented that they liked the idea of spending

“less time finding, more time writing”

Strong sense ofPay it forward

“If I do it now, it will help others later” “If others do it, it will help me when I need it”

Empty restaurant syndromeSome fears of being the first to contribute

if they did not see evidence that others were doing the sameStronger indication they’d contribute

if they saw that everyone else was doing it

44

It seems pretty easy to “buy” student participation

Even the one student who had consistently said she wouldn’t be likely to contribute quickly checked off all 4 “Campus Credit” concepts as motivating.

45

Primary barriers to contributing

Many (but not all) are unwilling to support freeloaders.

However, they do like being able to freeload themselves and do see the connection that someone must contribute for others to freeload.

Worried about being accused of plagiarism makes students reluctant to share with peers.

Course-related sharing may need to be sanctioned by the professor of that course to allay these fears.

46

Strategies for ensuring quality

Patron authentication: fosters more measured and insightful commentsIt is not possible to leave anonymous comments in our systemEvery username is associated back to a real user authenticated against the library

database

Aggregation: helps you not draw conclusions from a single sourceAbility to see all reviews by particular reviewerAbility to view reviews on an item by many different customers.

Design a marketplace of ideas: self-managed system, not editorial reviewCapture positive/negative sentiment in a structured wayReview the reviewsExpose more attributes than binary hasReview vs. lacksReview

“3 students in your class found this reading helpful to prepare for the midterm exam”

“4 faculty included this item in course reading lists”

47

Vision for BiblioCommons’ academic product

Provide a smart catalogue that offers an outstanding user experienceWhen searching, for most of the people, most of the time, less is moreConditional display of request actions, no buttons leading to a dead endSurface curated content: lists, course reading lists, student bibliographies, etc.Provide ways to get answers to questions that users didn’t know how to ask

Organize by courses, assignments; not LCSH or broad subject guidesEnsure UGC data elements are course-centric, assignment-centricProvide course hubs that unify access to disparate course-related links

Break down artificial barriers between data silosLibrary web siteElectronic resource A-Z listsCourseware (e.g., WebCT)Article databases

48

Our research and development with Queen’s

Priorities grounded in user researchSignificant new development for BiblioCommons2008/2009: Iterative beta release process

49

Questions?

Martha WhiteheadAssociate University Librarian, Queen’s University [email protected]

Steve ToubProduct Manager, Academic Services, [email protected]