ALISE, January 22-25, 2013 oclc/research/activities/synergy.html

The world’s libraries. Connected.

ALISE, January 22-25, 2013http://www.oclc.org/research/activities/synergy.html

Marie L. Radford

Chair, Department of Library & Information ScienceRutgers University

mradford@rutgers.eduLynn Silipigni Connaway

Senior Research ScientistOCLC

connawal@oclc.org

ALISE/Bohdan S. Wynar Research Paper Award

The world’s libraries. Connected.

Seeking Synchronicity & Cyber Synergy

The world’s libraries. Connected.

Transcript Analysis

Seeking Synchronicity

Live Chat Transcripts 2004-2006 (QP1)

• Total QuestionPoint & 24/7 = 651,687

• 850 randomly selected

• 550 QP & 300 24/7

Cyber Synergy

Live Chat & Qwidget Transcripts 2010 (QP2 & QW)

• Total QP2 & QW = 296,797• QP2 221,158 & QW 75,639

• 560 randomly selected

• 350 QP2 & 210 QW

2004-2006 (QP1)

2010(QP2 & QW)

The world’s libraries. Connected.

Research Questions – Cyber Synergy

• How have the mix & frequency of types of VRS questions changed?

• How frequently are ready reference questions asked in VRS?

• How has the rate of accuracy of answers to VRS ready reference questions changed?

• What differences are there in mix, frequency, & accuracy for live chat compared to IM/QW VRS modes?

The world’s libraries. Connected.

Types of Questions

Seeking Synchronicity & Cyber Synergy

Transcript Analysis Results

Types of questions

The world’s libraries. Connected.

Types of VR Questions

• Subject Search

• Ready Reference

• Procedural

• No Question

• Holdings

• Research

• Inappropriate

• Directional

• Reader’s Advisory

The world’s libraries. Connected.

Subject Search or Specific Search

Where can i find information about

ADHD? (QP2–005)

How was the labeling theory important in

deinstitutionalization? (QW–023)

can you help me find poems by

Maya Angelou and criticism on them?

(QP1–230)

“Almost always takes the form of giving the user a document, for example, a list of citations, a book, or a report”

(Arnold & Kaske, 2005).

The world’s libraries. Connected.

Ready Reference

“These are the typical ready-reference or data queries that require only a single, usually uncomplicated, straightforward

answer...Who? What? When? Why? Where?” (Arnold & Kaske, 2005).

who won the world cup game between south africa and france?

(QW–024)

Who was Bentonville, NC named after?

(QP1–147)

How do i cite a political talk show in MLA

format?

(QP2–013)

The world’s libraries. Connected.

Procedural

“Questions pertaining to the policies or procedures within the library system” (Arnold & Kaske, 2005).

Can I get summer long

term renewals online?

How? (QP1–004)

what is the max for

checkout on blu ray

dvids? (QW–033)

need access logon info

to lexis from scool

library website (QP2–

027)

The world’s libraries. Connected.

Holdings

“Questions about specific holdings of a library in print or digital form” (Arnold & Kaske, 2005).

I was wondering if you have textbooks to rent for

an hour or 2? (QP2–172)

Do you have any books on Paris Hilton?

(doing a project on her)

(QP 1–195)

hi there; i was wondering if you guys

have Moby Dick in stock?

(QW–014)

The world’s libraries. Connected.

Research

“Research questions… involve trial-and-error searching or browsing… [and] are usually identified as coming from an adult specialist who is seeking detailed information to assist in specific work” (Arnold & Kaske, 2005).

Looking for information on national traffic survey and

incident reports. (QP2–169)

Hi, I am trying to find out information on how the

Learn Direct initiative was started, the criteria that was set and if local government

had to be involved in the first stage. (QP 1–155)

Hi I’m looking for studies similar to one

I’ve already found. How do I go about this?

(QW–013)

The world’s libraries. Connected.

Inappropriate

“Questions which are not appropriate for a reference service including personal questions” (Radford, 2005).

How do I have sex? (QP1–159)

WAT DOES it mean when a guy said he like me more as a

friend doesnt it mean like a gf? (QP2–153)

AreYou Typing Warand Peace

(QW–020)

The world’s libraries. Connected.

Directional

“The general information or directional question is of the information booth variety…” (Arnold & Kaske, 2005).

what is the URL for the summer reading program

for teens? (QP2–032)

I heard that you have adapted book kits for ESL speakers. Where

can I find these online?

(QW–008)

Where is the 67th street library? Is it on the west

or east side? (QP1–387)

The world’s libraries. Connected.

Reader’s Advisory

Reader’s Advisory questions are “focused on helping readers find materials they want to read, listen to, or view

for pleasure” (Ross, Nilsen, & Radford, 2009).

Hello I am looking for recommendations for the

author jonathan kellerman. (QP1–218)

I was wondering if u would happen to know

any good fantasy books…

(QP2 – 050)

Hi, my 7 year old son is looking for a copy of the Hobbit suitable

for children, but we can’t seem to find anything in the library

catalogue. Can you please offer any advice? Thanks.

(QW– 120) (from Australia)

The world’s libraries. Connected.

Subject Search Ready Ref-erence

Procedural No Question Holdings0%

5%

10%

15%

20%

25%

30%

35% 32%

27%

18%

11%

8%

Query Type: 2004-2006

'04 - '06 (n = 915)

n=293

n=243

n=162

n=104

n=77

The world’s libraries. Connected.

Query Type: 2010

Ready Ref-erence

Procedural Subject Search Holdings No Question0%

5%

10%

15%

20%

25%

30%

35%31.1% 31.1%

16.9%

8.5%

4.3%

'10 (n=575)

n=179 n=181

n=97

n=49

n=25

The world’s libraries. Connected.

Subject Search Ready Ref-erence

Procedural No Question Holdings

-5%

0%

5%

10%

15%

20%

25%

30%

35%n=293

n=243

n=162

n=104

n=77

n=97

n=179 n=181

n=25

n=49

Query Type: 2004-2006 vs. 2010

'04 - '06 (n = 915)

'10 (n = 575)

32%

8%

4%

11%

27%

31%31%

17%18%

9%

The world’s libraries. Connected.

Ready ReferenceAccuracy

The world’s libraries. Connected.

Accuracy in VR Ready Reference

• How has the rate of accuracy of answers to VRS ready reference questions changed?

• Is the 55% rule in effect?

(Hernon & McClure, 1987)

The world’s libraries. Connected.

Accuracy Transcript Analysis

• Subset of ready reference questions

• 8/2004 - 10/2006

• n=162 ready reference queries

• 6/2010 - 12/2010

• n=168 ready reference queries

The world’s libraries. Connected.

Checking for Accuracy

• Teams of coders checked responses

• Authoritative websites

• Subscription-based databases

• Any citations/links provided by the librarian/staff member

You were interested in the

kids’ (up to age 12) summer reading,

right?

YesYes

Here is a link to info on it. http://host8.evanced.info/yorklibraries/sr/homepage.aspHopefully, that page will answer your questions.

(QP2-024)

The world’s libraries. Connected.

Categories

1. Correct with Citation

2. Correct without Citation

3. Correct without Citation, No Citation Needed

4. Incorrect with Citation

5. Incorrect without Citation

6. Incorrect (without specific information requested)

7. Other

The world’s libraries. Connected.

Ready Reference Questions (2004-2006 vs. 2010)Received by Type of Chat Service

Consortium Academic & Law

Public National Other0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

n=45

n=34

n=59

n=20

n=4

n=68n=65

n=33

n=1 n=1

'04-'06 (n=162)

'10 (n=168)28%

40%

21%

39%

36%

20%

12%

<1%2%

<1%

The world’s libraries. Connected.

Ready ReferenceAccuracy Results

The world’s libraries. Connected.

Improving Accuracy: 2004-2006 vs. 2010

Correct Incorrect Other0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

n=141

n=36

n=3

n=151

n=7

n=10

'04 - '06 (n=180)

'10 (n = 168)

78%

20%

4%2%

6%

90%

The world’s libraries. Connected.

Ready Reference Accuracy Correct Answers and Citations for 2004-2006 vs. 2010

Correct w/ Citation Correct w/o Citation Correct No Citation Needed0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

n=125

n=9 n=7

n=114

n=21n=16

'04 - '06 (n=141) '10 (n = 151)

89%

75%

6%

14%

5%

11%

The world’s libraries. Connected.

Ready Reference is Alive!

• % of Ready Reference Queries

• Relatively stable

• Slight increase over time (4%)

• Accuracy much higher than 55% benchmark

• 78% (‘04-’06) to (‘10) 90%

• Both ready reference accuracy & percentages for correct w/ citation improved

The world’s libraries. Connected.

Explanations for Increase

• VRS service providers more facile in chat & IM environment

• Providers feel less rushed

• Have followed authors’ previous recommendations?

The world’s libraries. Connected.

Where to Go from Here

The world’s libraries. Connected.

Recommendations to Improve Accuracy

• Clarify question

• Provide citations

• Check for broken links

• Make sure all parts of a query are answered

• Make sure specific information requested in presented before pushing a general website

improve

The world’s libraries. Connected.

Future Research Directions

Investigate:

• Accuracy in other question types & SQA

• Data-derived taxonomy of accuracy

• SQA accuracy

• Level of difficulty (using READ model)

• Staffing models

• Both FtF & VR desks

• Managing multiple chat windows

• Lean staffing in tight budgets

• More collaboration/crowd-sourcing

• Mobile access

ask a librarian

The world’s libraries. Connected.

References

Arnold, J., & Kaske, N. (2002). Chat with a Librarian. Computers in Libraries Annual Conference and Exhibition Usa-, 1-6.

Arnold , J., & Kaske, N. K. (2005). Evaluating the quality of a chat service. portal: Libraries and the Academy, 5(2), 177-193.

Connaway, L. S., Dickey, T. J., & Radford, M. L. (2011) “If it is too inconvenient I'm not going after it:” Convenience as a critical factor in information-seeking behaviors. Library & Information Science Research, 33(3), 179-190.

Connaway, L. S., Radford, M. L., & OCLC Research. (2011). Seeking synchronicity: Revelations and recommendations for virtual reference. Dublin, Ohio: OCLC Research. http://www.oclc.org/research/activities/synchronicity/default.htm

Hernon, P., & McClure, C. R. (1987). Unobtrusive testing and library reference services. Norwood, New Jersey: Ablex Publishing.

Janes, J. (2003). What is reference for? Reference Services Review, 31(1), 22-25.

Nummien, P., & Vakkari, P. (2009). Question types in public libraries' digital reference service in Finland: Comparing 1999 and 2006. Journal of the American Society for Information Science and Technology, 60(6), 1249-1257.

Pearce, A., Collard, S., & Whatley, K. (2010). SMS reference: Myths, markers, and modalities. Reference Services Review, 38(2), 250-263.

Pew Internet & American Life Project. (2012a). Nearly half of American adults are smartphone owners. Retrieved on April 13, 2012 from http://www.pewinternet.org/Reports/2012/Smartphone-Update-2012/Findings.aspx

The world’s libraries. Connected.

References

Pew Internet & American Life Project. (2012b). Teens, smartphones & texting. Retrieved on April 13, 2012 from http://www.pewinternet.org/~/media//Files/Reports/2012/PIP_Teens_Smartphones_and_Texting.pdf

Radford, M. L., & Connaway, L. S. 2005–2008. Seeking Synchronicity: Evaluating virtual reference services from user, non-user, and librarian perspectives. Funded by the Institute of Museum and Library Services (IMLS), Rutgers University, and OCLC http://www.oclc.org/research/activities/synchronicity/default.htm

Radford, M. L., & Connaway, L. S. (2013). Not dead yet! A longitudinal study of query type and ready reference accuracy in live chat and IM reference. Library & Information Science Research, 35(1), 2-13. [Available: http://www.oclc.org/resources/research/publications/library/2012/radford-connaway-lisr.pdf]

Radford, M. L., Connaway, L. S., Confer, P., Sabolsci-Boros, S., & Kwon, H. (2011). “Are we getting warmer?” Query clarification in live chat virtual reference. Reference & User Services Quarterly, 50(3), 259-279.

Radford, M.L., Connaway,L.S. & Shah, C. (2011-2013). Cyber synergy: Seeking sustainability through collaboration between virtual reference and social Q&A sites. Funded by the Institute of Museum and Library Services (IMLS). http://www.oclc.org/research/activities/synergy/default.htm

Radford, M. L., & Vine, S. (2011). An exploration of the hybrid service model: Keeping what works. In: Diane Zabel (Ed.), Reference Reborn: Breathing New Life into Public Services Librarianship, pp. 79-89. CO: Libraries Unlimited.

Ross, C. S., Nilsen, K., & Radford, M. L. (2009). Conducting the reference interview (2nd ed.). New York: Neal-Schuman.

Stacy-Bates, K. (2003). E-mail reference responses from academic ARL libraries. Reference & User Services Quarterly, 43(1), 59-70.

The world’s libraries. Connected.

Funding & Acknowledgements

Cyber Synergy: Seeking Sustainability through Collaboration between Virtual Reference and Social

Q&A Sites

• $250,000 for 2011-2013

• Funded by IMLS, OCLC, & Rutgers University

• Co-PIs Marie Radford (RU), Lynn Silipigni Connaway (OCLC), & Chirag Shah (RU)

http://www.oclc.org/research/activities/synergy.html

• We thank Alyssa Darden, Lisa DeLuca, Kathy Juliano, Stephanie Mikitish, Janet Torsney, & Jacqueline Woolcott for assistance in coding, analysis, & data presentation.

The world’s libraries. Connected.

Questions & Discussion

Marie Radford

mradford@rutgers.edu

Lynn Silipigni Connawayconnawal@oclc.org