ALISE, January 22-25, 2013 oclc/research/activities/synergy.html

The world’s libraries. Connected.

ALISE, January 22-25, 2013http://www.oclc.org/research/activities/synergy.html

Marie L. Radford

Chair, Department of Library & Information ScienceRutgers University

[email protected] Silipigni Connaway

Senior Research ScientistOCLC

[email protected]

ALISE/Bohdan S. Wynar Research Paper Award


Seeking Synchronicity & Cyber Synergy


Transcript Analysis

Seeking Synchronicity

Live Chat Transcripts 2004-2006 (QP1)

• Total QuestionPoint & 24/7 = 651,687

• 850 randomly selected

• 550 QP & 300 24/7

Cyber Synergy

Live Chat & Qwidget Transcripts 2010 (QP2 & QW)

• Total QP2 & QW = 296,797• QP2 221,158 & QW 75,639

• 560 randomly selected

• 350 QP2 & 210 QW

2004-2006 (QP1)

2010(QP2 & QW)


Research Questions – Cyber Synergy

• How have the mix & frequency of types of VRS questions changed?

• How frequently are ready reference questions asked in VRS?

• How has the rate of accuracy of answers to VRS ready reference questions changed?

• What differences are there in mix, frequency, & accuracy for live chat compared to IM/QW VRS modes?


Types of Questions

Seeking Synchronicity & Cyber Synergy

Transcript Analysis Results

Types of questions


Types of VR Questions

• Subject Search

• Ready Reference

• Procedural

• No Question

• Holdings

• Research

• Inappropriate

• Directional

• Reader’s Advisory


Subject Search or Specific Search

Where can i find information about

ADHD? (QP2–005)

How was the labeling theory important in

deinstitutionalization? (QW–023)

can you help me find poems by

Maya Angelou and criticism on them?

(QP1–230)

“Almost always takes the form of giving the user a document, for example, a list of citations, a book, or a report”

(Arnold & Kaske, 2005).


Ready Reference

“These are the typical ready-reference or data queries that require only a single, usually uncomplicated, straightforward

answer...Who? What? When? Why? Where?” (Arnold & Kaske, 2005).

who won the world cup game between south africa and france?

(QW–024)

Who was Bentonville, NC named after?

(QP1–147)

How do i cite a political talk show in MLA

format?

(QP2–013)


Procedural

“Questions pertaining to the policies or procedures within the library system” (Arnold & Kaske, 2005).

Can I get summer long

term renewals online?

How? (QP1–004)

what is the max for

checkout on blu ray

dvids? (QW–033)

need access logon info

to lexis from scool

library website (QP2–

027)


Holdings

“Questions about specific holdings of a library in print or digital form” (Arnold & Kaske, 2005).

I was wondering if you have textbooks to rent for

an hour or 2? (QP2–172)

Do you have any books on Paris Hilton?

(doing a project on her)

(QP 1–195)

hi there; i was wondering if you guys

have Moby Dick in stock?

(QW–014)


Research

“Research questions… involve trial-and-error searching or browsing… [and] are usually identified as coming from an adult specialist who is seeking detailed information to assist in specific work” (Arnold & Kaske, 2005).

Looking for information on national traffic survey and

incident reports. (QP2–169)

Hi, I am trying to find out information on how the

Learn Direct initiative was started, the criteria that was set and if local government

had to be involved in the first stage. (QP 1–155)

Hi I’m looking for studies similar to one

I’ve already found. How do I go about this?

(QW–013)


Inappropriate

“Questions which are not appropriate for a reference service including personal questions” (Radford, 2005).

How do I have sex? (QP1–159)

WAT DOES it mean when a guy said he like me more as a

friend doesnt it mean like a gf? (QP2–153)

AreYou Typing Warand Peace

(QW–020)


Directional

“The general information or directional question is of the information booth variety…” (Arnold & Kaske, 2005).

what is the URL for the summer reading program

for teens? (QP2–032)

I heard that you have adapted book kits for ESL speakers. Where

can I find these online?

(QW–008)

Where is the 67th street library? Is it on the west

or east side? (QP1–387)


Reader’s Advisory

Reader’s Advisory questions are “focused on helping readers find materials they want to read, listen to, or view

for pleasure” (Ross, Nilsen, & Radford, 2009).

Hello I am looking for recommendations for the

author jonathan kellerman. (QP1–218)

I was wondering if u would happen to know

any good fantasy books…

(QP2 – 050)

Hi, my 7 year old son is looking for a copy of the Hobbit suitable

for children, but we can’t seem to find anything in the library

catalogue. Can you please offer any advice? Thanks.

(QW– 120) (from Australia)


Subject Search Ready Ref-erence

Procedural No Question Holdings0%

5%

10%

15%

20%

25%

30%

35% 32%

27%

18%

11%

8%

Query Type: 2004-2006

'04 - '06 (n = 915)

n=293

n=243

n=162

n=104

n=77


Query Type: 2010

Ready Ref-erence

Procedural Subject Search Holdings No Question0%

5%

10%

15%

20%

25%

30%

35%31.1% 31.1%

16.9%

8.5%

4.3%

'10 (n=575)

n=179 n=181

n=97

n=49

n=25


Subject Search Ready Ref-erence

Procedural No Question Holdings

-5%

0%

5%

10%

15%

20%

25%

30%

35%n=293

n=243

n=162

n=104

n=77

n=97

n=179 n=181

n=25

n=49

Query Type: 2004-2006 vs. 2010

'04 - '06 (n = 915)

'10 (n = 575)

32%

8%

4%

11%

27%

31%31%

17%18%

9%


Ready ReferenceAccuracy


Accuracy in VR Ready Reference

• How has the rate of accuracy of answers to VRS ready reference questions changed?

• Is the 55% rule in effect?

(Hernon & McClure, 1987)


Accuracy Transcript Analysis

• Subset of ready reference questions

• 8/2004 - 10/2006

• n=162 ready reference queries

• 6/2010 - 12/2010

• n=168 ready reference queries


Checking for Accuracy

• Teams of coders checked responses

• Authoritative websites

• Subscription-based databases

• Any citations/links provided by the librarian/staff member

You were interested in the

kids’ (up to age 12) summer reading,

right?

YesYes

Here is a link to info on it. http://host8.evanced.info/yorklibraries/sr/homepage.aspHopefully, that page will answer your questions.

(QP2-024)

http://host8.evanced.info/yorklibraries/sr/homepage.asp

http://host8.evanced.info/yorklibraries/sr/homepage.asp


Categories

1. Correct with Citation

2. Correct without Citation

3. Correct without Citation, No Citation Needed

4. Incorrect with Citation

5. Incorrect without Citation

6. Incorrect (without specific information requested)

7. Other


Ready Reference Questions (2004-2006 vs. 2010)Received by Type of Chat Service

Consortium Academic & Law

Public National Other0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

n=45

n=34

n=59

n=20

n=4

n=68n=65

n=33

n=1 n=1

'04-'06 (n=162)

'10 (n=168)28%

40%

21%

39%

36%

20%

12%

<1%2%

<1%


Ready ReferenceAccuracy Results


Improving Accuracy: 2004-2006 vs. 2010

Correct Incorrect Other0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

n=141

n=36

n=3

n=151

n=7

n=10

'04 - '06 (n=180)

'10 (n = 168)

78%

20%

4%2%

6%

90%


Ready Reference Accuracy Correct Answers and Citations for 2004-2006 vs. 2010

Correct w/ Citation Correct w/o Citation Correct No Citation Needed0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

n=125

n=9 n=7

n=114

n=21n=16

'04 - '06 (n=141) '10 (n = 151)

89%

75%

6%

14%

5%

11%


Ready Reference is Alive!

• % of Ready Reference Queries

• Relatively stable

• Slight increase over time (4%)

• Accuracy much higher than 55% benchmark

• 78% (‘04-’06) to (‘10) 90%

• Both ready reference accuracy & percentages for correct w/ citation improved


Explanations for Increase

• VRS service providers more facile in chat & IM environment

• Providers feel less rushed

• Have followed authors’ previous recommendations?


Where to Go from Here


Recommendations to Improve Accuracy

• Clarify question

• Provide citations

• Check for broken links

• Make sure all parts of a query are answered

• Make sure specific information requested in presented before pushing a general website

improve


Future Research Directions

Investigate:

• Accuracy in other question types & SQA

• Data-derived taxonomy of accuracy

• SQA accuracy

• Level of difficulty (using READ model)

• Staffing models

• Both FtF & VR desks

• Managing multiple chat windows

• Lean staffing in tight budgets

• More collaboration/crowd-sourcing

• Mobile access

ask a librarian


References

Arnold, J., & Kaske, N. (2002). Chat with a Librarian. Computers in Libraries Annual Conference and Exhibition Usa-, 1-6.

Arnold , J., & Kaske, N. K. (2005). Evaluating the quality of a chat service. portal: Libraries and the Academy, 5(2), 177-193.

Connaway, L. S., Dickey, T. J., & Radford, M. L. (2011) “If it is too inconvenient I'm not going after it:” Convenience as a critical factor in information-seeking behaviors. Library & Information Science Research, 33(3), 179-190.

Connaway, L. S., Radford, M. L., & OCLC Research. (2011). Seeking synchronicity: Revelations and recommendations for virtual reference. Dublin, Ohio: OCLC Research. http://www.oclc.org/research/activities/synchronicity/default.htm

Hernon, P., & McClure, C. R. (1987). Unobtrusive testing and library reference services. Norwood, New Jersey: Ablex Publishing.

Janes, J. (2003). What is reference for? Reference Services Review, 31(1), 22-25.

Nummien, P., & Vakkari, P. (2009). Question types in public libraries' digital reference service in Finland: Comparing 1999 and 2006. Journal of the American Society for Information Science and Technology, 60(6), 1249-1257.

Pearce, A., Collard, S., & Whatley, K. (2010). SMS reference: Myths, markers, and modalities. Reference Services Review, 38(2), 250-263.

Pew Internet & American Life Project. (2012a). Nearly half of American adults are smartphone owners. Retrieved on April 13, 2012 from http://www.pewinternet.org/Reports/2012/Smartphone-Update-2012/Findings.aspx

http://www.sciencedirect.com/science/article/pii/S0740818811000375

http://www.oclc.org/research/activities/synchronicity/default.htm

http://www.pewinternet.org/Reports/2012/Smartphone-Update-2012/Findings.aspx


References

Pew Internet & American Life Project. (2012b). Teens, smartphones & texting. Retrieved on April 13, 2012 from http://www.pewinternet.org/~/media//Files/Reports/2012/PIP_Teens_Smartphones_and_Texting.pdf

Radford, M. L., & Connaway, L. S. 2005–2008. Seeking Synchronicity: Evaluating virtual reference services from user, non-user, and librarian perspectives. Funded by the Institute of Museum and Library Services (IMLS), Rutgers University, and OCLC http://www.oclc.org/research/activities/synchronicity/default.htm

Radford, M. L., & Connaway, L. S. (2013). Not dead yet! A longitudinal study of query type and ready reference accuracy in live chat and IM reference. Library & Information Science Research, 35(1), 2-13. [Available: http://www.oclc.org/resources/research/publications/library/2012/radford-connaway-lisr.pdf]

Radford, M. L., Connaway, L. S., Confer, P., Sabolsci-Boros, S., & Kwon, H. (2011). “Are we getting warmer?” Query clarification in live chat virtual reference. Reference & User Services Quarterly, 50(3), 259-279.

Radford, M.L., Connaway,L.S. & Shah, C. (2011-2013). Cyber synergy: Seeking sustainability through collaboration between virtual reference and social Q&A sites. Funded by the Institute of Museum and Library Services (IMLS). http://www.oclc.org/research/activities/synergy/default.htm

Radford, M. L., & Vine, S. (2011). An exploration of the hybrid service model: Keeping what works. In: Diane Zabel (Ed.), Reference Reborn: Breathing New Life into Public Services Librarianship, pp. 79-89. CO: Libraries Unlimited.

Ross, C. S., Nilsen, K., & Radford, M. L. (2009). Conducting the reference interview (2nd ed.). New York: Neal-Schuman.

Stacy-Bates, K. (2003). E-mail reference responses from academic ARL libraries. Reference & User Services Quarterly, 43(1), 59-70.

http://www.pewinternet.org/~/media/Files/Reports/2012/PIP_Teens_Smartphones_and_Texting.pdf

http://www.oclc.org/research/activities/synchronicity/default.htm

http://www.oclc.org/resources/research/publications/library/2012/radford-connaway-lisr.pdf

http://www.oclc.org/research/activities/synergy/default.htm


Funding & Acknowledgements

Cyber Synergy: Seeking Sustainability through Collaboration between Virtual Reference and Social

Q&A Sites

• $250,000 for 2011-2013

• Funded by IMLS, OCLC, & Rutgers University

• Co-PIs Marie Radford (RU), Lynn Silipigni Connaway (OCLC), & Chirag Shah (RU)

http://www.oclc.org/research/activities/synergy.html

• We thank Alyssa Darden, Lisa DeLuca, Kathy Juliano, Stephanie Mikitish, Janet Torsney, & Jacqueline Woolcott for assistance in coding, analysis, & data presentation.

http://www.oclc.org/research/activities/synergy.html


Questions & Discussion

Marie Radford

[email protected]

Lynn Silipigni [email protected]

Documents

ALISE, January 22-25, 2013 oclc/research/activities/synergy.html