Embed Size (px)
Citation preview
Creating Chat Connections: E-valuating Virtual Reference Transcripts
Marie L. Radford
ACRL Delaware Valley Chapter
November 2, 2007
Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives
Project duration: 2 ½ Years (10/05-3/08)
Four phases:I. Focus group interviews
II. Analysis of 850 QuestionPoint live chat transcripts
III. 600 online surveys
IV. 300 telephone interviews
Phase II: Transcript Analysis
• Random sample 7/04 to 11/06 (18 months) 500,000+ pool of transcripts 30-50 per month = 850 total sample
• 746 usable transcripts Excluding system tests & technical problems
• 372 classified by age/educational level 146 “Screenagers” (Middle & High School) 226 “Others” (College/Adult)
6 Analyses
• Geographical Distribution– Originating library– Librarian respondents
• Type of Library• Wait Time & Session Time• Type of Questions
– Katz/Kaske Classification
• Subject of Questions– Dewey Decimal Classification
• Interpersonal Communication– Radford Classification
VRS Session Times
• Wait time– Mean – 1.87 Minutes– Median – 1 Minute– Minimum – 1 Second– Maximum – 67 Minutes
• Session time– Mean – 12.42 Minutes– Median – 12 Minutes– Minimum – 12 Seconds– Maximum – 71 Minutes
234
0
137
40
7
37
63
2
262
187172
165
64
37
13 10
0
50
100
150
200
250
300
Public QuestionPointBackup
Academic Consortium National Law State Other Special
Num
ber
of T
rans
crip
ts
Originating library (N=723) Respondent (N=707)
N = 723
VRS Transactions by Library Type
US-Pacific, 40.0%
US-Northeast, 17.3%
US-Southeast, 13.1%
Australia, 9.8%
England, 3.2%
Canada, 2.8%
Other countries, 1.2%
US-West, 2.2%
US-S. Central, 2.9%
US-N. Central, 6.4%
Non-US, 17.0%
N = 730
VRS Questions by Location of Originating Library
US-Pacific, 35.8%
US-Northeast, 21.4%
US-Southeast, 20.5%
Australia, 10.5%
England, 3.7%
Canada, 2.0%
Other countries, 0.8%
US-West, 1.7%US-S. Central, 3.1%
US-N. Central, 3.4%
Non-US, 17.0%
N = 712
VRS Questions by Location of Librarian Respondents
30 seconds or less 37.2%
31-90 seconds 37.8%
1.5 to 8 minutes 21.6%
9 to 67 minutes 3.3%
N = 658
MEAN = 1.87 minutesMEDIAN = 1 minute
Wait Time for VRS Users
Academic, 1.04
Public, 1.13
Other Special, 1.22
Consortium, 1.71
State, 1.80
National, 2.78
Law, 9.03
0 1 2 3 4 5 6 7 8 9 10
MinutesN = 657
QuestionPoint Backup, 1.61
VRS Mean Wait Time by Library Type
National, 11.67
Public, 13.68
Academic, 14.07
State, 14.41
QP Backup, 14.52
Law, 14.82
0 2 4 6 8 10 12 14 16 18 20
Minutes
N = 577
Consortium, 16.60
VRS Mean Session Times by Library Type
Directional, 0.2%
Inappropriate, 1.4%
Research, 2.6%
Holdings, 7.8%
No Question, 11.3%
Subject Search, 30.0%
0% 5% 10% 15% 20% 25% 30% 35%
Percent of OccurrenceN = 810
Ready Reference, 26.8%
Policy and Procedural, 20.6%
Reader's Advisory, 0.1%
VRS Questions by Type
Inappropriate, 1.0%
Religion, 1.2%
Language, 1.3%
Literature, 3.5%
Science, 7.0%
Technology, 7.1%
Other, 12.2%
Procedural, 17.6%
Social Sciences 23.8%
0% 5% 10% 15% 20% 25% 30%
Percent of Occurrence
N = 761
History & Geography, 13.6%
Arts & Recreation, 5.1%
Compupter Science & General, 4.1%
Philosophy & Psychology, 1.0%
VRS Questions by Subject
Interpersonal Communication Analysis
Theoretical FrameworkWatzlawick, Beavin & Jackson (1967)
Pragmatics of Human Communication
– All messages have both content & relational dimension.
• Content = Information (WHAT)• Relational = Relationship Aspects (HOW)
Method
• Qualitative Analysis of Transcripts• Development of category scheme• Careful reading/analysis• Identification of patterns
Interpersonal Communication Research Questions
• What relational dimensions are present in chat transcripts?
• Are there differences in relational dimensions/patterns of chat users & librarians? If so, what are they?
Results
• Relational Facilitators– Interpersonal aspects of the chat conversation
that have a positive impact on the librarian-client interaction and that enhance communication.
• Relational Barriers– Interpersonal aspects of the chat conversation
that have a negative impact on the librarian-client interaction and that impede communication.
Transcript Examples – Relational Facilitators
“The Size of an Atom”
Question Type: Subject Search
Subject Type: Life Sciences, Biology (DDC:570)
Duration: 40 min.
“Diabetes”
Question Type: Subject Search
Subject Type: Business
Duration: 43 min., 15 sec.
Transcript Example – Relational Barriers
“Mesopotamian Government”Question Type: Subject SearchSubject Type: History of Ancient World (DDC:930)Duration: 27 min.
“Telekinetic Powers”Question Type: Subject SearchSubject Type: Parapsychology & OccultismDuration: 7 min., 29 sec.
Facilitators – VRS Users Screenagers (n=146) vs. Others (n=226)
• Lower numbers/percentages per transcriptS O
Thanks 21% (75) vs. 77% (175)
Agreement to try what 32% (46) vs. 51% (116)
is suggested
Closing Ritual 32% (47) vs. 49% (111)
Self Disclosure 42% (61) vs. 55% (125)
Seeking Reassurance 39% (57) vs. 49% (111)
Admit lack knowledge 19% (13) vs. 21% (47)
Facilitators – VRS Users
Screenagers (n=146) vs. Others (n=226)
• Similar numbers/percentages per transcript S O
Alternate Spelling/ 28% (41) vs. 27% (60)
Abbreviated Words
Informal Language 9% (13) vs. 9% (21)
Offering Confirmation 8% (11) vs. 8% (13)
Empathy 3% (4) vs. 4% (8)
Barriers – VRS Users Screenagers (n=146) vs. Others (n=226)
• Higher numbers/percentages per transcript
S O
Impatience 8% (12) vs. 6% (13)
Rude or Insulting 6% (9) vs. 4% (9)
Facilitators - Librarians Screenagers (n=146) vs. Others (n=226)
• Lower numbers/percentages per transcript L to S L to O
Offering Opinion/Advice 29% (43) vs. 37% (83)
Explaining Search Strategy 6% (9) vs. 14% (31)
All Lower Case 11% (63) vs. 18% (43)
Encouraging Remarks 12% (18) vs. 17% (39)
Facilitators - Librarians Screenagers (n=146) vs. Others (n=226)
• Higher numbers/percentages per transcript L to S L to
O
Seeking Reassurance 61% (89) vs. 51% (115)
Greeting Ritual 52% (76) vs. 48% (108)
Asking for Patience 39% (57) vs. 35% (80)
Explaining Signing off 5% (8) vs. 1% (2)
Abruptly
Facilitators - Librarians Screenagers (n=146) vs. Others (n=226)
• Similar numbers/percentages per transcript L to S L to O
Polite Expressions 57% (83) vs. 56% (127)
Inclusion 33% (48) vs. 34% (76)
Thanks 22% (32) vs. 23% (51)
Makes Sure User Has 18% (27) vs. 20% (45)
No More Questions
Interjections 8% (11) vs. 9% (20)
Barriers - Librarians Screenagers (n=146) vs. Others (n=226)
• Higher numbers/percentages per transcript L to S L to O
Abrupt Endings 16% (23) vs. 9% (20)
Limits Time 6% (9) vs. 0% (1)
Sends to Google 5% (8) vs. 0% (0)
Reprimanding 4% (6) vs. 0% (1)
Failure/Refusal to 5% (7) vs. 2% (5)
Provide Information
Strategies that Work!All Modes of Reference
• Basic interpersonal skills• Recognizing that user may need reassurance
– Providing reassurance• Awareness of appropriate self-disclosure
– When to disclose– Acknowledgment of user’s self-disclosure
• Humor – importance of acknowledgment
More Strategies
• Greetings & Closings.– Beware negative closure!– Beware robotic scripts!
• Inclusion (use of we, let’s, etc.).• Mirror relational strategies.• Don’t b afraid 2 use informal language,
abbreviations & emoticons as appropriate :)
Boost Satisfaction
• Collaborate across generations
• End encounter on a positive note.
• Ask “Have I answered your question completely?”
• Avoid “Negative Closure”
• Invite to return to desk or e-service if further help needed.
Bottom Line
• Communication critically important!– Difficult process– Generational differences add to
complexity!!– Use your experience & intuition as
guides.
End Notes
• This is one of the outcomes from the projectSeeking Synchronicity: Evaluating Virtual Reference Services
from User, Non-User, and Librarian Perspectives
• Funded by IMLS, Rutgers University, & OCLC Online Computer Library Center, Inc.
• Special thanks to Lynn Silipigni Connaway, Patrick Confer, Timothy Dickey, Jocelyn DeAngelis Williams, Julie Strange, Janet Torsney, & Susanna Sabolski-Boros.
• Slides available at project web site: http://www.oclc.org/research/projects/synchronicity/
