Upload
others
View
7
Download
0
Embed Size (px)
Citation preview
Lessons Learned from Applying Customer Satisfaction Criteria to Virtual Reference
Cat Silvers Florida State University Libraries 2012
Assessment Background • Wanted to see what types of questions we
were getting and how operators were performing
• “Pilot” • Based on RUSA Guidelines for Implementing and
Maintaining Virtual Reference Services
• October 2011 • 492 chat transcripts coded of 550
Data Outcome Quantitative
• Chat Volume, October-December 2011 – FSU (#1 in Florida; 1,310) – USF (#2 in Florida; 915)
• Question Types – General – Strozier-specific
• Peak Days – Wednesdays; 23 – Mondays; 21.8
• Peak hour 4-5pm
Qualitative
• Customer Service quality (RUSA guidelines): 53%
• Question accuracy: 95% • Question completeness:
94%
Guidelines
Evaluation Criteria
1. Did the operator refer the patron to a more appropriate librarian if needed to answer the question?
2. Did the operator explain the search strategy to the patron, in addition to the sources used?
3. Did the operator explain how to use the source?
4. Did the operator receive validation from the patron? (“Thank you!”)
Completeness of Answer
Assessment Conclusions • Peak times and days
to facilitate proper staffing
• Identify strengths and weaknesses
• What customer service aspects need to be address via training
Difficulties Experienced
Quantitative Section
• categories did not reflect all types of questions experienced
• Some categories were too general
Qualitative Section • Customer Service quality
marked “Satisfactory,” “Unsatisfactory” or “Unknown” – 7 requirements to be
considered satisfactory – Judgment call
Three weeks, one person
Future Changes
• New categories needed for quantitative section • Increased question clarity in qualitative section • New categories for customer service section • Additional coders needed • Coder training for consistency
Benefits of the Study
• Set customer service expectations • Grading rubric to score customer service
categories • Individualized operator progress reports • Stronger staffing
Cat Silvers Florida State University Libraries Undergraduate Services Overnight Coordinator [email protected]