Embed Size (px)
Citation preview
Implementing LibQUAL+Implementing LibQUAL+®® Massachusetts LSTA OrientationMassachusetts LSTA Orientation
MCCLPHEI ConsortiumMCCLPHEI Consortium
Boston, MANovember 1, 2006
Martha KyrillidouAssociation of Research Libraries
old.libqual.org
old.libqual.org
OverviewOverview
• Research Foundation and Current Uses of LibQUAL+®
• Preparing to Implement the Survey
• The Survey Process: Initial Steps
• Mapping Disciplines
• Approving and Running Your Survey
• Understanding Your Results: An Overview
• General Discussion and Q&A
old.libqual.org
Total CirculationTotal Circulation
old.libqual.org
Reference TransactionsReference Transactions
old.libqual.org
AssessmentAssessment
“The difficulty lies in trying to find a single model or set of simple indicators that can be used by different institutions, and that will compare something across large groups that is by definition only locally applicable—i.e., how well a library meets the needs of its institution. Librarians have either made do with oversimplified national data or have undertaken customized local evaluations of effectiveness, but there has not been devised an effective way to link the two.”
Sarah Pritchard, Library Trends, 1996
old.libqual.org
Multiple MethodsMultiple Methodsof Listening to Customersof Listening to Customers• Transactional surveys*• Mystery shopping• New, declining, and lost-customer surveys• Focus group interviews• Customer advisory panels• Service reviews• Customer complaint, comment, and inquiry capture• Total market surveys*• Employee field reporting• Employee surveys• Service operating data capture
*A SERVQUAL-type instrument is most suitable for these methods
Note. A. Parasuraman. The SERVQUAL Model: Its Evolution And Current Status. (2000).
Paper presented at ARL Symposium on Measuring Service Quality, Washington, D.C.
old.libqual.org
The The LibQUAL+LibQUAL+®® Premise Premise
PERCEPTIONS SERVICE
“….only customers judge quality;
all other judgments are essentially
irrelevant”
Note. Zeithaml, Parasuraman, Berry. (1999). Delivering quality service. NY: The Free Press.
old.libqual.org
13 LibrariesEnglish LibQUAL+™ Version
4000 Respondents
QUAL
QUAN
QUAL
QUAL
QUAN
QUAL
PURPOSE DATA ANALYSIS PRODUCT/RESULTDescribe library environment;build theory of library service quality from user perspective
Test LibQUAL+™ instrument
Refine theoryof service quality
Refine LibQUAL+™ instrument
Test LibQUAL+™ instrument
Refine theory
Unstructured interviewsat 8 ARL institutions
Web-delivered survey
Unstructured interviews at Health Sciences and the Smithsonian libraries
E-mail to surveyadministrators
Web-delivered survey
Focus groups
Content analysis:(cards & Atlas TI)
Reliability/validityanalyses: CronbachsAlpha, factor analysis,SEM, descriptive statistics
Content analysis
Content analysis
Reliability/validity analyses including Cronbachs Alpha,factor analysis, SEM, descriptive statistics
Content analysis
VignetteRe-tooling
Iterative
Emergent2000
2004315 Libraries English, Dutch, Swedish,
German LibQUAL+™ Versions160,000 anticipated respondents
LibQUAL+LibQUAL+®® Project Project
Case studies1
Valid LibQUAL+™ protocol
Scalable process
Enhanced understanding of user-centered views of service quality in the library environment2
Cultural perspective3
Refined survey delivery process and theory of service quality4
Refined LibQUAL+™ instrument5
Local contextual understanding of LibQUAL+™ survey responses6
old.libqual.org
DimensionsDimensions
2000 2001 2002 2003-200641 items 56 items 25 items 22 items
Affect of Service Affect of Service Service Affect Service Affect
Library as Place Library as Place Library as Place Library as Place
Reliability Reliability Personal ControlInformation Control
Provision of Physical Collections
Self-RelianceInformation Access
Access to Information
Access to Information
old.libqual.org
Survey Structure – Page 2Survey Structure – Page 2(Detail View)(Detail View)
old.libqual.org
Rapid Growth in Other AreasRapid Growth in Other Areas
• Languages– American English– British English– French– Dutch– Swedish– Norwegian– Finnish– Danish
• Consortia– Each may create 5 local
questions to add to their survey
• Types of Institutions– Academic Health Sciences– Academic Law– Academic Military– College or University– Community College– European Business– Hospital– Public– State
• Countries– Australia, Canada, Denmark,
Egypt, Finland, France, New Zealand, the Netherlands, Norway, South Africa, Sweden, Switzerland, UAE, U.K., U.S.
old.libqual.org
LibQUALLibQUAL++®® Participants Participants
12
43
164
316
206 210
0
50
100
150
200
250
300
350
2000 2001 2002 2003 2004 2005
old.libqual.org
LibQUALLibQUAL++®® Surveys by Type Surveys by Type Years
2000 2001 2002 2003 2004 2005
Academic law 1 25 10
Academic military 6 1
College or university 13 41 111 244 150 191
Community college 16 29 3 14
European business 5
Family history library 1
Health sciences library 1 35 23 13 12
Hospital 10 1
New York Public library 1
Public 4 1 1
Smithsonian 1 1
State 1 1
University/TAFE 2
Electronic 1
Natural Resources 2
FFRDC 5
old.libqual.org
LibQUALLibQUAL++®® Languages Languages
American English
Dutch EnglishFrench Canadian DutchSwedish
Swedish(British English)
Afrikaans
DanishFinnishGerman Norwegian
British English
Continental French
old.libqual.org
LibQUALLibQUAL++®® Surveys by Language Surveys by Language
Years
2000 2001 2002 2003 2004 2005
American English 13 42 164 285 176 207
British English 20 22 31
Continental French 1
Dutch 1
Dutch English 1 1
French Canadian 2 1 4
Swedish 5 2
Swedish English (A.E.) 2 1
Afrikaans 4
old.libqual.org
LibQUALLibQUAL++®® Surveys by Consortia Surveys by Consortia 2000 2001 2002 2003 2004 2005
AAHSL 1 35 21 14 12
AJCU 20
Alabama Academic (NAAL) 10 1 8
CES 6
City University of New York 19
CUC 8
EBSLG 6
FFRDC 5
Hospital/MLA 7
MERLN 6
NELLCO 8
NY3Rs 76 2
Oberlin 12 9 13
OhioLINK 57 45 1 14
SCONUL 20 17 16
State Universities of Florida 6 2
University of Wisconsin System 14
VALE 12
old.libqual.org
Participating Libraries by CountryParticipating Libraries by Country
Country 2000 2001 2002 2003 2004 2005
Australia 1 6
Canada 1 3 4 8 10 15
Denmark 1
Egypt 1
France 1
Ireland 1 1
Netherlands 1 1
Sweden 3 4
Switzerland 2
U.A.E. 1
UK 20 17 16
South Africa 12
old.libqual.org
Surveys by Session 2004-2005Surveys by Session 2004-2005
Year Session 1 Session 2
2004 202 2
2005 199 56
old.libqual.org
World World LibQUALLibQUAL++®® Survey Survey
old.libqual.org
Preparing for the Survey ImplementationPreparing for the Survey Implementation
old.libqual.org
Why is Your Library Participating Why is Your Library Participating in in LibQUAL+LibQUAL+® ® ??
• Institutional goals:
– what do you want to get out of the survey?
• Survey requirements:
– people, supplies, technology
old.libqual.org
Get Permission from your Get Permission from your Institutional Review BoardInstitutional Review Board
• Local group that approves human subject research
• May also be called
– Committee for Human Subject Research
– Committee for the Protection of Human Subjects
• Not all institutions need to obtain permission
old.libqual.org
Get Permission from yourGet Permission from yourInstitutional Review BoardInstitutional Review Board
• If permission is required:
– Seek well in advance of survey
– Supply a copy of the survey, if requested
– Inform them that results will be shared among participants
• No need to inform LibQUAL+® of decision
old.libqual.org
Determine Whom to SurveyDetermine Whom to Survey
• Random sample
or
• Entire population
old.libqual.org
If You Sample…If You Sample…
• Recommendations: – At least 1,200 random email addresses for
each user group– Separate sample groups for
undergraduates, graduates, faculty, and staff
– Think about survey fatigue if you want to repeat the survey regularly
• Keep notes on your methodology as you will be asked to describe it in the Post Hoc Questionnaire
old.libqual.org
If You Survey the If You Survey the Entire Population….Entire Population….
• Recommendations:
– Think about survey fatigue if you want to repeat the survey regularly
• Keep notes on your methodology as you will be asked to describe it in the Post Hoc Questionnaire
old.libqual.org
Obtain Email AddressesObtain Email Addresses
• Typical sources include:
– Campus computing office
– Campus administrative records office
– Library patron database
old.libqual.org
Promoting your SurveyPromoting your Survey
– Place ads in campus newspaper
– Write article for library newsletter
– Post flyers around campus
– Present at faculty meetings & student orientations
– Create a survey Web site and feature on library’s home page
– Take the survey to where users are: dining halls, study rooms, dorms
old.libqual.org
Become Familiar with Become Familiar with LibQUAL+LibQUAL+®® Resources Resources
• Management Centerhttp://www.libqual.org/Manage/index.cfm
• LibQUAL+® Procedures Manualhttp://www.libqual.org/Manual/index.cfm –Updated Version – September 2006–Much more detail
• Discussion [email protected]
old.libqual.org
The The LibQUAL+LibQUAL+®® Management CenterManagement Center
old.libqual.org
The Survey Process: Initial StepsThe Survey Process: Initial Steps
old.libqual.org
Mapping DisciplinesMapping Disciplines
old.libqual.org
Mapping DisciplinesMapping Disciplines
• Results notebooks summarize findings by user group and provide a chart for, both, standard and custom disciplines
• Standard disciplines
• Customized disciplines
– Recommend no more than 10 disciplines, if possible
old.libqual.org
The Standard DisciplinesThe Standard Disciplines
• Agriculture/Environmental Studies
• Architecture• Business• Communications/
Journalism• Education• Engineering/Computer
Science• General Studies• Health Sciences
• Humanities• Law• Military/Naval Science• Other• Performing & Fine Arts• Science/Math• Social Sciences/
Psychology• Undecided
old.libqual.org
Customized DisciplinesCustomized Disciplines
• Use your local terminology to map to the standard disciplines
• Cautions:
– Need to provide representativeness data for each discipline
– Too many choices present challenges to users
old.libqual.org
Standard Disciplines: Standard Disciplines: A PA Academic LibraryA PA Academic Library
old.libqual.org
Customized Disciplines: Customized Disciplines: A PA Academic LibraryA PA Academic Library
old.libqual.org
Getting Practice with MappingGetting Practice with Mapping
Let’s practice mapping your disciplinesLet’s practice mapping your disciplines
old.libqual.org
Approving and Running Your SurveyApproving and Running Your Survey
old.libqual.org
Understanding Your ResultsUnderstanding Your Results
old.libqual.org
Results Notebooks
• Sections for Overall, Undergraduates, Graduates, Faculty, Staff, Library Staff include:– Demographic Summary– Core Questions Summary– Dimensions Summary– Local Questions– General Satisfaction Questions– Information Literacy Outcomes Questions– Library Use Summary
• Appendix describing changes in the dimensions and the questions included in each dimension.
old.libqual.org
Results
• Results Notebooks (PDF)http://old.libqual.org/Manage/Results/index.cfm
– Group Notebook (PDF)
– Unified Notebook (PDF)
– User Comments
– Excel Data File
SPSS data files (if requested at registration) will be e-mailed to the primary contact two to three months after the session has closed.
old.libqual.org
Core Items and DimensionsCore Items and Dimensions
22 core items (i.e., questions)
Three dimensions:
• Affect of Service – 9 questions
• Information Control – 8 questions
• Library as Place – 5 questions
old.libqual.org
Understanding a Radar ChartUnderstanding a Radar Chart
old.libqual.org
Key to Radar ChartsKey to Radar Charts
old.libqual.org
What Do the Colors Mean?What Do the Colors Mean?
GreenAnd
Blue
old.libqual.org
What Do the Colors Mean?What Do the Colors Mean?
Red
old.libqual.org
What Do the Colors Mean?What Do the Colors Mean?
Little Yellow
old.libqual.org
What Do the Colors Mean?What Do the Colors Mean?
A lot of Yellow
old.libqual.org
Dimension SummaryDimension Summary
old.libqual.org
Library Use Question (Unified)Library Use Question (Unified)
old.libqual.org
2006 2006 LibQUAL+LibQUAL+®® Highlights: Highlights: OverallOverall
old.libqual.org
Qualitative Analysis: Qualitative Analysis: User CommentsUser Comments• About one-half of users include comments on their
surveys
• User Comments available on the LibQUAL+® Web site– Download comments in Excel or text file
• Skim the comments
• Conduct ATLAS.ti analysis
old.libqual.org
General Discussion and Q&AGeneral Discussion and Q&A
old.libqual.org
Summary and ClosureSummary and Closure
old.libqual.org
LibQUAL+LibQUAL+®® Resources Resources
• LibQUAL+® Web site:http://old.libqual.org
• Publications:http://old.libqual.org/publications
• Events and Training:http://old.libqual.org/events
• Gap Theory/Radargraph Introduction:http://old.libqual.org/Information/Tools/libqualpresentation.cfm
• LibQUAL+® Procedures Manual:http://old.libqual.org/Publications/index.cfm
old.libqual.org
The The LibQUAL+LibQUAL+®® Team Team
• MaShana DavisTechnical Communications [email protected]
• Richard GrovesCustomer Relations [email protected]
• Martha KyrillidouDirector, Statistics and Service Quality [email protected]
