Upload
sydney-bowen
View
216
Download
3
Tags:
Embed Size (px)
Citation preview
4th International JISC/CNI Conference
Service Quality Assessment in a Service Quality Assessment in a Digital Library Environment Digital Library Environment
Edinburgh, ScotlandJune 27, 2002
Joseph Boykin, Fred Heath, Duane Webster
Overview of DiscussionsOverview of Discussions
1. New Models for Understanding and Describing Library Success
2. ARL’s New Measures Initiative
3. LibQUAL+ Project Development
4. Experience with LibQUAL+
5. Conclusions and Next Steps
The Association of Research Libraries
www.arl.org
ASSOCIATION OF RESEARCH LIBRARIES
Mission: Shaping and influencing forces affecting the
future of research libraries in the process of
scholarly communication.
Members: 123 major research libraries in North America.
Ratios: 4% of the higher education institutions
providing 40% of the information resources.
Users: 3 million students and faculty served.
Expenditures: $2.35 billion annually, $727 million for
acquisitions of which 9% is invested in access
to electronic resources.
The Problem of Assessment in The Problem of Assessment in Research LibrariesResearch Libraries
• ARL Membership Criteria Index variables emphasize inputs, primarily expenditures
• To rise in the ARL Index it is only necessary to spend more
• No demonstrable relationship between expenditures and service quality
• The lack of metrics describing performance
Systems Model for Organization Design
MISSION
VISION
WORK PROCESSES
RESOURCES
HUMAN ASPECTS
INPUTSSUPPLIERS
OUTPUTS
FEEDBACK
CUSTOMERS
FEEDBACK
ENVIRONMENT
STRUCTURE / SYSTEMS PEOPLE / SKILLS
CULTURE / CLIMATE
OUTCOMES &
FROM: Association of Research Libraries, Office ofLeadership and Management Services, Washington DC
STAKEHOLDERS
FUTUREENVIRONMENT
ARL New Measures Initiative
• Collaboration among member leaders with strong interest in this area
• Specific projects developed with different models for exploration
• Intent to make resulting tools and methodologies available to full membership and wider community
ARL New Measures Projects
• Demonstration project for service effectiveness measures (LibQUAL+)
• Project to define usage measures for electronic information resources (includes institutional outcomes)
• Investigation of role libraries play in support of the research process
• Investigation of role libraries can play in campus learning outcomes activities
• Identification of cost-drivers and development of cost-benefit studies
AssessmentAssessment
“The difficulty lies in trying to find a single model or set of simple indicators that can be used by different institutions, and that will compare something across large groups that is by definition only locally applicable –i.e., how well a library meets the needs of its institution. Librarians have either made do with oversimplified national data or have undertaken customized local evaluations of effectiveness, but there has not been devised an effective way to link the two” Sarah Pritchard, Library Trends, 1996
LibQUAL+LibQUAL+™™ Description Description
LibQUAL+TM is a research and development project undertaken to define and measure library service quality across institutions and to create useful quality-assessment tools for local planning.
The Purpose of Our Research
To fill a knowledge void in modeling the dimensions of library service quality from a user perspective
Based upon the model, to develop a web-delivered, effective total market survey instrument equivalent for service quality assessment in academic libraries
Using the derived instrument to recommend a process for an ongoing program of comparative outcome measurement for academic libraries
LibQUAL+LibQUAL+™™ Project Goals Project Goals
1. Development of web-based tools for assessing library service quality
2. Development of mechanisms and protocols for evaluating libraries
3. Identification of best practices in providing library service
4. Establishment of a library service quality assessment program at ARL
Project ResourcesProject Resources
LibQUAL+TM is an ARL/Texas A&M University joint effort. The project is supported in part by a 3-year grant from the U.S. Department of Education’s Fund for the Improvement of Post-Secondary Education (FIPSE).
LibQUAL+LibQUAL+TM TM TeamTeam
ARL
• Duane Webster
• Martha Kyrillidou
• Kaylyn Hipps
• Julia Blixrud
• Jonathan Sousa
• Consuella Waller
TAMU
• Fred Heath
• Colleen Cook
• Bruce Thompson
• Yvonna Lincoln
• Trey Thompson
• Julie Guidry
LibQUAL+LibQUAL+TM TM Project HistoryProject HistorySep
t. 19
99
Jan20
00
July
200
0
Sept
. 200
0
Oct
. 200
0
Jan.
200
1
June
200
1
9/99 - ARL launches “New Measures Initiative” which includes the study of service effectiveness known as SERVQUAL spearheaded by Texas A&M University.
1/00 - Initial 12 institutions begin the SERVQUAL study led by the Texas A&M team.
7/00 - LibQUAL+ as a distinct library-based assessment tool is presented.
9/00 - ARL and Texas A&M awarded a FIPSE grant to fund further development of the LibQUAL+ project.
10/00 - The ARL symposium, “New Culture of Assessment in Academic Libraries Measuring Service Quality” attracts a group of 170 people.
1/01 - Representatives from 43 research and university libraries participating in the Spring 2001 Implementation meet in Washington, DC during ALA midwinter.
6/01 - National Science Foundation awards grant to ARL and Texas A&M to adapt LibQUAL+ for NSDL
LibQUAL+LibQUAL+™ Contribution™ Contribution
• Grounded questions yield data of sufficient granularity to be of local use
• Normative data across cohort group at first and higher order levels
• Surfaces “Best Practices”• Web approach makes little demand of local
resources while compiling robust dataset
Relationships: perceptions, Relationships: perceptions, service quality and satisfactionservice quality and satisfaction
….only customers judge quality;all other judgments are essentiallyirrelevant”
Zeithaml, Parasuraman, Berry. (1999). Delivering quality service. NY: The Free Press.
LibQUAL+LibQUAL+TM TM ParticipantsParticipants
Year 1
Year 2
Year 3
Spring 2001Spring 2000 Spring 2002
12 Participants
43 Participants
170 Participants
For More Information about Participants:
Visit the LibQUAL+ web site.
SURVEYINSTRUMENT
Dimensions of LibraryDimensions of LibraryService QualityService Quality
Affect of Service
Empathy
Information Access
Personal Control
Responsiveness
Symbol
Utilitarian space
Assurance
Content/Scope
Timeliness
Convenience
Library as Place
LibraryServiceQuality
Refuge
Reliability
Ease of Navigation
Convenience
Modern Equipment
_____________________________________________________________________________ Factor_ ______ _ No. I II III IV Item Core _____________________________________________________________________________ 32 .84947 .12848 .24465 .13335 1 Willingness to help users33 .80847 .13662 .25348 .14147 1 Giving users individual attention 7 .80757 .17881 .12781 .21125 1 Employees deal with users caring fashion50 .79273 .19288 .18847 .12497 1 Employees who are consistently courteous31 .77262 .16358 .26461 .20061 1 Employees have knowledge answer questions 5 .74072 .14754 .18453 .29624 1 Employees understand needs of users 3 .74052 .15102 .17296 .20793 1 Readiness to respond to users' questions18 .71718 .19757 .18289 .26766 1 Employees who instill confidence in users43 .62487 .22402 .29970 .28256 0 Dependability handling service problems 20 .16556 .87679 .11430 .16236 2 A haven for quiet and solitude 2 .17739 .83172 .08498 .13901 2 A meditative place19 .22362 .83147 .14705 .22566 2 A contemplative environment25 .16013 .80492 .18894 .16628 2 Space that facilitates quiet study41 .20398 .80204 .17599 .20255 2 A place for reflection and creativity 37 .22528 .12353 .78405 .15466 * website enabling me locate info on my own28 .19602 .09611 .75780 .13173 * elec resources accessible home or office14 .33339 .16156 .60389 .31109 * access tools allow me find on my own45 .30467 .23784 .59090 .28919 3 Modern equip me easily access info I need17 .35390 .18467 .55690 .41864 * info easily accessible for independent use29 .30136 .21018 .55341 .38474 4 Convenient access to library collections 11 .13494 .23183 .18868 .73636 3 Comprehensive print collections39 .14894 .23743 .29367 .60350 3 Complete runs of journal titles16 .29445 .19831 .22384 .60107 3 Interdisciplinary library needs addressed 9 .27782 .05333 .16331 .57866 4 Timely document delivery/interlibrary loan 8 .22850 .18484 .13137 .56343 0 Convenient business hours
________________________________________________________________
LibQUAL+ Core Questions Y1LibQUAL+ Core Questions Y1
Affect of ServiceAffect of Service
• Emerged as the dominant factor early in our work
• Absorbed several of the original SERVQUAL questions measuring Responsiveness, Assurance and Empathy
• In the current analysis also includes Reliability
• All in all: the Human Dimension of Service Quality
Library as Place
• Transcends the SERVQUAL dimension of Tangibles to include the idea of the library as the campus center of intellectual activity
• As long as physical facilities are adequate, library as place may not be an issue
Personal ControlPersonal Control
• How users want to interact with the modern library
• Personal control of the information universe in general and web navigation in particular
Access to InformationAccess to Information
• Ubiquity of access: information delivered in the format, location and time of choice
• Comprehensive collections
Project DeliverablesProject Deliverables
Print and web-based results include:– Aggregate Summaries – Demographics by Library– Item Summaries– Dimension Summaries– A copy of the survey instrument– Dimensions measured for survey
implementation
Two Interpretation FrameworksTwo Interpretation Frameworks
• Score Norms
• Zone of Tolerance
Score Norms
• Norm Conversion Tables facilitate the interpretation of observed scores using norms created for a large and representative sample.
• LibQUAL+TM norms have been created at both the individual and institutional level
C
C
Zone of ToleranceZone of Tolerance
• The area between minimally acceptable and desired service quality ratings
• Perception ratings ideally fall within the Zone of Tolerance
ZONE OF TOLERANCE: DIMENSIONS
LibQUAL+™ Fundamental Contributions to the Measurement of Effective Delivery
of Library Services
• Shift the focus of assessment from mechanical expenditure-driven metrics to user-centered measures of quality
• Re-ground gap theory for the library sector, especially academic libraries
• Determine the degree to which information derived from local data can be generalized, providing much needed “best practices” information
• Demonstrate the efficacy of large-scale administration of user-centered assessment transparently across the web
LibQual+ 2000 and 2001 Results
L I B Q U A L +
C a t e g o r y : A c c e s s t o I n f o r m a t i o n
2 0 0 1 2 0 0 2
A c c e s s t o I n f o r m a t i o n A l l A l l C h a n g e
T i m e l y d o c u m e n t d e l i v e r y / i n t e r l i b r a r y l o a n 0 . 1 9 0 . 5 0 . 3 1
C o m p r e h e n s i v e p r i n t c o l l e c t i o n s - 0 . 6 7 - 0 . 0 6 0 . 6 1
I n t e r d i s c i p l i n a r y l i b r a r y n e e d s b e i n g a d d r e s s e d 0 . 2 4 0 . 2 6 0 . 0 2
C o m p l e t e r u n s o f j o u r n a l t i t l e s - 0 . 9 7 - 0 . 1 2 0 . 8 5
C o n v e n i e n t B u s i n e s s H o u r s - 0 . 0 3 0 . 6 2 0 . 6 5
P o s i t i v e G a p s 0 . 6 0 o r h i g h e r N e g a t i v e G a p s ( r e g a r d l e s s o f v a l u e )
L I B Q U A L +
C a t e g o r y : A f f e c t o f S e r v i c e
P o s i t i v e G a p s 0 . 6 0 o r h i g h e r N e g a t i v e G a p s ( r e g a r d l e s s o f v a l u e )
2 0 0 1 2 0 0 2A f f e c t o f S e r v i c e A l l A l l C h a n g eR e a d i n e s s t o r e s p o n d t o u s e r s ' q u e s t i o n s 0 . 5 4 0 . 6 3 0 . 0 9E m p l o y e e s w h o u n d e r s t a n d t h e n e e d s o f t h e i r u s e r s 0 . 3 6 0 . 5 5 0 . 1 9E m p l o y e e s w h o d e a l w i t h u s e r s i n a c a r i n g f a s h i o n 0 . 7 4 0 . 7 8 0 . 0 4E m p l o y e e s w h o i n s t i l l c o n f i d e n c e i n u s e r s 0 . 7 6 0 . 9 1 0 . 1 5E m p l o y e e s w h o h a v e t h e k n o w l e d g e t o a n s w e r u s e r q u e s t i o n s 0 . 4 4 0 . 4 1 - 0 . 0 3W i l l i n g n e s s t o h e l p u s e r s 0 . 6 1 1 . 0 0 0 . 3 9G i v i n g u s e r s i n d i v i d u a l a t t e n t i o n 0 . 7 6 0 . 8 9 0 . 1 3D e p e n d a b i l i t y i n h a n d l i n g u s e r s ' s e r v i c e p r o b l e m s 0 . 4 0 0 . 5 0 0 . 1 0E m p l o y e e s w h o a r e c o n s i s t e n t l y c o u r t e o u s 0 . 6 3 0 . 7 4 0 . 1 1
L I B Q U A L +
C a t e g o r y : L i b r a r y a s P l a c e
P o s i t i v e G a p s 0 . 6 0 o r h i g h e r N e g a t i v e G a p s ( r e g a r d l e s s o f v a l u e )
2 0 0 1 2 0 0 2L i b r a r y a s P l a c e A l l A l l C h a n g eA c o n t e m p l a t i v e e n v i r o n m e n t 0 . 1 2 0 . 5 0 0 . 3 8A h a v e n f o r q u i e t a n d s o l i t u d e 0 . 0 7 0 . 5 2 0 . 4 5S p a c e t h a t f a c i l i t a t e s q u i e t s t u d y 0 . 1 6 0 . 3 9 0 . 2 3A c o m f o r t a b l e a n d i n v i t i n g l o c a t i o n 0 . 6 4 0 . 6 5 - 0 . 0 1A p l a c e f o r r e f l e c t i o n a n d c r e a t i v i t y 0 . 3 8 0 . 5 4 0 . 1 6
L I B Q U A L +
C a t e g o r y : P e r s o n a l C o n t r o l
P o s i t i v e G a p s 0 . 6 0 o r h i g h e r N e g a t i v e G a p s ( r e g a r d l e s s o f v a l u e )
2 0 0 1 2 0 0 2P e r s o n a l C o n t r o l A l l A l l C h a n g eE a s y - t o - u s e a c c e s s t o o l s t h a t a l l o w m e t o f i n d t h i n g s o n m y o w n 0 . 2 8 0 . 2 6 - 0 . 0 2M a k i n g i n f o r m a t i o n e a s i l y a c c e s s i b l e f o r i n d e p e n d e n t u s e 0 . 1 5 0 . 3 1 0 . 1 6M a k i n g e l e c t r o n i c r e s o u r c e s a c c e s s i b l e f r o m m y h o m e o r o f f i c e - 0 . 0 5 0 . 1 9 0 . 2 4A l i b r a r y w e b s i t e e n a b l i n g m e t o l o c a t e i n f o r m a t i o n o n m y o w n 0 . 1 4 0 . 1 2 - 0 . 0 2M o d e r n e q u i p m e n t t h a t l e t s m e e a s i l y a c c e s s t h e i n f o r m a t i o n I n e e d - 0 . 1 1 0 . 3 4 0 . 4 5C o n v e n i e n t a c c e s s t o l i b r a r y c o l l e c t i o n s 0 . 4 1 0 . 3 2 - 0 . 1 1
NSF GrantNSF Grant
• Assess service quality in digital libraries
• 3 year period
• Adopt LibQUAL+ instrument for use in the Science, Math, Engineering and Technology Education Digital Library community (NSDL)
NDSL LibQUAL+ GoalsNDSL LibQUAL+ Goals
• Define dimensions of digital library service quality from the users’ perspectives
• Develop tool for measuring user perceptions and expectations of digital library service quality across NSDL digital library contexts
• Identify digital library best practices that permit generalizations across operations and development platforms
NDSL LibQUAL+ ActivitiesNDSL LibQUAL+ Activities
• 120-200 qualitative interviews to contribute to identifying dimensions of digital library service quality
• Test and refine dimensions of digital library service quality and self-sufficiency through development of total market survey
• Implement survey across variety of organizational and digital library implementations
LibQUAL+LibQUAL+TM TM Related Related DocumentsDocuments
• LibQUAL+TM Web Site http://www.arl.org/libqual/
• LibQUAL+TM Bibliographyhttp://www.coe.tamu.edu/~bthompson/servqbib
• Survey Participants Procedures Manual
http://www.arl.org/libqual/procedure/lqmanual2.pdf
LibQUAL+LibQUAL+ Related Related DocumentsDocuments
• LibQUAL+ Web Site http://www.arl.org/libqual/
• LibQUAL+ Bibliographyhttp://www.coe.tamu.edu/~bthompson/servqbib
woofwoof