Embed Size (px)
Citation preview
www.arl.org
ARL Statistics and Measurement
Library Assessment Thessaloniki, Greece
June 13-15, 2005
Brinley Franklin, University of ConnecticutMartha Kyrillidou, Association of Research Libraries
www.arl.org
The Association of Research Libraries • Mission: influences the changing environment of scholarly
communication and the public policies that affect research libraries and the communities they serve.
• Members: 123 major research libraries in North America.
• Ratios: 4% of the higher education institutions providing 40% of the information resources.
• Users: 3 million students and faculty served.
• Expenditures: $3 billion annually of which 25% is invested in access to electronic resources.
www.arl.org
The Association of Research Libraries
• Began in 1932• 123 Members
– US Academic– Canadian Academic– Non-university
• Statistics was early program– collected beginning 1961/62– descriptive and quantitative data gathered since
1908
www.arl.org
ARL Roles
www.arl.org
ARL Statistics and Measurement
…To describe and measure the
performance of research libraries and their contribution
to teaching, research, scholarship and community service …
www.arl.org
ARL Statistics and Measurement
…To describe and measure the performance of research libraries and their
contribution to
teaching, research, scholarship and community
service …
www.arl.org
Web Presence
www.arl.org
Descriptive and Quantitative Measures
• Inputs– Collection size– Expenditures– Staffing
• Outputs– Services– People served
• Ratios (inputs outputs)– e.g., expenditures per FTE
www.arl.org
Traditional Statistics
• Longest and consistent data series (1908- )• Describe level of investment• Define membership for the Association of
Research Libraries• Used for budget justification purposes• Used for benchmarking with other peer
institutions• Describe trends in scholarly communication
and for research libraries in general
www.arl.org
• The Trends
www.arl.org
Library Expenditures as a Percent of University Expenditures for 40 ARL Libraries, 1982-2002
2.4000
2.6000
2.8000
3.0000
3.2000
3.4000
3.6000
3.8000
1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002
Copyright Association of Research Libraries, 2005
www.arl.org
Supply and Demand in ARL Libraries, 1986-2003
-50%
0%
50%
100%
150%
200%
1986 1988 1990 1992 1994 1996 1998 2000 2002
% C
hang
e S
ince
198
6
Interlibrary Borrowing (+214%)
Interlibrary Lending (+108%)
Grad. Students (+78%)
Faculty (+14%)
Total Students (+27%)
Serials Purchased (+14%)
Monographs Purchased (0%)
Serials Received (+167%)
Copyright © 2004 Association of Research LibrariesSource: ARL Statistics 2002-03, Association of Research Libraries, 2003
www.arl.org
Monograph and Serial Costs in ARL Libraries, 1986-2003
-50%
0%
50%
100%
150%
200%
250%
1986 1988 1990 1992 1994 1996 1998 2000 2002
Serial Unit Cost (+215%)
Serial Expenditures(+260%)
Monograph Unit Cost (+82%)
MonographExpenditures (+66%)
Serials Purchased (+14%)
Monographs Purchased (0%)
CPI (+68%)
Copyright © 2004 Association of Research LibrariesSource: ARL Statistics 2002-03, Association of Research Libraries, 2003
www.arl.org
Expenditure Trends in ARL Libraries, 1986-2003
-10%
20%
50%
80%
110%
140%
170%
200%
230%
260%
1986 1988 1990 1992 1994 1996 1998 2000 2002
% C
hang
e S
ince
198
6
Serial Expenditures(+260%)
Library Materials (+206%)
Operating Expenditures (+84%)
TOTAL (+128%)Total Salaries (+116%)
CPI (+68%)Monograph Expenditures (+66%)
Copyright © 2004 Association of Research LibrariesSource: ARL Statistics 2002-03, Association of Research Libraries, 2003
www.arl.org
Resources per Student in ARL University Libraries, 1986-2003
-50%
0%
50%
100%
150%
200%
1986 1988 1990 1992 1994 1996 1998 2000 2002
InterlibraryBorrowing (+172%)
Volumes Held (+28%)
Vols. Added (-15%)Total Staff (-21%)
Serials Purchased (-1%)
Monographs Purchased (-32%)
Copyright © 2004 Association of Research LibrariesSource: ARL Statistics 2002-03, Association of Research Libraries, 2003
www.arl.org
Printed Book Circulation: All ARL LibrariesPrinted Book Circulation: All ARL Libraries
Note. M. Kyrillidou and M. Young. (2002).ARL Statistics 2000-01. Washington, D.C.: ARL, p.7.
www.arl.org
Research Behavior: Research Behavior: Personal ControlPersonal Control
When searching for When searching for printprint journals for journals for research:research:
• Only 13.9% ask a librarian for assistanceOnly 13.9% ask a librarian for assistance• Only 3.2% consider consulting a librarian a preferred Only 3.2% consider consulting a librarian a preferred way of identifying information way of identifying information
Note. Digital Library Federation and Council on Library and InformationResources. (2002). Dimensions and Use of the Scholarly Information Environment.
www.arl.org
Reference Decrease: All ARL LibrariesReference Decrease: All ARL Libraries
Note. M. Kyrillidou and M. Young. (2002).ARL Statistics 2000-01. Washington, D.C.: ARL, p.7.
www.arl.org
The Challenge of Assessment in The Challenge of Assessment in Academic LibrariesAcademic Libraries
• Traditional statistics emphasize inputs, expenditures, acquisitions, holdings, etc.
• National Rankings are often misleading
• No demonstrable relationship between expenditures and service quality
• The lack of metrics describing outcomes: success from the user’s point of view
www.arl.org
Assessment
“The difficulty lies in trying to find a single model or set of simple indicators that can be used by different institutions, and that will compare something across large groups that is by definition only locally applicable—i.e., how well a library meets the needs of its institution. Librarians have either made do with oversimplified national data or have undertaken customized local evaluations of effectiveness, but there has not been devised an effective way to link the two.” Sarah Pritchard, Library Trends, 1996
www.arl.org
Measures that Matter
• Input Output Outcome or Impact• Consistent with organizational mission, goals
and objectives and integration with program review
• Establish accountability• Collection and use of reliable and valid data
– Benchmarking with peers– Over time within own library
www.arl.org
Why New Measures for ARL
• Increased demand for quality instead of quantity measures
• Growing customer and stakeholder service expectations and need to balance customer, stakeholder, and employee interests and needs
• Increasing demand for libraries to demonstrate outcomes/impacts in areas of importance to institution
• Exploding growth in use and applications of technology
• Competition for scarce resources• Increasing pressure to maximize use of resources through
benchmarking or best practice resulting in:– Cost savings– Reallocation
www.arl.org
ARL New Measures Initiative
• Collaboration among member leaders with strong interest in this area
• January 1999 retreat to define topics (resulted in 8 areas for investigation)
• Specific projects developed with different models for exploration -- most self-funded
• Intent to make resulting tools and methodologies available to full membership and wider community
www.arl.org
8 Areas of Investigation and Current ARL Projects
Market penetration
Customer satisfaction
Extent of access
Teaching and Learning
Research
Cost effectiveness
Space
Organizational Capacity
LibQUAL+
E-Metrics
Learning Outcomes
Research Outcomes
Cost Drivers
www.arl.org
Future of New Measures Initiative
• Incorporation of some new data elements in traditional surveys
• Development of workshops and consulting services for performance measures
• Establishment of data gathering and statistical analysis tools
www.arl.org
A vision
StatsHome www.libstats.org
www.arlstatistics.orgwww.arlstats.org
LibQUAL+DigiQUAL(/digiqual)
MINES(/mines)
SAILS(/sails)
E-Metrics(/emetrics)
ARL Statistics(/arlstats)
ARL ARL S&M Site
www.libqual.org
www.arl.org
StatsQUAL™
StatsHome
LibQUAL+DigiQUAL(/digiqual)
MINES(/mines)
SAILS(/sails)
E-Metrics(/emetrics)
ARL Statistics(/arlstats)
Interactive Statistics
(/interactive)?
Login
UserProfile
InstitutionProfile
www.libqual.org
SurveyManagement
www.arl.org
New Measures
• LibQUAL+™
• E-metrics … Project COUNTER
• MINES™
• DigiQUAL™
• SAILS
• Learning Outcomes Working Group
• Task Force on New Ways of Measuring Collections
www.arl.org
Measuring Collections
In an environment with increasing emphasis on digital resources, what are the metrics that are appropriate for describing and characterizing collections, and what collection trends are particularly important to identify and track over time?
www.arl.org
The Value of the Qualitative Perspectives
“Only with in-depth, local, qualitative, ‘culture’ studies can libraries know and understand what compels some to remain as far away from the library as possible, while others refrain from engaging library staff in their own search for proficiency and self-reliance” (Lincoln, p. 15).
www.arl.org
“If Foucault is correct that we in the West live in surveilled societies, then what function does self-reliance serve? …the library user who wishes to navigate resources with as little help as possible – seeks a kind of privacy from the surveillance of librarian help …Having found the relative anonymity of cyberspace and a virtual world, this self-reliant user now seeks the same independence and lack of surveillance in the text-based and digitized universe of information resources known as the library” (Lincoln, p. 12).
Cultural Perspective – Self-reliance
www.arl.org
• “…the library needs to welcome them in. It needs to make them feel like this is a place where they can be in almost a haven, a refuge” (Business professor)
• “…It’s beyond the ease [with] which you can find information, just because the library experience is something like Greece or Athens…” (Undergraduate)
• “writing an undergraduate thesis with this big dome over his head…he felt really like a scholar” (Linguistics professor)
• Writing a dissertation in a particular library for another scholar “was an emotional experience”
Cultural Perspective – Library as Place
www.arl.org
“In the physical [vs. virtual] reality, ‘texture’ has become important. Density of collections becomes important, and, if collections are not complete, users want to know where they can find missing volumes, journal articles, and/or how swiftly interlibrary loan will work for them” (Lincoln, p. 11).
Cultural Perspective - Collections
www.arl.org
“…as users have metamorphosed from penitents to self-reliant information surfers, the rules of engagement have changed. Service is not something dispensed; rather, it is enacted as an elaborate cultural ritual, the texture and fabric of which is changing in front of us. Service may now embody multiple overlays of meaning, many too dense for anything but an anthropological fieldwork study to uncover” (Lincoln, p. 15).
Service as Performance
www.arl.org
Think outside of the box
survive
