Making Library Assessment Work: Practical Approaches for Developing and

Sustaining Effective AssessmentPhase I Update

ARL VISITING PROGRAM OFFICERS 2004-06Steve Hiller

Library Assessment Coordinator, University of Washington Jim Self

Director, Management Information Services, University of Virginia &

Martha KyrillidouARL, Director of Statistics and Measurement Program

ARL New Measures 1999- Key Areas

• User Satisfaction/Service Quality – LibQUAL+™– Measuring Library Service Quality Lyceum

• Market Penetration • Ease and Breadth of Access • Library Impact on Teaching and Learning

– Project SAILS, Learning Outcomes

• Library Impact on Research • Cost Effectiveness • Library Facilities/Space • E-Metrics

– MINES for libraries– ARL supplementary statistics

Barriers to Management Information and Using Data Effectively in Libraries

• Organizational culture/leadership support• Time/Staff/Resources• Data issues – Too much, compatibility, validity• Establishing priorities• Knowing what to measure and methods to use• Inexperience, perceived lack of skills and expertise• Understanding, presenting and knowing what to do

with the results

Hiller, S. and Self, J. (2004). From Measurement to Management: Using Data Wisely for Planning and Decision-Making. Library Trends.

Making Library Assessment Work: Practical Approaches for Developing and

Sustaining Effective Assessment

• 2 year ARL project to assist libraries with moving assessment efforts forward

• Led by Steve Hiller (UW) and Jim Self (UVa) • Recognition that libraries understand value of assessment

but have trouble sustaining efforts and using results• Site visits conducted to evaluate assessment and develop

practical approaches to effective local practices• Identify common barriers and facilitators to assessment• 7 libraries participating in Phase I Spring 2005; 12-14

during 2005-06 academic year; final report in 2006

Phase I Participants

• University of Arizona February 2005• Arizona State University February 2005• University of Connecticut March 2005• University of Illinois at UC May 2005• New York University March 2005• University of Notre Dame June 2005• University of Oregon May 2005

Phase I Process

• Each library chooses a contact person • Survey sent to contact person 2 months before visit

– Assessment activities, motivators, organizational structure– What’s worked/hasn’t, specific areas to address, expectations

• Schedule of meetings established• Site visit• Short report completed within 30 days

– Specific recommendations made on moving forward

• Library evaluates process after final report received

Sample Site Visit Schedule

• Meet with University Librarian/Contact person• Presentation on effective assessment

– 90 minutes to 2 hours with Q&A and discussion

• Group Meetings– with management/administrative group

– assessment-related group (if formed

– different departments

– functional areas (e.g. info literacy)

• Wrap-up

ARL ProjectPreliminary Observations after Phase I Visits

• No “cookie-cutter” libraries; diverse organizational cultures • Sustainability seen as resource issue• Management information system seen as crucial underpinning• Integrating data use effectively into library management• Many internal statistics (non-ARL related) kept• Few libraries allocate sufficient resources (staff and funding) for

assessment or management information• Facilities renovation as catalyst to good assessment practices• Good assessment work being done but not published/

disseminated or coordinated• Need for basic/practical training on assessment methods/analysis• Little understanding of what’s happening outside the library in

data warehousing/management

Phase I Feedback from Participants

• One day is too short • More resource materials would be helpful• More examples presented• Follow-up activity would maintain momentum• Establishing an assessment-related “community” of

practitioners would assist ongoing communication and development

Phase II Changes

• Site visit time increased from 1 day to 1 1/2 days• Appropriate resource materials provided• Consulting on a follow-up activity

– Assessment plan implementation

– Specific assessment effort

• Individual meeting at a professional conference • Group meeting at a professional conference• Stronger project Web presence

Phase II Participants

• Confirmed– Cornell

– Emory

– Kansas

– Louisville

– Pennsylvania State

– Southern California

– Texas Tech

– Wayne State

– Western Ontario

• Willing to commit (not confirmed)

– Boston Public Library

– Colorado State

– Nebraska

– Purdue

• Interest pending (maybe)

– Alabama

– Alberta

– Marquette

Future Reporting

• Upcoming Presentations– Northumbria International Conference on Performance

Measurement, Durham, England, August 22-25– 3rd International Evidence-Based Librarianship Conference,

Brisbane, Australia, October 16-19– ARL Membership Meeting, Washington, D.C., October 25-28

or in May 2006• Making Library Assessment Work Web Site

– http://www.arl.org/stats/newmeas/VPO_Hiller_Self.html• Final Report late summer 2006• ARL Newsletter• Conference in Fall 2006?