LIBQUAL+® AND THE EVOLUTION OF LIBRARY AS PLACE AT RADFORD UNIVERSITY, 2002-2008 Eric Ackermann...

LIBQUAL+® AND THE EVOLUTION OF “LIBRARY AS PLACE” AT RADFORD UNIVERSITY, 2002-2008

Eric AckermannRadford UniversityLibrary Assessment ConferenceUniversity of WashingtonAugust 3-7, 2008

Introduction

Me, the 15% Assessment Librarian

Radford University McConnell Library

Background

Changes to Library as Place continuous since 2002

Examples Coffee shop Designated quiet areas Group study areas “No cell phone” zones New furniture Improved signage

Goal of this study

Are these physical changes reflected in our LibQUAL+® survey results?

Study design

Data types: Score (rankings) & comments Data source: 2002, 2005, 2006, 2008

LibQUAL+® surveys Respondent groups:

Faculty Graduate students Undergraduates

Level of analysis: “Library as Place” dimension

Methodology

Meta-analysis A data reduction method used to

combine results across studies using

Effect size meta-analysis Statistical procedure that converts

incomparable metrics into a common effect size metric.

Data analysis

Organize the data & respondent types into

comparison groupsFaculty: 2006 v. 2002Grad students: 2005 v. 2002, 2008 v.

2005Undergraduates: 2005 v. 2002, 2008 v.

2005All groups: 2005/6 v. 2002, 2008 v. 2005/6

Data analysis (con’t)

Determine effect size for each comparison group using mean adequacy gap (score data) odds ratio (positive: negative

comments) Determine final weighted average

effect size d for each respondent group for 2002-2008

Results reported as d, .95 CI, and BESD

How did we measure success?(aka, Practical significance)

Practical significance

SuccessLevel Significanc

e

Action criteria Action

d BESD

Complete

Userssatisfied

0.31 or

more

15.1% or

moreCelebrate!

PartialUsers

somewhatsatisfied

0.3 to -0.3

15% to -15%

Look for improvement

s;implementchanges

Un-success

ful

Usersunsatisfie

d

-0.31 or

less

-15% or less

High Priority review/chang

e

What did we expect to find?

In the literature…

Nothing found about specifically using LibQUAL+® to measure satisfaction with changes in Library as Place

All reports found about measuring change due to drastic overhaul of building or new construction.

Conclusion…

If dramatic changes to the Library as Place yield dramatic, immediate increases in user satisfaction, then incremental changes will yield modest gains in user satisfaction over time.

What did we actually find?

Response rate & representativeness Response rate (average of 2002-2008)

Faculty: 15.7% Grad students: 22.4% Undergraduates: 26.4%

Representativeness (average of 2002-2008) Faculty: 8.2% Grad students: 7.4% Undergraduates: -13.1%

So what?(Practical significance again)

Practical significance

Partial success (Users somewhat satisfied) Action Criteria: d + 0.30 (or BESD + 15%)

Faculty (0.01 or 0.6%) Grad students (0.19 or 9.2%) Undergrads (0.08 or 4.1%)

Action Looked for improvements (e.g., examined

“Suggestion” coded comments, conducted focus groups)

Implemented changes (see next slide…)

Review/change

Improvements since 2008 LibQUAL+ survey Created Front Desk from Media Services

and Circ desks Reorganized quiet study floor Reduced noise level on quiet study floor

More improvements planned New furniture for the lobby Walls to be painted Reference area reorganization

Conclusion

Effective Method is statistically defensible Told us what we did (and did not) want to hear

Practical Metrics have practical meaning at the local

level Provide criteria for actionable results

Sustainable Commitment to assessment by library

administration Funded by university

Questions?

Contact information

Eric AckermannReference/Instruction and Assessment

Librarian egackerma@radford.edu 540-831-5488 Box 6881, McConnell Library, Radford

University, Radford, VA 24142