BIX - The Library IndexBIX - The Library Index

Roswitha PollChair of ISO TC 46 SC 8: Quality – Statistics

and performance evaluation

Roswitha PollChair of ISO TC 46 SC 8: Quality – Statistics

and performance evaluation

Hamar2010

a benchmarking system for public and academic libraries

since 1999 (public libraries) resp. 2004 (academic libraries)

participation voluntary, but the indicators a must

annual fee: 170 €

organized by: Bertelsmann Foundation (start); German Library Association (DBV)

Participants 2009: 257 libraries• 177 public libraries (of ca. 2.200)• 80 academic libraries (Germany: 68 of ca. 350; 12 from Austria))

BIX – The Library Index

2 separate indices (public/academic)

17 indicators each

4 dimensions = Balanced Scorecard • resources • customer focus (use) • efficiency • development (potentials)

BIX – The Library Index

Kaplan, R.S./Norton, D.P.:Kaplan, R.S./Norton, D.P.: The Balanced Scorecard: Translating Strategy into Action.

Boston 1996

4 dimensional scores and ranks

1 overall score and ranked list

8 categories - public libraries by size (5 categories) - academic libraries by organisation - integrated library systems

- two-tier-systems (central libraries only) - polytechnics

BIX – Ranking

Indicators for academic libraries are not

weighted !

Library

BIX results

LibraryLibrary

Total rankTotal rank

Total scoreTotal score

resourcesrank and score

resourcesrank and score

customer focusrank and score

customer focusrank and score

Indicatorvalues

Indicatorvalues

Indicatorvalues

Indicatorvalues

efficiencyrank and score

efficiencyrank and score

► BIX-magazine► press releases► blueprints for participants‘ press releases ► seminars► individual analysis /consultancy

► BIX website

BIX – Publication of results

http://www.bix-bibliotheksindex.de/

BIX-Magazine

•Tables of results

•Best Practices

•Stories

•Testimonials/ Interviews

Methods of quality assessment in libraries

Performance indicatorsmeasure the effectiveness and cost-efficiency of library services: quantitative, objective

User satisfaction surveysmeasure the perceived quality, the users‘ impression of library services: qualitative, subjective

Outcome assessmenttries to show the benefits, the value for individual users and society

BIX – benchmarking with performance indicators

The performance indicators• Should have informative content: show whether the service or activity that is „measured“ is „good“ or „bad“

• Results should be comparable between libraries of similar structure and

clientele

• The data for the indicators should for the

most part be taken from the national library statistics: practical

Only a few indicators - but for all

services and all

stakeholders

Collections of performance indicators for libraries

ISO 11620, 2nd ed. (2008), Information and documentation – Library performance indicators (45 indicators)

ISO TR 28118 (2008), Information and documentation – Performance indicators for national libraries (30 indicators)

Poll, R. and te Boekhorst, P., 2nd ed. (2007), Measuring quality, performance measurement in libraries, Saur, München (IFLA Publications 127) (40 indicators)

BIX indicators for public libraries

Resources• Media in the collection per capita• User area in m2 per 1,000 capita• Library staff per 1,000 capita• Workstation hours available per capita• Internet services• Events per 1,000 capita

Customer Focus (Use)• Visits per capita• Loans per capita • Collection turnover rate• Opening hours per year per 1,000 capita

Efficiency• Acquisitions budget per loan• Employee hours per opening hour• Visits per opening hour• Total expenditure per visit

Development• Renewal rate (additions to stock) • Training as percentage of all employee hours• Investment budget per capita

Indicators for public libraries:Internet services

Sum of yes-answers

• Library website • WEB–OPAC • Interactive functions (account, reservations, etc.) • e-mail reference service • Electronic collection • Active information services (news, events…)

Probably most public libraries will have such services

Resources / Infrastructure

Square meters of user area

Library staff

Expenditure on literature and information

per 1,000 membersof the population

Percentage of that expenditure spent on the electronic collection

Opening hours per week

Academic libraries

Usage

Library visits per capita(physical visits)

Electronic usage(virtual visits; central counting of homepage and OPAC page

visits)

User training attendances per 1,000 members of the population

Immediate availability(Immediate loans as a percentage of total loans including reservations and ILL)

User satisfaction rate(Identical online survey in all libraries, not used at the moment)

Efficiency

Library expenditure per active user(acquisitions, material costs, staff)

Ratio of acquisitions expenditure to staff costs

Efficiency of processes(Example: Media processing. Processed media per FTE staff)

Efficiency of processes(Example: Lending. Processed loans and ILL per FTE staff)

Development (Potentials)

Days of training per staff member

Percentage of the university budget allocated to the library

Percentage of library means received through third-party funds, special funds and income generation

Percentage of library staff providing and developing electronic services

BIX - A method for counting virtual visits

Counting sessions on the homepage and the Counting sessions on the homepage and the OPAC start page by a counting pixel downloadedOPAC start page by a counting pixel downloadedfrom a central serverfrom a central server

(Sebastian Mundt,(Sebastian Mundt, Stuttgart Media University Stuttgart Media University mundt@hdm-stuttgart.de)mundt@hdm-stuttgart.de)

The advantage of this method is that it easy to use The advantage of this method is that it easy to use

The disadvantage is that access to other pages The disadvantage is that access to other pages

is not countedis not counted

Advantages of benchmarking

"The value of benchmarking as a proven tool to achieve quality management should be rated very highly indeed"

► Explains the library‘s own results

► Shows „best practice“

► Detects problems in processes and organisation

► Higher attention for the results in the public, the media and funding institutions

► Higher credibility of the library’s reports

BIX: Survey 2010

“More indicators”, “less indicators” and “anonymous participation” all lower than 40 %.

► 1.570 libraries; responses 694 = 44 %

► Which data are most important for your daily work? 1. A practical user satisfaction survey 2. Data showing cost-efficiency 3. Reliable measuring for use of electronic services 4. Data for impact and outcome

► In what direction should BIX develop? 1. More participants (ca. 80 %) 2. No ranking, only groups (ca. 51 %) 3. Additional analysis of the results (ca. 47 %)

BIX: Satisfaction with the system(1 to 5, 1 = best)

Results presented as ranking 1,9

The set of indicators 2,4

Number of participants / option of comparison within my group 2,9

Representativeness of the results 3,0

Usefulness for external presentation 2,2

Internal effect for management 2,3

Transparency / tangible calculations and results 2,7

Effort of data collection in the library 2,3

Organisation of the BIX procedures 1,9

Cost-benefit ratio 2,2

Play alone and you're bound to win

Arabian Proverb