An analysis of the performance of research libraries in Poland

– project description

Elżbieta GórskaThe Polish Librarians’ Association

Leistungsmessung und Benchmarking in BibliothekenInternationale Konferenz, Bratislava 23.9.2008

Library statistics and library standard in Poland

there is no institution to develop tools and methods of evaluating the work of libraries.lack of uniform criteria with which to assess and compare libraries performance. The Central Statistical Office investigates the work of libraries only fragmentarily, CSO sends out questionnaires to libraries every two years

– lack of financial aspects of the performance of libraries, – lack of their infrastructure and such areas as educational activity

and publishing – the data gathered in such a way is insufficient for comparative

studies.The State Accreditation Commission, which evaluates the level of education in given subjects, monitors the performance of libraries merely on a general level, Rankings of educational establishments cause much annoyance and dissatisfaction among librarians.

Research Libraries in Poland

1. The National Library,2. The academic libraries,3. The libraries run by the Polish Academy

of Sciences,4. The libraries of branch R&D units5. Selected public libraries6. Other libraries listed by the Minister

of Culture.

Task Group for Standardization

appointed in 2001, initially as an informal group Since 2004 based on “Agreement on Realising the ‘Analysis of the Performance of Research Libraries in Poland’ Project,” The Group currently consists of 10 representatives of research libraries, representing academic, public and special libraries. The project coordinator is the Library of the University of Poznań, and it is partly financed by the Ministry of Science and Higher Education. the Group’s work has been a subject of interest by and support from the Conference Library Directors of Academic Schools in Poland

The general objectives of the project

to formulate methods of evaluating the performance of research libraries in Poland, to obtain data for a computer database in order to define indicators and standards for research libraries in Poland, to compare the results of analyses with the indicators and standards governing the performance of research libraries abroad, to prepare and publish annual reports.

International standards and recommendations used

ISO 11620:1998 AD1:2003 Information and documentation. Library performance indicators=PN-ISO 11620:Informacja dokumentacja – Wskaźniki funkcjonalności bibliotek

ISO/TR 20983:2003 Information and documentation. Performance indicators for electronic library services PN-EN ISO 2789:2005 Informacja i dokumentacja –

Międzynarodowa statystyka biblioteczna

Poll R.,te Boekhorst P.Measuring Quality.International Guidelines for Performance Measurement in Academic Libraries. Polish ed. Wrocław 2004

Questionnaire (1) We ask about the environment, processes and services in the following spheres:

staff, collection, budget, infrastructure, circulation, reference services, educational activity, publications and databases, library cooperation, events,staff professional activities,

selected performance indicators

Questionnaire(2)

48 different type questions – statistical data e.g. size of collection, number of users – questions about services YES/NO answers e.g. electronic

ordering of books 86 indicators

– 19 provided by libraries – 67 evaluated automatically – list of indicators

http://ssk2.bu.amu.edu.pl/standaryzacja/AFBN_wskazniki.pdf

Performance indicators – why so many?

the need for a comprehensive analysis of the performance of libraries, the need to include in the research all the aspects covered by the questionnaires, plans to produce standards governing the evaluation of libraries in successive years, on the basis of the indicators currently analysed, a “triple” calculation of selected indicators – there is no uniform definition of a statistical student.

The performance indicators provided by libraries – examples

the percentage share of specific expenditures in the overall budget, the percentage share of specific revenues in the overall budget, expenditures by libraries per student/user, expenditures on the purchase of collection/books per student/user, book processing speed, the percentage of books produced with the aid of computers, the percentage of catalogue records taken from outside resources.

The software

libraries can complete, modify and send out questionnaires on-line, selected performance indicators can be calculated automatically on the basis of the information on the questionnaire. the correct completion of the questionnaire can be verified automatically, the Task Group for Standarization may conduct comparative research, each library can analyse the data and the performance indicators for a given year or for several years, and compare them with the mean values of libraries of the same type.

A fragment of the questionnaire form

Explanations

Numerical data

Yes/No question

Most important performance indicators computed automatically

ratio of total library staff to the university staff ratio of users registered to potential library users number of items/books added per student/user number of students/users per library staff member total library area per one student/user number of students/users per one study place in reading rooms number of loans per registered user number of loans per library staff member ratio of interlibrary loans to the total amount of loans.

Internet application

website – information about project, a set of instructions, a questionnaire, useful links, survey resultsmodule for libraries

– work with questionnaire – analysis of one’s own library data – compare own library’s data and performance indicators

with “average type” the same library type an administrator’s module – registration of libraries, contacts with librarians the database to incorporate and register data from questionnaire a module for the Task Group for Standardization – statistical analyses on data and performance indicators

About project

Set of instructions

Questionnaires (pdf)

Surveyresults

Access to questionnaire (password)

List of indicators

Analysis for Groupor registered library

Choice of indicator

or data for group

Analysis results

Own library result

Other libraries’ results

indicator

Analysis by the Task Group results

Results by library types

Indicator

Participation of libraries in the survey by number of questionnaires certified in 2002–2006

Libraries year

2002 2003 2004 2005 2006

State universities 6 8 8 8 8

State technical universities 8 10 10 10 12

Other state universities (economic, medical ) 1 4 6 5 4

Non state universities 1 3 3 14 15

Public libraries 1 2 2 3 3

others – 2 2 3 2

17 29 31 44 44

Participation of libraries in the survey

Registered libraries(August 2008) – 94includes:

87 academic libraries: 55 state and 32 non state 4 public libraries 3 others

Certified questionnaires by year 2002 – 17 (including 17 academic libraries) 2003 – 29 ( 25 academic) 2004 – 31(27 academic) 2005 – 44 (38 academic) 2006 – 44 (39 academic)

Difficulties with completing the questionnaire

an absence of the statistical data needed to complete the questionnaire, an absence of comparative data on the use of electronic sources (differences in the statistics generated by providers), divergent organisational structures and models of financing educational establishments, difficulties with certifying the questionnaires – errors requiring correction, a misunderstanding of data, misinterpretation of questions.

The collective analysis of the data

available to the public on the project websiteindicators from the questionnaires (calculated by the applicants) indicators calculated by the programme automatically, the percentage share of budget expenditures, data broken down into library categories, results: arithmetical average, median value, maximum and minimum values.

Analysis by the Group –

By library types,financing,years

Database statistics and survey results available to the public

The results of analysis by type of indicators

• general, • financial, • indicators regarding

collections, • indicators regarding

services and their use • indicators regarding

staff, organisational structure and vocational activity,

• remote catalogue services, own publications, cooperation between libraries, events.

The results of the analysis by the Task Group – example

Use of the results of the surveys

registered libraries which submit questionnaires,

Task Group for Standardization,

librarianship practicians and theoreticians,

institutions which assess libraries.

Plans

the production of a domestic set of performance indicators in compliance with ISO 11620,

a survey into user satisfaction on the basis of a standard questionnaire,

the formulation of a nationwide set of standards and a definition of indicators for each of them.

Thank you