PM7Workshop_de Jager en Self

7/29/2019 PM7Workshop_de Jager en Self

1/42

Measures that matter:

Designing and developing yourown Balanced Scorecard

Karin de J ager & J im Self

7th Northumbria International Conference

on Performance Measurement

in Libraries and Information Services

An IFLA Satellite Conference


2/42

August 2007 7th Northumbria Conference, Spier,South Africa 2

Programme:

14h00 14h40:

Introduction to the Balanced Scorecard. The South Africanenvironment. Implementing a Scorecard with measures thatmatter. Karin de J ager

14h40 - 15h20:The Balanced Scorecard in practice: how it works and what it cantell us. Problems, pitfalls & solutions.J im Self

15h20 16h40: Small group work:Develop the outline of a Balanced Scorecard for a particularinstitution. Specify three metrics (with targets) for the four

quadrants. 16h40 17h30:

Reportback session with comment & feedback from speakers &participants.


3/42


Measures that matter in SA: an

Introduction Higher Education Evaluation in SA

Dealing with too much data

Assessing Quality Different Perspectives

The Balanced Scorecard

Measures & Targets Components of a Scorecard


4/42


Higher Education Evaluation

2004 HEQC responsible for Quality Assurance

Quality Audits mandated

CHELSA established own QA Subcommittee

Two documents: Measures for Quality; and a Guideto the Self Review of University Libraries, to assist

Large number of possible indicators. but not

prescriptive Individual libraries to select most appropriate for

own circumstances


5/42


Data, data everywhere

Over-complexity often a cause for failure

Useful information can drown in too muchdetail

Collecting and sorting through too muchinformation can become more trouble than itis worth

One way of attempting to manage this delugeof data, is to focus on only a small number ofmeaningful indicators


6/42


What is Quality?

Fitness for purpose

Never static or finally achieved

Accurate measurement essential If something cant be measured, it cant be

understood. If it cant be understood, it cant

be controlled; if it cant be controlled, it cant

be improved (Harrington)


7/42


Quality in libraries seen from

different points of view

The consumers of library services: Diverse in nature: from first year to advanced

research

Library workers: Have to be competent in wide range of services

Institutional funding authorities:

Objectives addressed & value for money? Overall strategic perspective: Ensure fitness for purpose into the future


8/42


Balanced Scorecard:

One approach

Commercial management tool

Defines organization's long-term strategy

Specific, measurable goals

From different perspectives

Attempts to integrate large amounts of diversedata into a single system


9/42


Scorecard perspectives

User perspective: "how well is the library meetingthe needs or expectations of its users?

Finance perspective: "how well are the library'sfinances managed to achieve its mission?"

Internal process perspective: "how do the library'sinternal processes function to deliver librarycollections and services efficiently?"

Future, learning or growth perspective: "how wellis the library positioned to ensure that goals are metin the future?"


10/42


Balanced Scorecard illustrated

Financeperspective

User

perspective

Processperspective

Future/Learning

perspective


11/42


Vision at the centre

E.g. a First Class African research andteaching university

What does this mean?

First class? Benchmark internationally

African?

How African are we really, and how can wedemonstrate & measure this?

Evidence of support for research & teaching?


12/42


Indicators must be measurable

Surveys Users, e.g. LibQUAL+

Availability

Financial expenditure Totals or per unit

Time expended E.g. on acquisitions, cataloguing, ILL

Counts and tallies of processes and services

Targets to see how well one is doing


13/42


Constructing a Balanced

Scorecard

Select a limited number of meaningfulindicators for each quadrant

Select targets for each indicator

Four quadrants:

User perspective

Internal processes perspective

Finance perspective

Future/growth perspective


14/42

August 2007 7th Northumbria Conference, Spier,South Africa

14

User Perspective (1)

User satisfaction

LibQual+ comparative data

satisfaction with the overall quality of the service

provided by the library

Adequate support for teaching & learning(HEQC)

Information literacy activities? How many classes or students are taught at

present?


15/42


15

User Perspective (2)

Adequate support for research?

ILL provides research materials for postgraduatestudents & academics

Extent of use:

Increase in use ofelectronic resources (increasein full-text downloads, or downloads per capita)

Access to library portal or web page, Loans per student?

Market penetration?


16/42


16

Internal Processes Perspective

Efficiency of library processes:

Turnaround time how long does it take to order;process; shelve?

Ease of accessing resources?

Usability testing?

Staff ratings of own internal services?

Internal surveys?

What do YOU want to manage & improve?


17/42


17

Financial Perspective

Library budget as % of institutional budget?

Unit costs & ratios, e.g.

Library expenditure per student? Books purchased per student?

Spend on e-resources as % of acquisitions spend?

Fundraising?

Set target amounts


18/42


18

Future/Growth Perspective

Staff development? Staff participation: further education;

Staff participation: in-service training

Staff participation: conferences, workshops Continuous alignment with institutional vision? African materials purchased?

Investment in infrastructure Replacement rate for library computers?

% of library computers/copiers/printers out ofaction?


19/42


19

Importance of Targets

Measure quantitatively

Set high but achievable targets

Consider two sets of targets: Complete success

Partial success

Aggregate regularly to provide feedback

Address problems that are revealed


20/42


20

Targets at a glanceUser Perspective

40%

40%

20%

Target met

Partial success

Target not met

Internal Perspective

34%

33%

33%

Target met

Partial success

Target not met

Finance Perspective

25%

50%

25%

Target met

Partial success

Target not met

Future/Growth Perspective

25%

25%

50%

Target met

Partial success

Target not met


21/42


21

The Balanced Scorecard in

Practice

Jim Self

Director, MIS

University of Virginia Library


22/42


22

The BSC at the U.Va. Library

Implemented in 2001

Results tallied FY02 through FY06

Completing metrics for FY08 Tallying results for FY07

A work in progress


23/42


23

Choosing the Metrics

--Reflecting Values

What is important?

What are we trying to accomplish?


24/42


24


--Diversity and Balance

Innovations and operations

Variety of measurements


25/42


25


--Ensuring validity

Does the measurement accurately

reflect the reality?


26/42


26


--Being Practical

Use existing measures when possible

Use sampling

Collect data centrally Minimize work by front line


27/42


27

What Do We Measure at U.Va.?

Customer survey ratings

Staff survey ratings

Timeliness and cost of service

Usability testing of web resources

Success in fund raising

Comparisons with peers


28/42


28

A closer look

Impact of the Scorecard


29/42

August 2007 7th Northumbria Conference, Spier,

South Africa29

Process improvementIdentifying and dissecting shortfalls


30/42


South Africa

30

Metric U.4.B: Turnaround time for

user requests

Target1: 75% of user requests for new booksshould be filled within 7 days.


Result FY06: Target1.

79% filled within 7 days.


31/42


South Africa

31

Metric VII: Turnaround time for

user requests



Result FY03: Target not met.

17% filled within 7 days.


32/42


South Africa

32

Accountability

Institutionalizing customer feedback


33/42


South Africa

33

Metric U.1.A: Overall rating in

student and faculty surveys

Target1: An average score of at least 4.00(out of 5.00) from each of the majorconstituencies.

Target2: A score of at least 3.90.

FY06 Result: Target1 Graduate students 4.08

Undergraduates 4.11


34/42


South Africa

34

Metric I.3.B.Staff Survey Rating

of Internal Customer Service

Target1: A composite rating of at least 4.00,with no unit rated below 3.50.

Target2: A composite rating of 3.50, with nounit below 3.00.


Composite score of 4.09. Individual unit ranged from 3.52to 4.57.


35/42


South Africa

35

Examining priorities


36/42


South Africa

36

Metric F.2.A: Unit Cost of

Electronic Serial Use

Target1: There should be no increase in unitcost each year.

Target2: Less than 5% annual increase inunit cost.

Result FY03-FY05: Target1.

Result FY06: Target not met.

8.8% increase ($2.10 vs. $1.93)


37/42


South Africa

37

Gaining resources


38/42


South Africa

38

Metric F.1.B. Library spending

compared to University

expenditures

Target1: : The University Library willaccount for at least 2.50% of the

Universitys academic division expenditures. Target2: : The Library will account for at

least 2.25% of expenditures. Result FY06: Target1.

2.57% ($25.2M of $972M)

Metric F 1 C Amount of


39/42


South Africa

39

Metric F.1.C.Amount of

unrestricted development receipts.

Target1: Increase unrestricted (or minimallyrestricted) giving by 10% each year.

Target2: Increase of 5% per year.


FY06 unrestricted receipts were $774,000; target was

$374,000.


40/42


South Africa

40

Metric L.2.C.Compare staff

salaries to peer groups.

Target1: Library faculty salaries should rankin the top 40% of salaries at ARL libraries.

Target2: Rank in top 50%.


Ranked 33 of 113. (Top 28%)


41/42


South Africa

41

To summarize

The Balanced Scorecard

Reflects the organizations vision

Clarifies and communicates the vision

Provides a quick, but comprehensive, pictureof the organizations health


42/42

August 2007 7th Northumbria Conference, Spier, 42

Assessing the BSC

What are the benefits?

Is it worth the costs?

Are we measuring the right things? What have we learned?

Where do we go from here?

Documents

PM7Workshop_de Jager en Self