Building on sand?Using statistical measures to assess

the impact of electronic services

Dr Angela Conyersevidence base

research & evaluationUCE Library Services

UCE Birmingham

• Why collect usage statistics?• What statistics to collect?• How to collect them?• How to analyse them?• How can libraries be assisted in

this task?

Key questions

• E-measures project funded by HEFCEworking with 25 UK higher education librarieswww.ebase.uce.ac.uk/emeasures/index.htm

• NESLi2 study of usage statistics funded by JISC working with 17 UK higher education librarieswww.ebase.uce.ac.uk/projects/NESLi2.htm

• Because they’re there• Because SCONUL has asked for them• To help with promotion and user

support• For budgeting and decision-making• To aid bench-marking

Why collect usage statistics?

1. Identify type of resource:

• Serials• Databases• E-books• Digital documents• Virtual visits• Electronic enquiries

What usage statistics to

collect?

2. Identify reliable statistics:

COUNTER compliance:

Serials: JR1 report – number of full-text article requests – 40 publishers/suppliers

Databases: DB1 report – total searches and sessions by month and database – 13 publishers/suppliers - DB2 report – total searches and sessions by month and service – 15 publishers/supplier

E-books: new draft COUNTER code of practice

What usage statistics to

collect?

• Identify sources of statistics• Collect passwords• Download• Establish where gateway statistics to be

added• Encourage all suppliers to be COUNTER

compliant

How to collect?

1. Identify supplementary information required:

– Cost of deal (subscribed titles and e-access costs)

– Lists of subscribed titles– Number of FTE users– Total library serials budget

How to analyse?

2. Identify groupings:

– Usage range– Price band– Subject category– Subscribed/unsubscribed titles

How to analyse?

Nil and low use (under 10 requests)nil use1-4 requests5-9 requests

Medium use (10-99 requests)10-49 requests50-99 requests

High use (100 or more requests)100-499500-9991000-999910,000 plus

Usage range

Unpriced: those for which no price could be found

Low price – under £200

Medium price - £200-399

High price - £400-999

Very high price - £1,000 and over

Price band

Subject category:STM (science, technology & medicine)

HSS (humanities and social sciences)

Subscribed and unsubscribed titles

Other groupings

• Average cost per request – all requests• Average cost per request – subscribed titles• Average cost per request – unsubscribed

titles• Cost per FTE user• Cost per title• Yield per £ for access to unsubscribed titles

Assessing value for money

NESLi2 study main recommendations to the JISC:

– Clearer, more transparent deals– Review of pricing and cancellations policies– Portal site for publisher usage statistics– More support to libraries in monitoring usage-

toolkit, workshops etc

How can libraries be assisted with

the task?

NESLi2 study main recommendations to the higher education library community:

– Libraries to ensure they have sufficient internal resources to manage and analyse usage data

– Libraries to be encouraged to take a more joined-up approach to the management of print and e-journal resources

How can libraries be assisted with

the task?

• Workshops• Tool-kits• Benchmarking groups• Outsourcing

Angela ConyersAngela.Conyers@uce.ac.uk

www.ebase.uce.ac.uk

The future?

Some results –full-text article requests

Total usage (all libraries, all publishers)

0

500000

1000000

1500000

2000000

2500000

Jan-Jun 2003 Jul-Dec 2003 Jan-Jun 2004

Some results – costs

Cost ranges of titles

0%

5%

10%

15%

20%

25%

30%

no price Low (under£200)

Medium(£200-399)

High (£400-999)

Very high(£1000 or

over)

Some Conclusions

• Number of full text article requests considerably higher in large old universities in this study

• Overall increase in usage 2003-4• Balance of usage and costs generally led to similar range of

costs across large and small universities• Low average cost per request• Small % of titles generated high usage• Low use titles were mostly low price• High use titles were generally high cost• STM titles most heavily used.• Data limitations