What’s the Big Deal? Collection Evaluation at the National Level

April 21, 2023 NAME OF EVENT 1

What’s the Big Deal? Collection Evaluation at the National Level

Clare Appavoo, Executive Director,

Canadian Research Knowledge Network (CRKN)

April 21, 2023 ALA 2013 2

Introduction to CRKN

Collection evaluation at the national level: questions to be answered

April 21, 2023 ALA 2013 3

Big Deal Exit Checklist

Leaving a Big Deal – Assess potential institutional impacts

Leaving a Big Deal – Exit implementation guidelines

April 21, 2023 ALA 2013 4

Big Deal Cost-Per-Use

April 21, 2023 ALA 2013 5

Limits of usage and exploring the California Digital Library Journal Value Metrics

The CDL Model

“How much value does our institution derive from Journal X compared to other journals that we license in the same discipline?”

•Value is defined as quality, utility and cost effectiveness

•Evaluating 8,600 journals in 36 licensed packages

•A numerical score is assigned to each journal depending on whether it is above or below the benchmark in a subject area

April 21, 2023 ALA 2013 6

Journal Value Metrics - Method

April 21, 2023 ALA 2013 7

CRKN Pilot – Adapted Journal Value Metrics

Institution FTE Region

Library A 1,300 Ontario

Library B 3,300 Atlantic

Library C 4,200 Prairie/Pacific

Library D 12,500 Quebec

Library E 18,220 Atlantic

Library F 22,000 Ontario

Library G 27,000 Prairie/Pacific

Library H 29,000 Quebec

April 21, 2023 ALA 2013 8

Does the value of journals remain constant regardless of varied institutional characteristics?

Top 100 JournalsNo. of titles in institution’s top 100 and overall top 100

Library A 65

Library B 62

Library C 69

Library D 68

Library E 71

Library F 74

Library G 74

Library H 74

Comparing Quartiles

April 21, 2023 ALA 2013 9

Combined individual list price of the top quartile of journals compared to total package price

April 21, 2023 ALA 2013 10

Do the top 100 journals as determined by JVM differ from the top 100 journals as determined by use?

Discipline Journals in top 100 - JVM

Medicine 31

Life Sciences 22

Social & Behavioral Sciences 16

Psychology 13

Nursing, Dentistry & Healthcare 4

Agriculture, Aquaculture & Food Science 3

Business, Economics & Finance 3

Mathematics 3

Earth & Environmental Science 2

Chemistry 1

Computer Science 1

Law & Criminology 111

Discipline Journals in top 100 - Usage

Medicine 33

Life Sciences 32

Chemistry 7

Social & Behavioral Sciences 5

Psychology 5

Physical Science & Engineering 5

Nursing, Dentistry & Healthcare 4

Agriculture, Aquaculture & Food Science 3

Veterinary Medicine 3

Earth & Environmental Science 2

Mathematics 1

Do the top 100 journals as determined by JVM differ from the top 100 journals as determined by use?

JVM – top 100 journals Usage – top 100 journals

April 21, 2023 ALA 2013 12

Benefits of evaluating collections collaboratively

April 21, 2023 ALA 2013 13

Journal Value Metrics – Next Steps and Applications

Findings:

Without movement from the vendors on individual list prices, a centrally negotiated Big Deal continues to represent the best value for a comprehensive suite of content.

When evaluating participation in a Big Deal, or when financial pressures necessitate cancellation, usage is not a sufficiently multifaceted unit of measure to determine key journals for research and teaching.

April 21, 2023 ALA 2013 14

Next Steps:

Questions?

Clare Appavoo: cappavoo@crkn.ca

Eva Jurczyk ejurczyk@crkn.ca for study specifics.