Transcript Document

What’s the Big Deal?
Collection Evaluation at the National Level
Clare Appavoo, Executive Director,
July 22, 2015
Canadian Research Knowledge Network (CRKN)
NAME OF EVENT
1
Introduction to CRKN
July 22, 2015
ALA 2013
2
Collection evaluation at the national level:
questions to be answered
Is there a benefit to
evaluating
collections
collaboratively?
July 22, 2015
Does the value of
journals and
packages remain
constant
notwithstanding
institution’s individual
characteristics?
ALA 2013
Is there an affordable
way to dismantle Big
Deals while
continuing to foster
research and
promote access?
3
Big Deal Exit Checklist
Leaving a Big Deal – Assess
potential institutional impacts
Leaving a Big Deal – Exit
implementation guidelines
Loss of
Access
Collection
Management
(Workflows)
Define
objectives
Interlibrary
Loan
Exiting a Big
Deal
Exiting a Big Deal
Consider
compliance
and
accreditation
needs
Needs for
teaching
Collections
Budget
Define
resources for
title-level
management
Communicate
with users and
other
stakeholders
Needs for
research
July 22, 2015
Define
resources for
transition
Identify titles
ALA 2013
4
Big Deal Cost-Per-Use
Gather COUNTER JR1
usage statistics
Decide what constitutes
“high-use” (e.g. 50, 100
uses)
Divide the # of high-use
titles by total package price
to determine maximum
“ideal price”
Compare ideal price with
list price. If list price is
lower than ideal price, a
move to individual
subscriptions may be
warranted
Determine average list
price for single
subscriptions using vendor
pricing data
Eg. if 300 titles are high
use, and the package price
was $100,000, the ideal
price is $333. Individual
subscriptions can be NO
MORE than $333
July 22, 2015
ALA 2013
5
Limits of usage and exploring the California
Digital Library Journal Value Metrics
Favors
resources in
STM
disciplines
Does not
consider
content
needs for
accreditation
Favors
resources
used for
teaching
Limits to the
cost-per-use
methodology as
an assessment
measure
Does not
consider
value solely
for research
Does not
consider
journal quality
Can change
with
unpredictable
spikes in use
patterns
July 22, 2015
The CDL Model
“How much value does our
institution derive from Journal X
compared to other journals that we
license in the same discipline?”
• Value is defined as quality,
utility and cost effectiveness
• Evaluating 8,600 journals in 36
licensed packages
• A numerical score is assigned
to each journal depending on
whether it is above or below the
benchmark in a subject area
ALA 2013
6
Journal Value Metrics - Method
Utility
Quality
Cost
Effectiveness
July 22, 2015
Full text
usage
Impact Factor
Cost per use
ALA 2013
Faculty
publications
SNIP (Source
Normalized
Impact per
Paper)
Cost per
SNIP
7
CRKN Pilot – Adapted Journal Value Metrics
Institution
FTE
Region
Library A
1,300
Ontario
Library B
3,300
Atlantic
Library C
4,200
Prairie/Pacific
Library D
12,500
Quebec
Library E
18,220
Atlantic
Library F
22,000
Ontario
Library G
27,000
Prairie/Pacific
Library H
29,000
Quebec
July 22, 2015
ALA 2013
8
Does the value of journals remain constant regardless of
varied institutional characteristics?
Comparing Quartiles
Top 100
journals
overall were
within the top
quartile for
each library
Top quartile
(340) for each
institution was
virtually
identical
Bottom
quartile (340)
for each
institution was
virtually
identical
July 22, 2015
Top 100 Journals
No. of titles in
institution’s top
100 and overall top
100
Library A
65
Library B
62
Library C
69
Library D
68
Library E
71
Library F
74
Library G
74
Library H
74
ALA 2013
9
Combined individual list price of the top quartile of
journals compared to total package price
July 22, 2015
ALA 2013
10
Do the top 100 journals as determined by JVM differ
from the top 100 journals as determined by use?
Discipline
Journals in top
100 - JVM
Discipline
Journals in top
100 - Usage
Medicine
31
Medicine
33
Life Sciences
22
Life Sciences
32
Social & Behavioral Sciences
16
Chemistry
7
Psychology
13
Social & Behavioral Sciences
5
Nursing, Dentistry & Healthcare
4
Psychology
5
Agriculture, Aquaculture & Food Science
3
Physical Science & Engineering
5
Business, Economics & Finance
3
Nursing, Dentistry & Healthcare
4
Mathematics
3
Agriculture, Aquaculture & Food Science
3
Earth & Environmental Science
2
Veterinary Medicine
3
Chemistry
1
Earth & Environmental Science
2
Computer Science
1
Mathematics
1
Law & Criminology
1
11
Do the top 100 journals as determined by JVM differ
from the top 100 journals as determined by use?
JVM – top 100 journals
July 22, 2015
Usage – top 100 journals
ALA 2013
12
Benefits of evaluating collections collaboratively
Determine
where to commit
consortia’s
resources
Shared
resources for
information
gathering
Data will
determine
negotiation
priorities
Shared
management of
journal data
July 22, 2015
ALA 2013
13
Journal Value Metrics – Next Steps and
Applications
Full text
usage
Utility
Findings:
Without movement from the
vendors on individual list prices, a
centrally negotiated Big Deal
continues to represent the best
value for a comprehensive
suite of
Quality
content.
Next Steps:
All publishers asked
Simplified
JVM(Source
SNIP
to provide
model forNormalized
use at
COUNTER JR1 stats
institutions
Impact per for all members
Paper)
Ensure the right Big
When evaluating participation in a
Evaluating all
Deals, containing the
Big Deal, or when financial
order to
right titles, are being
Cost- packages inCost
pressures necessitate cancellation,
per
use
for the right
usage is not a sufficiently
effectivenessmake comparisons? licensedprice
multifaceted unit of measure to
determine key journals for research
and teaching.
July 22, 2015
ALA 2013
14
Questions?
Clare Appavoo: [email protected]
Eva Jurczyk [email protected] for study specifics.