Transcript IFLA 2001 Presentation

Next Generation Statistics for Libraries:
CAUL Forum
20 January 2003
Sydney, NSW
The ARL E-Metrics Project
Mary E. Jackson
Senior Program Officer for Access Services
Association of Research Libraries
[email protected]
Association of Research Libraries
www.arl.org
ARL New Measures Initiative
• Collaboration among member
leaders with strong interests
• Specific projects explore different
models
• Self-funded projects
Association of Research Libraries
www.arl.org
ARL New Measures Initiative
• Make resulting tools and
methodologies available to full
membership and wider community
• Continue to collect, but limit/freeze
modifications to existing
descriptive measures
Association of Research Libraries
www.arl.org
ARL New Measures Projects
• Demonstration project for service
effectiveness measures
– LibQUAL+
• Identification of measures that
demonstrate a library’s contribution to
student learning outcomes
• Investigation of role libraries play in
support of the research process
Association of Research Libraries
www.arl.org
ARL New Measures Projects
• Development of tools to address
cost effectiveness of library
operations
– staff allocation, Assessing ILL/DD
Services study
• Project to define usage measures
for electronic information
resources
– E-metrics Project
Association of Research Libraries
www.arl.org
Electronic Serials Expenditures
Total Electronic Serials Expenditures, 1995-2001
140,000,000
120,000,000
100,000,000
80,000,000
60,000,000
40,000,000
20,000,000
1995
1996
1997
1998
1999
2000
Year
Association of Research Libraries
www.arl.org
2001
Percentages of Acquisitions Dollars
Devoted to Electronic Resources
1 8. 00%
1 6. 00%
1 4. 00%
1 2. 00%
1 0. 00%
8. 00%
6. 00%
4. 00%
2. 00%
0. 00%
1 993
1 994
1 995
1 996
Association of Research Libraries
1 997
1 998
1 999
2000
www.arl.org
2001
Average Yearly Increases in Electronic
Resources and Total Library Materials
Expenditures
50
47.46%
45
40
37.28%
32.69%
35
27.06%
30
23.22%
24.81%
22.5%
25
20
12.86%
15
8.48%
10
9.18%
9.12%
5.53%
4.18%
5
6.35%
3.4%
4.32%
0
1992-93
1993-94
1994-95
1995-96
1996-97
Average Increase in Library Materials Exp end itu res
Association of Research Libraries
1997-98
1998-99
1999-2000
2000-01
Average Increase in Electronic Resou rces Exp end itu res
www.arl.org
Project Leaders
• Co-Chairs
– Rush Miller, University of Pittsburgh, and Sherrie
Schmidt, Arizona State University
• Information Use Management and Policy
Institute, Florida State University
– Charles R. McClure, Wonsik “Jeff” Shim, and
John Carlo Bertot
• ARL
– Duane Webster and Martha Kyrillidou
Association of Research Libraries
www.arl.org
Project Participants
•
•
•
•
•
•
•
•
•
•
•
•
Alberta
Arizona State
Auburn
Chicago
Connecticut
Cornell
Illinois - Chicago
Library of Congress
Manitoba
Maryland
Massachusetts
Nebraska
Association of Research Libraries
•
•
•
•
•
•
•
•
•
•
•
•
New York Public
Notre Dame
Pennsylvania
Penn State
Pittsburgh
Purdue
Southern California
Texas A & M
Virginia Tech
Western Ontario
Wisconsin
Yale
www.arl.org
Vendor Statistics Working Group
12 major ARL vendors met with project team
in Denver prior to 2000 ACRL Meeting
Academic Press/IDEAL *
Elsevier/Science Direct
Lexis/Nexis
Ovid
Bell & Howell
Gale Group
ISI * †
netLibrary
Silver Platter *
EBSCO
JSTOR
OCLC/First Search
* Unable to attend Denver Meeting
† Nonparticipant in project.
Association of Research Libraries
www.arl.org
Working Definition of
Networked Services
Electronic information resources and/or
services that users access electronically
via a computer network:
1. From on-site in the library
2. Remote to the library – but from a
campus facility
3. Remote from the library & campus
Association of Research Libraries
www.arl.org
Networked Information
Resources
• Locally Licensed Databases
• Regional or Statewide Consortia
Licensed Databases
• Aggregated Databases
• Publishers Databases
• Publicly Available (Web) Resources
Association of Research Libraries
www.arl.org
ARL E-Metrics Project:
Phase 1
May-October 2000
What do we know?
Inventory of current practices at ARL
libraries of statistics, measures,
processes, and activities that pertain to
networked resources and services.
Association of Research Libraries
www.arl.org
ARL E-Metrics Project:
Phase 2
November 2000-June 2001
What can we collect?
Identified and field tested an initial draft set
of statistics and measures.
Association of Research Libraries
www.arl.org
ARL E-Metrics Project:
Phase 3
July 2001-December 2001
What difference does this make?
Build linkages to: educational outcomes/impact,
research, technical infrastructure.
Association of Research Libraries
www.arl.org
Recommended Statistics &
Measures
•
•
•
•
•
Patron Accessible Electronic Resources
(R1-3)
Use of Networked Resources & Services
(U1-5)
Expenditures for Networked Resources &
Related Infrastructure (C1-3)
Library Digitization Activities (D1-3)
Performance Measures (P1-3)
Association of Research Libraries
www.arl.org
Patron Accessible
Electronic Resources
• R1 – Number of electronic full-text
journals
• R2 – Number of electronic reference
sources
• R3 – Number of electronic books
Association of Research Libraries
www.arl.org
Use of Networked Resources &
Related Infrastructure
• U1 – Number of electronic reference
transactions
• U2 – Number of logins (sessions) to
electronic databases (db)
• U3 – Number of queries (searches) in
electronic db
• U4 – Items requested in electronic db
• U5 – Virtual visits to library’s website and
catalog
Association of Research Libraries
www.arl.org
Expenditures for Networked
Resources & Related Infrastructure
C1  Cost of electronic full-text journals
C2  Cost of electronic reference sources
C3  Cost of electronic books
C4  Library expenditures for bibliographic
utilities, networks, & consortia
• C5  External expenditures for bibliographic
utilities, networks, & consortia
•
•
•
•
Association of Research Libraries
www.arl.org
Library Digitization Activities
• D1 – Size of library digital collection
• D2 – Use of library digital collection
• D3 – Cost of digital collection
construction & management
(Collecting these data requires staff familiar with the digital
environment.)
Association of Research Libraries
www.arl.org
Performance Measures
• P1 – Percentage of electronic
reference transactions of total
reference
• P2 – Percentage of virtual visits of all
library visits
• P3 – Percentage of electronic books to
all monographs
Association of Research Libraries
www.arl.org
Characteristics of Each
Recommended Measure
•
•
•
•
•
•
•
Definition
Rationale
Unit of Measure
Data source
Frequency
Process
Related Issues
Association of Research Libraries
www.arl.org
R1 – Number of Electronic
Full-text Journals
• Definition - Number of electronic full-text
journal subscriptions – by individual
institution or consortia licensing.
• Rationale – Documents degree of
expansion of electronic subscriptions
available – can be used to show good
coverage & need for more funding.
• Unit of measure – the journal subscription
Association of Research Libraries
www.arl.org
R1 – Number of Electronic
Full-text Journals
• Data source – local or vendors.
• Frequency – annual, monthly, etc.
• Process – parse into database or
spreadsheet, update dynamically from
local catalog or vendor record.
Association of Research Libraries
www.arl.org
U1 – Number of Electronic
Reference Transactions
• Definition - number of electronic reference
transactions – via e-mail, WWW form, etc.
• Rationale – libraries are interested in
tracking the development of new electronic
services. Attempt to measure reference
transactions through new electronic tools
and services.
• Unit of Measure – request count, time it took.
Association of Research Libraries
www.arl.org
U1 – Number of Electronic
Reference Transactions
• Data Source – local server, manual tally, email count.
• Frequency – daily, monthly, annually, etc.
• Process – clarify process, identify activity
points, identify collectors of data, consolidate
data.
• Related Issues –This measure may have to
broken down into additional data types –
time, type of query, type of interaction,
scheduling issues, measures of quality and
reliability.
Association of Research Libraries
www.arl.org
Outcomes of the
Investigation
• Mixed results
• Participants wanted more definitive answers
• Confirmed how difficult, but not impossible,
to come to consensus
• Realized how dependent libraries are for
vendors to supply data
• Recognized need for ongoing effort
• Reaffirmed importance of using data for
decisions
Association of Research Libraries
www.arl.org
E-Metrics Next Steps
Call for participation among members to test
proposed measures for 2002/2003 (over 35
participants to date)
– Examination of the deliverables from the first phases
– Collect FY02 totals
• Compilation
• Data analysis
• Distribution for discussion
– Analysis of approaches
– Best practices for work processes
• From E-metrics project
• Locally developed
Association of Research Libraries
www.arl.org
E-Metrics Next Steps
• Continued work with vendors through
international COUNTER project
• Continued work with international
standards activities
• Workshops and training to develop
necessary data analysis skills
Association of Research Libraries
www.arl.org
Project Documents
• Measures for Electronic Resources (EMetrics)
Part 1: Project Background and Phase
One Report
Part 2: Phase Two Report
Part 3: E-Metrics Instructional Module
Part 4: Data Collection Manual
Part 5: Library and Institutional Outcomes
• www.arl.org/stats/newmeas/emetrics/
Association of Research Libraries
www.arl.org
Additional Questions?
• Please contact:
Martha Kyrillidou
Senior Program Officer for Statistics
and Measurement
Association of Research Libraries
[email protected]
Association of Research Libraries
www.arl.org