Transcript Document

Calculating ROI in
Special Libraries and
Information Centers:
Consequences of NOT
Having an Information
Center
June 2, 2007
Special Library Association
Annual Conference
José-Marie Griffiths, Ph.D.
Sarah E. Aerni
University of North Carolina at
Chapel Hill
Topics Covered in
Workshop
• Return-on-Investment (ROI) defined
• Two examples of ROI projects
– ROI of entire special library
– ROI of access to journal collections:
print and electronic
• ROI methods
• Communicating results
Role of Evaluation
• Planning
• Resource allocation and management
• Justification of new or existing
resources
• Advocacy, marketing and public
relations
Conceptual
Framework for
Evaluation
Object of Evaluation
•
•
•
•
•
Library
Function
Service
Activity
Resource
Conceptual Framework for Library Economic Measures
Measurement Perspectives
Library
User
Organization
Industry/
Sector
Society
Conceptual Framework for Library Economic Measures
Measurement Perspectives
Library
User
Specific Measures
Inputs
(Resources)
Amount
Cost
Attributes
Organization
Industry/
Sector
Society
Conceptual Framework for Library Economic Measures
Measurement Perspectives
Library
User
Specific Measures
Inputs
Outputs
(Resources)
(Products/Services)
Amount
Attributes
Amount
Cost
Attributes
Quality
Timeliness
Availability
Accessibility
Organization
Industry/
Sector
Society
Conceptual Framework for Library Economic Measures
Measurement Perspectives
Library
User
Organization
Industry/
Sector
Specific Measures
Inputs
(Resources)
Outputs
(Products/
Services)
Usage (Use & Nonuse)
Amount
Factors affecting use/nonuse
Ease/cost of use (price paid)
Available alternatives
Purpose of use
Importance of and satisfaction with
attributes of output
Awareness
Society
Conceptual Framework for Library Economic Measures
Measurement Perspectives
Library
User
Organization
Industry/
Sector
Specific Measures
Inputs
Usage
(Resources)
(Use & Nonuse)
Outputs
(Products/
Services)
Outcomes
(Consequences of Use and Nonuse)
Time saved
Improved learning
Improved productivity
Improved quality of work
Improved timeliness of work
Value derived
Effects on organization goals
Higher order effects
Society
Conceptual Framework for Library Economic Measures
Measurement Perspectives
Library
User
Organization
Industry/
Sector
Society
Specific Measures
Inputs
Usage
(Resources)
(Use & Nonuse)
Outputs
(Products/
Services)
Outcomes
(Consequences
of Information)
Domain
(Environmental Characteristics)
Target population
User/nonuser population
characteristics
User/nonuser
needs/requirements
Externalities
Conceptual Framework for Library Economic Measures
Measurement Perspectives
Library
User
Organization
Industry/
Sector
Society
Specific Measures
Inputs
Outputs
Usage
Outcomes
(Resources)
(Products/
Services)
(Use & Nonuse)
(Consequences
of Information)
Performance
Domain
Derived Measures
Conceptual Framework for Library Economic Measures
Measurement Perspectives
Library
User
Organization
Industry/
Sector
Society
Specific Measures
Inputs
Outputs
Usage
Outcomes
(Resources)
(Products/
Services)
(Use & Nonuse)
(Consequences
of Information)
Performance
Effectiveness
Domain
Derived Measures
Conceptual Framework for Library Economic Measures
Measurement Perspectives
Library
User
Organization
Industry/
Sector
Society
Specific Measures
Inputs
Outputs
Usage
Outcomes
(Resources)
(Products/
Services)
(Use & Nonuse)
(Consequences
of Information)
Performance
Effectiveness
Cost-Effectiveness
Domain
Derived Measures
Conceptual Framework for Library Economic Measures
Measurement Perspectives
Library
User
Organization
Industry/
Sector
Society
Specific Measures
Inputs
Outputs
Usage
Outcomes
(Resources)
(Products/
Services)
(Use & Nonuse)
(Consequences
of Information)
Performance
Effectiveness
Cost-Effectiveness
Domain
Derived Measures
Impact
Conceptual Framework for Library Economic Measures
Measurement Perspectives
Library
User
Organization
Industry/
Sector
Society
Specific Measures
Inputs
Outputs
Usage
Outcomes
(Resources)
(Products/
Services)
(Use & Nonuse)
(Consequences
of Information)
Performance
Effectiveness
Cost-Effectiveness
Cost/Benefit; Return-on-Investment
Domain
Derived Measures
Impact
Evaluation Involves
Comparison
• Over time
• Between services
• Between the current and the
anticipated
• Between inputs and
outputs/outcomes
ROI is a Comparison
Compare Returns and Investments
• Investments or Costs
- to the library (or service within)
- to the library user
- to the organization
• Returns
- to the library
- to the library user
- to the organization
ROI is a Comparison
Investments or Costs
• to the library (or service within):
$ spent
• to the library user:
time and $ spent
• to the organization:
total time and $ spent
ROI is a Comparison
Returns
Important to distinguish between:
outputs - produced as result of
expenditure, e.g., larger collection
use - extent to which outputs are used,
e.g., increased use of library materials
outcomes - consequences of use, e.g.,
learned something new, saved time,
etc.
Use contingent evaluation
Return-on-Investment
Defined
• Investment
– Library expenditures and organization
overhead
– User time in wages and overhead
– Cost of other relevant resources
• Return
– Contingent valuation of the additional cost to
users if there were no library services
– Changes in user outcomes e.g., productivity,
information needs satisfied, and other relevant
indicators
Definition of Contingent
Valuation
• Contingent valuation is an economic
method used to assess the benefits
of non-priced goods and services
(e.g., libraries or specific library
services) by examining the
implications of not having the
product or service
Case Study 1:Library ROI
(Availability & Use)
Investment - Total
• Library expenditures
• User expenditures
• Total organization investment
Special Library
Investments
• Library expenditure
– $610 per professional
• User cost to use library services
– $1,090 per professional
• Total organization investment
– $1,700 per professional
Contingent Valuation Library Availability
• User cost to use library services
• User cost to use alternative source
for service or information
• Net benefit or return (availability) is
difference between current cost to
use library and anticipated cost to
use alternatives
Special Library
ROI
Net benefit or return:
User additional cost to use
alternatives
$5,010 per professional
Library ROI (availability)
($5,010÷$1,700) or 2.9 to 1
Returns from Library Use
• Amount of use of information
resource or service
• Benefits derived from that use (time
saved, $ saved, etc.)
• Assume a fixed willingness to invest
• When using costlier alternatives,
some uses would be “lost”
Library ROI
• Benefits derived
– $310 per journal article reading
– $650 per book reading
– $1,090 per internal report reading
• Total savings across all readings
– $31,300 per professional for reading
journal articles
– $28,000 per professional for reading
books
– $42,500 for reading internal reports
Library ROI
• Labor ROI
–
–
–
–
–
Labor savings divided by time spent reading
8.3 to 1 for reading journals articles
7.9 to 1 for reading books
14.2 to 1 for reading internal reports.
Overall ROI (Labor) is 8.3 to 1
• Lost Benefits
$12,240 per professional
• Library ROI (Use)
– Ranges from 5 to 1 …. 16 to 1
Outcomes of Special Library Use
• Special libraries help increase
productivity
– User time and/or other expenditures are
saved in over one-third of library uses
– User time and/or other expenditures are
saved in about 40 percent of readings of
library documents
– Five indicators of user productivity are
correlated with amount of library
Outcomes of Special Library
Use
• Special libraries contribute to users’
quality of work
• Users indicate that that the library services
are absolutely essential to their work for
nearly 40 percent of uses
• Users indicate that the library services
improves their quality of work in nearly
60 percent of uses
• Users whose work has been recognized
through awards, etc. use libraries more
than cohorts and non-award winners
Case Study 2: Library
Journal Collection ROI
Trends in Special Library
Journal Collection Use
• Examine patterns of information
seeking
• Examples of scientists and engineers
• Amount of journal reading by
scientists is up
• Reading from special library journal
collections is up
• Increases due in part to electronic
journals
Patterns of Information
Seeking
•
•
•
•
•
•
•
Focus on article reading
Time spent reading
How read articles are identified
How the articles are obtained
Format of the articles
A trend in the influence of libraries
Effect of e-journals on library cost
Many Ways to Identify
Articles
• Browsing through print or electronic
journals (mostly for current awareness)
• Searching in search engines in
bibliographic and e-journal databases
(mostly for research and writing)
• Follow-up of citations in journals and
other publications
• Recommendations from colleagues, etc.
• Other (e.g., alerts, preprint services, etc.)
Many Sources of Articles
•
•
•
•
Personal subscriptions
Library collections
Authors, colleagues, etc.
Other
Where Do Readers Obtain
Articles?
Trends
Personal Subscriptions
Library Collections
From Another Person
Author Websites
Preprint and other
Dow
n
Up
Sam
e
Smal
l
Smal
Amount of Reading Depends
on Where Readers Works
• University scientists average reading more than
non-university scientists served by special
libraries
– University scientists: 252 readings/year
– Non-university scientists: 113 readings/year
HOWEVER
• Non-University scientists account for 75 percent
of all reading in U.S.
AND
• University scientists write nearly 75 percent of
articles published by U.S. authors
Trends in Reading Patterns
of Scientists Served by
Special Libraries
• The appear to be reading more
• They rely on libraries more
• Reasons for increased library use
Average Annual Reading by
Scientists Served by Special
Libraries
106
Average Number of Readings Per Scientist
120
100
75
113
80
80
60
40
20
0
1985-89
1990-93
1994-98
Year of Surveys
2000-2001
Average Annual Reading by
Scientists Served by Special
Libraries by Source of Articles
Read
70
61
60
Average Number of Readings Per Scientist
63
59
50
47
50
50
40
30
30
Library Collections
Other Sources
24
20
10
0
1985-89
1990-93
1994-98
Year of Surveys
2000-2001
Reasons for Shift to
Reading from Special
Library Collections
• Decrease in personal subscriptions
• More reading of articles identified by
online bibliographic searches
• Electronic collections have
broadened access to articles
Average Subscriptions Per Scientist
Average Number of Personal
Subscriptions per Scientist
4
3.70
3.5
2.89
3
2.44
2.5
1.93
2
1.5
1
0.5
0
1985-89
1990-93
1994-98
Year of Survey
2000-2001
Average Number of Readings Per Scientist
Average Number of Articles
Identified by Online Search
12.8
14
13.9
12
10
8
5.6
6
4
1.5
2
0
1985-89
1990-93
1994-98
Ye ar of Survey
2000-2001
Electronic Collections
Contribution
• Personal subscriptions - 90% print
• Library collections
– 80% electronic
– Broadens journal availability
– Saves readers about 20 hours per year
• Breadth of reading has increased
– Read from about 13 journals in 1977
– Over twice that amount now
Library Journal Collection
Investment
• Library expenditure (amount
allocated to faculty and staff use)
– $1.87 million
• Faculty and staff cost to use the
library collection
– $1.56 million
• Total organization investment
– $3.43 million
Library Journal Collection
Return (Availability)
• User cost to use alternative sources
of article information ($11.38
million in time, $2.1 million in
purchases)
$13.48 million
• Return/net benefit
($13.48 million - $3.43 million)
$10.05 million
Library Journal Collection
ROI (Availability)
• ROI (Availability)
($10.05 million ÷ $3.43 million)
2.9 to 1
PLUS
• Potential lost benefits in savings,
productivity, etc.
Library Journal Collection
Return (Use)
If the journal collection were not available,
professionals said they:
Did not know
37%
Would use another library
28%
Would use another source
21%
Would purchase the item
6%
Would take another course of action 8%
Library Journal Collection
Return (Use)
About 25% of readings of library-provided
journal articles saved the user time and/or
money
Average amount of savings
$310 per article reading (special
library studies)
$385 per article reading (national
studies of scientists and engineers)
ROI (Use)
8.3 to 1
Library Journal Collection
Return (Use)
Reasons for savings
Avoided having to do some work
Provided confirmation of work
Stopped unproductive line of work
Modified research or design
Modified analysis methods
49%
27%
10%
12%
16%
Total Time Saved
• Total time saved
– 250,000 hours
– 114 FTEs
• Electronic remote access savings in
time
– 50,000 hours
– 23 FTEs*
* Based on an average of 2,200 hours worked
ROI Methods
• Library ROI Methods
• Journal Collection ROI Methods
Library ROI Methods
• Mailed survey to organization
professionals
–
–
–
–
Letter from high level executive
Publicity by library
Promise results to users
Reminders by cards
• Universe is visits (critical incidents)
Library ROI Methods
Observations include:
• Library use (including last use)
–
–
–
–
Amount of use
Services used
Time spent using library
Time and cost to use alternative sources
if the library were not there
Library ROI Methods
Observations include:
• Section on purposes and
consequences of using the library
– Purposes (e.g., research, writing, etc.)
– Ways information/services affect
purposes
– Savings in time and/or money from
information
– Importance of information/services
Library ROI Methods
Observations include:
• Section on demographics
– Education level achieved, discipline
– Awards received
– Authorships
Methods: Cost of Users’ Time
• We use the average salary plus an
amount for overhead
• From other studies, we estimate that
professionals average about 2,200
hours of work per year.
• This allows us to calculate an hourly
rate per hour for users.
Survey of Special Library Use
Your responses are confidential and data will
be reported only in aggregated form.
Because your answers are extremely
important to the accuracy of our study,
please submit the questionnaire even if
you are unable to answer all the
questions. We have tried to keep the
questionnaire as short and simple as
possible and yet achieve our study
objectives. If you have any questions,
please contact….
Survey of Special Library Use
Section 1: Library Use
1. In the past year (12 months)
approximately how often have you used
your (company, agency, laboratory)
library? Uses include visiting the library
in-person or remotely including access to
the library’s bibliographic databases,
electronic journal collections, to request
services, etc.
Number of times the library was used the past
year: _______ times
Survey of Special Library Use
The following questions in this section refer to the
last use of the library, whether in-person or
remotely. Note that this last use may not be typical,
but will help us establish the range of uses of your
library.
2. Was this last use in-person or remote?
– In-person
– Remote
• By e-mail
• By telephone
3. How long ago was this last use?
4. What services were used during this last use?
Survey of Special Library Use
5. About how much time did you spend on this last
use of the library?
6. If you did not have the library, what would you
have done to obtain the information or service
obtained during this last use?
– I would not bother getting the information (skip to
question 7)
– I need the information, but do not know where else to
get it (skip to question 7)
– I would obtain the information from another source.
Please specify source here: _____________________
Journal Collection
ROI Methods
• Survey of professionals
• In-depth cost of library collection
purchase and processing
Reader Survey Methods
• Mailed survey…
• Universe is readings (critical
incident)
• See questionnaire sample in handout
Journal Collection Costs
Include All Resources
Used
•
•
•
•
•
Collection purchases
Staff
Facilities
Equipment and systems
Photocopies, binding, etc.
Allocating Library Staff
Costs
• Prepare a list of relevant journal
processing activities
• Prepare a staff activity log
• Instruct library staff for filling out
the logs
• Establish detailed calculation
methods
Total Cost and Cost Per Use of
Collection Access Services
Life-Cycle Cost per Title
• Electronic collection - $180 per title
• Print collection - $580 per title
– Current collection - $190 per title
– Backfile collection - $390 per title
Annual Cost Per Reading
• Compare the Unit Cost of Services
– Electronic - $3.00 per reading
– Current Periodicals - $13.60 per
reading
– Bound Backfiles - $15.60 per reading
– ILL - $8.40 per item
– ILB - $12.60 per item
Life-Cycle Cost Per Reading
• Electronic - $7.30 per title
• Print - $23.50 per title
Communicating
Results
Target Audiences
• Identify your target audiences – who
do you want to pay attention to your
information?
Target Audience
Characteristics
Summarize key characteristics of your
target audience(s)
• Current key issues and concerns
• Language/educational level
• Preferred communication media
Need “Sound Bytes”
•
•
•
•
Core messages
“So therefore…”
Meaning and impact
Comprehensibility
Core Messages
• List of 1-sentence statements of main
points, NOT long paragraphs
• Maximum of 4 to 5 statements
• Sequenced from most important to
least
• Need to have a 1-sentence summary
for all
“So Therefore…”
• Your core messages must include not
only facts and figures but also what
it means
• This is true, so therefore
–
–
–
–
You should…
We will…
This proves…
Etc.
Core Messages: Meaning
and Impact
• Your core messages must answer the
question for your target audience:
“What does it mean and why should I
care??”
Core Messages:
Comprehensibility
• Simplify, simplify, simplify!
• Just because you know big words doesn’t
mean you have to use them
Core Messages:
Comprehensibility
“Insofar as the comprehensive research is too
voluminous to delineate here, if you and/or your
corporation, organization, etc. are apprehensive
as to the pedagological or methodological
veracity, authenticity or rationale supporting the
aforesaid investigation please reference the
relevant available electronic resource.”
Compared to: “Please check our website for more
information.”
Core Messages:
Comprehensibility
Avoid being misquoted in the press (including the
academic press!) by asking the reporter to:
• E-mail you his/her questions before the interview.
• Repeat back to you what you have said
• Call you back with any additional questions, as
necessary.
• Provide you with a copy of your quotes before
the article is published. Quickly correct any
inaccuracies that you find. (Some reporters wil
not provide you with pre-publication quotes, but
some will. And it won't hurt to ask.)
Craft Your Message
• Identify the desired results of your
communication
Target
Audience
Organizational
leadership
Know
Feel
Do
The results
of the study:
the numbers
and their
implications
The
library/information
service is
well
worth the
cost
Support
additiona
l funding
requests
Craft Your Message
• Attention and sequencing - pyramid
approach
• Find or create visuals
• Humanize your message
• Try out your message with your
harshest critic and don’t argue with
their comments - listen and learn
Attention and Sequencing
• Must have an opening line (headline,
first line of interview,etc.) that is
attention getting —focuses on
impacts rather than data
• Need excellent headline/title
• Visuals
Visuals Communicate
Time of One
Information
Professional
Saves the
equivalent of
FIVE
professionals’
time
Without Special Library
With Special Library
Follow Up
• Every communication is the
foundation for the next
• Be clear about asking for what you
want
• Be appreciative
José-Marie Griffiths, PhD
Sarah E. Aerni
• Email: [email protected]
• Email: [email protected]
• School of Information and Library Science
University of North Carolina at Chapel Hill
• phone: (919) 962-8363
Thanks for coming!
• Please submit the evaluation forms
to us so that we can send them back
to SLA.
• Enjoy the rest of the conference!