Transcript E-Medien in Deutschland: Benutzung, Statistiken, Zukunft
Ten years after:
„Measuring Quality“ revised
Roswitha Poll Münster, Germany
Measuring Quality 1st edition 1996 English, German, French, Russian, Italian only academic libraries 17 indicators
2/21
Measuring Quality 2nd revised edition 2007 academic and public libraries 40 indicators Authors: Peter te Boekhorst, Roswitha Poll
3/21
Measuring Quality: Why?
So what are you doing at the moment?
Writing a book about performance indicators for libraries
What’s that, performance indicators?
Methods for seeing whether the library is good things right
For example?
Okay, sounds easy
But why do you need a book for that?
That all the books are always in their right place
Yes, that’s important. And such things show you whether you are good?
Well, not quite, but I can compare with other libraries
And if they are worse, you know that you are good?
That’s about it 4/21
Changes in the library world
electronic services predominant the teaching library as new role cost-effectiveness crucial competition of the institutions library as physical place widens to meeting and communication 5/21
Trends in performance measurement
stakeholders‘ views quantitative and qualitative measures standardisation has proceeded - ISO 11620 will comprise e-measures user surveys find broad acceptance impact/outcome measures 6/21
More use of performance measures?
More data available -
For electronic services
- resources - expenditure usage
In national statistics
- reference transactions - data for library space But: • Visits still only physical visits • Digitisation missing - attendances at user training and events 7/21
More use of performance measures?
Joint benchmarking projects
BIX –Library Index
German public and academic libraries CASL (Council of Australian State Libraries) public libraries
Swedish Quality Handbook
all types of libraries HELMS (UK Higher Education Library Management Statistics) academic libraries
Netherlands University Libraries
8/21
More use of performance measures?
Joint benchmarking projects Projects:
5
Indicators:
55
Corresponding to ISO 11620:
25
Used in more than one project:
20
9/21
More use of performance measures?
Interactive national statistics allow combinations of data (e.g. per capita) Less reports from individual libraries But: service level agreements with performance indicators on websites Performance measures have been integrated 10/21
The handbook: structure
Balanced Scorecard chapter on impact/outcome bibliography for each indicator examples for possible scores 11/21
The handbook: How to choose indicators
The set of indicators should cover the full range of library services consider traditional and electronic services if possible „merged“ indicators include indicators for „potentials and development“ consider the view of different stakeholders 12/21
Library quality: Stakeholder views
Users
Access to information worlwide Delivery of information to the desktop Speed and accuracy of delivery Good in-library working conditions Responsiveness of staff Reliability of services 13/21
Library quality: Stakeholder views
Financing authorities
Cost-effectiveness Clear planning, effective organisation Positive outcome on users Benefits for the institution’s goals Effective cooperation High reputation of the library 14/21
Library quality: Stakeholder views
Staff
Good working conditions Clear planning, straight processes Systematic staff development High reputation of the library 15/21
The handbook: How to choose indicators
Only one indicator for one question?
Example: How intensively is the collection used?
Indicators: collection turnover loans per capita percentage of stock not used Each indicator gives a different picture and might serve different purposes of the evaluating library 16/21
The indicators
A. Resources, infrastructure: What does the library offer?
Library as place for learning and research
1. User area per capita
Collections
2. Seats per capita 3. Opening hours compared to demand 4. Expenditure on information provision per capita 5. Availability of required titles 6. Percentage of rejected sessions 7. Ratio of requests received to requests sent out in interlibrary lending 8. Immediate availability
Staff Website
9. Staff per capita 10. Direct access from the homepage
17/21
Opening hours compared to demand
Definition
The actual number and time of opening hours compared to opening hours as desired by users Method: Survey Possible formula: A/B A = the number of present opening hours B = the number of hours which the users state that they need (present hours + additional hours) If a library opens 60 hours per week and in the survey users ask for 10 hours more, the score would be 60:70 = 0.86
18/21
Immediate availability
Definition
The percentage of immediate loans of total loans
Method:
Immediate loans = loans minus reservations Total loans = loans plus ILL loans received
Question:
The probability that a user’s loan request will be fulfilled immediately 19/21
Direct access from the homepage
Definition
The availability of the most frequently used resources and services via the homepage of the library’s website, measured by the number of clicks necessary and the comprehensibility of the terms used
Method
Cognitive walk-through: A small group of experts simulates user behaviour
Question
Whether the homepage leads directly or very quickly to the most frequently needed information 20/21
Direct access from the homepage
Academic libraries
Address Way to the library Opening times Online catalogue Lending service User card User account ILL/document delivery Reference Databases E-journals Subject access User training News, events 21/21
Direct access from the homepage
Public libraries
Address Way to the library Opening times Online catalogue Lending service User card User account Electronic collection Link collection Reference Fees Services for children Branch libraries Community services News, events 22/21
Access via the homepage
Service direct
information on the homepage, e.g. times open, address, catalogue search
unmistakable term
on the homepage
unmistakable term
on the homepage
unmistakable term
on the homepage
unmistakable term
on the homepage
ambiguous term
on the homepage
ambiguous term
on the homepage
Clicks 0 1 2 3
1 3
1 Points 10 8 6 2 0 4 0
The indicators
B. Use: How are the services accepted?
General Library as place Collections
1. Market penetration 2. User satisfaction 3. Library visits per capita 4. Seat occupancy rate
Information services
5. Number of content units downloaded per capita 6. Collection use (turnover) 7. Percentage of stock not used 8. Loans per capita 9. Percentage of loans to external users 10. Attendances at training lessons per capita
Cultural activities
11. Reference questions per capita 12. Attendances at events per capita
24/21
Library visits per capita
Definition
The total number of library visits per year, either physical or virtual, by members of the population, divided by the number of persons in the population
Method
Physical visits: turnstile Virtual visits: web browser visits IP visits homepage visits 25/21
Library visits per capita: A method for counting homepage visits Counting the deliveries of a Transparent GIF placed on the home page for counting purposes The number of deliveries of the pixel is identical to the number of page deliveries. The advantage of this method is that it easy to use. The disadvantage is that access to other pages is not counted.
German Benchmarking Project BIX http://www.bix-bibliotheksindex.de/ 26/21
The indicators
C. Efficiency: Are the services offered cost-effectively?
General Collection costs Processes - speed Processes - reliability
1. Cost per user 2. Cost per visit 3. Cost per use 4. Ratio of acquisitions costs to staff costs 5. Cost per database session 6. Cost per download 7. Acquisition speed 8. Media processing speed 9. Employee productivity in media processing 10. Lending speed 11. Interlibrary loan speed 12. Correct answer fill rate 13. Shelving accuracy
27/21
Cost per use
Definition
The total operating or recurrent expenditure of the library during the reporting year divided by the number of loans + in-house use + downloads from the electronic collection
Method
loans = lending system in-house use = sampling downloads = suppliers and library servers
Question
The indicator assesses the library costs per case of collection use and therewith the cost-efficiency of library services 28/21
The indicators
D. Potentials and development: Is the library fit for future?
Electronic services
1.Percentage of acquisitions expenditure spent on the electronic collection 2. Percentage of library staff providing end developing electronic services
Staff development Budget
3. Attendances at training lessons per staff member 4. Percentage of library means received by special grants or income generation 5. Percentage of institutional means allocated to the library
29/21
Attendances at training lessons per staff member
Definition
The number of of library staff attendance hours of staff members at formal training lessons during one year divided by the total number
Method
- number of attendants at each training - duration of a training A training of 3 hours with 12 attendants would be 36 attendance hours.
Question
What priority does the library give to staff training and therewith to the library’s ability to cope with development?
30/21
Staff training: What has been counted?
Hours of training per staff member Days of training per staff member (hours calculated in days) Percentage of staff members who received training during the year Number of training lessons per staff member Training hours as percentage of total staff working hours Expenditure for staff training 31/21
Measuring Quality: What we tried to do
new indicators indicators for all services „merged“ indicators
find the ultimate set of measures ??
bibliography for each indicator practical examples for each indicator 32/21
It is difficult to catch a black cat in a dark room especially when
it is not there
Chinese Proverb