Transcript Leistungsmessung und Bibliotheksinnovation

Measuring Innovation in Libraries and Information Services
Sebastian Mundt
Stuttgart Media University
7th Northumbria International Conference on Performance Measurement in Libraries and Information Services
Stellenbosch – Aug. 15, 2007
Mundt_Measuring Innovation_2007-08-15_1
Overview
1. Drivers of innovation in libraries
2. Innovation as an object of measurement
3. Potentials and shortfalls of existing frameworks
4. Need for additions and improvement
5. Proposed framework for measuring innovation
Mundt_Measuring Innovation_2007-08-15_2
Drivers of Innovation in Libraries

Increasingly high innovation rate in the media field
►

Libraries deliver value added on media
►

Pressure on libraries to compete in order to „remain visible“
Dependency to provide technological and organizational solutions
Libraries invest in evolving media and technologies
►
Are these „capital expenditures“ justified and worthwile?
Mundt_Measuring Innovation_2007-08-15_3
Definition of „Innovation“

Defined in different contexts

often used interchangeably with creativity, flexibility, future potential, invention or improvement
“Innovation is the embodiment, combination, or synthesis of knowledge in original,
relevant, valued new products, processes, or services” (Luecke/Katz 2003)
successful
in the market
new or changed substantially, i.e. must add
significant value in functional characteristics, ease of
use, interaction, sourcing of supplies, or any other
dimension
Mundt_Measuring Innovation_2007-08-15_4
Imitation
Introduction of a product or process

as a reaction on a specific innovation of another institution

which is aiming at similar markets or addressing similar audiences

where critical technologies or processes are copied from the original innovation generated by
the other institution.
Mundt_Measuring Innovation_2007-08-15_5
Examples: Service/process innovations
Online interlibrary lending
►
customers can order 24/7
►
quicker delivery
►
automated status reports
►
easier handling of orders
Implementation of RFID
►
quicker check-out / shorter waiting times
►
less check-out errors
►
better protection of materials against theft
►
longer life period than conventional barcodes
Mundt_Measuring Innovation_2007-08-15_6
Example: Service innovation
Search interface for e-resources
► higher comfort in searching different resources
► likely more useful search results
Example for an imitation: Catalogue enrichment
►
supports users‘ choice of media
►
first introduced by Online Bookstores
Mundt_Measuring Innovation_2007-08-15_7
Measures in context of strategic management
Task
Select, organize and make literature
and information visible for users
Strategic
Aims
Make all relevant resources
available electronically
Provide users with electronic
versions of current serials
Objectives
Replace print subscriptions by
electronic licenses as far as possible
Activities
Mundt_Measuring Innovation_2007-08-15_8
Existing frameworks (1)
Performance indicators

set of standardized statistics
and performance measures

„Objectives of library statistics
can be defined as … to monitor
trends over time and the results
of innovation“ [ISO 2789:2003]
►
mostly deal with established products and services
►
outcome and impact indicators not widely tested yet
►
developments are usually compared over time
Mundt_Measuring Innovation_2007-08-15_9
Existing frameworks (2)
Balanced Scorecard

BSC indicators usually focus
the institutional perspective

„Potentials and development“

►
Percentage of library means
received by special grant
or income generated
►
Percentage of library staff
providing electronic services
Resources
Potentials
Usage
Efficiency
measure development potential, but not actual innovative performance
Mundt_Measuring Innovation_2007-08-15_10
Measuring innovation as a process
Phase 2:
Phase 1:
Creativity
Potentials
Phase 3:
Marketing
Development
Efficiency
Research output
Articles/Papers

Finances

Time budget


Personnel

Financial budget
Performance
more input =
more innovative
output (likely)
Control resource
allocation
higher efficiency
= more success
(likely)
Provide feedback
to staff

Functionalities

Cost reduction
higher output
= more value
to users
Market relevance

Population reached

user satisfaction
Document value
to funding inst.
Mundt_Measuring Innovation_2007-08-15_11
Measurement aims
Institution-based

to measure how innovative a library is
Project-based

to measure the success of an innovation in terms of input, processes and results
Mundt_Measuring Innovation_2007-08-15_12
Indicators to describe Potentials (1)
Number of R&D staff members per year (per innovation project)

Library staff allocated part or full time to one or more innovation projects

in management, researching, creative, developing or administrative functions

Human resources external to the library should be excluded (but included in R&D
expenditure).

counted in Full Time Equivalents (FTE)

indicates the workload put into a specific project
Mundt_Measuring Innovation_2007-08-15_13
Indicators to describe Potentials (2)
Project expenditure as a percentage of library means

broken down per year per innovation project

including relevant capital expenditure, R&D staff costs (FTE, actual or normalized) including
travel costs, IT and consulting expenditure

indicates the financial resources put into a specific innovation project

cf. definition of capital expenditure in ISO 2789:2003
Mundt_Measuring Innovation_2007-08-15_14
Indicators to measure development (processes)
Adherence to cost projection
Adherence to time schedule

relates actual with pre-planned resource input

could be broken down to single phases of an innovation
project if intermediate assessment is required

indicates how likely it is that the project will be finished within
the pre-planned resource allocation

however: at least in publicly funded projects, project funding
and time schedules are often planned precisely in advance.
Mundt_Measuring Innovation_2007-08-15_15
Measuring success: Difficulties
Multi-causality

result (value for an indicator) depends on various management decisions and/or on different
external factors
Attribution

one innovation can have several impacts

an innovation can trigger mutual dependencies
Time lag

to many users, mid or long-term impact becomes visible only after an introductory period of
unknown duration (e.g. e-books lifecycle)
Mundt_Measuring Innovation_2007-08-15_16
Indicators of success: Performance
Specific service performance, e.g.

Portal: Percentage of titles directly searchable in the portal

RFID: check-outs per member of lending staff, check-out error rate, percentage of lost or
misplaced items on shelf

Online ILL: Speed of interlibrary lending (ISO 11620)

mostly specific indicators (needing careful preparation)

compared over time at a pre-defined interval

indicates service and/or process improvement
Cost reduction

no standardized measures available

indicates financial resources saved by the innovation
Mundt_Measuring Innovation_2007-08-15_17
Indicators of success: Relevance
Percentage of population reached with the service
Customer satisfaction regarding specific (survey) items
Number of libraries imitating the service

at pre-defined interval

indicate the market relevance of an innovation

usually not relevant for process innovations
Mundt_Measuring Innovation_2007-08-15_18
Summary: Simple working steps

Before starting the project, describe the core functionalities of the innovation.

Define a set of indicators suitable to evaluate the functionalities.

Set their target values.

Agree upon an evaluation deadline.

During and/or after the project, measure.

Compare the results with your target values.
(This procedure also works for the assessment of “normal” improvements.)
Mundt_Measuring Innovation_2007-08-15_19
Measurement aims
Institution-based

to measure how innovative a library is
Project-based

to measure the success of an innovation
Mundt_Measuring Innovation_2007-08-15_20
Assessing the innovative institution (1)
Library staff with research and development tasks as a percentage of total library staff

staff allocated to innovation projects of the library or that the library contributes to (e.g. in field
tests)

counted in „Full Time Equivalents“ (FTE)

per year

alternatively count in terms of standardized staff costs

similar: Percentage of library staff providing electronic services (ISO 11620)
Mundt_Measuring Innovation_2007-08-15_21
Assessing the innovative institution (2)
Expenditure on R&D as a percentage of library means

all means allocated to innovation projects of the library or that the library contributes to

including relevant capital expenditure, R&D staff costs (FTE, actual or normalized) including
travel costs, IT and consulting expenditure

broken down per year

similar: Percentage of library means received by special grant or income generated (ISO 11620)
Mundt_Measuring Innovation_2007-08-15_22
Assessing the innovative institution (3)
Refereed articles and conference papers per member of R&D staff

per year for all innovation projects

Book publications excluded (results will usually have been published in articles before)

could also be counted in relation to one innovation project
Mundt_Measuring Innovation_2007-08-15_23
Summary and outlook


Different perspective to established performance measurement
►
continuity less relevant than short handed success
►
specific measures apply
Can all libraries afford to be „innovative“?
►

Large libraries are more likely „carriers of innovation“.
Impact / outcome?
►
Do „innovative“ libraries have a greater impact on their community than others?
►
What do “innovative” libraries change?
Mundt_Measuring Innovation_2007-08-15_24
Sebastian Mundt
Stuttgart Media University
Faculty for Information and Communication
Wolframstraße 32
D-70191 Stuttgart
Tel. +49 (0)711 25706-263
Fax +49 (0)711 25706-300
[email protected]
www.hdm-stuttgart.de/iuk
Mundt_Measuring Innovation_2007-08-15_25