Transcript PERFORMANCE EVALUATION OF ACADEMIC LIBRARIES

PERFORMANCE EVALUATION OF
ACADEMIC LIBRARIES
IMPLEMENTATION MODEL
Luiza Baptista Melo
University of Porto - Faculty of Science - Libraries
and CIDEHUS – University of Évora
[email protected]
Cesaltina Pires
University of Évora - Management Department
and CEFAGE – University of Évora
[email protected]
Summary





Introduction
Overview CAF, Balanced Scorecard
(BSC) and Analytic Hierarchy
Process (AHP)
Proposed model and the
implementation
Results
Conclusions
Introduction

Model to measure the performance of libraries

Based in the CAF, Balanced Scorecard and
Analytic Hierarchy Process (AHP)

Performance indicators based in ISO 11620 and
ISO 2789

Relative weights for each performance measure
calculate using AHP. They are computed in two
steps:


Comparing (pairwise) the performance measures
under each criterion
Comparing (pairwise) the major criteria of the
proposed model
CAF – Common Assessment
Framework
ENABLES
RESULTS
Human
Resources
Management
Leadership
People
Results
Process
And
Change
Management
Policy &
Strategy
Partnerships
& Resources
Customer
Oriented
Results
Key
Performance
Results
Impact
on
Society
INNOVATION and LEARNING
European Foundation for Quality
Management Excellence Model
(EIPA. 2006 CAF - Common Assessment Framework.
Maastrich: EIPA)
Evaluation scale from 0 to 5
Balanced Scorecard (BSC)
an implementation in a library
How are the library's finances
managed to use resources in a
cost in a effective way?
Finance
Perspective
How are the library
services to meet the
needs of users?
Customer Perspective
Library Vision and
Strategy
Innovation and Learning
Perspective
(Kaplan, R.S. & Norton, D.P. 1992. The
balanced scorecard-measures that drive
performance. Harvard Business Rev.,JanFev, pp. 71-79)
How is the library fitness
to ensure that goals are
met in a future?
How are the library’s
internal process organised
to get efficient and quality
services?
Internal Process
Perspective
Analytic Hierarchy Process (AHP)
1. Design a hierarchy structure
2. Determine the weights at each
hierarchy level
GLOBAL
2.1 Construction of pairwise
comparison matriz
2.2 Synthesis
2.3 Consistency ratio estimation.
MEASURE
CRITERIA
B1
B2
Bm
C1
E1
F1
C2
E2
F2
..
..
..
..
..
..
Cn
Er
Fs
(1st level)
PERFORMANCE
• Lower hierarchy level study –
determining the weight of each
indicator in a given criterion
D
PERFORMANCE
INDICATORS
(2nd level)
• Upper hierarchy level study –
determining the weight of each
criterion in the global
performance measure (D)
(Saaty, T.L. 1990 Decision making for leaders: the analytic hierarchy
process for decisions in a complex world. Pittsburgh: Univ. of Pittsburgh.)
Hierarchy Analyse of
the problem D
Mixed model CAF-BSC- AHP
model implementation
(Melo, L.B. 2005. Avaliação de desempenho das bibliotecas da
Universidade do Porto. Évora. Inf. Sc. Ms. Dissertation – Univ. Évora)
(Melo, L.B., Pires, C. & Taveira, A. 2008 Recognizing best practice in
Portuguese Higher Education Libraries. IFLA Journal, 34(1), pp. 34-54.)
Proposed model and the
implementation
To compute the Global Performance Measure:
•Choose the set of criteria Bi
(7 criteria)
•Choose the set of performance indicators Ci
(23 performance indicators)
•Calculate their relative weights (AHP)
(using the opinion of 10 university librarians)
Mixed model CAF-BSC- AHP
model implementation

Calculations were estimated with
the software Excel for Windows.
Results
Global Performance Measure (D)
CRITERIA
1st level
Leadership
Strategy &
0,182
Planning
0,076
PERFORMANC
E INDICATORS
2nd level
External
Partnership &
Resouces
Process and change
management
Client
0,100
0,218
0,069
R.1 – 0,521
M.1 – 0,640
M.4 – 0,663
M.7 – 0,677
M.10 – 0,684
M.2 – 0,238
M.5 – 0,229
M.8 – 0,225
M.11 – 0,218
M.3 – 0,123
M.6 – 0,107
M.9 – 0,098
M.12 – 0,098
R.2 – 0,212
R.3 – 0,108
R.4 – 0,092
Impact on
Society
Finance
0,161
0,194
R.6 – 0,545
R.7 – 0,214
R.8 – 0,150
R.10 – 0,549
R.11 – 0,351
R11a – 0,100
R.9 – 0,091
R.5 – 0,068
Enablers
Results
Hierarchic structure of the AHP and results how, the inquired librarian,
weight the different criteria used to computed the gobal performance
measure (D), and how they weight the various performance indicators used
to evaluate each criterion
RESULTS OF THE RELATIVE
WEIGHT OF THE CRITERIA
Customer perspective - 0.218
Impact on society - 0.194
Relative Weights of the Criteria
Leadership - 0.182
Customer
Impact on society
Financial perspective - 0.161
Process and change
management - 0.1
Criterion
Leadership
Financial
Process and change management
Strategy and planning – 0.076
External partnership and
resources - 0.069
Strategy and planning
External partnerships and resources
0
0,05
0,1
0,15
Relative Weight
0,2
0,25
Results of the relative weights
of the PI in the Global Performance Measure D
Relative Weights of the Performance Indicators in the global performance measure
Devel opment and f or mul at i on of a vi si on and a mi ssi on
Li br ar y vi si t s per capi t a
Rat e of t he st udent s success
Cost per t eacher
M edi an t i me of document acqui si t i on
Cost per st udent
Syst emat i c gat her i ng i nf or mat i on about needs and expect at i ons of user s
Number of monogr aphs, e-books, j our nal and e-j our nal s
Performance Indicator
Loans per capi t a
Devel opment or gani sat i onal st r uct ur e i n accor dance wi t h t he t asks
Rat e of pr of essor s and r esear cher s publ i cat i on
Rat e st af f l i br ar y par t i ci pat i on i n i nt er nal di scussi on gr oups, i nt er nat i onal meet i ngs
Over al l user sat i sf act i on
P r omot i on and t r ai ni ng t o i mpr ovement t he act i vi t i es
M edi an t i me of document r et r i eval f r om Open A ccess A r ea
Over al l st af f sat i sf act i on
Degr ee of compl i ance wi t h envi r onment al pr i nci pl es
Reor gani sat i on and i mpr ovement st r at egi es and met hods act i vi t i es
Cost per l i br ar y vi si t
Openi ng hour s f or a week
Level s of absent eei sm or si ckness
A ssess mar ket penet r at i on of el ect r oni c ser vi ce
Devel opment and appl yi ng met hods t o measur e t he per f or mance
I dent i f i cat i on of st r at egi c par t ner s and t he nat ur e of t he r el at i onshi ps
0,000
0,020
0,040
0,060
0,080
Relative Weight
0,100
0,120
0,140
Conclusions
•The evaluation model proposed in this study is based
on well-known instruments, CAF, BSC and AHP.
•Measuring academic libraries inputs, outputs,
processes and outcomes is not an easy task and it is a
great challenge.
•The discussion of these subjects among librarians is
useful and allows us to obtain data for the evaluation of
libraries and information services.
Evaluation
of quality
is a
•It is possible to build partnerships
to create
projects,
process
to be definitely
to share knowledge and data, to
get performance
incorporated in the activities of
evaluation results so as to improve
quality services in
the information services
university libraries.
THANK YOU!
ευχαριστώ