Transcript Slide 1

4th International Public Management
Summer Institute
August 25- 29, 2008 Tukums, Latvia
“Measuring Performance in Public Administration”
Monitoring, reporting and
evaluation – usage and benefits of
performance indicators
Egle Rimkute
Service delivery and accountability
Public
(service users)
Government
(service provider)
Lines of accountability
Lines of scrutiny
Legislator
(parliamentary scrutiny,
audit)
What is monitoring, evaluation and
reporting?
• Monitoring is the regular observation,
systematic collection and storage of data. It
enables activities, projects, programs, plans, and
strategies to be evaluated and reported upon.
• Evaluation is a systematic research conducted
to assess the linkages between programme
inputs, activities, outputs and outcomes or
factors in the programme environment.
• Reporting is the documentation of results of
monitoring and evaluation and the presentation
of them to appropriate audiences at specified
times
Purpose - support and improve resource
allocation, policy decisions and accountability
Principles of monitoring, reporting
and evaluation
Key principles are that monitoring, evaluation and reporting
should:
• be useful for all partners
• focus on few critical aspects of performance
• be simple, cost-effective, affordable and practical by:
- avoiding duplication of effort,
- using data for multiple purposes,
- ensuring that users can obtain the data, and,
- ensuring that users can easily find out whether suitable data already exist.
• provide comparative information
– historical trend information
– comparative information
•
•
•
look foreword as well as back
specify assumptions within strategies and decision making processes
integrate financial and non-financial information.
Fly exercise
X
A spectrum of performance approaches
Performance measurement
Basis for specifying expectations and for
monitoring
Performance targets
Notifies managers of results they are expected to
achieve and will be assessed
Performance reporting
Compares expected and actual performance;
provides transparency and accountability to public
Performance audit
Assessment of reliability of performance reports,
of adequacy of performance management
Performance benchmarks
Comparing performance with other providers;
targets set with reference to performance
achieved by best
Performance contracting
Agreement between government and service
provider of inputs to be used and
outputs/outcomes to be achieved
Performance-based pay
Linking employees’ pay to performance
Performance budgeting
Allocating resources on baisis of expected
performance
Performance measurement in
monitoring, evaluation and reporting
• Performance measurement is an elegant way of
shaping accountability and provides basis for
monitoring and reporting on performance
• It is important that:
– information is relevant
– information about performance is measured
systematically and quantified
– information can easily be communicated
– information can be supplied at the same time each
year
– information is fair and understandable
– PIs describe whether the service has achieved the
goals
Usage of PIs in monitoring,
evaluation and reporting
The use of PIs has been justified:
• Clear statement of what has been
achieved
• Focus on results delivery
• Basis for assessing what does and does
not work
• Better accountability
What makes a good PI?
To achieve effective monitoring, reporting and evaluation,
PIs should be:
• balanced
–
–
–
–
•
•
•
•
•
Volume
Quality of services
Efficiency
Financial performance
Comparable
Robust
Amendable
Reliable
Cost-effective
Performance indicators and users
KPIs are of
wide interest
Level of
information
Goals
(outcomes)
Programmes
Indicators
(outcomes, outputs)
Users of
information
Public, Parliament,
Media, Management
Interested
organisations
Users
Processes
(outputs)
Actions, resources
(inputs)
Supervisors,
implementers,
staff
Operational
indicators
are of interest
internally
Benefits of PIs in monitoring,
evaluation and reporting
• Easy to explain to the public, Parliament, other
stakeholders
– Graphs: UK – progress at a glance
– Dashboards: USA – scorecard, traffic-lights
•
•
•
•
Focus management attention on major issues
Concentrate on outcomes and outputs
Clear way of assessing if services are improving
Transparency – an organisation can make clear
what products/services/impacts it has delivered
• Learning – PIs, when monitored and reported
for, encourage to ask questions
Benefits of PIs in monitoring, evaluation
and reporting (cont.)
• Appraising:
– rewarding – PIs allow to reward good performance
– sanctioning – PIs enable to sanction for insufficient
performance
• Self-assessment – PIs enable comparison benchmark - between similar programmes,
organisations or different jurisdictions
• Devolved management – accountability for
results
• Intrinsic motivation compared to top-down
management control
• Improvement – PIs pull through radical
improvement
Problems Using Performance
Indicators
• Measuring outputs and outcomes
– unclear goals and objectives
– hard to measure concepts like well being
– summing multiple objectives
– distortions if only partial measures
– gaming – any measure, once selected as a
performance indicators will change its
behaviour (“Any observed statistical regularity
will tend to collapse once pressure is placed
upon it for control purposes”, Goodhart’s Law)
Problems Using Performance
Indicators (cont.)
• Quasi inputs/creaming
• Attributing causality
– exogenous factors
– need to look for value added
• The best are made better, the worst worse
How can PIs improve the situation?
•
•
•
•
•
Internal and external scrutiny
Political accountability
Hierarchial accountability
Quasi-market
Self-improvement
Things to remember: only performance indicators
will never be enough
for monitoring, reporting and evaluation