Transcript Quality Assessment

Government’s Expenditure
Review Initiative
Progress 2002 - 2004
Lunchtime seminar of Irish Evaluation Network
10 March 2005
Conor McGinn, Department of Finance
([email protected])
Content
Expenditure Review history and processes
Management structures
Report on progress, 2002-2004
Views of Depts & Quality Assessors
Conclusions and recommendations
What next?
Some initial scepticisms
Departments reviewing themselves,
making case for more resources
Why evaluate when the answer is
obvious?
ERI yet another bureaucratic papergenerating-machine
Evaluation not a priority for top
management and at political level
ERI events since 1997
The Strategic Mangement Initiative and
the beginnings of expenditure review
Objectives of the ERI
C&AG value for money study of the ERI,
October 2001
Reforms to the process, 2002-2004
Stages under each round of ERI
Selection of topics by each Dept’s
Management Advisory Committee
In consultation with Dept of Finance
Topics submitted for Government approval
Local steering committees for each review
appointed
ERI stages (cont’d)
Terms of reference approved by relevant
Secretary General
Data collection & analysis; report drafted
Draft report quality assessed externally
Publication; laying before Oireachtas
And hopefully…
… review recommendations implemented
Expenditure Review Central
Steering Committee (ERCSC)
Chaired by Secretary General,
Department of Finance
Vets progress in Departments regarding:
selection of topics
Progress with reviews, etc.
Monitors quality and assesses impact
Reports and makes recommendations to
Minister & Government
And the name…?
First called the Central Steering
Committee, the CSC …
… ‘til the Civil Service Commission
became upset!
ERCSC: You pronounce it!
Lord of the Rings: the Orcs
The ‘real’ CSC no longer exists
All very ERC-some
Expenditure Reviewers’ Network
Our own mini-version of the IEN
For civil servants carrying out reviews - all
Depts/Offices represented: 250+ members
Includes students of CMOD Masters
course in public policy analysis
Overseeing committee chaired by Dept of
Finance
Offers training, network events, & extranet
Why produce a report now?
A good thing to do …
(Still feels that way after being called
before an Oireachtas Committee?)
To put focus on Departments
To take stock of progress of reforms &
indicate areas for further change
To consider how ERI interlinks with other
reforms/initiatives
Report was based on
Discussions with Secretaries General
Views of Expenditure Reviewers’ Network
Committee’s own deliberations
Questionnaire responses received from
the Quality Assessors
Secretaries General on role &
impact of ERI
Reviews help provide better information
and so contribute to policy development
Clarify objectives of programmes reviewed
Highlight operational & efficiency
improvements
Help set context for subsequent decisions
Contribute to the Estimates process
Foster a culture of evaluation
Comments by Quality Assessors
Selection of topics for review
Quality of review report
Terms of reference
Evaluation approach
Quality Assessors, cont’d
Planning and managing of reviews
Steering committees, external input
Evaluative capacity
Response to quality assessment process
Criteria used by Quality Assessors
in assessing reviews
1. Are ToR appropriate to the ERI?
2. Does report comprehensively address ToR?
3. Adequacy of analytical approach;
robustness of methodologies used
4. Addresses future performance indicators?
5. Conclusions and recommendations
supported by analysis?
6. Structure, presentation, clarity of report
Some Departments currently
showing strong engagement
Agriculture & Food
Foreign Affairs
Social and Family Affairs
Enterprise, Trade and Employment
New kid on the block: Community, Rural and
Gaeltachts Affairs
Some Departments that are still
building evaluation capacity
Education and Science
Health … though many evaluation
reports/studies produced
Communications, Marine & Natural Resources
Environment & Local Government
Arts, Sports and Tourism
Finance
ERCSC’s main findings
Slippages in completing reviews on time
Number of reviews OK, taking other forms
of evaluation into account
Implementation of reforms to the ERI has
taken longer than expected
A need to build impact of reviews on
resource allocation
Main findings (cont’d)
Planning, managing and resourcing the
review process has been a problem
Evaluation culture in Departments is
variable, but improving
Synergies are possible with other
elements of public service modernisation
programme
Report recommendations
1. Improving timeliness of reviews
2. Changes to structures & reporting
arrangements within Depts/Offices
3. Detailing progress in Modernisation
Action Plan updates under Sustaining
Progress
4. Stating of reasons for each review
Recommendations (cont’d)
5. Making review steering committees more
independent
6. Strengthening of central supports
7. Smaller Offices
8. Systems to follow-up on review
recommendations
Recommendations (cont’d)
9. Annual Reports of Departments/Offices
to detail review activity
10. Laying review reports before Oireachtas
Committees
11. Linkages with Management Information
Framework
12. Linkages with Policy Analysis training
Where now?
Report was accepted and is being implemented
Call for 2005-2007 round of reviews has issued,
with focus to be on important, key areas
ERI building interlinkages with other initiatives:
– NDP/CSF evaluation,
– policy analysis training,
– Management Information Framework
Minister’s recent call for debate on Budgetary
process will have implications for the ERI
Longer-term issues
Moving beyond expenditure programmes
– Administrative expenditure
– Tax expenditures
Handling cross-departmental issues
Encouraging politicians to ask the right
questions
Perhaps the time is ripe for another
external evaluation of the ERI?