Transcript COMPAS 2006

COMPAS 2006
Evaluation of Agency Effectiveness
Presented by
Camille Gaskin-Reyes
Deputy Manager & Manager a.i.
Development Effectiveness and Strategic Planning Department
Inter-American Development Bank
February 7, 2007
Purpose and Role of COMPAS

The Common Performance Assessment System
(COMPAS) is a joint MDB initiative to monitor
MDB contributions to development effectiveness.
It is the product of the MDB Working Group on
Managing for Development Results.
 It is intended to answer the question of how
MDBs contribute to development effectiveness
and results on the ground.
1
The Key Principles







A common framework with standardized criteria
and indicators
Strong ownership by all MDBs
Comparability and comparisons of data (no
rankings)
Credibility - objective measurements and indicators
Simplicity, limited transactions costs, based on
existing data availability
Continuity and stability (to measure progress over
time)
Transparency (full disclosure)
2
Evolution of COMPAS
Indicators
2005 and 2006 COMPAS Matrices Compared
2005 COMPAS
2006 COMPAS
Category 1
Country Level Capacity
Development
Category 1
Country Capacity to
Manage for Development
Results (MfDR)
Category 2
Performance-based
Concessional Financing
Category 2
Country Strategies
Category 3
Country Strategies
Category 3
Allocation of Concessional
Resources
Category 4
Projects and Programs
Category 4
Projects (includes project
evaluation)
Category 5
Monitoring and Evaluation
Category 5
Institutional Learning and
Operational Experience
(includes evaluation of
operational experience)
Category 6
Management Adoption of
learning Incentives
Category 6
Results-focused Human
Resources Management
Category 7
Inter-agency Harmonization
Category 7
Harmonization Among
Development Agencies
3
2006 COMPAS -- Main Findings

Country capacity to manage for development
results – growing demand in BMCs for MDB assistance
to strengthen BMC capacity to manage for development
results

Country Strategies - Where compliance is monitored,
the finding is that there is significant room for
improvement in the design of Country Strategies

Allocation of Concessional Resources -- All
MDBs (except the EBRD, which does not provide
concessional financing) allocate concessional resources on
the basis of performance (as reflected in, among other
things, policies, institutions, and portfolio performance)
4
Main Findings (continued)
Projects (Design, Supervision) –

–
–
All MDBs conduct periodic reviews of project quality
at entry. Between 50% and 100% of the projects
reviewed received overall ratings of satisfactory or
better.
MDBs are taking various approaches to improve the
quality of project supervision. The WB has had
quality-of-supervision assessments for about ten
years, the AfDB started a similar exercise in 2006,
and the IDB has started validating the accuracy of
supervision reports.
5
Main Findings (continued)
Projects (Completion and thereafter) –

–
–
All MDBs keep track of portfolio performance. The
percentage of projects under implementation that
have suffered from unsatisfactory progress and/or
whose development objectives are unlikely to be
achieved varies among MDBs from about 3% to
about 25%.
All MDBs have procedures for reporting on the
results of their operations as soon as possible after
completion. The share of completion reports that
were actually prepared as a percentage of the number
that were due in a given year varies from 57% to
100%.
6
Main Findings (continued)

Institutional Learning from Operational
Experience –
– All MDBs have independent evaluation offices whose
mission is to help promote lesson learning and
accountability within MDBs
– The scope of their work includes, among other things:
evaluations (and evaluability assessments) of MDB
interventions (individual operations, sectors, themes,
and country strategies and programs; and assistance to
BMCs interested in strengthening their own evaluation
capacity
7
Main Findings (continued)


Results-focused Human Resources Management –
– The performance of MDB staff is routinely assessed by
comparing expected results and actual results
– All MDBs have mechanisms in place to link salary
increases to accomplishment of agreed-upon objectives
Harmonization among Development Agencies –
– Harmonization areas include: procurement, financial
management, evaluation, country performance
assessments, and environmental issues
– Joint activities include: country portfolio reviews;
Country Procurement Assessment Reports (CPARs),
Country Financial Accountability Assessments
(CFAAs), and Public Expenditure and Financial
Accountability (PEFA) reviews
8
Other Initiatives for MfDR

DAC member peer reviews –
– DAC “peer reviews” monitor individual members'
policies and efforts in the area of development cooperation. Each member is critically examined
approximately once every four years

The MDB Common Performance Assessment
System (COMPAS) –
– It is important to note that the purpose of the COMPAS
is to report on MDB performance – not on countrylevel results, which are a joint product of several actors,
including the MDBs
9
MfDR Initiatives (continued)

The World Bank Aid Effectiveness
Review 2006 –
– Broadens the monitoring framework to treat more fully
the 56 Partnership Commitments included in the Paris
Declaration by providing a qualitative assessment of
progress organized around the other 10 indicators
– The objective of the review is to help client countries,
Bank country teams and other development assistance
agencies share a common evaluation of progress and
jointly direct action and resources to strengthen
ownership, alignment, harmonization, results and
mutual accountability
10
MfDR Initiatives (continued)

The MfRD Source Book –
– Documents the principles and emerging good practice
in MfDR
– Provides illustrative examples of how MfRD is being
used in practical ways at the country, program, project,
agency, and interagency levels
– In the LAC region, examples of MfDR at the national
include Chile and El Salvador

DAC Joint Venture on Monitoring the Paris
Declaration –
– Tasked with coordinating the 2006 Survey on
Monitoring the Paris Declaration
11
Mutual Accountability






Mutual Assessments crucial
MfDR a work in progress
Dynamism of the development process requires
flexibility and client responsiveness
IDB currently in the process of realignment to
enhance its response to client demand
Several surveys of client satisfaction carried out
by IDB and other MDBs and have provided
feedback from BMCs on product and development
needs
The evaluation of agency effectiveness is best
done by those who work in partnership with
12
funding institutions
IDB Accountability Mechanisms

Medium Term Action Plan for Development
Effectiveness
 Development Effectiveness Overview
 Periodic country client surveys and
consultations (MIC example)
 Ongoing dialogue and feedback instruments
(Project Completion Workshops)
 Independent (Third Party) Audit and
Procurement Compliance Reviews
13