Transcript Phase II Approach Paper GENERAL APPROACH FOR PHASE II OF THE EVALUATION OF THE PARIS DECLARATION ON AID EFFECTIVENESS.

Phase II Approach Paper
GENERAL APPROACH FOR PHASE II OF
THE EVALUATION OF
THE PARIS DECLARATION
ON AID EFFECTIVENESS
Main Elements of the Approach
• Building Blocks (Ch. II)
• Scope and Focus (Ch. III)
– Conceptual Frameworks (III-B)
– The Evaluation Model (III-C)
– Critical Substantive Topics (III-D)
– Methodological Issues (III-E; Day 2)
• Governance and Architecture (Ch. IV; Day 3)
• Timetable and Budget (IV-F/G; Day 3)
2
Building Blocks (Ch. II)
• Booth and Evans, Options Paper (2006).
• Framework ToR for First Phase Evaluation (2007).
• Wood, etal, Synthesis Report on First Phase of the
Evaluation of Implementation of the PD (2008).
• Stern, etal, The Paris Declaration, Aid Effectiveness, and
Development Effectiveness (“linkages study”) (2008)
• Jones and Kotoglu, Applicability of the Paris Declaration in
Fragile and Conflict-affected Situations (2008)
3
Scope and Focus: Conceptual
Frameworks (III-B)
• Broad Evaluation Questions
1) Degree of adaptation of PD to country & aid circumstances
2) Value added to stakeholders of using PD partnership
arrangements
3) Could other strategies achieve results more effectively and
efficiently than the PD?
• Indicative Evaluation Framework - 2006 (Figure 1)
• Policy-focused Eval. Framework - 2008 (Figure 2)
• Some comparisons between the two frameworks
4
Figure 1. Indicative Evaluation Framework, Paris Declaration (from Fig. 2, Booth, etal, May 2006)
Level 1:
INPUTS
by
donors
and
partners
political support,
peer pressure and
coordinated action
political support,
peer pressure and
coordinated action
OWNERSHIP
ALIGNMENT
HARMONISATION
Level 2:
OUTPUTS
Countries define
strategies and
exercise
leadership
Donors base support
on country strategies,
and use/strengthen
country systems
Common
arrangements, better
division of labour and
supportive internal
incentives
political support,
peer pressure
and coordinated
action
political support,
peer pressure and
coordinated action
RESULTS
MANAGEMENT
Programming is
focused on results and
uses information for
improvement
Exogenous
influences: e.g.
other donor
country actions,
political change,
disasters
Level 3:
OUTCOMES 1
OUTCOMES 2
Level 5:
IMPACTS
Outcomes 1: Strengthened country capacity to make and implement policies and
programs focused on development results, making optimal use of concessional
finance and aid
Outcomes 2: Efficient and equitable public investment and service provision,
plus regulation and institutional development/coordination for private investment
Sustainable economic growth and transformation, resulting in attainment of
Millennium Development Goals and other national-development objectives
political support,
peer pressure and
coordinated action
MUTUAL
ACCOUNTABILITY
Country and mutual
accountabilities are
strengthened
Exogenous
influences: e.g.
other donor
country actions,
political change,
disasters
Figure 2. Policy-Focused Evaluation Framework. (from Fig. 3.2, Stern, etal, November 2008)
Actors
Intentions &
Priorities
Contexts
Country Level
Country
Characteristics
Government,
Parliament,
Private Sector,
NGOs
Inputs and Outputs
PD
Configuration
Poverty, human
development
growth
governance
Aid
Management
Analysis,
planning,
budgeting
Aid-related
Policies &
Programmes
Relevant
Inclusive
Targeted
Country level
Outcomes & Impacts
Poverty Reduction/
Achievement of
MDGs
Effective, efficient &
sustainable
Development
Outcomes
Human & economic
Partnership
Working
Donors
Goals and
priorities of
donors &
extent of
coordination
Aid Scenarios
Donor
engagement, aid
volumes &
dependency, aid
composition &
modalities
Capacity
Developmen
t
State Building
Public management
inclusion &
institutionalisation
Country Policies & Policy Making
Policy making,
governance &
institutions
Developmentrelated
policies
International
Outcomes & Impacts
ODA Legitimation
Donor ‘policy
learning’
Some comparisons between the
Two Frameworks
• Both frameworks give importance to outcomes,
impacts, & external factors
• Fig. 1 “unpacks” outcomes and impacts
• Feedback loops in Fig.2 suggest learning potential
for countries and donors (e.g. “state-building”)
• Outcomes and impacts on donor policy and ODA
are delineated in Fig. 2
• Fig. 2 unpacks inputs and associated “mechanisms
of change” (AP para 45 from Stern, etal. P. 47)
7
Phase 2 Evaluation Model
Two complementary foci:
1)Implementation focus (expanded Phase 1)
•
•
•
•
•
Larger, diverse set of countries
Wider range of stakeholder groups
More attention to capacity building
Assess donor aid to analytic & advisory activities
Self-evaluation of PD implementation at HQ level by
several additional donors not covered in Phase 1
8
Phase 2 Evaluation Model
Two complementary foci (continued):
2) Results focus: identify PD influence on development
effectiveness (outcomes & results): this should be a key
value added of the Phase 2 Evaluation
• Challenges
–
–
–
–
Brief time span since PD endorsed (March 05)
Demanding data requirements
External influences make attribution difficult
Rigorous methodology required
9
Phase 2 Evaluation Model
Two complementary foci:
2) Results focus (continued)
• Dealing with the challenges
– More feasible to identify PD-related policy changes that
“point” to likely changes in development effectiveness
– Focus on only 1 or 2 sectors and a few MDGs
– “Backward-track” from current results to “PD-like” actions in
previous years
– Identify “control” entities or areas (a methodological
approach to be discussed on Day 2)
10
Evaluation “Propositions”
 21 propositions (tentative hypotheses)
from “Linkages Study” (AP Box 1, p. 16)
 About 10 relate directly to results
(e.g. 2, 7, 9, 10, 11, 12, 13, 17, 19)
 Do these propositions capture the key
potential PD – results linkages?
o
E.g., have PD-like instruments, such as
SWAps, yielded development results?
11
Critical Substantive Topics
1) Accra Agenda for Action – September 2008
2) Capacity Strengthening
3) Fragile Situations
12
Critical Substantive Topics
Accra Agenda for Action (AAA)


Proposed that Phase 2 assess AAA progress
The AAA makes 48 Commitments under 3 broad
headings (and 12 sub-headings – AP para 47):
1) Strengthening Country Ownership over Development
2) Building More Effective and Inclusive Partnerships for
Development
3) Delivering and Accounting for Development Results
(2 of the 4 commitments relate directly to results)
13
Critical Substantive Topics
Capacity Strengthening


A means to realize MDGs & other development goals
Possible evaluation questions:
1)
2)


What changes in capacity attributable to the PD?
What effects have these changes had on development
outcomes and results?
3) How effective has donor assistance been in strengthening
partner country capacity?
4) What factors explain effectiveness of this aid and what lessons
can be learned?
Development results of some capacity strengthening will be
observable only in long-run.
Issue: Should Phase 2 concentrate only on a few MDGs?
Is including capacity strengthening too ambitious?
14
Critical Substantive Topics
Fragile Situations – Phase 1 Thematic Study


Given wide range of such situations, this study
needs to be taken into account
Among the study’s major conclusions are:
1) In post-conflict or improving settings, the PD can be
applied incrementally, but in deteriorating settings,
it may be of limited or declining relevance.
2) Harmonization is a “key entry point” for improving
aid effectiveness in these situations.
3) Aid effectiveness in a fragile situation must include
“state building,” taking into account the political and
conflict-resolving nature of the process.
15
Main Elements of the Approach
(“√” covered today; next 3 topics to be covered in subsequent sessions)
Building Blocks (Ch. II)
Scope and Focus (Ch. III)
Conceptual Frameworks (III-B)
The Evaluation Model (III-C)
Critical Substantive Topics (III-D)
• Day 2: Methodological Issues (III-E)
• Day 3: Governance and Architecture (Ch. IV)
• Day 3: Timetable and Budget (IV-F/G)
16