Transcript Evaluation Total ODA Impact Stefan Molund, SIDA Fourth meeting of the DAC Network on Development Evaluation, Paris, 30 – 31 March 2006

Evaluation Total ODA Impact
Stefan Molund, SIDA
Fourth meeting of the DAC Network on Development Evaluation, Paris,
30 – 31 March 2006
Executive summary
• The proposal ripe for go/no go decision
• Main alternatives:
1. Introduce the proposal to interested partner countries
2. Put it aside for the time being
• Remaining questions about design and process
best discussed with partner countries
Consultation process
• Presentation of original proposal. Paris, November
2004
• Assessment of technical feasibility by team of
development economists (Bigsten, Gunning, Tarp)
• Multidisciplinary seminar on evaluation of total ODA
and alternative approaches to macro-evaluation.
Stockholm, November 2005
• Assessment of proposals by enlarged evaluation task
force. Edinburgh, February, 2006
Original proposal
• Focus on development impact
• Broad coverage
• Long-term perspective
• Concern with change mechanisms
• Country development starting point: tracing links from
impacts backward to inputs rather than the other way around
(as in most evaluations).
• Focus on combined donor support (the distinctive feature of a
total ODA evaluation)
• Impacts assessed in partner country perspective.
Original proposal (contd.)
• Joint donor-partner country evaluation
with partner country actively involved
• Independent evaluation team
• Multidisciplinary evaluation team well
grounded in country context
Key conclusions from consultations
• Original proposal too broad in scope
• Evaluation technically feasible provided
that statistical ‘gold standards’ for causal
analysis are replaced by more realistic
standards of process tracing, ‘plausible
association’, inference to best explanation,
etc. Assessing total ODA impact very
different from assessing project impact on
restricted target group by experimental or
quasi-experimental methods. Eclectic
multidisciplinary approach required.
Key conclusions from consultations
(contd.)
• Partner country development strategies
not suitable as primary evaluation criteria.
Generally accepted measures of
development more appropriate. (However,
assessing usefulness of total ODA to
partner country development efforts
would still be very relevant.)
• Inviting partner countries to join the
evaluation is the logical next step.
Revised evaluation design
• Focus on development impact
• Focus on impact on restricted set of development variables (New)
• Long-term perspective
• Focus on change mechanisms
• Country development starting point
• Focus on combined donor support
• Impacts assessed against generally accepted welfare criteria. (New)
The process requirements are the same as before. The revised proposal
would serve as a framework for open discussions between the partners.