Country Program Evaluation (CPE) Methodology

Anis Dani March 20, 2013

IEG Methodology for CPEs

CPEs CPEs Objective Accountability and Learning Scope: Long-term Review (2+ CAS cycles) Evaluation Questions identified by some CPEs to address country specific issues and to provide lessons for similar countries Outcome Rating: 1. Relevance and 2. Efficacy 3. Client ownership of international development priorities 4. WBG corporate advocacy priorities, such as safeguards/Replaced with strengthening country systems 5. Institutional development impact 6. Risk to Development Outcome

3

CPE methodology and data sources

► Review of international literature and country level analytical work and project documents ► Review and analysis of IEG data from previous evaluations • including ISRs, ICRs, ICR Reviews, PPARs, AAA ►

Interviews with key stakeholders

• Learning Week with current and former country teams • In-country stakeholders and other development partners ►

Field visits and beneficiary feedback from project locations

Afghanistan CPE: Evaluation Questions

►

To what extent was WBG assistance in the initial years relevant and commensurate with country’s needs to build confidence in the state?

►

Relevance and effectiveness of strategic pillars over the past decade:



WBG support for building the capacity of state institutions and its accountability to citizens?

 

WBG assistance in promoting growth of the rural economy and improving rural livelihoods?

WBG efforts to support growth of the formal private sector, including through infrastructure development?

► ►

How effective were efforts in harmonization and alignment among donor organizations?

What lessons can be derived from Afghanistan for other fragile and conflict-affected states?

►

To what extent has the WBG been effective in building institutions likely to be sustainable and resilient to the risks facing the country?

5

Methodological constraints and solutions for Afghanistan CPE

PROBLEM: SOLUTIONS:

►

Security constraints restricted mobility and limited the size and duration of field missions

Cluster-based Assessment of all 107 Advisory and Analytical Activities(AAA) products ► Background paper on gender ► Case study of institutional capacity (second civil service) ►

Primary data collection:

• Beneficiary survey (radio & mobile phones) • Social media and Facebook survey • Focus group discussions with partner organizations (NSP, public health) and media representatives ►

Comparative analysis with results from other FCS countries

6

Limitations and Challenges

► Performance assessed against strategic choices reflected in CAS/CPS objectives but not designed to capture “errors of omission” in strategy ► The current methodology depends heavily on triangulation of evidence on a single country but is not designed for comparative analysis ► The methodology is based on the country model as the primary vehicle for delivery of Bank Group assistance and is less effective in assessing the relevance of regional or global issues