Transcript Joint Measurement of Results

UN support to conflict affected countries in
the context of UN Peace Operations
NORAD and EO/UNDP
(Oslo 22 May 2013)
Stefan Rummel-Shapiro
Senior M&E Advisor, UN Peacebuilding Fund (PBF/PBSO) New York
Topics for discussion
•
•
•
•
The scope for PBF country evaluations
Measuring results to assess the effectiveness of PBF
country support and global performance
How did evaluations trigger change?
Remaining challenges to improve the effective use of
evaluations
The scope for PBF country evaluations
Some key features:
•
Peace Building Support Office (PBSO): Established in 2005/06 by the
General Assembly with a direct reporting line to the Secretary General
•
PBF mandate: to support countries
- recovering from conflicts being at risk of (re-) lapsing into conflict
- addressing immediate needs for bridging funding gaps
•
•
Institutional arrangements
- Global Fund: Transfer to UN agencies as major fund users in-country
Two decision making levels
HQ: Budget approval
Field: Selection of Fund Recipients for project implementation
Priority areas for PBF country engagements
….cont. scope for PBF country evaluations…?
(1) Support the implementation of peace agreements and political dialogue
• Security Sector Reform
• Rule of Law
• (DD)R
• Political dialogue for Peace Agreements
(2) Promote coexistence and peaceful resolution of conflict
• National reconciliation
• Democratic governance
• Management of natural resources (including land)
(3) Revitalise the economy and generate immediate peace dividends
• Short-term employment generation
• Sustainable livelihoods
(4) (Re)-establish essential administrative services
• Public administration
• Public service delivery (including infrastructure)
Findings of the final evaluation (2010) of
PBF’s project portfolio in Burundi
...highlighting
• PBF’s specific added value: Overall project portfolio
1) Catching the right momentum to intervene (speed, risk taking)
2) Enhancing the coherence of Un agencies in their support to
peacebuilding efforts of the government
3) Achieving strategic results going beyond the scope of project
achievements of individual UN agencies
4) Programmatic response to conflict dynamics (large scale)
5) Catalytic effects: Funding leverage, unleashing political processes!
•
PBF’s specific added value: Project related
Approx. 30% of projects contribute significantly to the peacebuilding
processes in Burundi. Urgent call for improvements!
Findings of the final evaluation of PBF in Burundi…
….recommendations for improvements
.continuing evaluation findings….
•
Strategic guidance (PBSO/PBF) to the JSC and RUNOs?
1) Quality of programme design and measurements? Context specific?
2) How to trigger peace relevant change? Strategic entry point for Fund
Recipient Agencies?
3) Logic of intervention HQ versus field level? Coherent project selection?
4) Institutional arrangements for decision making at national level: More
project-based than strategic!
•
Implementation performance
1) Clarity in how projects can trigger peacebuilding processes?
2) Capacities in strategic management (planning, M&E, RBM)?
3) Efficiency of implementation process within risky environment?
4) Operational procedures flexible enough? Losing the ‘momentum’!
• The formulation of PBF Business Plan 2011 – 2013:
The opportunity to address evaluation findings (BDI)
The Performance Management Plan (PMP): The backbone for
• Quality assurance (programme design stage)
- Reflecting PBF core mandate (…reduced risk of…)
- Improving programme effectiveness and organizational performance
through stronger design, monitoring and reporting on results
- Budget approval, visibility, strategic positioning: Results-oriented
- PMP indicators: a ‘tool for dialogue’, identify leverage point for ‘change’
•
Streamlining M&E systems for assessing ‘quality of change’
- Roll-out mid 2011: Independent evaluations major MoV
- Reporting against PMP indicators in 2011 (baseline) and 2012
- Annual Report to SG, Performance Reviews (on donor demand)
Measuring results: Evaluations highlight the evidence for positive
contributions to peacebuilding in-country
(1) Outcome achievements: ‘Public safety improved’
Project support: SSR, RoL and DDR
Joint command / communication structures in place (BDI, SL)
Transitional justice systems with particular attention to women and girls (UGA)
Ex-combatants in armed forces or communities reintegrated (CAR, DRC, NEP)
(2) Outcome achievements: ‘Electoral process peaceful and votes respected’
Project support: Kick-start processes for reconciliation and social cohesion
Inclusive national dialogue initiated (BDI, GUI, SL)
‘Code of conduct’ developed for media and political parties (GUI)
Engagement of non - state actors for keeping civil oversight (SL)
(3) Outcome achievements: ‘Risks of relapse into mob violence reduced’
Project support: Mediation processes / skills enhanced:
Youths / women groups trained in conflict mediation (GUI, KYR)
Assessing PBF’s global performance: Aggregated
peacerelevant outcomes (project related)
Table 3: Verified significant contributions of PBF-supported projects (% of total investments) in the areas of SSR,
Rule of Law and DDR
Peacebuilding
outcomes
Projects Budget % of total % of total projects with
Countries with
(total #) allocation projects significant contributions to significant contributions
($m) 'on target' peacebuilding (per outcome) of PBF to peacebuilding
Public security
increased
Access to
judiciary systems
improved
14
31.7
64%
29% (4/14)
19
26.7
68%
26% (5/19)
Ex-combatants
successfully
integrated
TOTAL
15
24.1
80%
40% (6/15)
48
82.5
71%
31% (15/48)
(27% of total budget)
Burundi, Guinea, Liberia,
Sierra Leone
Burundi, Central African
Republic, Guinea,
Comoros, Nepal
Burundi, Democratic
Republic of Congo,
Nepal, Sudan
9 countries in total
How did evaluations trigger change within
PBSO/PBF?
 Strategic Guidance for Fund user
o PBF’s Global Results framework (PMP): A reference for improved
programming and streamlined reporting on results
o Guidance on how to design, monitor and report on results
(PBF Application Guidelines; www.unpbf.org)
o Efforts to build consensus with Representatives from Governments,
UN agencies, CSOs (Regional Workshop, July 2013)
o Conditions in place for an improved measuring of peace relevant
results and knowledge building on what works, what does not?
o Evaluation function strengthened
o 50% of country programmes evaluated to date (since 2010)
o Alignment of business processes: More results oriented budget
approval and quality assurance at HQ
o Meta-evaluation for internal knowledge management (draft)
o PBF global review 2013: Focus on ‘business model’ and ‘strategic
positioning’ (start date: June 2013)
Challenges to improve the effective use of evaluation
findings
 Quality assurance: The precondition for the evaluability of PBF’s added value
o How to apply minimum standards for good practice?
o Trade-off between quality assurance of e.g. sound conflict analysis and
speed to capture the momentum?
o PBF: Global Fund without field presence: What is ‘within PBF control’?
o Cascade of decision making: HQ versus field (JSC, RUNOS, IP)
o Time needed untill the new guidelines materialize in better results
 Measuring results and means of verification
o Quality of independent evaluations: To be improved
o Predictability of results (timebound)? Cumulative trends more accurate
than annual snapshots?
o How reliable are perception surveys for measuring ‘qualitative change’?
….challenges to improve…
o Methodology for measuring performance and reporting on results
o What does ‘strong’ versus ‘weak’ performance mean? Are ‘30%’
already the ceiling within a fragile environment?
o Where to set priorities: On performance improvements and
accountability for results, or better knowledge building / sharing?
o Substance of reporting on results?
 Ensuring the complementarity of the ‘E’ and ‘M’ function
o Insisting on better quality design is crucial but not enough for
ensuring programme evaluability.
o Early warning systems: Monitoring and reporting must be aligned
towards evaluation requirements (internal performance reviews)
o The right partnership between M&E matters building on trust and
confidence among all partners at HQ and in-country!
o External evaluations are one major data source, but not the only
one for providing reliable evidence of peace relevant results!