Transcript UNFCCC Meeting on Experiences with Performance Indicators for Monitoring and Evaluation of Capacity-Building in Developing Countries More Enabling and Less Controlling! Rio De Janeiro, 6-7

UNFCCC Meeting on Experiences with
Performance Indicators for Monitoring and
Evaluation of Capacity-Building in Developing
Countries
More Enabling and Less Controlling!
Rio De Janeiro, 6-7 November 2008
Lichia Saner-Yiu
CSEND, Geneva, www.csend.org
Objectives of Presentation



To urgently draw attention to the need to
review CB performance indicators in the
context of an institutionalised M&E processes
in the partner countries
To clarify that monitoring should not be
“mini-evaluation” but rather a management
tool!
To propose a monitoring tool which will
support greater effectiveness of the CB
projects/programme!
Field Experience






Capacity Building for the Public Administrative
Reforms – China (1988-1997), Slovenia (1994-1996) etc.
Building Business Training Centres in Russia (1995-1997)
Building Negotiation Capacities in the International
Arenas: UNFCCC, WTO, FTAA.. (2000-2003)
Strengthening management capacities of UN Agencies &
INGOs (ILO, UNICEF, WHO, UNHCR, Action Aid …)
Thematic Training and short term consulting inputs for
20+ years
Continue to teach in different universities on topics
related to institutional performance management and
transformation
Contexts



Decision 2/CP.7
The Paris Declaration for Aid Effectiveness (2005)
The ACCRA Agenda for Action (2008) calling
for accelerated progress in areas of country
ownership, inclusive partnerships and achieving
development results. Specifically, “Developing
countries will strengthen their capacity to lead and manage
development”
Framework | Scope of priority areas (UNFCCC, 2007)
1.
2.
3.
4.
5.
6.
7.
8.
Institutional CB
(National Focal Point)
Enabling environment
National
Communications
National climate change
programmes
GHG inventories
Vulnerability and
adaptation assessment
CB for implementation of
adaptation measures
Assessment for
implementation of
mitigation options
9.
10.
11.
12.
13.
14.
15.
Research and systematic
observation
Development and transfer
of technology
Decision-making and
participation in
international negotiations
CDM
Article 4.8 and 4.9 (LDC
Work Programme, NAPA)
Education, training and
public awareness
Information and
networking
The Challenge of Capacity
Development (CD)


“Capacity Development is a fundamental component
of development and aid effectiveness and a key
element in achieving the Millennium Development
Goals” (OECD/DAC 2006).
In particular CD is critical to achieving objectives of
ownership, aid alignment and mutual accountability, as
set in the Paris Declaration on Aid Effectiveness
(March, 2005).
The Challenge of Capacity
Development (CD)
 However,
the traditional approach to CD (as a
technical process or transfer of knowledge or
institutions from North to South) rarely leads to
the expected results i.e. sustainable capacity
within partner countries.
 Increase in funding is not sufficient to address
the challenge of capacity development. We need
a fundamental change in development practice
and the way that we look at the role and function
of M&E.
The need for Adequate CD
Parameters
Focusing on capacity as an endogenous process (led but
not exclusively by country actors) including M&E
components.
 Agreeing at country level on clear (institutional) capacity
objectives.
 Making effective use of existing capacities and
harmonising external support within the MEA framework.
 Monitoring outcomes from the perspective of beneficiaries
(OECD/DAC Recommendations).
 Enhancing institutional learning and knowledge creation
through M&E processes for all development
partners.

Shortcomings of Current Monitoring Practices

It is more of controling than for enabling.

It relies more on self-reporting, than evidence and
documented records.

It remains short in providing opportunities to the
partners and external (non-national) stakeholders
in taking charge of the CD process and in being
held accountable for what has been achieved and
what has not.
Shortcomings of Current Monitoring Practices
-2



The process for the QA of TBCD remains
exogenous rather than endogenous.
The learning remains at best temporal, at worst
short-lived on the account of the partners since
there is no mechanism to ensure
institutionalisation of learning.
Monitoring remains as activities to be checked off,
rather than an integral part of the day-to-day
process and mangement system
Confusing Use of Terminology



Mixing terms of “evaluation”, “inspection”,
“supervision” and “monitoring”
Monitoring in the parlance of the QM is about
“ensuring the CB process is being managed and
implemented as required so as to provide objective
evidence that the CB process is effective in
meeting requirements.”
Monitoring, therefore, should involve reviewing
the entire CB process at each stage of the project
cycle (from design to implementation) and not just
mid-term review or end-of the pipe evaluation.
Evaluation




About the past. Who remembers the W’s after 3
or 4 years of project duration?
Lessons learnt for whom? Project partners
(beneficiaries, donors, experts) move on to new
jobs
Very costly and very heavy for beneficiaries
(evaluation tourism)
Against Paris Declaration (e.g.EIF: multiple
evaluation methods, LDC overwhelmed with
compliance duties)
OECD Survey on Monitoring and Paris
Declaration (2008).

Les than 10% of countries have sound
frameworks to monitor and assess
development results. While some
progress has been made since 2005, an
enormous effort will be required to meet
the target of 35% by 2010
Urgent Need to Invest



Less evaluation, more monitoring!
Change our mindset and approach
to monitoring
Invest in a management information
system that monitors and enables
In order to accelerate the CB and the attainment
of UNFCCC on "preventing dangerous anthropogenic
interference with Earth's climate system“ and MDG 7 –
Environmental Sustainability
Proposal by Engel, Land, Keijzer
(2006)1

A balanced framework to define the term “Capacity
Building” in clear terms (capability to produce development
results, to self-organise and act, to adapt and self-renew, relate& create
adequate operating space, and to achieve coherence/vision/ strategy)


An excellent framework to develop indicators for
baseline studies, progress assessment and benchmark
Yet, there remains a missing link – an
institutionalised management system and
database to support the emergence of these essential
capabilities through practice and to facilitate
effective management decision making and self
regulation through evidence and data
1. “A balanced approach to monitoring and evaluating capacity and
performance”
CSEND Proposal
Capacity Building model
of Monitoring
The Three-Tiered Monitoring
Framework at the global level
Tracking Aid for
CB Flows
Donor SelfAssessment
National SelfAssessment
Quantitative
Information
Qualitative
Information
UNFCCC
CB
Review
CSEND’s Non Fiduciary AfT Monitoring
Framework (within borders)
Tracking CB
Implementation
Process
Verification of
Participating Donor
In-Country Inputs
Verification of
Executing Agency
In-Country Inputs
Verification of
National Inputs
Documentation
of InputThroughputOutput &
Outcome data
Validation of AfT
Processes
Knowledge
Creation
Based on Paris Declaration Principles
Continuous
Improvement
Project Cycle
CB programming
process
(framing the scope)
1.
Defining strategic
interests & needs of a DC
5.
Evaluating project
outcome
Monitor


2.
Initiating & formulating
project proposal
National
4.
Implementing project
3.
Appraising & approving
project
(at country and global
levels)

Global
What Is Needed for An Effective
Monitoring System 


M&E protocol
Monitoring methodology (focusing on
organisation, management and institutional
collaboration)
Third party verification according to the
agreed CB objectives and measure
A matrix structure and delineating
levels of responsibility for monitoring







Institutional actors and roles
Level of responsibilities regarding monitoring
Periodicity of monitoring actions
Inputs needed and from
Outputs produced
Monitoring criteria & indicators
Records
Example of M&E System (EIF)
EIF
Actors
Level of
Responsi
bilities
regarding
Monitorin
g
Periodi
c
it
y
NSC
Oversight on the Use
of EIF
resources for
the country
Setting national policy
regarding
trade
development
and
performance
targets
Supervise the
performance
of National
Implementatio
n Organs and
mechanisms
6 months
Outputs
Inputs





National
Trade
Development
Plan and
Priorities
PRSP
Country
Strategy and
Programme
Priorities if
existing
DTIS and
Action Matrix
NIU quarterly
implementati
on reports
(both
technical and
financial)
NIU process
audit reports



Country EIF
programme,
strategies,
workplans
Semester
Country EIF
Implementation
Review (Tier 1
& 2)
Annual Country
EIF
Programme
Performance
Report
Monitoring
Criteria &
Indicators
Monitoring Criteria


Compliance to
National EIF
programme
formulation
procedure
Compliance to
National EIF
project approval
procedure
Records





Decisions on
Tier 1 workplan
and Tier 2
programme
priorities
EIF Country
Programme
Monitoring
Records
NSC audit
records
Request for
Corrective or
Preventive
actions
Corrective
action report

It aligns CB with DC/LDC country development and
ministerial performance objectives within the context of
national strategy

It provides a process management tool for learning transfer,
continuous improvement of capacity building processes
and outcome (higher pay-off)

It gives ownership and accountability to the beneficiary
countries and counterparts

It supports intermittent programme evaluation (monitoring
means collecting in-depth data and rich information source)

It facilitates multilevel knowledge creation
and knowledge management amongst donors, beneficiaries
and experts.
Added Value of a CB based
monitoring management system





Safeguards against sub-optimal use of resources
Ensures result-based accountability of the
capacity building function
Documents quantifiable data for process
improvement
Anchors CC agenda within recipient country’s
development agenda, planning and
implementation processes
Knowledge management in the hands of ALL
partners
Vision to Achieve
Capacity building should assist developing
countries to build, develop, strengthen,
enhance, and improve their capabilities
to achieve the objective of the Convention
through the implementation of the provisions
of the Convention and the preparation for
their effective participation in the Kyoto
Protocol process (italic added).
Thank You !!