Transcript Title

EVALUATION
Evaluation Office
THE INDEPENDENT EVALUATION OFFICE OF THE
UNITED NATIONS DEVELOPMENT PROGRAMME
REFLECTIONS ON CONTRIBUTION TO
TRANSPARENCY, LEARNING AND
ACCOUNTABILITY
Indran A. Naidoo
Director
June 2013
2
Setting the stage




Demand from Members states and management.
The Evaluation Policy of the UNDP clarifies roles
and responsibilities for evaluation
The Evaluation Office (EO) of the UNDP, the
largest in the UN system, produces independent
evaluations at the corporate, programme and
country level.
It extends its influence by managing the UNEG
Secretariat, producing guidance and standards,
engaging with networks and supporting
evaluation capacity building across the globe
1. Mandate and functions
1999
EO was
established
2004
GA 59/250
Resolutions for
UN System
2005
UNEG Norms &
Standards
2006
UNDP Evaluation
Policy and
Independence
2011
UNDP Evaluation
Policy (revised)
 Reports directly to the Executive Board of the UNDP
 Supports the Administrator in her substantive accountability
function
 Operates within the UNEG Norms and Standards and ethical
guidelines, and engages stakeholders in the conduct of
evaluations to ensure transparency, learning and accountability
3
Evaluation Office
4
In practice this means…




Evaluations are planned and answer questions
within the Strategic Plan - has been RELEVANCE,
EFFECTIVENESS, EFFICIENCY and SUSTAINABILITY
DO UNDP INTERVENTIONS MAKE A CHANGE?
EO staff lead all evaluations, drawing on advisory
panels and using engagement processes with evaluands
and stakeholders as a means to enhance credibility.
As an independent office, the Director signs off on all
evaluations.
2. Contributions to other organisational tier
support and professionalasation)
UNDP: Support to decentralized evaluation
function and products
• Setting standards; Guides; Assessment of evaluation
quality Setting standards; Guides; Assessment of
evaluation quality
United Nations Evaluation Group:
Coherence in UN system-wide evaluation
• Norms and standards; Guidelines; Methodology; Peer
Reviews; UNEG Secretariat. (www.unevaluation.org)
Professional Networks: Advancing
Development Evaluation
• Regional: AfrEA, RELAC, Malaysia Eval Society, IPEN
• International: NONIE; ECG; IDEAS; IOCE; Eval Partners
5
Evaluation Office
6
For evaluators it requires …




Commitment to evaluator
professionalisation
Ability to engage in a supportive yet
independent manner
High levels of methodological skill and
content expertise
Strong strategic and communication skills
7
The Evaluation Office





Has produced over 100 evaluations since 2000,
and 80 country level evaluations. All have a
management response.
Management uses evaluations to review policies,
programmes and approaches – shown in 92%
uptake on recommendations.
Independence respected, and evaluation on the
agenda at key top management meetings.
Board allocates significant time to engage with
evaluation findings, management responds fully.
8
Transparency and stakeholders



The EO engages fully as evaluation processes are
as important as the “big report”
Learning can come through accountability
processes, these are not opposed to each
Stakeholder workshops held at country level, led
by EO, with government as key stakeholder.
9
Where UNDP operates




Globally, across a broad mandate
In complex situations where interventions are
difficult to embed and hard to measure
Implications for evaluation: difficult to work out
additionality in terms of attribution or contribution
Context plays an important role, not easy to
superimpose systems to measure results, but not
impossible. Involves high levels of engagement to
agree on what constitutes success
UNDP Programmes and Operations
177 countries in 5 Regions
National
Goals
and
Priorities
10
United
Nations
Development
Framework
UNDP Focus Areas
 Poverty and MDG
 Democratic governance
 Crisis prevention and
recovery
 Environment and
sustainability development
Programs
• Global
• Regional
• Country
• Others
Projects
Non-Project
operations
(advisory,
advocacy,
standard
setting/
normative,
coordination,
mobilization)
Evaluation Office
11
What does this mean?




Evaluations need to be context specific, yet meet
evaluation norms and standards.
At the country level they inform the next UNDP
programme, and draw on lessons from the last
programme period
Aggregation for synthesis needs to consider the
question of scale, variability and the challenges of
validity
Stakeholder workshops demonstrate to
government UNDP commitment to transparency
and accountability
12
Independence, a credibility question



Credibility goes beyond the tools and methods. It
rests on the leadership that directs evaluation
processes. This comes about through transparent
and logical evaluation plans and processes which
ensure engagement opportunities throughout and
across the spectrum to reduce bias
Results must be engaged publically
Independence is central for the reasons above, to
ensure credibility and authenticity of reports
Transparency
 Defined
in UNEG norms and standards
 Consultations with Stakeholders


TORS > Inception Report on Scope, Design and plan for data
collection and analysis > Stakeholder Meetings > Draft Reports
The Audit Trail <> significance for mutual understanding and final
decisions by EO
 Public




13
Access
All UNDP plans and evaluations in the Evaluation Resource Center (l)
EO evaluations in the EO website
Management response and tracking system (ERC)
Ratings on quality of decentralized evaluations
Evaluation Office
Independence - Structural
Functions, and staff - organizationally
independent from operations and policy
units and decision making.

Executive Board



UNDP
Administrator
Reporting



14
Director reports to the Executive Board
(2 terms and no re-entry into UNDP)
Board approval of programme of work
and budget
(independent of programme budget)
Executive Board
Evaluation reports are the
responsibility of the Director
Transmitted directly to the Board
following review and comment by
management
Senior managers safeguard the
independence: EO has access to all
records and information
Evaluation Office
Director
Evaluation Office
Examples from …2013
As of June 2013 the Evaluation Office of UNDP presented eight independent evaluations to the UNDP
executive board, which will contribute towards the development of the new UNDP Strategic Plan.
Five Regional Programme Evaluations
Evaluation of UNDP
Strategic Plan
15
Evaluation of South-south and
Triangular Cooperation
Evaluation of Global
Programme of UNDP
Evaluation Office
Country Programme Evaluation
(Assessment of Development Results – ADRs)
Evaluation Office has conducted 80 ADRs since 2002
16
Evaluation Office
17
Acting on evaluations


Engaging with decision-makers – engagement for
quality and credibility (management responses
and update)
Statistics on uptake – 92%
18
Global influence
Support of professional networks, associations and
events, to profile evaluation:
Supportive of IDEAS, Barbados 2013 (key notes,
panels courses)
Supportive of IPDET – course presentation
Supportive of EvalPartners – management board
Support to continental and regional networks (AFREA,
APEN, …
Hosting of the National Evaluation Capacity
workshops, next San Paulo, September 2013
www.NEC2013.org
19
Conclusions
Evaluation is not an event, but a difficult journey
requiring constant push, reappraisal, strategising to
ensure relevance. It does not occur naturally and requires
drivers
An evaluation function must be independent, which can
be supported when it is both inward and outward
focused. The outward – through events like these brings in
critical insights necessary for ongoing revision. An
outward orientation is the only way that one get
professionalisation that is necessary for evaluation.
Thank You
www.undp.org/evaluation
20
Evaluation Office