Transcript Sixth Meeting of the DAC Network on Development Evaluation, 27-28 June 2007, Paris Country-led Joint Evaluation Dutch ORET/MILIEV Programme in China Chen Zhaoying NCSTE , China Hans.

Sixth Meeting of the DAC Network on Development
Evaluation, 27-28 June 2007, Paris
Country-led Joint Evaluation
Dutch ORET/MILIEV Programme
in China
Chen Zhaoying
NCSTE , China
Hans Slot
IOB, The Netherlands
Presentation Outline
■ A brief note on the evaluation
■ The response from stakeholders
and the dissemination of the
evaluation
■ Issues for discussion
2
Joint Evaluation of
Aid Programs
Definition
DAC Glossary defines joint evaluation as:
An evaluation to which different donors and/or partners
participate.
Four categories of joint evaluation
■ Donor + Donor
Donor + Partner Country
■ Multi-Donor + Multi-Partner
■ Partner + Partner
3
Joint Evaluation of
Aid Programs
Overview
■Partnership in development leads to partnership in
evaluation. Joint Evaluation has been on the international
development agenda since the early 1990s.
■Country-led Joint Evaluation of ORET/MILIEV
Programme in China is regarded by DAC Network study
report as one of the important joint evaluation cases since
1990.
4
Country-led Evaluation
■ Ownership in development leads to ownership of
evaluation.
■ The World Bank, UNDP, the DAC, as well as some
donor governments, have been developing approaches
to promote country-led evaluations (CLEs) and to
increase partner country ownership.
■ The majority of evaluations of development aid
programmes are still donor evaluations: Led by donors
and are carried out to meet donors’ requirements
■ What is needed are evaluations from the perspective of
the partner country. Country-led evaluations will provide
that.
5
The Country-led Joint
Evaluation in China
■ About the title of the evaluation
■ The recipient country leads the process, supported by
the donor and both sides are jointly responsible for the
evaluation
The evaluation covers the period 1991-2003. A total project portfolio
of 84 projects were approved. 40 projects were finished and 44 were
ongoing. Of these 44, 17 completed the transaction but still had to
produce the final report.
The grant amount of the transaction is about € 200 million, the
total transaction amount of the evaluated portfolio € 470 million.
6
How is the Evaluation
Initiated?
■ After the experiences of joint evaluation of aid
development, NCSTE would like to have a more
pronounced role in the development evaluation
and IOB would like to be involved in a recipientled joint evaluation.
■ The joint evaluation was co-initiated by IOB
and NCSTE.
7
The Evaluation Aims to
Assess to what extent the programme (through its
activities / projects) has fulfilled the policy objectives,
needs and priorities of the Netherlands and China
Verify whether the funds have been
appropriately and efficiently used
Provide information for both the
Chinese and the Dutch that could be
used to improve the programme, as
well as for policy formulation
8
Evaluation Criteria
■ The relevance of the Programme
■ The efficiency of the Programme
■ The effectiveness of the Programme
■ The impact of the projects implemented under
the Programme.
9
How is the evaluation
organized?
Governance Structure
An equal governance structure was established
for this particular type of evaluation.
■ The Steering Committee (SC)
Make decisions on the evaluation
■ Team Leaders (TL)
Organise the fieldwork and other tasks
■ The Reference Group (RG)
Provide advice and support
10
How is the evaluation
conducted?
Four steps of the joint evaluation
■ Design/TOR
■ Setting up & organization
■ Implementation & Final Draft Report
■ Report dissemination/Following up
11
How Is the Evaluation
Conducted?
Evaluation Methodology
12
■ Desk Study: Cover all the 84 projects
■ Field Visits: 35 projects were chosen, 19 finished and 16
ongoing projects
■ Questionnaires: 68 were returned from end users and
60 from suppliers
■ Stakeholder Workshops: 17 provincial-level workshops
were held during the field visits. More than 300
stakeholders were invited to participate in the workshops
■ Cross-verified & In-depth Analysis: The information in
the fact sheets and the evidence collected from the field
visits were cross-verified to support in-depth analysis; the
database of initial evidence for the 35 visited projects has
been built
The Responses
from Stakeholders
■ The responses from Chinese stakeholders
■ The responses from Dutch stakeholders
■ The responses from other development
agencies
■ The policy reaction in China and the
Netherlands
13
The Dissemination
of the Evaluation
The dissemination has been given attention from
the beginning. Some dissemination activities are
planned and some are not predicted.
■The final evaluation reports published in both
Chinese and English
■ The dissemination workshop in Beijing China,
March 29, 2007
■ Communicate this evaluation case at the
evaluation training courses (IPDET, ShIPDET and
other national evaluation training workshops in
China)
14
Comments for
the Evaluation
■ The key benefits of the evaluation
■ The limitation of the evaluation
■ Potential challenges of the country-led joint
evaluation
■ Making the decision to undertake a joint
evaluation
15
The key benefits of
the Country-led Joint Evaluation
■ Offer great opportunities for mutual capacity
development
■ Increase ownership, the evaluation reflects
better the voice of the partner countries
■ More relevant to the needs of both sides
■ Sharing the burden of work of evaluation
16
The limitation of
the Country-led Joint Evaluation
Institutional limitation in China
■ A national evaluation system has not been set up
■ No one in-charge of the official policy reaction
■ Lack of special budget for evaluation
17
The Potential challenges
of the Country-led Joint Evaluation
■ The biggest challenge comes from different
evaluation cultures and evaluation systems
■ Ensure that the evaluation meets the needs of
both countries
■ Build consensus to design and conduct
evaluation (such as: Identifying the key Issues in
writing TOR, choice of field study cases)
■ Engage the key stakeholders in the design,
conduct, and interpretation of evaluation
18
The Potential challenges
of the Country-led Joint Evaluation
■ Develop capacity of partner countries to do
AND use evaluation
■ Ensure that evaluation is USED appropriately
■ Enable partner countries to evaluate
themselves rather than having evaluation done
to them.
19
Listen to
your Opinions and Advice
■ What do you think of this joint evaluation? What are the
advantages and weaknesses?
■ How to promote the country-led joint evaluation as a
successful evaluation model?
■ What is the driving force for this kind of evaluation?
■ What are the major barriers to the country-led joint
evaluation ? Lack of demand or weak evaluation
capacity ?
■ Donor’s role in CLEs, particularly in the “first generation”
of CLEs, by nurturing the country’s demand and facilitating
evaluation activities.
20
Sixth Meeting of the DAC Network
on Development Evaluation
27-28 June 2007, Paris
Thank you
Any Questions, Opinions & Advices… …
21