Transcript www.3ieimpact.org

Evaluation Capacity Building
in South Asia: Experiences,
Lessons Learned and Way
Forward
R.S. Goyal
Professor, Institute of Health Management
Research, Jaipur, India
(email: [email protected])
Backdrop…..
 Most South Asian countries now have monitoring and
evaluation systems in place, however, these are largely
concerned with;
the progress of implementation, rather than
assessing the social, economic and environmental
impacts of projects.
 Also, there seem to be no systems that assess the impact of
policy interventions emerging from recent macro-level
measures, such as;
liberalization, privatization, and the preservation
of women's rights.
 Further, donor agencies have played a key role in
planning, implementing and financing various
socio-economic development programs and
projects in these countries.
 In many cases, the outcomes of these
interventions do not match the intended objectives.
 It has also been observed that due to the lack of
ongoing evaluation many governments fail to learn,
in time, the way a project is unfolding and the
manner in which it is generating benefits.
 There are also many who simply do not see the
benefits of evaluation and, consider it to be a
donor-driven activity of no management use.
 Those donors who do see evaluation as an
important tool to improve investment quality are
now seeking to build evaluation as mandatory
management style- result based management.
 Success of these initiatives seems to have been
constrained, among other things, by;
 lack of a unified approach;
 inadequate appreciation and analysis of governmental
culture;
 confusion about concepts and methodologies;
 lack of long-term commitment; and
 lack of either interest or resources—or both—from the
recipient governments.
 Lack of professional evaluation capabilities to
commission, manage, implement and use
evaluation has also been sited as one of the major
constrains.
 Strengthening evaluation capacity in the South Asia region
has therefore becomes critical to achieve results of
development projects.
 Institutionalization of evaluation training in
academic institutions
viable strategy.
is identified as one of the most
 With support from UNICEF ROSA, a group of select
institutions in South Asia, embarked upon a path to offer
professional evaluation training in a sustainable manner.
 This paper discusses the process, outcomes, lessons learned and
future directions of this initiative.
The Process…..
 Assessment of demand for professional evaluation
training.
 A well published call for expression of interest sent
by UNICEF ROSA indicated a widespread need of
professional evaluators among donors and desire to
participate in the capacity building process among
leading institutions in the region.
 The feasibility assessment led to identification of
six institutions with essential facilities, expertise
and commitment to participate in the capacity
building program.
 A key strategy of this initiative was to seek support
from centers of excellence in evaluation training in the
North and foster a North-South collaboration.
 Among numerous Northern institutions showing
interest; Carlton University (Canada) and London
Metropolitan University (UK) were identified.
 To promote ownership and cross fertilization of ideas
among participating institutions, a consortium of
institutions on evaluation training in South Asia
was formed.
 To broaden the scope this initiative, the policy makers,
commissioners of evaluation and evaluation
professionals in South Asia were also brought in the
loop.
 It was agreed that training program would
commence with an executive level course of
one week duration.
 The training curriculum was developed
jointly by North and South institutions.
 A training of trainers program was held in
Dec. 07 at the author’s institute to test the
training module and orient the faculty from
South Asian institutions on delivering the
course.
 This program was attended by institutions
from Bangladesh, India, Pakistan and Sri
Lanka.
 The feedback of this program was critically
examined with reference to contents,
training pedagogy and reading material
provided.
 The analysis shows that mode of delivery
was good. However, contents need
modification to suite the expectations of
programs and professionals in the region.
 The next step was setting
up procedures by
participating institutions to offer executive course on
regular basis.
 Several institutions managed this process efficiently.
These include;
BARC University, Dhaka, Institute of Health
Management Research, Jaipur, Lahore University
of
Management
Sciences,
Lahore,
Sri
Jayawardanapura University, Colombo.
Other institutions are in the process of
making arrangements.
 The participating institutions started offering
evaluation trainings in their respective areas of
specialization.
 Author offered the first course on Evaluating
Healthcare Interventions on Oct. 13-17, 08. It was
attended by 18 participants from South Asian region.
 The training focused on;
evaluation design, ethics and standard of
evaluation, result based management, types of
evaluation and links to monitoring, developing
terms of reference, process of carrying out
evaluation and utilization of evaluation findings for
program management.
 The feedback of this training shows;
 it met the expectations of participants to a large extent,
 helped in enhancing the skills in planning evaluations,
writing TOR, carrying out and utilizing evaluation
findings,
 further skill building was however, sought on research
design and data analysis process.
 In the light this feedback the training module was
modified and the next executive level training
was offered on March 2-6, 09.
 This training attracted a very positive response and
attended by 59 evaluation professionals from South
Asia.
 The feedback of second program was even more
encouraging:
 Over 95 percent participants observed that course was
well structured and practical oriented.
 Ninety five percent participants opinioned that course
learning would help them in their immediate and future
assignments.
 Over 96 percent participants observed that course
material was very relevant to their work.
 Almost all participants wanted to extend the training
period.
Lessons Learned…..
 The regional approach adopted in this initiative has
been a strength in its implementation and in moving it
further.
 It
enabled adoption of common applicable and
culturally acceptable curriculum and training pedagogy.
 Coming together of institutions as a consortium for
evaluation training has provided a region-wide forum
and an authenticated body.
 It facilitated the program to move forward as a body
beyond the tenure of assistance of original donor agency
and developing a plan for diploma program.
 The collaboration between North and South Asian
institutions played a vital role in the success of this
initiative.
 The
former brought the state of art knowledge
and approaches which later adapted to South
Asian learning context and needs.
 In developing diploma program in evaluation
this experience will be taken further through
a greater engagement of South Asian
partners and advanced knowledge and
experiences from the North.
 The main success of this capacity building
process was its sustainability.
 The South Asian Institutions were able to
run the course on a cost recovery basis.
 Thus with infusion of a small start up
funding together with catalytic actions,
this intervention presents a viable model to
address the need of
professional
education.
Future Directions…..
 Planning has been initiated to offer a on campus
and online diploma in evaluation.
 Consolidation and standardization of training
module of one week executive level course for
wider replicability.
 Evolution of a regional network of professional
evaluators to support the evaluation activities.