Transcript KOINADUGU DISTRICT HEALTH MANAGEMENT TEAM

Impact Evaluation of the Child
survival Project
Presenter:
Bockarie Sesay-DM&E Officer
Koinadugu District-CARE International
Sierra Leone
Co-Authors:
Ahmed A.G Aboubacrine-DM&E Coordinator, Yuki Suehiro-Health
sector Coordinator, Sowo Lebbie-CSP Project Manager, Sayoh
Francis- Assistant CSP Project Manager
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
Project location
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
Project beneficiaries
 19,198 - children
under five years
 27,101 - women of
reproductive age
 54 Direct and 429
indirect communities
of five chiefdoms
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
Project Goal and Objectives
 To improve the health status of children
under five and women of reproductive age in
Koinadugu District
 Strengthened family and household knowledge
and decision-making skills
 Enhanced community capacity to form groups
and institutions that sustain health initiatives,
demonstrate social cohesion, and promote
good governance mechanisms;
 Improved quality and accessibility of services
provided by MOHS personnel and MOHS
extension services; and,
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
Project implementation focuses on the
following four interventions through the CIMCI strategy:

The expanded
programme on
immunization (EPI)
Intervention focuses on
raising vaccination
coverage of children and
pregnant women. CARE is
working with communities
and MOHS to promote EPI
outreach through the
Community Health Clubs
(CHCs). The second
prong of the EPI strategy
is to encourage CHCs to
use appropriate BCC
strategies to increase
demand for and utilization
of EPI services.
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
Interventions conts.
 The nutrition
intervention works
through CHCs;
Community Based
Growth Promotion
volunteers, also
members of CHC; and
PHU staff to promote
early initiation of
breastfeeding, exclusive
breastfeeding,
complementary feeding
and improved Vitamin A
(VA)/iron intake for
women and children.
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
Interventions conts.
 The malaria
intervention The
strategy is to train PHU
staff to recognize
malaria and provide
standard case
management, capacity
building of CHC
members and through
them the community
members, while
promoting active
involvement of VDCs in
promotion and
distribution of ITNs.
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
Interventions conts.
The Maternal and
Newborn Care (MNC)
intervention
The strategy focused on
capacity building of CHCs,
involvement of TBAs in
promotion of institutional
deliveries, formation of
pregnant women support
groups, birth waiting homes
and later establishment of
VS&L groups.

7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
Project Strategies
 Capacity-building of institutions and
partners
 Behavior change communication (BCC)
 Quality assurance initiatives
 Health system strengthening
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
The Impact evaluation-August 2008
 Findings of the evaluation
 Best practices and overarching
lessons learned from the project
 Contribution of participatory impact
evaluation methodology to an
effective learning process
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
METHODS AND PROCESSESS USED IN
THE IMPACT EVALUATION
Three step process
 Health facility assessment (April 08): Client Oriented
Provider Efficient (COPE) Assessment was done in
collaboration with DHMT.
 Quantitative (June 08): A standard KPC 2000+ 30x15
cluster survey was done by an external consultant.
 Qualitative (August 08): Participatory qualitative
assessment was done which involved an external
consultant, stakeholders and partners.
 Key questions to be addressed – Why and
How? Are the changes brought by the project
sustainable? If yes, to what extent?
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
Purpose of Final Evaluation
 To assess if the project met the stated goals and
objectives
 To review the effectiveness of the project technical
approach as well as cross-cutting approach
 To identify the most important achievements
 To determine the sustainability of the project’s health
achievements
 To analyze the determinants of success, constraints
and other factors affecting project performance
 To elaborate best practices and overarching lessons
learned from the project
 To come up with recommendations
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
FE Qualitative:
Participatory Process (1)
FE is an effective learning experience which involved all
partners. Assessment team included:
 External consultant (Team leader)
 MOHS representative
 DHMT members
 District Council representative
 Community representatives
 NGO representatives – IRC, CAUSE Canada, CAD
 Radio Bintumani representative
 CARE Sierra Leone staff across different projects
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
FE Qualitative:
Participatory Process (2)
 Key informant interviews with DMO, DHMT, District
Councilor, MOHS, UNICEF, USAID, and Radio
Bintumani
 Review of background information – project overview,
outputs, and quantitative survey key findings
 Preparation and finalization of FGD/IDI tools
 Training of the assessment team on FGD/IDI
 5 day field work to conduct FGD/IDIs - split into 5
teams, one per chiefdom
 2-day synthesis session to reach the consensus on
findings and recommendations
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
Key findings
By intervention specifics:
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
Nutrition
100%
Baseline
86%
82%
Endline
80%
70%
68%
68%
67%
60%
60%
40%
20%
18%
20%
8%
0%
Early initiation of
breastfeeding
7/16/2015
Exclusive
breastfeeding 0-6M
Children 6-23M
received Vit A
Iron supplements for Post-natal Vitamin A
pregnant w omen
NONIE, AfrEA, 3IE Cairo
Conference 2009
Has malnutrition decreased?
30.0%
26.5%
25.0%
Baseline (CDC 1978)
Endline (CDC 1978)
18.8%
Endline (Anthro 2005)
20.0%
17.4%
15.0%
10.0%
5.0%
0.0%
Children 0-23 months w ho are underw eight
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
Malnutrition by age group
comparison CSP KPC 08 vs MOHS 07
30.0%
Underweight < -2SD KPC 08
22.8%
22.8%
Underweight < -2SD MOHS 07
23.0%
23.0%
20.9%
20.0%
15.7%
14.7%
11.6%
10.0%
0.0%
7/16/2015
0-5M
NONIE, AfrEA,6-11M
3IE Cairo
Conference 2009
12-23M
12-23M
Malaria
100%
83%
77%
80%
Baseline
Endline
60%
51%
40%
31%
27%
20%
1%
0%
children 0-23M slept under ITN
7/16/2015
children 0-23M received ACT w ithin mothers took IPT during pregnancy
48 hrs after
fever
NONIE, AfrEA,
3IE Cairo
Conference 2009
Maternal Newborn Care
80%
75%
Baseline
60%
Endline
44%
38%
40%
34%
20%
15%
7%
0%
w omen 15-49 know 2 symptoms
that need referal for EmoC
7/16/2015
mothers able to report 2 know n
neonatal danger signs
NONIE, AfrEA, 3IE Cairo
Conference 2009
children w hose births attended by
skilled personnel
Has institutional deliveries increased?
3000
2575
2262
2500
2000
1734
2003
1500
2007
788
1000
500
0
Deliveries assisted by MCH Aides
Deliveries assisted by TBAs
(Source: DHMT Koinadugu)
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
Expanded Program on Immunization
100%
78%
80%
Baseline
66%
Endline
60%
47%
46%
40%
20%
0%
mothers received at least 2 TT bef ore the birth of
youngest child
7/16/2015
children 12-23 f ully vaccinated bef ore 1st birthday
NONIE, AfrEA, 3IE Cairo
Conference 2009
Morbidity patters in Koinadugu District
(Facility based data: 2004-June 2008)
18000
16000
15671
Malaria
Diarhoea
14000
12000
12740
ARI
11828
10708
10445
STIs
10325
10000
8008
8000
7545
6233
6000
4767
4357
3458
4000
3111
2653
3114
3359
4129
3896
2880
2062
2000
0
2004
7/16/2015
2005
2006
NONIE, AfrEA, 3IE Cairo
Conference 2009
2007
2008
(Source: DHMT Koinadugu)
Mortality trend in Koinadugu District
(Facility based data: 2003-June 2008)
DEATHS UNDER FIVE 2003 TO JUNE 2008
180
160
140
120
100
80
60
40
20
0
161
91
65
63
59
44
7/16/2015
2003
2004
2005
2006
NONIE, AfrEA, 3IE Cairo
Conference 2009
2007
2008
(Source: DHMT Koinadugu)
sustainable
Component
Baseline
figure
End line
figure
Project
target
Health Status of the
participating population
37%
64.2%
44%
Health Services
Characteristics
32%
80.1%
43%
Local organizational capacity
42%
86.7%
57%
Local organizational viability
10%
35.0%
50%
Community capacity
70%
100%
80%
Political and policy
environment
26%
29.1%
40%
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
Sustainability Dashboard
Comparison between Baseline, endline & project target
Health staus
100
80
Political & policy enviroment
60
Health Services
40
20
Baseline
0
End line
Project Target
Community capacity
Organizational Capacity
Organizational Viability
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
How Sustainable?
 Community structures is strong, will ensure
sustenance of positive behaviors and demand for
services.
 Though behaviors will sustain as benefits are very
apparent to the community, there is a need for
focused plans for supervision and monitoring from
MoHS and DHMT. The District HMIS needs to capture
levels of community support through simple indicators
 Strong savings and loan groups serve emergency
needs, however mobility, outreach, access suffers due
to road conditions
 Health providers capable, competent and confident,
will however require refresher trainings, continued
supportive supervision and payment of salaries to
maintain level of activities.
 Continued engagement of VDCs and DLC by the DHMT
will ensure the monitoring of community level
activities
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
Continuing Challenges
 Roads
 Communities which are not close to PHUs or
outreach points, continue to have a barrier in
accessing immunization services on a regular
basis. The timeliness of the vaccination
seems to be affected by this factor despite
increased knowledge and acceptance of the
behavior.




Economic constraints and shortage of food
Uniform availability of all the entitlements
ITNs for males and older children
Availability of adequate trained human resources
and their remuneration
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
Key Lessons Learnt







CHC strategy led to an amplified community action,
promoted community led advocacy and scaling up of
activities
Involvement of men ensured that positive health decisions
were being made for mothers and children
VS&L is a good strategy to overcome economic barriers to
health care and provides resources for other developmental
activities
While bye laws of VDCs have helped promote health, they
require monitoring to ensure appropriateness.
Complementary activities by other NGOs and governmental
bodies contributed to the impact of the program
The partnership of TBAs and PHU worked positively for both
partners
Involvement of local authorities made maternal neonatal
health as everybody’s agenda
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
Contribution of participatory impact evaluation to an
effective learning process

The District Medical Officer (DMO) during the debriefing with
national-level stakeholders presented the findings of the final
evaluation in the presence of MOHS senior officials, donors,
and other NGO representatives. This exhibited a great
ownership of the government of the project as well as the
evaluation findings, which was acknowledged as one of the
project successes by the participants of the debriefing.

A national-level MOHS officer participated throughout the data
collection and synthesis for the final evaluation exercise. This
contributed to the perceived credibility of the evaluation
findings by senior MOHS officials and donors, which led to their
willingness to take the best practice / lessons learned /
recommendations forward. Interventions such as the birth
waiting homes, the pregnant women support group, the village
savings and loan were recognized as such best practices and
the discussion was held on the scale up to other parts of the
country.
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
Contribution of participatory impact evaluation to an
effective learning process- Continue

Above-mentioned MOHS participant further used his experience with
CSP final evaluation into another project implemented by another
NGO: He replicated the evaluation methodology to assist baseline
survey for the Health Unlimited child survival project in Bombali
district. He indicated the usefulness of the CSP final evaluation
questionnaire, sampling, and qualitative data collection methodology
as somewhat standard approach that could be used for any
reproductive and child health programming in future.

The donor who participated in the process of impact evaluation is now
spontaneously disseminating lessons learned from the project in
different national-level forum.

CARE-SL sister projects are using lessons learned during their
implementation. Also implications drawn from the impact evaluation
are made use of to develop new programme within CARE Sierra Leone.
These include infant and young child feeding programme; social
analysis on family planning to explore integration of family planning
into current programming; and advocacy and research project on
maternal health.
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
Live save calculator
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009
THE END
THANK YOU
7/16/2015
NONIE, AfrEA, 3IE Cairo
Conference 2009