Transcript Research, Advocacy & Spread

Arnab Acharya
8 February 2013
Authors: Acharya, Lalwani, Dutta,
Ruducha, Varkey, Rajaratnam,
Menezes, Mehta, Wunnava, Taylor
and Bernson

Sure-Start Project
◦ Program Description and Content
◦ Theory and Motivation
Basic Questions
 Evaluation Design and Survey
 ITT
 Mother’s group impact
 Sub-group Analysis
 Conclusions

3iE Presentation-Delhi
7-Jul-15
2
Discuss Sure Start as a learning project: A
large-scale project implemented in a random
way
Present results to detect the intention to treat
effect between the two randomised arms
Explore the plausibility of the theory of change
through LATE
Sub-group analysis: Use of PSM
Conclusions
3iE Presentation-Delhi
7-Jul-15
3





Implemented in 7 districts of Uttar Pradesh,
complemented and supported National Rural
Health Mission (NRHM), a GoI sponsored program
Promoted safe MCH behaviours and services
through community mobilization efforts
NRHM advocated activities
PATH implemented project using a partnership
model working with 5 lead partners at state level,
and 55 district-level partners
Funded by Gates Foundation as a learning project
3iE Presentation-Delhi
7-Jul-15
4




Project used community level structures set
up through NRHM
Raised awareness around essential maternal
and new born care behaviours and services
Mentored community-level health workers
called Accredited Social Health Activists
(ASHAs)
ASHAs mobilized and interacted with women
in Mother’s Group (MG) meetings and home
visits, in more intensive intervention areas
3iE Presentation-Delhi
7-Jul-15
5
It is a learning project: there are three
hierarchical levels of implementation
1. Advocacy Awareness generation, through media
campaign (Level 1)
2. Level 1+Direct Contact at the village level by
ASHA to promote safe pregnancy and neo-natal
period, along with VHSC strengthening the
support system for the frontline workers (Level 2)
3. The last level was not implemented fully
3iE Presentation-Delhi
7-Jul-15
6





Placed in 7 districts, 1-2.8 mil/dist
Level 1 in all areas: mid media
Level 2 in 40% of areas in each block (pop
100-150,000), implemented since 2008, in a
randomised manner
Randomised field implementation to learn
how well scaling up can work (NOT A RCT)
L2: Unit of Implementation
◦ Gram Panchayat Level
◦ Village level activities
3iE Presentation-Delhi
7-Jul-15
7

Uttar Pradesh
◦ One of the poorest places in the world
◦ Population the size of Brazil with 1/12th the land
mass
◦ GDP per capita of Kenya at $809, India $1,265
(2010)
 PPP: UP $2000, India: $3300
◦ Very poor health infrastructure, violence and
corruption associated with NRHM
3iE Presentation-Delhi
7-Jul-15
8





Small-scale trials in Nepal, Orissa and Uttar
Pradesh (Lassi et al. SR, 2013).
Community-based interventions with active
recruitment of mothers to group meetings
All of them did some interventions on the
supply side: payment to health workers or
health system strengthening (Nepal: both
control and treatment)
Small in size
SS: Recommended NRHM framework to learn
about large-scale implementation results
3iE Presentation-Delhi
7-Jul-15
9
L1
•Mid media
•Advocacy with
RKS and DHS
+
ANM
ASHA
AWW
Outcomes
Improved
women’s
awareness
knowledge of
birth
preparedness,
complication
readiness &
healthy
maternal and
newborn care
RKS
•Mid media +
•Mentoring of
ASHAs
•Facilitation &
capacity
building of
VHSCs
DHS
L2
Outputs
Increased interaction Between
Providers and Women and
Families, women attending
mother’s group
Sure Start
Interventions
Target Groups
VHSC
Inputs
3iE Presentation-Delhi
Impact
Improved
Women’s
MNH
practices
Decreased
NMR
Increased
utilization
of MNH
services
7-Jul-15
10



Do more intensive intervention areas have
better health impact? This is the ITT effect!
Was the pathways to change through
mother’s group meeting? Local Average
Treatment Effect!
Who attended the mother’s group? A subgroup analysis
3iE Presentation-Delhi
7-Jul-15
11
Did Sure Start interventions Improve:
 Maternal and Neo-Natal Health?
◦ Drop in NMR
◦ Delivery Complications (through safe practices)

Knowledge regarding pregnancy health?
◦ Recognition of danger signs during: pregnancy,
birth and post-natal policy
3iE Presentation-Delhi
7-Jul-15
12






5/7 Districts were chosen (Rae Bareli, Basti,
Gorakhpur, Hardoi and Balrampur)-two least typical
eliminated from the study
Each block had >10 clusters, 4 clusters (panchayat)
randomly assigned during implementation to L2, rest
L1
5 blocks were selected randomly, for the evaluation
5 panchayats each from L1 and L2 selected for the
evaluation
1 or 2 villages were randomly chosen from these
panchayats.
Pregnant women more or less surveyed as a census in
each villages, pregnant during: baseline 2007 and
endline 2010
3iE Presentation-Delhi
7-Jul-15
13

Survey of 100 villages in each of the 5
districts
◦ About 20 per village, But ended up with 12,000
women at baseline and endline

Administered surveys
◦
◦
◦
◦

pregnant women
ASHA health workers
VHSC members (ASHA, Pradhan, AWW, ANM)
Implementers (Block Medical Officer, District
Magistrate, Chief Medical Officer)
We use only the survey from pregnant women
in this presentation
3iE Presentation-Delhi
7-Jul-15
14
Sample Calculations, was based on 0.6 to fall
to 0.4*
 More or less balanced sample between level 1
and 2
 NMR is much lower than expected: we
thought the sample had enough slack if there
is no change in the control
 The control did change also
*Sample Calculation Issues can be quite
complex, a simple one is adopted

3iE Presentation-Delhi
7-Jul-15
15

NMR is related to the following as obtained
from baseline—also based on Bhalotra and
van-Soest (2008):
Education, wealth reduce mortality
First birth, and higher parity increase risk
Newborn care reduces NMR
Quality of ANC care is occasionally helpful in some
formulation
◦ Institutional Delivery has no effect
◦
◦
◦
◦
3iE Presentation-Delhi
7-Jul-15
16


We can be fairly confident that we
randomised. Baseline SES and key health
variable are similar between L2 and L1
SES and demographic variables were used for
the regressions measuring the impact
3iE Presentation-Delhi
7-Jul-15
17
Variable
Intensive Area
Less Intensive
Area
P-Value
4.52
4.33
0.607
NMR
NMR(%)
Labour and delivery
dangers signs
Post-partum danger
signs
0.33
0.88(1.28)
.87( 1.27)
0.14
1.45(1.66)
1.42(1.63)
Mother Literate
2.28(2.23)
33.43
2.25( 2.23)
34.5
0.207
Semi-Pucca
44.15
43.81
0.708
Age at first birth
20.46(2.86)
20.42(2.81)
0.25
First birth (alive)
22.44
23.53
0.15
During pregnancy
danger signs
3iE Presentation-Delhi
0.21
7-Jul-15
18

Overall Question: Did Sure Start (L2) village
women experience better health outcome
than (L1)? Intention to Treat (ITT) effect:
◦ Add village or Panchayat random effect as well a
following epidemiological approach, use endline
alone:
3iE Presentation-Delhi
7-Jul-15
19


Differences in Difference Measure in time
We use this as a check for the variables that
are continuous and those with prob mass
away from 0
3iE Presentation-Delhi
7-Jul-15
20
Intervention Area with
Intensive Program (L2)
Comparison Area with
Mid-media program
(L1)
NMR
Baseline
Endline
43.33
34.2
45.2
36.4
Pregnancy Compli
Baseline
Endline
37.96
21.18
38.16
25.51
Delivery Compli
Baseline
Endline
11.83
4.74
11.72
5.69
Post-partum Compli
Baseline
Endline
21.38
12.13
21.71
14.32
3iE Presentation-Delhi
7-Jul-15
21



A very large program, there may be variations
in implementation
Outcome may depend on health system which
was not part of the intervention
Intermediary steps may be good measures of
some program results
3iE Presentation-Delhi
7-Jul-15
22
Intervention Area
with Intensive
Program (L2)
Comparison Area with
Mid-media program
(L1)
20.04
28.52
19.47
25.16
Iron-folic Tablets
Baseline
Endline
44.94
76.75
45.72
70.75
Colostrum
Baseline
Endline
63.26
72.26
63.31
61.82
Breast Milk within/1
hr
Baseline
20.04
28.52 3iE Presentation-Delhi
Supplementary
Nutrition
Baseline
Endline
19.47
25.16
7-Jul-15
23

Some of the features of the program did not
reach all women; programmatic factors, % of
all women:
Intervention Area
with Intensive
Program (L2)
Comparison Area
with Mid-media
program (L1)
Heard SS message in
media
40
30
Heard about MG
Meeting
68
14.4
Actually attended MG
42
4.6
Have you seen SS logo
57.8
32.27
Did ASHA visit you at
home
81.39
89.31
3iE Presentation-Delhi
7-Jul-15
24


Carry out logistic regression for those that
have low probability
DD regression method can capture these
differences as well as time trend.
3iE Presentation-Delhi
7-Jul-15
25
Logistic
ODDS Ratios
Random Effect
Health
NMR
DD Coefficient and
Significance
0.954
Experiencing complication
Labor and delivery
-0.0345
Postpartum
-0.0879*
During pregnancy
Accessing Care
ANC
-0.152**
1.35**
.055641***
2 tetanus injections
1.307***
0.0322***
Iron-folic tablet
1.342**
0.0674***
0.95
0.01846
Supplementary nutrition
JSY received
3iE Presentation-Delhi0.0347***
7-Jul-15
1.165
26

Strategy to obtain MG impact
◦ Rely on Local Average Treatment Effect estimation
◦ The clear instrumental-variable for attending
mother’s group meeting is the assignment of L2
and L1 (as a percent of women attending MG in a
village)
◦ We capture block level variation by interacting block
identifiers with L2 and L1 assignment for the first
stage regression with a woman attending mother’s
group as the LHS variable
(See Bjorkmann and Svensson, 2007).
3iE Presentation-Delhi
7-Jul-15
27



The mother’s group impact should be
understood as local to the way we
implemented the program
However, it tells us that given the Sure Start
effort whether or not it was mother’s group
meeting that can contribute to better health
practices and health
Monotonicity conditions were met
(see Angrist, Imbens and Rubin, 1994)
3iE Presentation-Delhi
7-Jul-15
28




The main pathways to change in behavior is
mother’s group
Only 42% of the mothers attended the
mother’s group
Can we attribute the changes to mother’s
group activities?
Put it another way: Should we actually make
sure that we should increase the effort for
mother’s group attendance to get better
result?
3iE Presentation-Delhi
7-Jul-15
29
Logistic
ODDS Ratios
Random Effect
Health
NMR
DD Coefficient and
Significance
LATE
Local MG
impact (OLS)
0.954
-0.00148
Experiencing complication
Labor and delivery
-0.0345
-0.169**
Postpartum
-0.0879*
-0.435***
-0.152**
-0.716***
1.35**
.055641***
.1196236***
2 tetanus injections
1.307***
0.0322***
0.0590**
Iron-folic tablet
1.342**
0.0674***
0.147***
Supplementary nutrition
0.95
0.01846
.3318***
JSY received
1.165
3iE Presentation-Delhi
During pregnancy
Accessing Care
ANC
0.0347***
7-Jul-15
0.0911
30

Mother’s group were attended by schedule
caste at a greater rate than any other group
◦ General Population: 42% attended mother’s group
◦ 55% of SC/ST attend MG meetings while 27% and
37% of Muslims and other caste group attend MG
meetings

Question: Is the impact of L2 stronger on this
group than it was for everyone else in L2?
3iE Presentation-Delhi
7-Jul-15
31



Match the L2 SC/ST women with L1 women
who are SC/ST (as they have not been offered
the program at all)
Find propensity score and use the weights to
regress women’s outcome
The results confirm that for SC/ST the
benefits were stronger
3iE Presentation-Delhi
7-Jul-15
32
Outcome
General Population
Impact
(ITT)
SC/ST Impact
(ToT)
0.954
0.742
Labor and delivery
-0.0345
-0.0319
Postpartum
-0.0879*
-0.169**
-0.152**
-0.295***
1.342**
2.913***
1.165
1.939***
Health
NMR
Experiencing complication
During pregnancy
Accessing Care
Iron-folic tablet
JSY received
3iE Presentation-Delhi
7-Jul-15
33


We obtain no statistical significance results
for the main outcome of interest--NMR. It
may be that our study may not have been
powered enough to detect differences for the
change in time, when this change is so large
for both control and treatment areas
We can conclude that there is trend toward
better health in the more intensive program
compared to the mid-media program when
simply intention to treat is taken into account
3iE Presentation-Delhi
7-Jul-15
34



Mothers group stands as a pathway through
which change was achieved
This analysis is measured as LATE and as ToT
for a sub-group
Impact is present from this large-scale
intervention
3iE Presentation-Delhi
7-Jul-15
35

Short-comings from this analysis
◦ No spill-over effects have been analysed
◦ MG meetings were not attended by a lot of women;
can we find positive externality of information
being delivered to only a few women (Study in
Nepal reported 42% of women attending and high
rates of change in NMR).
◦ Could caste division be a barrier to spreading
information, as the UP impact is much lower?
3iE Presentation-Delhi
7-Jul-15
36
3iE Presentation-Delhi
7-Jul-15
37