Transcript Title of Presentation Subhrendu K. Pattanayak RTI

3ie Delhi Seminar Series
Shame or Subsidy
What explains the impact of Total Sanitation
Campaign
Sumeet Patil 1
S. K. Pattanayak 2, K Dickinson 3, J-C Yang 3, C. Poulos 3
1 NEERMAN, Mumbai (formerly RTI International)
2 Duke University, USA (formerly RTI International)
3 RTI International, USA
Study Team Partners
2

Rajiv Gandhi Drinking Water Mission, Govt of India

Orissa State Water and Sanitation Mission and Department
of Rural Development –

World Bank – Funders

WHO, USAID, UNICEF, ICMR - Multidisciplinary technical
advisory group

RTI International – Principal investigating agency

Duke University – Analysis and publication phase

NEERMAN – Analysis and publication phase

TNS Mode – Survey agency
Overview of Presentation
 Policy
Context for Study (4 slides)
 Study
Objectives (1 slide)
 Intervention
(3 slides)
 Methodology
 Results
and Implementation (9 slides)
(12 slides)
Approximately, 45 minutes
3
Race against Time
Source: WSP 2009 calendar
4
Policy Context for Study

Child Diarrhea - key underlying link for India’s MDG
targets

Sanitation is expected to break fecal-oral transmission
and thus improve health

Universal access to toilets (no open defecation) by
2022 is a goal of Nirmal Bharat Abiyaan (NBA)

Heated comparison between supply “pushed” subsidy
based TSC and CLTS based demand driven “no
subsidy” based approaches

Limited evidence to guide implementers and policy
makers
5
Govt M&E/MIS data highly unreliable
Source: Chambers and Von Medeazza (2013): working paper
6
Policy Context for Study

Evidence to make determination is very thin



Impact evaluation in sanitation sectors (are (were) few
Cross-sectional assessments (lacking baseline, control,
statistical power)
Need for rigorous impact evaluation

2005 RCT. Hammer and Spear (2013). Working Paper
 2006 RCT. Pattanayak et al. (2009). This paper
 2011 RCT. Patil et al. (2013). Working Paper
 2011 4-arm QE. We hope that endline happens
7
Overview of Presentation
 Policy
Context for Study
Study
Objectives
 Intervention
 Methodology
 Results
8
and Implementation
Study Objectives

Whether CLTS based behavior change coupled with
subsidy based intervention (TSC) impacts latrine use
and child health?



9
Track the logic chain from inputs to intermediate outputs
to outcomes to health impacts
Generate operational knowledge to guide policy
Ability to study the effect of Shame only and shame +
subsidy because of the TSC program design feature
Overview of Presentation
 Policy
Context for Study
 Study
Objectives
Intervention
 Methodology
 Results
10
and Implementation
Community-Led Total Sanitation in
Bhadrak
Knowledge
Links
11
Intervention: Community-Led Total Sanitation
(Kar, IDS)

Knowledge alone does not change behavior; need
to create “triggering events” and intensive Behavior
Change campaign
 “walk of shame”


12
“defecation mapping”
“fecal calculation”

TSC related Incentives for BPL for latrine
construction (Rs 1500)

Supply side: masons, rural sanitation mart, know
how, motivation, monitoring

Immediate outputs: Out of 20 villages, 9 resolved
to end OD, 2 agreed in principle, 5 decided to meet,
and 4 were unable to reach a consensus
CLTS Program – Logic Model
PROGRAM INPUTS
PROGRAM OUTPUTS
PROGRAM OUTCOMES
PROGRAM IMPACTS
Improvements in child
health
Subsidized labor & materials
Use of IHL
IHL construction know-how
Emphasis on
dignity & privacy
Number of IHL
constructed
Personal benefits
Satisfaction with IHL
Communication on
water-washed diseases
Broader welfare
impacts
Budging social norms
MEDIATING FACTORS
Knowledge
Attitudes
Preferences
Community-led
Total Sanitation (CLTS)
13
Overview of Presentation
 Policy
Context for Study
 Study
Objectives
 Intervention
Methodology
and
Implementation
 Results
14
Study Design

Randomized Control Experiment


Well controlled. Random and blind assignment of
treatment
Sample Size: 20 CLTS villages + 20 control villages and
25 HHs per village (with u5 children)

Baseline (2005) and Endline (2006)

Panel Surveys


Difference in Difference (DID) estimation of impacts

15
2 rounds, same season, same households
Difference: Before and After and With and Without
Sample Selection

Selected a district (Bhadrak) with adequate water

Selected blocks (Tihidi & Chandbali) without prior TSC

Restricted villages to have >70 HHs and < 500 HHs

Restricted to 1 village per GP to reduce spill over

Selected 40 villages & randomly assigned 20 to
treatment

Listed and mapped all households in 40 villages

Randomly selected and surveyed 25 households with
child < 5 yrs in each village
16
Study Villages
20 .- C o n tro lle d V illag e s o f
R T I - W B S tu d y
Naya nanda
10
0
10 K m .
B ir a b a r p u r
N
A ig iri a B u d h a p u r
M a n g r a jp u r
H a r ip u r
W
E
Te n tu l id a
B a r ik p u r
A rju n b i n d h a
T IH ID I
S
B L O C
K
Nuas ahi
Sa san kh as
Am arpur
B a d a p im p a li
P a d is a h i
S a n s am u ka b ed h i
B a li p a d a
S a tu ti
B h im p u r
R a jn a g a r
B a in c h a
O r a li
H e n g u p a ti
H a ta p u r
M adhupur
G o u rip ra sa d
Ta la b a n d h a
J a s h ip u r
Ta la d u m k a
B a li a rp u r
S a n a s in g p u r
Bahu
C H A N D B A L I
B L O C K
Guanal
B e g u n ia
B a li s a h i
J a la d h a r p u r
A m b o lo
B h u in b r u ti
D e u li g a a n
Te n tu l id a
J a y d u r g a p a tn a
D h u r b a p a h a lip u r
LEG END :
C o n tr ol v illa g e
Villa g es
17
Data: Measurements
18

Outputs, Outcomes, Impacts:
 Household pit latrines (IHL): constructed, operational and in-use
 Diarrhea frequency & severity (> 3 episodes in 24-hr, 2-week
recall)
 Child growth (anthropometrics – MUAC, weight, height)

Additional parameters:
 Individual - sex, age, class, caste, religion
 Household - family size and composition, education, housing
conditions, asset holdings, occupation and expenditures, services
 Community – roads, electricity, environmental sanitation,
employment, clinics, schools, credits, markets
 Institutional - main governmental and NGO programs, local
government size and composition
 Water quality (E. coli & total coliform) – community sources (all),
in-house (50%)
Data: Household Survey

Respondent - Primary Care Giver

Water samples collected from approx 50% of surveyed households

Modular questionnaire
 Knowledge, Attitudes
 Household SEC
 Sanitation Behaviors – outputs and outcomes
 Hygiene Behaviors
 Water Sources and their use
 Water Treatment/safety behaviors
 Food safety behaviors
 Environmental conditions – HH and community
 Budget constraints
 Community Participation
19
Data: Community Survey

Respondent – sarpanch, GP member, Informal leader, Doctor, etc

Water samples collected from up to 10 in-use drinking water
sources

Modular questionnaire design
 Background: population, households, area, arable land, major crop
grown






20
Public infrastructure: roads, water supply, sanitation, hygiene,
electricity, clinics, schools, STD booths, telegraph offices, post offices,
credits and markets
Environmental sanitation: general cleanliness, drainage, animal and
household waste, use of water sources, open defecation practices
WSS scheme: Swajaldhara, piped water, hand pumps, etc
Development Programs: Health, education, women support etc
Economy: employment opportunities, major governmental and NGO
programs, prices
Local government: structure, composition, activities
Survey Implementation - I

Schedule & Resources

1 month of data collection to catch the monsoons!!
 Field Teams – RTI (3 + 1 consultant) and TNS (30 field
people + 2 researchers)
21

Focus groups

Pre-testing (2 rounds of 50 household surveys)

Training (8 days. Mix of in-class and field
practice)(manuals prepared)

Supervision: Supervisors  executives 
Managers  Researchers. Back checks, spot
checks.
Fieldwork
22
Survey Implementation - II


23
Data Processing

On field editing, 100% scrutiny before data entry

CSPro based data entry

Cross-tabulation based cleaning
WQ Samples

50% HHs and up to 10 in use sources.

Sterilized bottles

Cold chain transport to lab within 24 hours
Overview of Presentation
 Policy
Context for Study
 Study
Objectives
 Intervention
 Methodology
Results
24
and Implementation
Baseline Balance - I
SEC
From scheduled caste
From other backward classes
Below poverty line
WASH
Used improved water sourced
Boiled or treated drinking water
Adults washed hands at 5 critical instances
Dumped garbage outside of house
Threw wastewater in the backyard
With individual household latrine
25
T
C
p-value
28
29
60
26
24
61
0.858
0.449
0.91
37
9
11
68
46
6
42
13
9
69
48
12.7
0.602
0.192
0.564
0.794
0.705
0.03
Baseline Balance - II
Attitudes
Completely dissatisfied with current sanitation
Water supply is most important improvement
Sanitation is most important improvement
Women lack privacy during defecation
Women are not safe defecating in the open
Government should bear the cost of sanitation
Health
U5 diarrhea in past 2 weeks
(MUAC)-for-age z-score for U5
height-for-age z-score for U5
weight-for-age z-score for U5
26
T
C
P-value
72
7
5
32
29
53
61
12
8
30
29
50
0.011
0.149
0.264
0.82
0.463
0.561
28
-1.3
-2
-2.2
23
-1.3
-1.9
-2.3
0.218
0.677
0.687
0.341
Estimation

27
% Households owning & using Toilets
(by intervention and year)
DID= 26%-0%= 26%***
40%
32%
30%
20%
10%
26%
13%
0%
13%
6%
0%
2006
2005
CLTS Villages
I indicates the 95% confidence interval.
28
2006
2005
Control Villages
E Coli Levels in HH Drinking Water
25
20
15
10
5
0
2005
2006
CLTS
29
2005
2006
Control
Elusive Health Impacts
U5 diarrhea Prevalence
MUAC-for-age z-score
Height-for-age z-score
Weight-for-age z-score
30
BL/EL
T
C
T-C
0
27%
23%
4.30%
1
14% 15% -0.60%
0
-1.34
1
-1.2 -1.32
0
-1.95
1
0
1
-1.33
-1.94
-2.01 -2.29
-2.16
-2.25
-2.22 -2.3
-0.011
0.123
-0.007
.273*
0.088
0.069
DID
-4.90%
0.133
0.281
-0.192
Shame or Subsidy?
Impact (mean test with EL)
Impact DID
31
Full sample
19
28.7
BPL
23.7
34.2
APL
12
20.7

Triple Difference to get the relative effect of
shame and subsidy

BPL = Subsidy + Shame and APL = Shame alone

DID for BPL – DID for APL = 34.2 – 20.7 = 13.5%

13 % effect (about 1/3rd) by the “subsidies”
Is this result replicable elsewhere?

Another RCT in Madhya Pradesh

A scaled up and more “realistic” program

50% to shame + “less subsidy” and 50% to
shame and “more subsidy (by Rs 2700)
Overall
Poor
Non-poor
32
Control
N
Mean
1514
0.22
375
0.17
1139
0.24
N
1525
300
1225
Treatment
Difference
0.19 (0.035) ***
0.32 (0.046) ***
0.15 (0.037) ***
Are effects sustainable?
2004
2005
2006
2006
2007
P e rc e n t o f H o u s e h o ld s O w n in g a L a trin e ,
T re a tm e n t V illa g e s
G oO D ata
H H S urvey
H H S urvey
C om m unity S urvey
G oO D ata
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
33
a
uk
du
la
Ta
la
Ta
m
nd
as
ba
kh
an
as
S
ha
i
ed
av
r
am
uk
ga
hi
na
aj
as
R
an
S
P
ad
hi
sa
hi
r
ua
sa
pu
N
hu
ad
M
H
at
ap
as
ur
ad
ur
G
ou
ra
pr
al
ah
pu
ub
ap
ha
od
B
hr
ip
r
r
pu
r
ra
pu
B
ira
ba
hi
m
B
un
ia
ur
ik
p
ar
eg
B
a
ad
B
al
ip
B
al
ia
rp
ur
la
pa
B
im
D
B
ad
ap
A
m
bo
la
0%
Findings from Mixed Methods - I

34
Some factors indicate “possibility of sustainability”

Increased satisfaction with sanitation situation

Increased belief that improving sanitation is the family’s
responsibility

Lack of knowledge of the “germ theory” is not the
most important BUT privacy and dignity are key

Households prioritize. Toilets may be “our” priority,
not theirs: 80% want health dispensary, 59% roads.
Compare to 7-9% for water supply and sanitation
Findings from Mixed Methods - II

Support structure – NGOs, district officials,
involvement of triggering team, village institutional
capacity are important success factors

Subsidies are tricky business
may have created an incentive for NGOs to “cut
corners” and produce lower quality latrines
 Concern that subsidies in general defeat the sense of
self-reliance
 Will subsidy be counteractive in long term?


How and when you give subsidies will matter

Community based incentives (e.g. NGP) instead of
individuals?
 Is “post” incentives practical for poor population?
35
7 years later…



36
Credible evidence that “shaming” works

BUT, so do subsidies

BUT, does the relative contributions depend on
“intensity” of CLTS or amount of subsidy?
Seems to be continued increase in toilet
coverage

BUT, what about use? And toilet maintenance?

BUT, will we reach 100% open defecation free status?

BUT, what about health impacts?
7 year later, we still stare at above critical
questions without credible answers
Thank You
37

Sumeet Patil:

Other papers
[email protected]

Pattanayak et al. (2010), “ “How valuable are
environmental health interventions?...” Bull WHO,
88:535-542.

Pattanayak et al. (2009), “Shame or subsidy
revisited:…” Bull WHO, 87:1-19.

Pattanayak et al. (2009), “Of taps and toilets….”, J of
Water and Health, 7(3): 434–451.

World Bank (2011). “Of Taps an Toilets”. WB report on
Evaluation of CDD program in RWSS.