RCT Case Study
Download
Report
Transcript RCT Case Study
Power to the People
Evidence from a Randomized Field
Experiment on Community-Based
Monitoring in Uganda
Martina Björkman, IGIER, University of Bocconi, & CEPR
Jakob Svensson, IIES, Stockholm University, NHH, &
CEPR
Background
Millions of children die from easily preventable
causes
Weak incentives for service providers
Top-down approach to monitoring also lacks
appropriate incentives
Recent focus on strengthening providers’
accountability to citizen-clients
Beneficiaries lack information
Inadequate participation by beneficiaries
Research Questions
Can an intervention that facilitates community-based
monitoring lead to increased quantity of health care?
Increased quality of health care?
Did the intervention increase treatment communities’
ability to exercise accountability?
Did the intervention result in behavioral changes of
staff?
Intervention
50 rural dispensaries in Uganda
Drawn from 9 districts
Households w/in 5 km catchment area
18 local NGOs
Provide communities with information on relative
performance
Encourage beneficiaries to develop a plan that identified
steps the provider and community should take to improve
service performance and ways to get the community more
actively involved in monitoring
Intervention Specifics
Pre-intervention survey data used to compile unique
“report card” for each facility
Translated into community’s main language
Posters by local artist for non-literate
Information provided to community through
participatory / interactive meetings
Community: suggestions summarized in action plan
Staff: review & analyze performance
Interface: contract outlining what needed to be done, how,
and by whom
Timing
Intervention intended to “kick-start” community
monitoring
Mid-term review after 6 months, but no other outside
presence in communities
Not able to document all actions taken by communities
Data
Pre-intervention survey to collect data for report
cards
Post-intervention survey 1 year after intervention
Quantitative service delivery data from facilities’ own
records
Households’ health outcomes, perceptions of health facility
performance parameters
Whenever possible supported by patient records
Child mortality (under 5)
Weight of all infants
Roughly 5000 randomly-sampled households in
each survey round
Evidence of Increased
Monitoring
More than 1/3 of Health Unit Management
Committees in treatment communities reformed or
added members; no change in control communities
70% of treatment communities had some sort of
monitoring tool (such as suggestion boxes,
numbered waiting cards, duty rosters); only 16% in
control communities
Performance of staff more often discussed at local
council meetings in treatment communities
NGO reports suggest that discussions shifted from general
to specific issues regarding community contract
Treatment Practices
At facilities in treatment communities significantly:
More likely to have equipment used during exam (19%
increase)
Shorter wait times (10% decrease)
Less absenteeism (14%age points lower)
More on-time vaccinations
Larger share received information on dangers of selftreatment and family-planning
Also possibility of less drug-leakage
Utilization
At facilities in treatment communities
significantly:
Higher utilization of general outpatient services
(16%)
More deliveries at the facility (68%)
From household surveys:
Consistent increases in use of treatment facilities
Reduction in visits to traditional healers & the
extent of self-treatment
Health Outcomes
Child mortality
3.2% in treatment communities
4.9% in control communities
90% confidence interval for difference ranges from 0.3%-3.0%
Corresponds to roughly 540 averted deaths (per 55,000
households in treatment communities)
Infant weight
Compare distributions of weight-for-age (z score)
Difference in means is 0.17 z score
Reduction in average risk of mortality based on risk of death from
infectious disease among underweight children estimated to be
8%
Institutional Issues
Did district or sub-district management react to
intervention?
Check that treatment & control communities have
comparable:
Monthly supply of drugs
Funding
Construction or infrastructure improvements
Visits from government or Parish staff
Employment (dismissals, transfers, hiring)
External Validity
Idiosyncratic process differed from
community to community in experiment
In another context, process could play out
entirely differently
Cultural factors key
Scaling Up
What actually caused the observed effects?
How to replicate the intervention?
Process dependent on NGO facilitators
No way to know which components of monitoring were
influential
An Alternative Explanation
Possible (but unlikely) that intervention directly
influenced providers’ behaviors
Outcomes not necessarily result of increased monitoring
Considered additional treatment of staff meetings
only but decided against it
Financial reasons
Ethical reasons
Conclusion
?
Impressive effects, but intervention difficult to
replicate
Important piece of causal chain undocumented