Transcript Impact Evaluation of Performance

Performance-Based Financing in
Rwanda
Agnes Soucat, Adviser HNP Africa Region
Background (1)
• Shortage of human resources for health
services
• No cash resources in health facilities
• Low levels of productivity and motivation
among medical personnel
• Low user satisfaction & poor quality of service
lead to low use.
• High levels of child and maternal mortality
2
Background (3)
• In 2005 , 4/10 births
attended by a health
professional.
• Infant Mortality : 86 per
1,000
• HIV : 3.1%
Source: Rwanda 2005: results from the demographic and health
survey. 2008. Studies in family planning, 39(2), pp. 147-152.
3
Strengthening accountability in the health sector in Rwanda
NATIONAL GOVERNMENT
VOIC
E
Umushyikirano, Citizen
Report Cards, Ombusdman
LOCAL GOVERNMENT
CLIENT POWER
Clients / Citizens
COMMUNITY
GOVERNANCE
COMMUNITY HEALTH
INSURANCES
Mutuelles
PERFORMANCE BASED,
CASH AND IN KIND
INVESTMENT INPUT
SUBSIDIES TRANSFERS
Performance
CONTRACTS
AUTONOMOUS
FACILITIES PROVIDERS
COMMUNITY HEALTH
WORKERS PROVIDERS
Rwanda has undertook major reforms
to strengthen accountability of all
institutional and individual actors for
MDGs related results...
..through a shift of paradigm..
- Decentralisation of health services with strong
governance structures based community
participation.
- Imihigo: Performance contracts between President
of Republic and mayor of Districts;
- PBF: Performance Based Financing;
- CBHI: Community Health Insurance;
- Autonomy of health facilities, including hiring and
firing of health personnel;
Decentralization
Fiscal and Financial Decentralization
70,000,000,000
60,000,000,000
50,000,000,000
Transfers to Districts
40,000,000,000
CDF
30,000,000,000
Transfers to Provinces
20,000,000,000
10,000,000,000
0
Dis
bu
r se
d2
00
2
Dis
bu
r se
d2
00
3
Dis
bu
r se
d2
00
4
Dis
bu
r se
d2
00
5
Bu
dg
et
20
06
Pr
oje
cte
d2
00
7
Amount in RWF
• Administrative, fiscal
and financial
decentralization has
provided huge sums of
money to local levels of
government and given
them much flexibility by
providing them with
block grants
Year
Total health personnel in publicly funded facilities has almost
doubled in 3 years …
Total staff
14000
13133
12000
10000
8000
Total staff
6961
Linear (Total staff)
6000
Linear (Total staff)
4000
2000
0
2005
2008
Financing has more than tripled in four years (going from USD 7.5 to 30.3 millions,
of which the PBF
has grown more than tenfold from USD 0.8 to 8.9 millions)
25
21.4
20
18.5
Basic salaries (Million USD)
15
Performance based financing (Million
USD)
11.2
Linear (Basic salaries (Million USD))
10
8.9
8
Linear (Performance based financing
(Million USD))
6.7
5
3.6
0.8
0
2005
2006
2007
2008
Rwanda: Scaling up of community
health insurance
%
Proportion of individuals enrolled in
health insurance
90
80
70
60
50
40
30
20
10
0
2002
2003
2004
Source: MOH Rwanda; 2005 EICV 2005
2005
2006
2007
2008
Results show Rwanda is now back on
track towards the health MDGs…
• Health outcomes
– Neonatal, infant and child mortality
– Malaria incidence and mortality
– HIV
– Improved financial access
– Reduction of catastrophic expenditures
• High Impact Interventions
– ITNs
– Family planning
– Assisted Deliveries
Rwanda is back on track to reach the MDGs
Under five mortality trends with MDG target for 2015
250
1990 level
U5MR per 1,000
200
MDG target for 2015
150
Observed
100
Trends since 1998
50
Trends required to
reach the 2015 target
0
1999
2001
2003
2005
2007
2009
2011
2013
2015
All income groups benefit but inequities still persist …
Under five mortality trends by income quintile (2005-2007)
250
211
U5MR per 1,000
204
195
200
161
170
149
150
132
141
122
100
84
50
0
Poorest
Q uintile 2
Q uintile 3
DHS 2005
Source: DHS 2005 and 2007.
DHS 2007
Q uintile 4
Richest
% of sick who sought care from modern health service provider
Rwanda Health Insurance
At all income 60
levels, those
50
enrolled in
mutuelles are 40
much more likely
30
to use health
20
services.
Source: Shimeles et al, 2009
enrolled
50
43
non-enrolled
42
40
42
33
31
26
25
25
21
14
10
0
Poorest
2
3
4
Quintiles
Best-off
Overall
%
Rwanda: Effect on MDGs High Impact
Interventions
100
90
80
70
60
50
40
30
20
10
0
2000
2007
% delivered in
a health
facility TOTAL
DPT3 (%)
Currently
Using any
modern FP
method (%)
% U5 who
slept under an
ITN the past
night
Rwanda : Increase in utilization of high
impact services
Proportion (%) of children under 5 years of age who have slept under a mosquito
bed-net during the night preceding the survey
Increase in utilization of high impact
services
Trends in assistance at delivery : Years 2000, 2005, 2007
Percentage (%) of women delivered by a health professional
Table 1: Output Indicators (U’s) and Unit Payments for PBF Formula
OUTPUT INDICATORS
Amount paid per unit (US$)
Visit Indicators: Number of …
1
curative care visits
0.18
2
first prenatal care visits
0.09
3
women who completed 4 prenatal care visits
0.37
4
first time family planning visits (new contraceptive users)
1.83
5
contraceptive resupply visits
0.18
6
deliveries in the facility
4.59
7
child (0 - 59 months) preventive care visits
0.18
Content of care indicators: Number of …
8
0.46
10
women who received tetanus vaccine during prenatal care
women who received malaria prophylaxis during prenatal
care
at risk pregnancies referred to hospital for delivery
11
emergency transfers to hospital for obstetric care
4.59
12
children who completed vaccinations (child preventive care)
0.92
13
malnourished children referred for treatment
1.83
14
other emergency referrals
1.83
9
0.46
1.83
Quality Conceptual Framework
What They Do:
(Quality)
Production
Possibility
Frontier
What They Know (Ability/Technology)
Goal: Use Pay for Performance to
Close Productivity Gap
Production Possibility Frontier
What They Do
Productivity Gap
Conditional on Ability
Actual
Performance
Ability/Technology
Researcher & Policy Maker Collaboration
A collaboration between the Rwanda Ministry of Health, CNLS, and SPH, the
INSP in Mexico, UC Berkeley and the World Bank
• Research Team
–
–
–
–
Paulin Basinga, National University of Rwanda
Paul Gertler, UC Berkeley
Jennifer Sturdy, World Bank and UC Berkeley
Christel Vermeersch, World Bank
• Policy Counterpart Team
–
–
–
–
Agnes Binagwaho, Rwanda MOH and CNLS
Louis Rusa, Rwanda Rwanda MOH
Claude Sekabaraga, Rwanda MOH
Agnes Soucat, World Bank
Evaluation Questions: Did PBF…
– Increase the quantity of contracted
maternal health services delivered?
– Improve the quality of contracted
maternal health services provided?
Evaluation Design
• During decentralization, phased rollout at district level
• Identified districts without PBF in 2005
• Group districts into “similar pairs” based
on population density & livelihoods
• Randomly assign one to treatment and other to control
• MOH reallocated some districts to treatment
• With decentralization, some new districts had PBF in an area of
the new district – must be treatment
• Unit of observation is health facility
26
Rollout of PBF in health centers in Rwanda, 2006 – 2008
27
Isolating the incentive effect
• PBF
– Performance incentives
– Additional resources
• Compensate control facilities with equal
resources
– Average of what treatments receive
– Not linked to performance
– Money allocated by the health center management
Sample: Panel 165 Facilities 2006-08
• 2145 households in catchment areas
– Random sample of 14 per clinic
Log Expenditures
Year
2006
2008
Treatment
15.812
(1.042)
16.906
(0.71)
Control
15.612
(1.007)
16.989
(1.08)
Difference
0.200
0.241
-0.083
(0.14)
P-Value
0.418
0.568
• Randomization balanced baseline
• Follow-up balanced, so difference in follow-up
outcomes due to incentives not resources
Baseline Expenditures & Staffing
Impact of PBF: Statistical methods
• Have balance at baseline on all key outcomes
• Use difference in differences analysis
– Not a pure randomized experiment
•
•
•
•
Clustered at district-year level
Facility Fixed Effects
Year dummy
Controls: age, parity, education, household size,
health insurance, land, value of assets
Impact on 4+ Prenatal Visits and
Facility Delivery
0.7
0.6
0.5
0.4
2006
2008 No PBF
0.3
2008 PBF
0.2
0.1
0
4+ prenatal vists
Facility Delivery
Proportion of of institutional deliveries
Delivery at the health facility increased overall in
Rwanda, but 7% more in PBF facilities ….
60.0
55.6
49.7
50.0
7.3 % increase
due to PBF
40.0
36.3
34.9
30.0
Baseline (2006)
Control facilities
Follow up (2008)
Treatment (PBF facilities)
35
Prenatal Competency & Quality
• Provider knowledge/competency
– Standardized vignette presented to provider
– Compare answers to Rwandan CPG
– Measure of ability/knowledge
• Process quality
– Patient exit interview of clinical services provided
– Clinical content of care
– Provider effort
Standardized Prenatal effort score
In the last years, PBF has increased prenatal care
quality significantly …
0.20
0.15
0.15
0.10
15 % Standard deviation
increase due to PBF
0.05
0
0.00
-0.05
-0.10
-0.15
-0.10
-0.13
Baseline (2006)
Control facilities
Follow up (2008)
Treatment (PBF facilities)
37
.2
.3
.4
.5
Kernel Non parametric regression practice-competency at baseline
Control facilities
Treatment facilities
.3
.4
Competence
.5
.6
.7
.8
.2
.3
.4
.5
Kernel Non parametric regression practice-competency at follow up
Control facilities
Treatment facilities
.3
.4
Competence
.5
.6
.7
Impact of PBF on Prenatal Care Quality
Impact of PBF on Prob of Child Preventive
Care Visit in Last 4 Weeks
0.5
0.45
0.4
0.35
0.3
2006
0.25
2008 NO PBF
2008 PBF
0.2
0.15
0.1
0.05
0
0-23 Months
24-47 Months
Impact of PBF on Child Health (z-scores)
Results Summary
• Balanced at baseline
• Expenditures same, so isolate incentives
• Impact on utilization
– Delivery & Child prevention, but not prenatal
• Impact on prenatal quality
– Bigger for better doctors
• Reduced child morbidity & Taller children
• Effect sizes bigger than
most other interventions
Discussion
• PBF Effect seen despite many other national level
intervention: possible bigger effect in other
countries
• Increase in utilization in country with national
campaigns:
–
–
–
–
Mutuelle
Imihigo
HIV services
Safe motherhood and PCIME
• Possible spill over effect to child health
45
Discussions/ Policy implications
• You get what you pay for !
• Returns to effort important
– Bigger effects in things more in provider’s control
– Patient or community health workers for prenatal
care/Immunization
• Provide incentives directly to pregnant women? (conditional
cash transfer program).
• Financial incentive to community health workers
• Low quality of care : additional training coupled with PBF
• Need to get prices “right”
• Evaluation feedback useful
46
Limitations
• The original randomized designed was
changed due to the political decentralization
process: But sample well balanced!
• Trend analysis with HMIS data ongoing
• No measure of all paid and some non paid
indicators : HMIS analysis
• Cost effectiveness analysis
47
Acknowledgments
• Funding by:
– World Bank
– Government of Rwanda (PHRD grant)
– Bank-Netherlands Partnership Program (BNPP)
– ESRC/DFID
– GDN