Transcript Health Facility Surveys and Quantified Supervisory

Health Facility Surveys and
Quantified Supervisory
Checklists
Health System Innovations Workshop
Abuja, Jan. 25-29, 2010
Health Facility Surveys – What are
they?
• Assessments of different types of health
facilities using a standardized questionnaire
• Usually done through a simple random sample
of all health facilities
• Surveyors are usually trained health workers
(often doctors)
• Usually look at many different aspects of
service delivery including technical quality of
care
2
Health Facility Surveys
Advantages
• Can assess quality
of care
• Can be
independent of
service providers
• Can be done more
frequently than
HHS
Disadvantages
• Complex to design
• Lots of data, can
overwhelm
• Cannot provide
information on
coverage, equity
3
An example from Afghanistan:
• 600+ facilities surveyed every year 2004 to 2008
by a team led by JHU
• Contents developed through consultative process
• Very careful quality assurance
• Each facility rated on a score of 0-100, can be
aggregated at county, state, national level
• Present results through “balanced scorecard”
4
What the BSC Looks At:
•
•
•
•
•
•
•
•
Presence of staff
Knowledge of staff
Quality of patient-provider interaction
Availability of drugs and supplies (also quality
on sample basis)
Patient satisfaction (different from HH results)
Waste management
Use of facilities, use by women, and the poor
etc.
5
Can Look at Provincial Progress – Color
Coded
6
80
32% Improvement in Total Scores in Contracted
Facilities (from health facility survey)
75
70
65
MOPH Alone
PPA Median
60
55
50
45
40
2004
2005
2006
2007
Looking at Provincial Progress on Total
Score
Balanced Score Card Results from 2007 Compared to Previous
Years
Province
2004 2005 2006 2007 Change from 2004
to 2007
Badghis
48.7 59.3 49.8 80.2
31.5
55
71
71.6 78.6
23.6
53.4
60
62.8 75.8
22.4
National Median 53.2
59
65.4 70.2
17
Balkh
PPA Median
8
Can Look At Areas Needing Attention
Index
Patient Counseling
Equipment Functionality
Family Planning Availability
Patient History & Exam
Proper sharps disposal
Obstetrical care
BHC's with >750 patients
HMIS Implementation
Provider Knowledge
Drug Availability
2004 2005 2006 2007
29.6
65.7
61.4
70.6
62.2
25.4
22.2
67.7
53.5
71.1
35.1
67
70
73.5
52
22.3
32.3
65.8
69
83.7
36.6
78.7
82.9
82.2
77.5
42.3
55
74.9
68.7
85.7
48.7
83.8
93.7
83.1
84.4
59.5
57.4
91.5
68.7
81
Change
2007-2004
19.1
18.1
32.3
12.5
22.2
34.1
35.2
23.8
15.2
9.9 9
Health Facility Assessment in Nigeria under Malaria + Program
covering 327 facilities
Illnesses: Fever/Malaria; Pneumonia; Dysentery and Diarrhea
13
Poor awareness of PMVs regarding
new Malaria treatment policy
What are the challenges with Health
Facility Surveys?
• Deceptively difficult to do
• Requires talented technical staff
experienced in survey design
• Need to do it every year or so to look at
changes
• Costs about $300,000 per year (more during
development)
17
What are the challenges with Health
Facility Surveys?
• Generates a lot of data (400+ questions
on each facility)
• Tough to explain to managers – need
means, like BSC, to summarize data
• Quality assurance is a real challenge
• Easy to do badly – consumers won’t
know
18
Quantitative Supervisory Checklist –
What is it?
• A reduced version of a health facility
assessment
• Objectively assesses a variety of indicators to
come up with total score.
• Takes about 2-3 hours to complete
• A copy of results left in the health facility, easy
to track progress
• QSC is both a management intervention and
tool for M&E
19
Example of a Quantitative
Supervisory Checklist
Date of Visit
Availability of Drugs (0-10)
Presence of staff (0-5)
HMIS implementation (0-10)
TB Case Detection Rate (0-5)
DPT3 coverage rate (0-10)
Consultations per capita (0-10)
Deliveries in facility (0-10)
TOTAL SCORE (out of 60)
Supervisor’s signature
HF in-charge signature
5/12
7/19
8/11
10/21
3
2
3
0
2
2
0
12
5
1
3
1
3
4
1
18
4
2
5
1
3
2
1
18
6
2
5
2
4
5
3
27
20
Development of QSC in the
Philippines
• New HMIS forms developed which were
supposed to facilitate supervision
• “Checklist Safari” in 7 provinces found:
– 25 different checklists
– 95 items, average 4.5 pages long
– Rarely used, never found in health facilities
– Designed in such a way to make follow up difficult
• Supervision was sporadic, not systematic,
mostly dreaded by health workers
21
Development of QSC in the
Philippines
• Discussions with key program managers led to
definition of 20 indicators.
• Indicators scored from 0-3 with specific
definitions and means of calculation
• Copy of QSC could be left in HF so future
supervisors & staff could track progress
• Copy with supervisor so s/he could track which
indicators were lagging
• Before & after assessments in 4 experimental
provinces and 6 control provinces
22
Example of a Quantitative
Supervisory Checklist
Date of Visit
Availability of Drugs (0-10)
Presence of staff (0-5)
HMIS implementation (0-10)
TB Case Detection Rate (0-5)
DPT3 coverage rate (0-10)
Consultations per capita (0-10)
Deliveries in facility (0-10)
TOTAL SCORE (out of 60)
Supervisor’s signature
HF in-charge signature
5/12
7/19
8/11
10/21
3
2
3
0
2
2
0
12
5
1
3
1
3
4
1
18
4
2
5
1
3
2
1
18
6
2
5
2
4
5
3
27
23
Evidence for the Effectiveness of QSC
% Change in Scores from Baseline
60
50
40
Control
Experimental
30
20
10
0
total
<3
3+
24
Other Findings from QSC
• Health workers liked it because it made it clear
what was expected. Supervisors not angry
• Supervisors liked it because made interaction
with HWs more focused on key results
• HWs tracked performance and became adept
at tracking their own performance
• Was launched nation-wide but fell into dis-use
after devolution
25
Quantified Supervisory Checklists
Advantages
• Can assess QOC.
• Can be independent
of service providers
• Can be done often
• Inexpensive
• Clarifies what is
expected of HWs
• Can be adapted to
conditions as they
change
Disadvantages
• Challenging to design
• Cannot provide
information on
coverage, equity
• Ensuring continued use
is difficult
26