Transcript Benchmarks of Fairness: A policy tool for developing countries

Benchmarks of Fairness
Workshop
Agenda
Sviavonga, Zambia
June 11-13, 2003
Zambian Work shop on Bench mark s
June 1 1 -13 , 2 0 03
Day One
830
900
9 3 0- 10 :45
1 0 :4 5 1 1 :0 0
1 1 :0 0 -12 3 0
Wednesday
Regist rati on
Inaugurat ion and intr oduct ions
Session One
Brief Ov ervi ew of benchm arks: Daniels
Present ati ons/ Discu ssion on Zambian healt h
sys t em, pro blem s and ref orm ef f or t s
( Zambian part ici pants)
Break
Session Two
Plenary d iscussion of key pro blem s in Zambia
healt h sect or; comment s f ro m f or eign
parti cipant s on similariti es and dif f erences
Short p resent at ion on Equit y Gauge-- Ngulube
1 2 30- 1 33 0
1 3 30- 1 51 5
Lunch
Session t hr ee
St ruct ur e and cont ent of Benchm arks -Daniels
Plenary Discus sion
1 5 15- 1 53 0
1 5 30- 1 73 0
Break
Session Four
Connect ing t he Bench mark s t o Zambian
cont ext:
Small gr oups wit h concentr at ion on dif f erent
benchm arks ( Facilita t ors: Daniels, Ngum,
Ndifo rchu, Chia
Plenary br ief repor ts
Wor kshop on Equit y Bench mark s
9- 1 1 D ecember, 2 0 02 , Hot el Renuka, Colombo .
Day t wo
8 3 0-1 0 30
Thursday
Session Five
Present ati on on Cameroo n – Drs Ngum,
Ndifo rchu, Chia
Evid ence Base: Wh at is it , how to const ruct it
-- Daniels
1 0 30- 1 04 5
1 0 45- 1 23 0
Cof f ee
Session Six
Evid ence Base: Discussion of Cameroo n
examples and relevance t o Zambia,
Bench mark s 1- 3
Daniels, Ngum, Ndifo rchu,
Chia
1 2 30- 1 33 0
1 3 30- 1 50 0
LUNCH
Session Seven
Evid ence Base: Discussion of Cameroo n
examples and relevance t o Zambia,
Bench mark s 4- 6 , Daniels, Ngum, Ndifo rchu,
Chia
1 5 00- 1 52 0
1 5 20- 1 73 0
Tea
Session Eight
Evid ence Base: Discussion of Cameroo n
examples and relevance t o Zambia,
Bench mark s 7- 9 , Daniels, Ngum, Ndifo rchu,
Chia
General comment s
Workshop on Bench mark s
1 1 -13 June, 2 0 03 Lusaka
Day th ree
8 3 0-1 0 30
Friday
Session Nine
Planning f or Applicati on in Zambia-- Daniels
Relati onship t o Equit y Gauge -- Ngulube
General discu ssion of k ey st eps
1 0 30- 1 04 5
1 0 45- 1 23 0
Cof f ee
Session Ten
Planning f or Applicati on in Zambia: small
groups
1 2 30- 1 33 0
1 3 30- 1 53 0
LUNCH
Session Eleven
Planning f or Applicati on in Zambia: reports of
small groups
Const ruct ion of o verall plan, inclu ding
coor dinat ion wit h ot her countr ies, raising of
f unds
Benchmarks of Fairness:
A policy tool for developing
countries
Norman Daniels
June, 2003
Zambia
Norman Daniels
[email protected]
Dept. Population and International
Health, HSPH
Development of BMs
•
•
•
•
1993 Clinton Task Force
1996 Benchmarks of Fairness OUP
Pilot work in Pakistan, 1997
1999-2000 Adaptation: Pakistan, Thailand,
Colombia, Mexico: Daniels, Bryant et al Bulletin
of WHO, June 2000
• 2001-3 Demonstration Phase:Mexico, Portugal,
Pakistan, Thailand; Vietnam Cameroon, Ecuador,
Nicaragua, Guatemala, Chile, Sri Lanka, Yunnan,
Bangladesh, Zambia
Some Common Concerns about
HS Trends
• Rising Costs
• Epidemiological Transition
• Privatization and structural reform
– External pressures, transitional economies
– “introduce new resources”
• BUT: undermine public resources
– “avoid state bureaucracy”
• BUT: strong state needed to regulate markets
• Lack of focus on equity, accountability: no integration
• In Grip of Ideology of Market
• Lack of Satisfactory Results
–The Adapted Benchmarks
–
–
–
–
–
–
–
–
–
1. Intersectoral public health
2. Financial barriers to equitable access
3. Nonfinancial barriers to access
4. Comprehensiveness of benefits, tiering
5. Equitable financing
6.Efficacy,efficiency,quality of health care
7. Administrative efficiency
8. Democratic accountability, empowerment
9. Patient and provider autonomy
Connections to social justice
• Equity
– B1Intersectoral public health, B2-3 Access,
B4Tiering, B5 Financing
• Democratic Accountability
– B8, B9Choice
• Efficiency
– B6 Clinical Efficacy and quality
– B7 Administrative efficiency
Equity
• Equality
• Vs Equity
– Benefit according to need
– Burden according to ability to pay
Efficiency
• Value for money
• Vs equity
• Promoting equity
Accountability
– Responsible to:
• Patient
• Institution
• public
• Agent
– Responsible for
• Act or outcome
Structure of BMs
• B1-9 Main Goals
– Criteria -- Key aspects
• Sub criteria-- main means or elements
• Evidence Base + Evaluation
– Indicators
– Scoring Rules
Some evaluation approaches
• Specific reseach on reform effects (e.g. Bossert on
decentralization, Hsiao on financing outcomes)
• Monitoring of reform process (e.g. PAHO monitoring
project)
• WHO Framework (index for cross country
performance comparisons, including focus on equity,
efficiency, responsiveness)
• Policy development approaches: Reich, Roberts,
Berman -- new book based on World Bank project to
get reform efforts right, including value clarification
WHO Framework vs BM
WHO
BM
Scope
Cross national
Nat, subnat
Objective
Current perform
Reform eval
Purpose
Motivate
Deliberate
Product
Index, ranks
Scores
Who uses
National pol mk
Various
Requires
Good info
Info, tr. people
Problems
Inform change?
Subjectivity?
Overlap
Move to reforms
complementary
Key Features of BM
•
•
•
•
Country Specific: national, subnational
Integrates Equity, Efficiency, Accountability
Evidence Based (Objectivity)
Pragmatic
– adapted locally to purposes, evidence
– focus on improvement
• Improves Deliberative Capacity
• Complements other approaches
EG and BM Interaction
• EG complements BM
– Strong intersectoral focus
(Chile, Burkina Faso, Kenya)
– Strong focus on path from info
collection to evidence based
interventions,w. accountability
– Strategies for sensitive info
– Good public private
collaborations
• BM complements EG
– Useful for G’s focusing on
health care systems and
budgeting as in Zambia,
Zimbabwe, Uganda, S.A./Cape
Town
– Help G’s to prioritize issues
and focus on advocacy
campaigns within G’s
– Further tool for advocacy
within G’s
B1: Intersectoral Public Health
• Degree to which reform increases per cent of
population (differentiated) with: basic nutrition,
adequate housing, clean water, air, worplace
protection, education and health education
(various types), public safety and violence
reduction
• Info infrastructure for monitoring health status
inequities
• Degree reform engages in active intersectoral
effort
B2: financial barriers to access
• Nonformal sector
– Universal access to appropriate basic package
– Drugs
– Medical transport
• Formal Sector Social/Private Insurance
–
–
–
–
Encourages expansion of prepayment
Family coverage
Drug, med transport
Integrate various groups, uniform benefits
B3: Nonfinancial barriers to
access
• Reduction of geographical maldistribution
of facilities, services, personnel, other
• Gender
• Cultural -- language, attitude to disease,
uninformed reliance on traditional
practitioners
• Discrimination -- race, religion, class,
sexual orientation, disease
B4: Comprehensiveness of
benefits and tiering
• All effective and needed services deemed
affordable by all needed providers, no
categorical exclusions
• Reform reduces tiering and achieves more
uniform quality, integrates services to all
B5: Equitable financing
• Financing by ability to pay
– If tax based scheme: how progressive (by
population subgroup), how much reliance on
cash payments (by subgroup)
– If premium bases scheme: community rated?
Reliance on cash payments?
B6: Efficacy, efficiency and
quality of health care
• Primary health care focus
– Population based, outreach, community participation, integration
with system, incentives, appropriate resource allocation
• Implementation of evidence based practice
– Health policies, public health, therapeutic interventions
• Measures to improve quality
– Regular assessment, accreditation, training
B7: Administrative efficiency
•
•
•
•
Minimize administrative overhead
Cost-reducing purchasing
Minimize cost shifting
Minimize abuse and fraud and inappropriate
incentives
B8: Democratic accountability
and empowerment
• Explicit public detailed procedures for evaluating
services, full public reports
• Explicit deliberative procedures for resource
allocation (accountability for reasonableness)
• Fair grievance procedures, legal, non-legal
• Global budgeting
• Privacy protection
• Enforcement of compliance with rules, laws
• Strengthening civil society (advocacy, debate)
B9: Patient and Provider
autonomy
• Degree of consumer choice
– Primary care providers, specialists, alternative
providers, procedures
• Degree of practitioner autonomy
Why is evidence base important?
• Evidence base makes evaluation objective
• Making evaluation objective means:
– Explicit interpretation of criteria
– Explicit rules for assessing whether criteria met and the degree to
which alternatives meet them
• Objectivity provides basis for policy deliberation
– Gives points of disagreement a focus that requires reasons and
evidence
Evidence Base: Components
• Adapted Criteria--convert generic benchmarks into
country-specific tool
– Reflect purpose of application
– Reflect local conditions
• Indicators
– Outcomes
– Process
– revisability
• Scoring rules
– Connect indicators to scale of evaluation
– Specify in advance
Process of selecting indicators
• Clarity about purpose
• Type of criterion determines type of indicator
– Outcomes vs process indicator appropriate
– Standard vs invented for purpose
– Requires clarity about mechanisms of reform
•
•
•
•
Availability of information
Consultation with experts
Final selection in light of tentative scoring rules
Further revision in light of field testing
Scoring Benchmarks
Reform relative to status quo
-5
0
Or use qualitative symbols, --- or +++
+5
Scoring Rules: General Points
• Map indicator results onto ordinal scale of
reform outcomes
• Final selection of indicators should be done
as scoring rules are developed, so
refinements can be made
• Scoring rules should be adopted prior to
data collection to increase objectivity, but
may have to be revised in light of problems
Structure of Scoring Rule:I
• Relative to scale from -5 to +5 with 0 as status quo
or point of reform
• If there is only one indicator for criterion and it is
outcome indicator:
– Estimate ideal outcome for this indicator, e.g., 100%
coverage for a population group for clean water or
vaccination
– Specify baseline value -- e.g., 60%
– Divide gap between baseline into proportions of scale
and assign score value to indicator outcomes
Structure of scoring Rule:II
If only one indicator and it is process and qualitative; e.g. to
measure transparency we count public reports issued for
performance of pharmacies or district hospitals
Could count % of such units issuing public reports -- then
scoring might work as in previous case
– May need to combine this indicator with another measuring public
access to the report (was it really available on request, did anyone
request it, did community or advocacy group request it, or actually
use it
What about disagreements?
• Won’t different groups using bm’s come up with different
scoring rules, different results?
– Yes, but specification of evidence base provides basis for
deliberation about disagreements
• Won’t different groups using same instrument have come
disagreements about evaluations?
– Yes, but specificity of evidence base provides basis for resolving
dispute?
Perfect information?
• Won’t benchmarks reflect only part of truth
about situation with policies under analysis?
– Better to seek comprehensive household survey
to get complete representative view?
– Settle for less perfect info but have a basis for
deliberation about disagreements - but requires
clarity about evidence base
Information Plus Process
• Many approaches aim to give excellent information input
but leave process of deliberation unaffected
• Benchmarks aim to improve process of deliberation itself
– Adaptation that includes developing evidence base is training in
what to look for when monitoring and evaluating and how to
derive conclusions about reform from that
– Improvement can take place at any level -- official policy makers,
institutions doing implementation and lower level planning,
community groups assessing effects
Cameroon
•
•
•
•
•
MoH supports evaluation officially
Subnational -- district level
Medical Student rotation -- training, fieldwork
Baseline, then repeats for monitoring, evaluation
evidence base: complex, capacity building, revisable,
limited resources
Cameroon constraints on
indicators: illustrating
compromises
•
•
•
•
Absence of survey data
Student investigators
District level sources
Risks to students
How to measure indicator
BENCH MARK 1
Variables
Basic Nutrition
Pool of possible Indicators
Where to get
information
Denomenator
Numerator
1 % of health districts iodised salt in the District/Province No of Health districts
market.
monitoring iodine in
All health Districts
salt
2 % of health units administering vit A District
No of HU administering All health units in the
Vit A
District
3 Proportion of children fed exclusively Health Center
No of children on
All children 0-4 months
with breast milk till 4 months of age
breast milk up to 4
(1.2% of the total
months
population)
4 Proportion of children monitored using Health Center
No of children 0-11
the Road to health Chart.
months monitored
all children 0-11months (4%
using the road to health of total population)
chart
Environment
5 Prevalence of water borne
District
diseases (bacterial dysentry &
cholera)
6
Education
and health
Education
7
8
9
Total No of cases
of bacilliary
The total population
dysentery &
of the District
cholera registered
Population: sanitary
District/Counc
No of sanitary technicians:Total
technician ratio
ils
population
Literacy by gender
Census board
or survey
% of health units carrying out Health unit
The no of HU
outreach activities on priority
carrying out
diseases within and without
outreach activities
Total no of health
the H/c
on priority
units
diseases (EPI,
HIV/AIDs,
Malaria)
Proportion of health
Health centers The no of HU
promotion activities carried
carrying out
out within and without the
outreach activities
Total no of health
H/C
on health
units
promotion (IWC,
FP, ANC,
Nutrition, CCB)
Degree to which 17 % of meetings coordinating MOH, District
reforms have
Agriculture, social affairs, womens
actively engaged affairs, education, territorial
inter sector
administration over the past 12 months
efforts
No of meetings held No of meetings planned
Bench mark 3
Variables
Reduction in
Geographical
misdistribution in
Facilities &
services
Selected Indicators
Where to get
information
21 The proportion of Health
District
Areas with Integrated Health
Centers
22 Proportion of Health Areas District
carved out according to norms
How to measure indicator
Numerator
Denomenator
No of Health Areas with Total no of Health Areas
Integrated Health
Centers
No of Health Areas
Total no of Health Areas
carved out according to
norms
23 Proportion of health Areas
Health centres No of health Areas with Total no of Health Areas
with > 30% of the population
> 10% of the population
living at more than 1 hour
living at more than 1 hour
walking distance from the
walking distance from the
health center
health center
24 Proportion of Integrated
District
No of Integrated Health Total no of Health
Health centers delivering a
centers delivering a
centres
minimum package of activities
minimum package of
activities
25 Number of districts in the
District
% of districts in the provine without District referral
provine without District referral
Hospitals
Hospitals
Appropriate
allocation of
resources to
Interactive
community
participation
including
vulnerable subgroups
41 Proportion of health services District
planning by objectives
42 Proportion of dialogue
structures holding general
assemblies
No of health services
planning by objectives
health area, #No of H/As holding
district
general asemlies,
#No of Districts holding
general assembly
43 Proportion of health units with District
No of Health areas with
vulnerable groups
women and youths
represented in dialogue
represented in dialogue
structures
structures
44 Proportion of health units that health unit No of HU with plans of
involved communities in the
actions drawn with the
planning process.
participation of
communities (veriable in
the minutes of
deliberations)
of health services
planning by objectives
#No of H/As in the
District,
#No of Districts in the
province
No of Hu with dialogue
structures
Total number of HU
Referral
mechanisms
(over the last 12
months)
45 percentage of patients in the
district hospital who have not
been referred
46 The proportion of referred
patients that are counterreferred
47 % of patients referred to the
hospital
Implementation 48
of evidence
based practice in
therapeutic
49
interventions
50
51
District
hospital
Number of patients not
reffered
District
Hospital/He
alth center
Health
center
Number of patients
referred
Total number of patients
consulted
Total number of patients
conterreferred tohealth
centres
Number of patients
Total no of patients
referred district hospital consulted in the health
centres
% of health units with
Health units No of health units with Total No of health units
personnel trained in the use of
trained personnel
guidelines
% of health units with
District/
Number of health units Total No of health units
available clinical guidelines. Health units with available guidelines
% of health units sending
District
Number of health unit
Total No of health units
complete reports (NHMIS,
sending complete
EPI, weekly epidemiological
monthly reports
reoprts)
% of health units keeping
Health unit Number of health units Total No of health units
copies of sent reports
with available copies of
reports
selected Indicator
Bench Mark
7
56 Percentage of health units with
Inventory of equipment
Administrativ
e efficiency
57 Proportion of health units with a
trained personnel responsible for
maintenance of equipment
58 Percentage of health units with
equipment below or above the
norms
1-Minimize
administrative
overheads
Numerator
Where to get
information
District (stores Number of health units
accountant)
with inventory
Denominator
Total number of health
units
District, hospital No. of health units with Total number of health
maintenance personnel units
Health unit
A- No. of health units Total number of health
with equipment below units
norms
B- No. of healthunits with equipment above norms
59
Percentage of health units with
functional equipment
district,
hospital,health
centrre
60 The percentage of health areas with
coverage respecting the health map.
District
61 % of personnel leaving the country
wthin 2years
National
A- No. of health units
with functional fridges,
B- No. of health units
with functional
microscope
No. of health areas with
coverage respecting the
health map.
No. of medical doctors
who left the country
within the last 2years
Total number of health
units
Total number of health
areas
Total No. of medical
doctors registered in the
public service within the
last 2 years
Selected indicator
66 % of health units providing
monthly management and
technical reports through the
dialogue structures
Where to get
information
Health centers
67 % of health units with ‘budget H/C
by objective’ approved by
management committees
68
% of health units with ethical
committees
69
% of health units with internal Health units
regulations.
70
% of health units enforcing
regulations
Health unit
Numerator
Denominator
Number of monthly
management and
technical reports
submitted through
dialogue structures
Number of health
units with approved
budget by objective
Total number of
health units with
dialogue structures
Number of health
units with ethical
committees
Number of health
units with internal
regulations
Number of health
unit enforcing
regulations
Total No of health
units
Total number of
health units with
budgets
Total number of
health units
Total No of health
units
71 % of planned district activities
carried out
72 Proportion of advocacy
meetings that involved
vulnerable groups.
73 % of health units in which
planned advocacy meetings
are in response to advocacy
groups request.
District
District
Number of planned
activities carried out
Number of
advocacy meetings
involving vulnerable
groups
Number of health
units that have held
planned advocacy
meetings in response
to advocacy group
request
Total No of planned
activities
Total No of
advocacy meetings
Total No of planned
advocacy meetings
requested by
advocacy groups
Cameroon Results
• Data from 8 districts, first group of students,
analyzed, May ‘03; data quality so far good
• Data from some bmks easier to collect than others;
waiting results from other districts and students
• Further steps: faculty workshop to refine criteria,
national workshop to present and adopt indicators,
nationwide implementation as framework, use by
DMOs as management tool
Current Projects: I
• Phase 2:
– Thailand--Supasit project-- RF-- report drafted;
electronic version soon available
– Pakistan--Khan project--RF--just beginning
– Mexico--Gomez Dantes--Mexican funding--publication
in prep, available electronically
– Portugal--Portuguese funding
• Phase 1+: Vietnam-- MoH proposal-- seeking RF
• Phase 1: (various stages)
– Underway for 1 yr or less months:
• Cameroon-- preliminary report available
• Nicaragua, Ecuador, Guatemala : indicators selected in
Ecuador, Guatemala, field testing this summer
• Kenya (informal, Bryant)
Current Projects: II
• Phase 1 Work
– Recently underway-- initial workshop held, plan
developed, working group carrying out adaptation, mix
of MoH, NGO, academics, varying degrees approval
from MoH
•
•
•
•
Chile -- November 2002; EG collab
Bangladesh-- January 2003; EG collab
Sri Lanka - December 2003; some WHO country support
Kunming- Jan 2003; seeking WHO country support
– Planning Stage
• African sites--Zambia, South Africa (June 2003, EG collab),
Zimbabwe, Botswana, Nigeria, Tanzania, Uganda
Project Needs
• Longer term support for regional
coordination (Asia, Africa, Latin America)
• Country level support for second phase
projects
• Midterm support for web page, training
manual (to be in place by end of 2003 with
RF support)