Transcript Project Name

Building a Performance Improvement
System in a Large Urban Public
Health Department: Linkages and
Learning Collaboratives
MLC Open Forum
Washington, DC
September 16th, 2010
Dawn Marie Jacobson, MD, MPH
Director, Performance Improvement
Los Angeles County Department of Public Health
Presentation Objectives
• Provide an overview of performance
improvement efforts at Los Angeles County
Department of Public Health
• Discuss linkages required for efficient analysis
and reporting
– Data
– Standards/Benchmarks
– Reports
• Describe the DPH Performance Improvement
Learning Collaborative (PILC)
Quality Improvement—LAC DPH
Quality Improvement Division
Office of the Medical Director
Organizational Development and
Training
Nursing Administration
Health Education Administration
Public Health Investigation
Physician Administration
Oral Health
Quality Improvement
Functions
1.
2.
3.
4.
Performance
Improvement
Professional Practice
Science Review
Service Quality
Gunzenhauser JD, Eggena ZP, Fielding JE, Smith KN, Jacobson DM, Bazini-Barakat N. The Quality Improvement Experience in a HighPerforming Local Health Department, Los Angeles County. Journal of Public Health Management and Practice, 2010, 16 (1): 39-48
Performance Improvement—LAC DPH
Key Elements
1. Strategic Planning
– determine priorities and goals
2. Performance Measurement
– data management
– reporting
3. Performance Improvement Projects
– Modified IHI Method for Improvement
– Other tools (RCA, Fishbone diagrams, etc)
1. Strategic Planning: Determine
Priorities and Goals
•
•
•
What are the priority
public health issues in
Los Angeles County?
What are the
behaviors and
outcomes related to
these issues that we
want for people who
live in LA County?
How can we measure
these conditions?
SPA/
Program
Strategic Plan
DPH Strategic Plan
County Strategic Plan
LAC DPH—Strategic Planning
– Quality Improvement Division
• Public Health Measures required Mission and Vision
Statements, Goal Setting, and Evidence-Based Strategies
(2004-2007)
– Office of Planning
• Department-Level Strategic Plan (2008-2011)
– Division and Program Level Strategic Plans
• Office of Planning may provide technical assistance
2. Performance Measurement:
Public Health Measures
• The LAC DPH approach based on Mark Friedman’s
“Results Accountability”
• 32 operational units identified population health
indicators linked to program performance measures
to follow over time
• Healthy People 2010 objectives often identified and
used as the “Standard” to achieve over time
• Organized by Essential Services of Public
Health/NACCHO Standards/Accreditation Domains
Public Health Measures
POPULATION INDICATORS
(measures of population-level
health outcomes and behaviors)
AND
PERFORMANCE MEASURES
(measures of program
effort and output)
Public Health
Measures
Selecting Indicators and Measures
Population Health
Program Performance
Population
Goals
Population
Indicators
Effective
Strategies
Performance
Goals
Performance
Measures
Goal 1
Indicator
Strategy 1
Goal 1
Measure 1
Indicator
Strategy 2
Goal 2
Measure 2
NACCHO
Standards
Federal, State,
or Local
Guidelines
Strategic
Plan
Healthy
People
2010/2020
Community Guide
Clinical Guide
Other Sources
Decision-Making in Public Health:
Evidence Review
Tier 1 Evidence
Type (ranked)
Research findings: syntheses, systematic reviews, meta-analyses
Research findings: individual studies (quantitative and qualitative)
Performance data such as program evaluation or peer review reports
Demonstrated to be effective in computer modeling, simulations, or exercises
Consensus recommendations of recognized experts either local or national
Anecdotal accounts such as practices of other public health jurisdictions alleged to
be effective, clinical narratives, or case reports
Decision-Making in Public Health:
Other Rationale
Tier 2 Other Rationale
Type (unranked)
Philosophical or conceptual bases such as an ethical
framework or a professional code of conduct
Regulations, laws, or public policies
Grant requirements
Community preferences
Necessary because of the political climate
Best hunches
Public Health Measures:
Population Indicators
•
•
•
•
•
•
•
•
Longer life span
Increased quality of life
Increased health equity
Less disease
Less premature death
Healthier choices
Safer environment
Healthier homes
POPULATION-LEVEL
HEALTH OUTCOMES
& BEHAVIORS
Public Health Measures:
Performance Measures
1. Who are our clients?
2. Which services do we
provide to our clients?
3. What evidence-based
strategies will lead to
positive change in our
clients?
4. How can we measure if our
clients are better off?
5. How can we measure if we
are delivering services well?
Quantity
Input /
Effort
Output
/ Effect
Quality
How
How Well
Much Did Did We Do
We Do?
It? (%)
(#)
How
Quality of
Much
Change?
Change?
(%)
(#)
Performance Measures
•
•
•
•
•
•
•
Policies Created
People Informed
Partners Engaged
Surveillance Performed
Investigations Completed
Increased Access to Services
Client satisfaction
MEASURES OF
PROGRAM
EFFORT & OUTPUT
Example: Immunization Program
Population Goal To reduce morbidity and mortality from vaccine-preventable
diseases by improving immunization levels
Population Indicator
Percentage of children, ages 19-35 months, who are fully immunized with one
of the series of the Advisory Committee on Immunization Practices (ACIP)
recommended vaccines
Effective, Evidence-Based Strategies (selected subset)
1. Change provider behavior through systems change—Provider recall/reminder
systems in clinics
2. Change provider behavior through education—multi-component interventions
with education
3. Increase demand and access to immunizations—reduce out-of-pocket costs
Performance Goal (NACCHO Standard 9)
Performance Measure
Percent of Immunization Program public and nonprofit clinic partners who
routinely meet the Standards for Pediatric Immunization Practices for provider and
client recall/reminder systems
Public Health Measures:
Data Management
• Standardized spreadsheet for reporting data
• Labeling System
– Population Indicators= letters
• organized by population goals
– Program Performance Measures= numbers
• organized by Accreditation domains
• Data Documentation
• Standard Documentation
Public Health Measures:
Data Measurement Worksheet
– Type of measure (PI or PM)
– Measure name and description
– How calculated
– Target
– Data source (Name, govt level, dept, program)
– Data collection instrument
– Data collection plan
– NACCHO Standard (if applicable)
Common Data Sources
Population Indicators
Program Performance
Measures
Examples:
Examples:
– Los Angeles County Health Survey
– Casewatch (STD, AIDS)
– LA FANS
– RASSCLE (lead surveillance)
– Disease specific surveillance
– EHMIS
systems
– TRIMS (TB control)
– Vital Records
– vCMR (outbreak reporting
– CA Health Interview Survey
and investigation)
– OSHPD (Healthcare Utilization data)
– Syndromic surveillance
– BRFSS
– Clinic utilization data
– YRBS
– Contracts and grants
– National Immunization Survey
management
– Project-specific databases
Common Standard Sources
Population Indicators
Program Performance
Measures
Examples:
– Healthy People
– State of CA plans
– County of LA plans (e.g.
Commission on HIV)
– Internal DPH
Examples:
– Healthy People
– CDC guidelines
– State of CA guidelines or
mandates
– Grant-specific guidelines
– Professional associations
– Internal DPH
Example: Immunization Program
Example: Immunization Program
Public Health Measures:
Data Management
• Data collected two times per year
• Data analyzed and reported one time per year
• Option to update content of Public Health
Measures one time per year
• Public Health Measures database in
development
Public Health Measures: Reporting
National Efforts
1. CDC Guidelines or
Performance
Measures
County Efforts
1. Performance Counts!
2. County Progress Report
State Efforts
1. State Performance
Measures
2. Mandates and
Regulations
Department Efforts
1. Annual Performance
Report
2. PI Project Reports
LAC DPH Annual Performance Report
• Internal report of a subset of Population
Indicators and Performance Measures
• Includes:
– Department-Level Report Card
– Program-Level Performance Snapshots
• In-Person Progress Review with Director and
Health Officer
Public Health Report Card
Public Health Report Card
Public Health Measures: Reporting
• Future
–
–
–
–
Linked to strategic plan objectives
More frequent reports using an automated database
Portfolio of services by NACCHO Standards
Accreditation Preparation
Total
PIs
224
PMs
736
NS1
NS2
NS3
NS4
NS5
NS6
NS7
NS8
NS9
NS10
NS11
51
(7%)
100
(14%)
110
(15%)
34
(5%)
22
(3%)
36
(5%)
142
(19%)
92
(13%)
61
(8%)
36
(5%)
87
(12%)
3. Performance Improvement Projects
Since repeated measurement by itself is not enough to
improve public health practice....
What are common processes in our Department? How
can we share best practices in common processes?
What support do staff need to use PI methods (e.g. rapid
cycle tests) in practice?
How do we spread a successful PI approach throughout
the Department?
Performance Improvement Projects
A Learning Collaborative Approach*
• Create an internal performance improvement learning
collaborative (PI LC) of a diverse group of DPH units
• Teams represent 8 of 32 department Divisions/Programs
• Teams learn and work together for a 10 month period
• Teams apply common PI methods to improve a priority area
selected by their respective units
*This project is part of the “Building the Evidence for Quality Improvement in
Public Health ” grant program funded by the Robert Wood Johnson Foundation.
The RAND Corporation is providing training and evaluation support.
PI Method and Tools:
The IHI Model for Improvement... Plus
1. Set the Aim
•
•
Population health improvement
Customer or service improvement
2. Measure Performance
•
•
Population Indicators
Program Performance Measures
3. Map the Process
4. Make Changes for Improvement
•
•
Evidence Review and Best Practices
PDSA cycles
5. Apply other Tools (RCA, Fishbone diagrams, etc.)
Set the Aim: PI LC Team Aims
• Improve Provider/Contractor Performance
–
–
–
–
–
Office of AIDS Policy and Programs
Children’s Health and Disability Prevention Program
Tobacco Control and Prevention Program
Substance Abuse Prevention and Control
Emergency Preparedness and Response
• Inform and Engage Community Stakeholders
– Office of Senior Health
– Acute Communicable Disease Control
– Office of Women’s Health
Measure Performance: PI LC Project Metrics
• Population Indicators
Examples:
– % of children who qualify for the CHDP program who receive
needed follow- up care
– Community incident rates of reptile associated salmonella
– Community rates of CVD among women
• Program Measures
Examples:
– % of CHDP forms with a condition needing referral that have a
referral identified
– % of Early Childhood Education providers receiving the
photonovela intervention
– # of new callers to hotline per week
Percent of current adult smokers and current youth
smokers in Los Angeles County (2001-2006)
HP 2010
Standard
(16%)
20%
Percent
15%
10%
14.5% 14.3%
14.4%
5%
13.9%
Data not
collected
where missing
14.3%
11.8%
HP 2010
Standard
(12%)
0%
2001
2002
2003
2004
2005
2006
Calendar Year
TCPP-A Percent of adults
TCPP-B Percent of youth
Data Sources: 1) Los Angeles County Health Survey (LACHS), LAC DPH, OHAE and 2) Youth Risk Behavior Survey (YRBS), CDC, NCCDPHP
Number of jurisdictions adopting a legislative-based policy that
prohibits smoking in outdoor areas (00-01 to 09-10)
12
1st Funding Cycle = 4
2nd Funding Cycle =
3rd Funding Cycle = 15
10
DPH and State of
CA Standard
(4 jurisdictions
per 3-year
funding cycle)
Number
8
6
10
4
5
2
0
0
1
2
5
6
6
3
1
00-01 01-02 02-03 03-04 04-05 05-06 06-07 07-08 08-09 09-10
Fiscal Year
Data Source: Internal Records, LAC DPH, TCPP
TCPP 5-1 Number of Jurisdictions
Map the Process: Examples
Make Changes for Improvement:
PDSA and Other Tools
PDSA cycles to improve subcontractor satisfaction with
Task Force meetings (complete)
Average score
January 2010
Average score
April 2010
(scale of 1-4: 4= strongly agree,
1= strongly disagree)
(scale of 1-4: 4= strongly agree,
1= strongly disagree)
1.
Task Force meetings are well facilitated.
3.67
3.8
1.
Task Force meetings provide an effective learning forum for my
campaign.
3.63
3.7
1.
The ideas/strategies discussed at the meetings are helpful to my
campaign.
3.75
3.9
1.
Ideas/strategies shared by other task force members are helpful to
my campaign.
3.71
3.8
1.
Information provided by TCPP staff (e.g. billing, announcements,
campaign strategies, etc.) are useful.
3.62
3.9
Make Changes for Improvement:
PDSA and Other Tools
PDSA cycles to improve subcontractor training for community
campaigns to pass tobacco policies and ordinances (in progress)
Average score
April 2010
(scale of 1-4: 4= very helpful, 1= not helpful)
Community assessment
3.5
Policy campaign strategy
3.8
Coalition building/ broadening
3.6
Policy campaign implementation and policy adoption
3.6
Policy implementation and enforcement
3.4
1
–
–
–
6
4
–
–
lo
pe
An
te
7
–
y
Va
lle
So
ut
h
M
et
ro
Ba
y
Ea
st
rn
an
do
rie
l
W
es
t
So
ut
h
SP
A
SP
A
8
SP
A
–
Ga
b
Fe
Sa
n
Sa
n
–
5
25
SP
A
2
3
SP
A
SP
A
SP
A
SP
A
Pareto Chart:
Adult smoking rate by SPA
2007
20
15
10
5
HP
2010
goal
0
2007
PI LC Evaluation Metrics
• Improve project team metrics
• Increase staff knowledge and use of QI methods
– % of senior managers reporting they are aware of the 4step Model for Improvement
– % of senior managers who say they are proficient in
selected quality improvement methods and tools
• Disseminate QI methods across the Department
– % of DPH Division/Program Directors who report using
rapid-cycle PDSA to improve performance in a priority area
each year
– % of staff who report they are encouraged to take risks
when implementing QI projects
Based on: 1) Senior Manager Survey of QI culture, QI knowledge and readiness
for change; 2) Key Informant Interviews of DPH Executives; 3) Monthly reports
from the 8 PI LC project teams; 4) DPH Annual QI Report Card
PI LC Early Lessons Learned
• Successes
– Team engagement with learning sessions
– Improved understanding of internal processes and
links to key measures with process mapping
– Individual team coaching
• Challenges
– Competing priorities (e.g., H1N1 response)
– Doing rapid small scale cycles is a very new
concept
– Lack of readily available, validated measures and
best practice tools for team aim areas
Performance Improvement Projects:
Future Plans
•
•
•
•
Have all operational divisions and units
working on at least one PI project each year
PI to track PI projects
Inclusion of key PI project measures in the
Public Health Measures
Reporting PI project results
– Monthly for selected programs
– Yearly summary for all others
Annual Timeline
Jan
Feb
Mar
Add/Drop//Modify
Public Health
Measures
Apr
Data
Update
May
June
Data Analysis
and Review
July
Aug
Sep
Prepare
Reports
Oct
Data
Update
Nov
Dec
Data Analysis
and Review
Prepare DPH
and CEO
Performance
Reports
Performance Improvement Projects
Progress Reviews with Health Officer
Performance Improvement Training
Summary
• A large health department needs to link many
sources of data, standards, and reporting
processes to build an efficient performance
improvement system
– This takes time to do properly
– Best with department-wide participation
– Need to communicate effectively across levels of
government and understand a wide variety of unit
demands
Summary
• A learning collaborative approach is essential
to explore common processes and small tests
of change
– Brings PI champions together which generates
enthusiasm
– Maximizes learning and sharing
– Promotes a culture of openness and transparency
– Creates a “centralized” opportunity for technical
assistance and coaching
Questions and Discussion