Transcript Building & Improving a Performance Management System

Building & Improving a Performance
Management System
Public Health Foundation
Paul Epstein, Results That Matter Team
Jack Moran, Public Health Foundation
Workshop for Kentucky Department for Public
Health and Local Health Departments
April 2013
Introduction
Public Health Foundation
www.phf.org
National, non-profit organization dedicated to achieving healthy communities through
research, training, and technical assistance for over 35 years
A key partner in national programs such as the National Public Health Performance
Standard Program & the CDC’s National Public Health Improvement Initiative
Provides tools and technical assistance on performance management, performance and
quality improvement and
Epstein & Fass Associates
Results That Matter Team
www.RTMteam.net
Measuring & improving public and nonprofit performance since 1985
Consultant with the Public Health Foundation, ASTHO, & NACCHO
Effective Community Governance and Community Balanced Scorecard (CBSC) methods
featured in The Public Health QI Handbook of PHF (ASQ Press, 2009)
CBSC tools used by state & local health departments & community health partnerships
for health improvement planning. strategic planning, & accreditation preparation
2
…PHF Mission:
We improve the public’s
health by strengthening the
quality and performance of
public health practice
www.phf.org
Innovative Solutions.
Measurable Results.
3
Workshop Objectives
 Learn to use a PM System model based on Quality
Improvement (QI) principles, the Essential Services of
Public Health, and the domains of PHAB to structure a
PM system for your public health agency
 Practice using parts of the model so you can help agency
program managers develop goals, objectives, and
performance measures in ways that support improving
health outcomes
 Target engagement of key stakeholders to help make PM
system development successful
4
Agenda
1. Turning Point-compatible PM System Model
based on Quality Improvement (QI) Cycles
2. Parts of a QI-based PM System:
Aligning goals and objectives for a selected program
Drafting performance measures for a selected program
3. PM System Development as QI: Design,
Deployment, Assessment, & Improvement
Analyzing stakeholders to engage in system development
4. Action Planning
5
Overview: Context
The Turning Point Framework
6
The Four Major Parts of Turning Point
• Performance Standards: Organizational or system expectations
to improve public health practices based on internal or external
goals or benchmarks
• Performance Measures: Clearly defined indicators for collecting
data to assess achievement of standards
• Reporting of Progress: Documenting and analyzing results vs.
expectations and communicating such information as feedback to
guide future performance improvement decisions
• Quality Improvement: A process to manage change and improve
performance in public health policies, programs, or infrastructure
based on standards, data, and reports
7
The Turning Point Framework
 A good description of what you need for performance
management but does not provide guidance on developing an
organization-wide Performance Management System
 Questions this model leaves unanswered are:
 How do we select standards & measure against them?
 What process do we use to determine what programs or
practices to improve?
 How do we make this a “system” to manage our
organization, not just more things to do?
 How do we do it in the PH context, e.g., incorporating
SHA/CHA, SHIP/CHIP, MAPP, Strategic Planning, and
Accreditation?
8
Performance Management &
Performance Management Systems
Performance Management:
Using performance information to help make better
decisions.
Performance Management System:
Using performance information on a regular basis as
part of a continually repeated cycle of performance
monitoring, analysis, and improvement, in which
measured results are fed back into decision making to
improve future performance.
9
QI “PDCA” Cycle Meets the PM System Definition
As applied to specific processes & practices
Decide on further
improvements
Develop improvement
plans & targets
Act Plan
Check Do
Monitor, analyze, &
interpret data
Implement improvements
& collect data
Sometimes called “PDSA” for
“Plan-Do-Study-Act”
10
Organization-wide PM Systems
 Applies a PM cycle to management and governance systems
throughout the organization, usually involving:
 Planning: strategic, policy, operational, & financial (e.g., budget)
 Operations: program & policy implementation
 Monitoring & analysis of performance at all levels: from strategic
to operational (sometimes to employee level)
 Decision making to sharpen plans & strategies and improve
performance at all levels
 Can incorporate PH context such as SHA/CHA, SHIP/CHIP,
the essential services of PH or domains of PHAB
 Can be viewed as a large-scale “systemic” QI PDCA Cycle
11
QI Framework for a Public Health Organization-wide PM System
Plan
Direction
Act
Performance-based
Decision Making
Decisions on Changes
to Improve Results
SHIP/CHIP (e.g., MAPP)
Strategic Plans
Operating Plans
Financial Plan (Budget)
Engagement of:
• Leaders & managers
• Employees
• Partners
• Other stakeholders
Expectations
Do
Program & Policy
Implementation
Operational QI
P
Info Technology Backbone
A
D
C
Check
Information
Data
Performance Monitoring
& Reporting for the
Population & Organization
(incl. SHA/CHA, MAPP)
Analysis of Results
12
MAPP in a QI-based Organizational PM System
MAPP: Partnership Approach to
Community Health Assessment,
Planning, & Improvement
Direction
Plan
Expectations
Do
Act
P
A
D
C
Information
Check
Data
MAPP Action Cycle as both
Operational QI & Systemic QI
13
Entire Governments Using QI Cycles for PM Systems
City of Austin’s
PDCA Government
Accountability
Model to manage
for Results
Plan
Strategic
Planning & Annual
Performance
Planning
Act
Do
Performance- based
Decision Making
Performance
Budgeting &
Service Delivery
Check
Performance
Monitoring &
Reporting
14
A Local Government’s Modified PDCA PM System Model
King County (WA)
PM System
Resource
Allocation
Strategic
Planning
Business
Planning
Implementation
Performance
Evaluation
15
A State Health Department’s PM Framework
16
QI Framework for a PH PM
System Includes All Four
Turning Point Quadrants
Direction
Plan
SHIP/CHIP (e.g., MAPP)
Strategic Plan
Operating Plans
Financial Plan (Budget)
TP: Performance Standards
& Performance Measures
Expectations
Act
Do
Performance-based
Decision Making
TP: Quality
Improvement
Decisions on Changes
to Improve Results
Program & Policy
Implementation
Operational QI
P
A
D
C
Check
Information
Data
Performance Monitoring
& Reporting for the
Population & Organization
(incl. SHA/CHA, MAPP)
Analysis of Results
TP: Reporting of Progress
17
At least 4 types of plans should be aligned:
they should mutually support each other
Plan
SHIP/CHIP (e.g., MAPP)
Strategic Plan
Operating Plans
Financial Plan (Budget)
Focuses on strategic change &
efforts to support SHIP/CHIP
Covers all programs or
organizational units
SHIP/CHIP
STRATEGIC PLAN
OPERATING PLANS
(“Business Plan,” “Service Plan,”
or “Performance Plan”)
BUDGET
18
Views from Different Parts of the System
Altitude
30,000 ft.
Vision
& Mission
SHA/CHA & Public
Health Policy
Priorities
20,000 ft.
10,000 ft.
SHIP/CHIP
with Health
Outcome Priorities
Strategic Plan with
Priority Change Goals
Operating Plan with Objectives,
Performance Measures,
Improvement Plans, & Initial Targets
Performance Budget with Negotiated
Targets
Ground
Programs, Services, Projects & Initiatives,
Performance Monitoring & Improvement
19
Alignment of Plans Across the Organization
In addition to alignment of the SHIP/CHIP
and strategic plan
The operating plans of all programs,
projects, or organizational units must be:
 Aligned with the public health agency’s mission
and goals
 Consistent (or not inconsistent) with the strategic
plan
20
Alignment of Operating Plans: Example
Strategic Changes for Some
Programs & Initiatives
Service
Service
Activity
Activity
State & Federal Mandates
Service
Service
Activity
Activity
Service
Service
Service
Activity
Activity
Common Purpose
=Program A
Goals &
Objectives
Common Purpose
=Program B
Goals &
Objectives
Common Purpose
=Program C
Goals &
Objectives
Program
Performance
Measures
Key Results A
Selected
Measure(s)
SHIP/CHIP
&
Strategic Plan
Common Purpose
= Division
Program
Performance
Measures
Key Results B
Selected
Measure(s)
Program
Performance
Measures
Key Results C
Selected
Measure(s)
Goals &
Objectives
Results =
Accomplishment of
 Key Results A
 Key Results B
 Key Results C
 Total or Net Cost
Common Purpose
Service
Service
Activity
Activity
Common Purpose
Service
Service
Service
Activity
Activity
Common Purpose
= Project D
Goals &
Objectives
=Initiative E
Goals &
Objectives
Project
Measures &
Milestones
Initiative
Measures &
Milestones
Key Results D
Selected
Measure(s)
=
Crossfunctional
Team
Goals &
Objectives
Results =
Key Results E
Selected
Measure(s)
A
G
E
N
C
Y
G
O
A
L
S
A
G
E
N
C
Y
M
I
S
S
I
O
N
Accomplishment of
 Key Results D
Key Results
E
Total
or Net Cost
21
Goal Alignment is NOT the Same as Goal Profusion
Alignment of Operating Plans
 The PHAB domains (including the Essential
Services of PH) provide a way to align all
operating plans of a PH agency
 Use a “top level strategy map” based on PHAB
domains for which program or organizational unit
maps can be “cascaded”
23
“Top Level Strategy Map” Based on PHAB Domains: Can Be Used to Align Strategies & Operating Plans
Perspectives
STRATEGIC GOALS & OBJECTIVES (“Generic Categories” for Program or Issue Objectives)
Community Health Status
H1. Improve
Health
Outcomes &
Eliminate
Disparities
H2. Improve
the Environment
for Health &
Reduce Health
Risks
#2
Investigate,
Contain PH
Problems &
Hazards
#6
Enforce Public
Health Laws
#3
Inform, Educate
About PH Issues
(Promote Health)
#7
Promote
Strategies to
Improve Access
to Services
#5
Develop PH
Policies &
Plans
#1
Assess, Disseminate Population
Health Status &
PH Issues
#9
Evaluate &
Improve Processes,
Programs, &
Interventions
# 10
Contribute to
& Apply the
Evidence Base
of PH
#11
Maintain
Administrative &
Management
Capacity
#12
Maintain Capacity
to Engage the PH
Governing Entity
Health Outcomes, Disparities,
Determinants, Risks.
Community
Implementation
Projects, services, & actions to
improve health or the conditions
for health.
Community
Process & Learning
All policy, planning, & decision
processes that can affect health.
Also, gathering & using
information.
#4
Engage the
Financial & non-financial resources
Community to ID
(e.g., people, organizations,
partnerships, facilities, equipment) & Address Health
Problems
and how they function to support
Community Assets
health strategies.
#8
Maintain a
Competent PH
Workforce
24
Cascading The Strategy Map by Program Or Unit
Top Level Strategy Map
Zoom in: Maps
for programs or
organizational
units
Structures the
performance mgt.
system
Zoom in again: Measures,
targets, timeframes for
program objectives
25
Possible Cascaded Map for Environmental Health Enforcement & Investigation Unit
Perspectives
STRATEGIC GOALS & OBJECTIVES
Improve EH
Outcomes &
Eliminate
Disparities
Community
Health
Status
Community
Implementation
#2
Investigate,
Contain Environmental Health
Hazards
#6
Enforce
Environmental
Health Codes
Community
Process &
Learning
#5
Develop
Policies that
Incentivize
Compliance
#1
Monitor
Environmental
Health Status
Community
Assets
#4
Engage the
Community to
Reduce Need for
Enforcement
Minimize EH
Risks &
Disparities in
Risk
#9
Continually
Improve
Quality
#8
Maintain a
Competent PH
Workforce
26
Performance Driver Relationships for EH Enforcement & Investigation Strategy
Perspectives
STRATEGIC GOALS & OBJECTIVES
Community
Health
Status
Improve EH
Outcomes &
Eliminate
Disparities
Community
Implementation
#2
Investigate,
Contain Environmental Health
Hazards
Community
Process &
Learning
Community
Assets
Integrated QI drives better
performance of monitoring,
policies, programs, and
practices
#9
Continually
Improve
Quality
#8
Maintain a
Competent PH
Workforce
Minimize EH
Risks &
Disparities in
Risk
#6
Enforce
Environmental
Health Codes
#1
Monitor
Environmental
Health Status
#5
Develop
Policies that
Incentivize
Compliance
#4
Engage the
Community to
Reduce Need for
Enforcement
27
Performance Driver Relationships for EH Enforcement & Investigation Strategy
Perspectives
STRATEGIC GOALS & OBJECTIVES
Community
Health
Status
Improve EH
Outcomes &
Eliminate
Disparities
Community
Implementation
#2
Investigate,
Contain Environmental Health
Hazards
Community
Process &
Learning
Community
Assets
Workforce development
is also an important
systemic driver of
success
#9
Continually
Improve
Quality
#8
Maintain a
Competent PH
Workforce
Minimize EH
Risks &
Disparities in
Risk
#6
Enforce
Environmental
Health Codes
#1
Monitor
Environmental
Health Status
#5
Develop
Policies that
Incentivize
Compliance
#4
Engage the
Community to
Reduce Need for
Enforcement
28
Possible Cascaded Map for Environmental Health Enforcement & Investigation Unit
Perspectives
STRATEGIC GOALS & OBJECTIVES
Improve EH
Outcomes &
Eliminate
Disparities
Minimize EH
Risks &
Disparities in
Risk
Community
Implementation
#2
Investigate,
Contain Environmental Health
Hazards
#6
Enforce
Environmental
Health Codes
# 6a
Enforce Food
Safety Code
Community
Process &
Learning
#5
Develop
Policies that
Incentivize
Compliance
#1
Monitor
Environmental
Health Status
#9
Continually
Improve
Quality
Community
Health
Status
Community
Assets
#4
Engage the
Community to
Reduce Need for
Enforcement
# 6b
Enforce NonFood EH Codes
#8
Maintain a
Competent PH
Workforce
29
Plan Alignment Exercise
For your group’s program:
1.
On the “top level strategy map,” circle the PHAB domains &
Community Health Status goals (“H1” & “H2”) that most
apply
2.
Revise the statements of goals & objectives you circled to be
more specific your group’s program; as needed, include
more than one goal or objective statement for the same
domain.
3.
Write the revised goal/objective statements on post-it notes
(include PHAB Domain #) and post them in the appropriate
“perspective” (i.e., “row”) of your “Strategy Map Worksheet.”
4.
Enter those statements into your Scorecard Template.
30
Cascading Strategy Maps by Program or Unit
Aligns the Performance Management System
Top-Level Strategy Map
Top level map creates
alignment of all unit or
program plans
F
F2
C
C1
P
LG
F
F2
C
C
P
LG
F
C1
C
P
L
F
P2
L
C
L
Program A Strategy
Map
C3
P3
P2
L1
F3
C
C
P4
L
L3
F
C3
P
Aligns & structures
these plans in the
performance
management system
C4
L2
F
LG
F3
C2
P
P
L1
P1
F1
F
F
C
C
P4
L
C
P
P
L3
F
L
Unit B Strategy Map
LG
C
C2
P
L
F
P3
L
C
P
L2
L
Program C Strategy
Map
31
National-State-Local Alignment in a PM System
Janie: I could help you tailor
this for Kentucky???
JANIE WILL EDIT FOR KY.
Cascading Strategy Maps Can Align Performance
Management Systems Across State & Local HDs
State & Local Top-Level Strategy Map
Top level map can
create alignment of all
LHD plans in the state
F
F2
C
C1
P
LG
F
F2
C
C
P
LG
F
C1
C
P
L
F
P2
L
C
L
LHD A Strategy Maps
C3
P3
P2
L1
F3
C
C
P4
L
L3
F
C3
P
Aligns & structures LHD
performance management
systems with the State’s PM
System
C4
L2
F
LG
F3
C2
P
P
L1
P1
F1
F
F
C
C
P4
L
C
P
P
L3
F
L
LHD B Strategy Maps
LG
C
C2
P
L
F
P3
L
C
P
L2
L
LHD C Strategy Maps
33
Plan
Expectations
A few strategic
performance measures
& targets here
Most operating
performance measures
& targets here
Expectations should be aligned via consistent
goals, objectives, &relevant measures
Health Improvement Priorities & Health Outcome
Goals & Measures in SHIP/CHIP
Change Priorities in STRATEGIC PLAN; Major
Organizational Goals & Standards
Program Goals, Performance Objectives, Measures, &
Targets in the OPERATING PLANS
Resources to Support Achieving Targets in the
BUDGET
34
Measurement Problem
 Top strategic goals & measures may be outcomefocused:
 Reduce number of smokers in our service area by 20,000/year
 But as deployment goes down to programs and
organizational units the measures often become more
output or process focused, e.g.,
 Number of stop smoking clinics held
 Number of pamphlets handed out at a Stop Smoking Fair
 A disconnect occurs:
 Is it credible that doing well on these program
measures will achieve the desired outcome?
35
Solving the Measurement Problem
 Help program staff understand that they need to:
 Use evidence to show that improving results on their output or
process measures really does drive outcomes, or …
 Develop new measures and targets that have an evidence-base,
or …
 Use the PDCA cycle to develop their own evidence.
 Ultimately, program measures
 Can be of any type (e.g., output, process, timeliness, customer
satisfaction, intermediate or “participant” outcomes)
 So long as improvement in the program measures logically
contribute to improving outcomes, i.e.:
 Improving program measures drives better outcomes
36
Or is a “Measurement Problem” Really a
“Strategy Design Problem”?
 Program measures may be inadequate to drive
outcomes because the program cannot adequately
address an issue on its own
 Many public health issues can only be adequately
addressed by multiple programs and partners
 That’s why there’s MAPP!
 Efforts of all partners must be strategically-aligned
and measurement of outcomes and drivers must
reflect efforts and accomplishments of all partners
37
Community
Assets
Community Process
& Learning
Community
Implementation
Community Health
Status
Perspective
BEHAVIORAL HEALTH STRATEGY
Issue-focused strategy map
Improve outcomes for people with, or at
for a collaborative strategy
risk of, behavioral health problems (H1)
of multiple programs &
partners, e.g.,
Reduce risk of behavioral health crises,
• Hospitals
including hospitalizations & suicides (H2)
• Drug treatment programs
• University
Improve access to
• Schools
Increase early
behavioral health
• Health dept.
identification &
treatment (7)
Develop & advocate for new or
improved programs & policies
on behavioral health (5)
Monitor needs by problem
type & demographics (1)
Educate “gatekeepers” (health workers,
educators, employers, first responders, &
others) about behavioral health (4)
resources (7)
Research & seek
funding for
opportunities to use
evidenced-based
best practices (10)
Continually update knowledge
& skills of behavioral health
professionals (8)
38
Highest Level Outcomes
Population
Outcomes
Place
Outcomes
39
Lagging & Leading Indicators
Population & place outcomes
tend to “lag” …
So you need performance drivers
that “lead” to enable mid-course
corrections to your strategy.
Some performance drivers are
“participant outcomes” or other
“intermediate outcomes”
40
Performance Drivers to Outcomes
Population
or Place
Outcomes
Performance Drivers, including
Participant Outcomes or other
“Intermediate Outcomes”
Performance
Driver
Chains of Success in a Strategic PM System
41
STRATEGIC OBJECTIVES
Perspective
Community
Health
Status
Community
Implementation
Community
Process &
Learning
Improve outcomes for people with, or
at risk of, behavioral health problems
Reduce risk of behavioral
health crises, including
hospitalizations & suicides
Increase early
identification &
treatment
Develop & advocate for new or
improved programs & policies on
behavioral health
Monitor needs by problem type
& demographics
Community
Assets
Educate “gatekeepers” (health workers,
educators, employers, first responders, &
others) about behavioral health
Improve access to
behavioral health
resources
Research & seek
funding for
opportunities to use
evidenced-based
best practices
Continually update knowledge & skills
of behavioral health professionals
42
COMMUNITY ASSETS
IMPLEMENTATION
COMMUNITY
COMMUNITY
HEALTH STATUS
Wood County Performance Measures for Highlighted Objectives
Improve outcomes for people with, or at risk of, behavioral health problems
Population Outcomes:
 Suicide rate
 Substance abuse rates for youth
 Hospitalization rates for behavioral health
Increase early identification & treatment
Performance Drivers:
 # of youth and adults who are screened for depression
 # of youth identified for early behavioral health concerns
 # of youth who receive treatment due to the screening process
Educate “gatekeepers” (health workers, educators, employers, first responders, &
other stakeholders) about behavioral health
 Performance Driver: # of “gatekeepers*” trained in identifying & responding to
behavioral health concerns
 Performance Driver: # of gatekeepers who report that the training enhanced their
ability to respond to behaviors of concern in others.
 Performance Driver (Participant Outcome): # of gatekeepers who report having
applied the knowledge learned in training situations in their life.
43
Types of Performance Measures for Environmental Health Enforcement & Investigation Unit
STRATEGIC OBJECTIVES
Community
Assets
Community
Process &
Learning
Community
Implementation
Community
Health Status
Perspective
Targeted Measures:
Population & place
outcomes.
Targeted Measures:
Performance drivers, including
some participant outcomes.
Some improvement initiatives
or QI Plans also likely.
Improvement
initiatives, QI Plans
and/or targeted
performance driver
measures
for these objectives.
Improve EH
Outcomes &
Eliminate
Disparities
#2
Investigate,
Contain Environmental Health
Hazards
#5
Develop
Policies that
Incentivize
Compliance
#4
Engage the
Community to
Reduce Need for
Enforcement
Minimize EH
Risks &
Disparities in
Risk
#6
Enforce
Environmental
Health Codes
#1
Monitor
Environmental
Health Status
#8
Maintain a
Competent PH
Workforce
Mostly lagging
indicators with
lower frequency
data
Leading indicators
with higher frequency
data; may have some
lagging indicators.
#9
Continually
Improve
Quality
44
Environmental Health Enforcement & Investigation
Perspective: Community HealthSTRATEGIC
Status OBJECTIVES, MEASURES, & INITIATIVES
Goal: Improve EH Outcomes & Eliminate Disparities
Outcome measure: Number
EH-related
illnesses
injuries per 1,000 residents
Improve
EH
Minimize&EH
Outcomes
&
Outcome measure: Number
food borne
illnesses
Risks per
& 1,000 residents
Goal: Minimize EH Risks & Eliminate
Disparities in Risk
Disparities in
Disparities
Risk
Outcome measure: Total outstanding EH violations
Perspective: Community Implementation
Objective: Investigate & Contain EH Hazards
# 2 outbreaks with
Performance driver: % suspected
# 6 cause identified within 48 hours
Investigate,
Enforce
Performance driver: Average days to mitigate
actual outbreaks
Contain EnvironObjective: Enforce Environmental
HealthEnvironmental
Codes
mental Health
Health Codes
Performance driver: Number
establishments in top safety tier for its type
Hazards
Performance driver: % inspections on schedule (including re-inspections)
Perspective: Community Process & Learning
# 5 Incentivize Compliance
Objective: Develop Policies that
#1
#9
Performance driver: % fee Develop
& fine schedules updated
&
approved
by BOH
Monitor
Continually
Policies that
Environmental
Objective: Continually Improve
Quality See
Notes for InitiativeImprove
Incentivize
Health Status
Quality
ComplianceHealth Status
Objective: Monitor Environmental
See Notes for Initiative
Perspective: Community Assets
Objective: Engage the Community
# 4 to Reduce Need for Enforcement
Performance driver: Number
targeted
that participate in training
#8
Engage
the establishments
Maintain a
Objective: Maintain a Component
CommunityPH
to Workforce
Community
Assets
Community
Process &
Learning
Community
Implementation
Community
Health Status
Perspective
Competent PH
Reduce requiring
Need for certifications
Performance driver: % positions
with up-to-date cert. staff See Op QI Plan
Workforce
Performance driver: % staffEnforcement
meeting their training & development plans
45
Environmental Health Enforcement & Investigation
Perspective: Community Health Status
Goal: Improve EH Outcomes & Eliminate Disparities
Outcome measure: Number EH-related illnesses & injuries per 1,000 residents
Outcome measure: Number food borne illnesses per 1,000 residents
Goal: Minimize EH Risks & Disparities in Risk
Outcome measure: Total outstanding EH violations (a driver of above outcomes)
Perspective: Community Implementation
Objective: Enforce Environmental Health Codes
Performance driver: Number establishments in top safety tier for its type
Performance driver: % inspections on schedule (including re-inspections)
Perspective: Community Process & Learning
Objective: Develop Policies that Incentivize Compliance
Performance driver: % fee & fine schedules updated & approved by BOH
Perspective: Community Assets
Objective: Engage the Community to Reduce Need for Enforcement
Performance driver: Number targeted establishments that participate in training
46
Measurement Alignment
Exercise
Show people they can color code
For your group’s program:
variance when showing spreadsheet
1.
For each “Health Status” goal, identify one or more performance
measures, especially outcomes, and enter them in Scorecard
under that goal. Copy ““ or ““ under “Desired Direction”
2.
For each “Implementation” objective, identify one or more
measures, especially “Performance Drivers” of the outcomes of
your Health Status goals, and enter those measures & “Desired
Directions” on your Scorecard under each objective.
3.
For each “Process & Learning” and “Assets” objective, determine
either an initiative(s), a measure(s), or both to drive performance of
one or more objectives above. Enter them in the Scorecard:

For each initiative, start with the word “Initiative” and end with a
completion date (e.g., “by 9/30/2013”).

For each measure, enter “Desired Direction.”
47
Measures Listed by Perspective (Sample, Excel)
Desired Direction and % Variance (Sample, Excel)
Show people they can color code
variance when showing spreadsheet
At bottom of spreadsheet:
49
Measures in Scorecard Format (Sample, Excel)
50
N M
I
I
N
Q
N
Trend Line of a Measure vs. Target
Percent inspections on schedule (including re-inspections)
Target = 95%
Actual
99%
98%
98%
84%
80%
88%
95%
99%
99%
52
N M
I
I
N
Q
N
Link to Maps with Data by Geographic District
% Establishments Inspected in Top Safety Tier
By Community Board District - 2013
% Establishments Inspected in Top
Safety Tier by Community District
20 – 39%
40 – 59%
60 – 79%
80 – 100%
54
Implementation
Perform consistent with expectations in how you do all that
you do, e.g.:
Do
Program & Policy
Implementation
•
•
•
•
•
•
•
•
•
•
•
•
Operational QI
P
A
D
C
P
A
D
C
Investigate & contain PH problems & hazards (PHAB 2)
Enforce PH laws (PHAB 6)
Inform & educate about PH issues (promote health) (PHAB 3)
Promote strategies to improve access to services (PHAB 7)
Develop PH policies & plans (PHAB 5)
Assess, disseminate population health status & issues (PHAB 1)
Evaluate & improve processes, programs, interventions (PHAB 9)
Contribute to & apply the evidence-base of PH (PHAB 10)
Engage the community to ID & address health problems (PHAB 4)
Maintain a competent PH workforce (PHAB 8)
Maintain administrative & management capacity (PHAB 11)
Maintain capacity to engage the PH governing entity (PHAB 12)
Operational QI
•
Use “high-frequency data” to make regular incremental quality
improvements to regular practices and processes
•
Ensure managers, staff, & partners have the tools & incentives to
keep improving performance
55
N M
I
I
N
Q
N
A “Stat” System as a High-frequency Operational QI Cycle
Plan
Plan & Map
Targeted
Improvements
Use partners as
needed for geographic,
demographic, or
programmatic
capabilities
Act
Do
Decide on Nature &
Targets of Changes
to Improve Results
Implement
Targeted
Changes
Check
Frequently Measure,
Map, Review,
& Analyze Results
57
Data Frequency Exercise
For your group’s program:
Identify any performance measures for which you think
you should get data more often than quarterly for
operations management. Enter the frequency (e.g.,
monthly, weekly, daily) under “Data Frequency”
58
Do
P
A
Collect DATA & make accessible at the frequency &
levels of detail needed by data users
 Relevant data to support analyses and decisions
 Reliable data to increase confidence in decisions & reduce risk
D
C
Data
Check
Check
Monitor & report results on, e.g.,
Performance Monitoring
& Reporting for the
Population & Organization
(incl. SHA/CHA, MAPP)
Analysis of Results
 Assessments every 3-5 years: SHA/CHA, MAPP
 Population & place outcomes (e.g., of SHIP/CHIP): lagging
 Performance drivers (of organization & partners): leading
 Financial performance & conditions
Analyze results for EVIDENCE to support decisions
59
Provide DIRECTION to
 Influence future plans & strategies
Direction
 Make future improvements more likely
 Improve the performance management system
Plan
Act
Decide on changes to improve results, e.g.,
 Redesign programs, revise operational plans
 Reallocate funds & other resources
 Improve capabilities of PM, QI, employees
From “performance-informed” to
“performance-driven” decisions
Act
Performance-based
Decision Making
Decisions on Changes
to Improve Results
Convert raw data to timely, useful
INFORMATION for decision makers
Information
 Move from analyses to interpretations
Check
 Interpretations clarify evidence for decision makers
60
QI Framework for a Public Health Organization-wide PM System
Plan
Direction
Act
Performance-based
Decision Making
Decisions on Changes
to Improve Results
SHIP/CHIP (e.g., MAPP)
Strategic Plan
Operating Plans
Financial Plan (Budget)
Engagement of:
• Leaders & managers
• Employees
• Partners
• Other stakeholders
Expectations
Do
Program & Policy
Implementation
Operational QI
P
Info Technology Backbone
A
D
C
Check
Information
Data
Performance Monitoring
& Reporting for the
Population & Organization
(incl. SHA/CHA, MAPP)
Analysis of Results
61
QI “PDCA” Can Also Guide PM System Development
Design the System
Improve the System
Act
Plan
Check Do
Review the System
Deploy the System
62
PLAN: Design the System
 Performance Management System functionality:
 Align operations with strategy (in SHIP/CHIP & strategic plan)
 Guide day-to-day operations & encourage regular, frequent QI
 Help identify improvements needed at all levels
 Support decisions to improve performance & strategy
 PM system development usually led by a DESIGN TEAM:
 Staffed with exec “owner” rep (HD Director), “cross section” of users
(Program & Division Staff), IT staff, consistent with PHAB 9.2.1 A:
 A current, functioning PM committee or team.
 Seeks broad engagement from stakeholders, as in PHAB 9.1.1 A:
 Engage staff at all organizational levels in establishing … a PM system.
 Defines purpose of the PM System & budget or resources to run it
 Decides whether to use the approach and spreadsheets provided here,
revise them, or use a different model
63
Performance Management System Team (PMST)
 Design team transitions into an ongoing “PM SYSTEM TEAM” to:
 Decide on software, hardware, user operating guidance
 Guide initial deployment & system improvement over time
 Ensure the PM system has adequate ongoing resources run effectively:
so it meets user needs on time, all the time.
 Meets PHAB Measure 9.1.2 A on an ongoing basis:
 A current, functioning PM committee or team.
 The PMST is an “System Ownership” group that:
 Has full accountability for the system
 Makes sure system meets the needs of its users to continually improve
performance
 Entertains & decides on user requests for changes
 Has people who spend significant time focused on the system long-term
(more than most Design Team members)
64
DO: Deploy the System
P
A
D
C
D
E
P
L
O
Y
M
E
N
T
F
E
E
D
B
A
C
K
65
Exercise: Barriers to Deploying a PM System
1. What are the barriers to developing
and implementing a performance
management system, and what
stakeholders are associated with
each barrier?
2.
What can be done to stimulate
greater support from the
stakeholders identified?
66
DO: Deploy the System
 For successful PM system deployment, assess
stakeholders and stakeholder groups for:
 Expected level of their support for system
development and deployments (e.g., what barriers, if
any, are they associated with?)
 Their level of influence: How badly you need them for
the PM system to succeed
 Involve stakeholders differently based on that
assessment.
67
Exercise: Stakeholder Analysis for Systemic Improvements
High: Will help
Expected Support
as they can
Medium: May
cooperate,
unlikely to
oppose
Low: Will not
cooperate &
may oppose
Low
Medium
High
Influence, Need for Success
68
How to Involve Different Stakeholders
High: Will help
Expected Support
as they can
Actively involve & ask them to help engage
others (especially those lower on this chart)
Medium: May
cooperate,
unlikely to
oppose
Use example of those providing “high
support” to try to get these
stakeholders on board
Low: Will not
Engage them to be sure they
understand their interests are
considered; try to keep them from
opposing the process & the plan
cooperate &
may oppose
Low
Medium
High
Influence, Need for Success
69
DO: Deploy the System
 PM System deployment should be part of a culture
change to make the entire organization results focused.
 In deployment, use development & revision of measures
& scorecards to drive a results-focused culture by:
 Engaging those accountable for measures in their development
 Assuring organizational performance measures align with priority
health outcomes to be achieved & showing linkages to all staff
 A cultural norm should be: “Everything we do must add
value to improving population health.”
 A culture will change when desired new behaviors are
measured, reported, recognized, rewarded, & celebrated.
70
CHECK: Review the System
 The PMST should assess the PM system at least yearly to
ensure it meets the needs & expectations of the organization
 Every few years conduct a more thorough system audit
 Focus of a review can include:
 Use of the system & its information by the users
 Changes in system requirements
 Alignment of expectations with a focus on results
 Relevant measures and reliable data
 Employee involvement and training
 Organizational guidance
 Review vs. other organizations’ PM systems
 What the “next level” of system advancement should be
71
CHECK: For Relevant Measures & Reliable Data
A Key to Relevance: Ask people if they use the measures.
A Key to Reliability: A clear measure definition.
72
CHECK: Assess System Maturity & “Next Level”
Determine where you are in the evolution of a mature
PM system
Determine the most strategic next level of development
for the PM system in your organization, community, &
state
Potential “Next Levels”:
• Build “Operational QI
Plans” into the PM System
• Enhance organization
learning through multiple
PM cycles
73
Sample Operational QI Plan
74
(4) 4-Year Major
Update of
Strategic Plan
4-Year Strategic Plan
•Strategic Goals
•Community Outcomes
• Strategies & Objectives
Every 4th Year:
Extensive Citizen Process
•Outreach Across County
•Strategic Issue Focus Groups
•Goal Task Forces
(3) Annual
Strategic
Plan Update
Measure &
Evaluate Results
•Performance Reporting
•Citizen Surveys
•Citizen Input
4 Cycles at Different
Time Scales
Performance Budget
•Funding Decisions
•Service Level Decisions
& Performance Targets
(2) Performance Data
for Annual Budget &
Mid-Year Adjustments
(1) Performance
Data to Improve
Service Results
Deliver &
Improve
Services
Governing for Results Cycles in Prince William County, VA 75
(4) 4-5 Year
Major Update
of Strategic Plan
PH Agency Strategic Plan
•Agency Vision, Mission, Values
•Strategic Goals, Objectives, Measures
• Key improvement targets
Every 3-5 Years:
MAPP Iteration Produces CHIP
•Organize Partners, Visioning
•ID Issues based on assessments
• Goals, strategies, plans
(3) Annual
Strategic
Plan Update
Budgets & Operating Plans
•PH agency budget &
performance targets, incl. MAPP
• MAPP partners’ resource &
performance targets
(2) Annual
& Mid-Yr
Adjustments
Measure & Evaluate Results
•3-5 Yrs: 4 MAPP assessments (CHA)
•Weekly, Monthly, Yearly: Results of
MAPP Initiatives & Agency
Performance vs. Targets
KEY
Blue: Long-term MAPP
Assessment & Planning
Red: MAPP Action Cycle
Cycles of MAPP & Agency
Performance Management
MAPP Action Cycle
(1) Operational
QI by PH agency
QI within
MAPP Action
Cycle
Implement
Plans,
Programs,
Services
76
ACT: Improve the System
 Based on the results of a system assessment or audit,
the PMST should decide on system improvements.
 To implement PM system improvements:
 Start the PDCA cycle again
 As appropriate, engage management, employees, & other
stakeholders in designing and implementing changes.
 Clearly communicate PM system changes to all users
before the changes are put into effect.
77
PM System Implementation: Suggested Progress Measures
78
PM System Action Planning
If there’s an assignment, add a
slide before this one describing
the assignment, as well as a
handout.
Then also put the assignment on
the Excel spreadsheet with the
Action Plan as well as on this
slide. Include “Due Date” under
“Target Date.”
79
Concluding Discussion, Assignment, Follow-up
Discussion:
May be more than
one slide.
 What did you learn today?
 What will you do with the knowledge & tools?
Assignment:
 Present here if not on earlier slide, or do a quick
summary review …
Follow-up Schedule
 Whatever it is …
80