Transcript Results and Performance Accountabilty, Decision

RBA
TM
Results-Based Accountability
The Fiscal Policy Studies Institute
www.raguide.org
www.resultsaccountability.com
Book - DVD Orders
amazon.com
resultsleadership.org
Agenda for Today
1.
Welcome and Opening Remarks



2.
3.
4.
5.
Jerry Jones, Interim CEO Santa Fe Community
Foundation
Mayor Javier Gonzales
Kathy Armijo-Etre, CHRISTUS St. Vincent Regional
Medical Center
Results-Based Accountability Presentation
Lunch Break
Turn the Curve Exercise (Population or Program
Performance)
Questions & Answers, Next Steps
4/27/2020
SIMPLE
COMMON SENSE
PLAIN LANGUAGE
MINIMUM PAPER
USEFUL
Results-Based Accountability
is made up of two parts:
Population Accountability
is about the well-being of
WHOLE POPULATIONS
for Communities – Cities – Counties – States - Nations
Performance Accountability
is about the well-being of
CUSTOMER POPULATIONS
for Programs – Agencies – Service Systems
Results-Based Accountability
COMMON LANGUAGE
COMMON SENSE
COMMON GROUND
Language Discipline
THE LANGUAGE TRAP
Too many terms. Too few definitions. Too little discipline
Benchmark
Outcome
Result
Modifiers
Indicator
Measurable Core
Urgent
Qualitative
Priority
Programmatic
Targeted
Performance
Incremental Strategic
Systemic
Measure
Goal
Objective
Target
Lewis Carroll Center for Language Disorders
Core Make
qualitative
strategic
objectives
up your
own jargon.
Measurable
urgent
systemic
indicators
DEFINITIONS
RESULT or OUTCOME
1. A condition of well-being for
Population
children, adults, families or communities.
Children born healthy
healthy, Children ready for school,
school
Safe communities,
communities Clean Environment,
Environment Prosperous Economy
INDICATOR or BENCHMARK
2. A measure which helps quantify the achievement
of a result.
Performance
Rate of low-birthweight babies
babies, Percent ready at K entry
entry,
rate air quality index,
index unemployment rate
crime rate,
PERFORMANCE MEASURE
3. A measure of how well a program, agency or service
system is working.
1. How much did we do?
Three types: 2. How well did we do it?
3. Is anyone better off? = Customer Results
From Ends to Means
From Talk
Talk to
From
toAction
Action
Population
RESULT
ENDS
Performance
INDICATOR
PERFORMANCE
MEASURE
Customer result = Ends
Service delivery = Means
MEANS
IS IT A RESULT, INDICATOR OR
PERFORMANCE MEASURE?
RESULT 1.
INDICATOR 2.
PERF. MEASURE 3.
RESULT 4.
Safe Communities
Crime rate
Average police response time
People live in safe stable affordable housing
INDICATOR 5. %
RESULT 6.
INDICATOR 7.
People have living wage jobs and income
% of people with living wage jobs and income
PERF. MEASURE 8. %
|
spending >30% of income on housing
of participants in job training who get living
wage jobs
Translation Guide/Rosetta Stone
Not the Language Police
Ideas
Group 1
Group 2
1. A condition
of well-being
for children,
adults, families
& communities
RESULT
OUTCOME
2.
3.
etc.
TRANSLATION
Back to the Idea
Group 3
etc.
GOAL
Results – Indicators – Performance Measures in
Amharic, Cambodian, Laotian, Somali, Spanish, Tigrigna, Vietnamese
POPULATION
ACCOUNTABILITY
Community Outcomes
for Christchurch, NZ
1. A Safe City
2. A City of Inclusive and Diverse Communities
3. A City of People who Value and Protect the Natural
Environment
4. A Well-Governed City
5. A Prosperous City
6. A Healthy City
7. A City for Recreation, Fun and Creativity
8. City of Lifelong Learning
9. An Attractive and Well-Designed City
Georgia Policy Council
for Children and Families
RESULTS
● Healthy Children
● Children Ready for School
● Children Succeeding in School
● Strong Families
● Self Sufficient Families
City of London Ontario
Priority Results
● A Strong Economy
● A Vibrant Diverse
Community
● A Green and Growing City
● A Reliable Infrastructure
● A Safe City
Source:
Draft Strategic Plan Sept 2011
CARDIFF, WALES
Community Outcomes
1. People in Cardiff are healthy
2. Cardiff has a clean attractive and sustainable
environment
3. People in Cardiff are safe and feel safe
4. Cardiff has a thriving and prosperous
economy
5. People in Cardiff achieve their full potential.
6. Cardiff is a great place to live, work and play.
7. Cardiff has a fair, just and inclusive society.
REPORT CARDS
Leaking Roof
(Results thinking in everyday life)
Experience
Cm of Water
BASELINE
Measure
Not OK
? Fixed
Turning the Curve
Story behind the baseline (causes)
Partners
What Works
Action Plan # 2
Results / Outcomes
Experience
Indicators
Baseline & Story
Criteria for
Choosing Indicators
as Primary vs. Secondary Measures
Communication Power
Does the indicator communicate to a broad range of audiences?
Proxy Power
Does the indicator say something of central importance about the result?
Does the indicator bring along the data HERD?
Data Power
Quality data available on a timely basis.
Choosing Indicators
Worksheet
Safe Community
Outcome or Result_______________________
Candidate Indicators
Communication
Power
Proxy
Power
Data
Power
Measure 1
H M L
H M L
H M L
H
H
H
H
H
L
Measure 2
Measure 3
Measure 4
Measure 5
Measure 6
Measure 7
Measure 8
Data
Development
Agenda
Three Part Indicator List for each Result
Part 1: Primary Indicators
● 3 to 5 “Headline” Indicators
● What this result “means” to the community
● Meets the Public Square Test
Part 2: Secondary Indicators
● Everything else that’s any good (Nothing is wasted.)
● Used later in the Story behind the Curve
Part 3: Data Development Agenda
● New data
● Data in need of repair (quality,timeliness etc.)
The Matter of Baselines
H
M OK?
L
Point to Point
History
Turning the Curve
Forecast
Baselines have two parts: history and forecast
Alcohol-Related Traffic Fatalities
United States 1975 to 2005
30
75 people
per day
25
45 people
per day
Tho usand s
20
15
MADD
10
28 people
per day
5
2005
2010
Source 1982 to 2005: Actual data from the NHTSA Fatality Analysis Reporting System (FARS)
Source 1975 to 1981: Estimate based on NHTSA data on % of fatality drivers with BAC of .10 or greater.
2000
1990
1982
1980
1975
0
Rebound
Newcastle, UK
Nov 08 –
Jan 09
8.5
Christchurch, New Zealand
Number of Graffiti Sites
FY 2002 to FY 2010
no. of Tagged Sites visited
2002/03
18000
16000
14000
12000
10000
8000
6000
4000
2000
0
2003/04
2004/05
2005/06
2006/07
2007/08
2008/09
20
01
/0
20 2
02
/0
20 3
03
/0
20 4
04
/0
20 5
05
/0
20 6
06
/0
20 7
07
/0
20 8
08
/0
20 9
09
/1
0
no. of sites
2001/02
year
2009/10
Hull, UK
"Woundings"
“If I include you,
you will be my partner.
If I exclude you,
you will be my judge.”
- Rosell
PERFORMANCE
ACCOUNTABILITY
“All performance measures
that have ever existed
for any program
in the history of the universe
involve answering two sets of
interlocking questions.”
Performance Measures
Quantity
Quality
How
Much
How
Well
did we do?
did we do it?
(#)
(%)
Performance Measures
Effort
How hard did we try?
Effect
Is anyone better off?
Performance Measures
Effort
How
Much
How
Well
Effect
Quantity
Quality
Input
Effort
How much
service did
we deliver?
How well
did we
deliver it?
Output
Effect
Performance Measures
How much
change / effect
did we produce?
What quality of
change / effect
did we produce?
Effect
Effort
Performance Measures
Quantity
Quality
How much
did we do?
How well
did we do it?
Is anyone
better off?
#
%
Effort
Education
Quantity
Quality
How much did we do?
How well did we do it?
Number of
students
Student-teacher
ratio
Effect
Is anyone better off?
Number of
high school
graduates
Percent of
high school
graduates
Effort
Education
Quantity
Quality
How much did we do?
How well did we do it?
Number of
students
Student-teacher
ratio
Effect
Is anyone better off?
Number of
9th graders who
graduate on time
and enter college or
employment after
graduation
Percent of
9th graders who
graduate on time
and enter college or
employment after
graduation
Effort
Pediatric Practice
Quantity
Quality
How much did we do?
How well did we do it?
Number of
children
treated
Percent of
patients waiting
less than
30 min in the
waiting room
Effect
Is anyone better off?
#
children
fully
immunized
%
children
fully
immunized
(in the practice)
(in the practice)
Effort
Drug/Alcohol Treatment Program
Quantity
Quality
How much did we do?
How well did we do it?
Number of
persons
treated
Percent of
staff with
training/
certification
Effect
Is anyone better off?
Number of clients Percent of clients
off of alcohol &
off of alcohol &
drugs
drugs
- at exit
- 12 months after exit
- at exit
- 12 months after exit
Effort
Fire Department
Quantity
Quality
How much did we do?
How well did we do it?
Number of
responses
Response
Time
Effect
Is anyone better off?
# of fires
kept to
room of origin
% of fires
kept to
room of origin
Effort
Waste Management Services
Quantity
Quality
How much did we
do?
How well did we do
it?
# tonnes of
residential
waste
Unit cost
per tonne
collected
Effect
Is anyone better off?
#/amt to land fill
% to land fill
#/amt diverted
from landfill
% diverted from
landfill
Effort
General Motors
Quantity
Quality
How much did we do?
How well did we do it?
# of production hrs
Employees per
vehicle
produced
# tons of steel
Effect
Is anyone better off?
# of cars sold
% Market share
$ Amount of Profit
Profit per share
$ Car value after
2 years
% Car value after
2 years
Source: USA Today 9/28/98
Effort
Not All Performance Measures Are Created Equal
Quantity
Quality
Quality
How much did we do?
How well did we do it?
Least
Also
Very Important
Important
Effect
Is anyone better off?
Most
Important
RBA Categories Account for All Performance Measures
(in the history of the universe)
Total Quality
Mgmt (TQM)
Cost
Process
Input
Quality
Effort
Quantity
Efficiency Admin overhead, Unit cost
Efficiency,
Staffing ratios, Staff turnover
Staff morale, Access, Waiting time,
Waiting lists, Worker safety
Customer Satisfaction
Product
Output
Impact
Effect
(quality service delivery
& customer benefit)
Benefit value
Cost / Benefit ratio
Return on investment
Client results or client outcomes
Effectiveness
Value added
Productivity
RBA Categories Account for All Performance Measures
(in the history of the universe)
Total Quality
Mgmt (TQM)
Cost
Process
Input
Quality
Effort
Quantity
Efficiency, Admin overhead, Unit cost
Staffing ratios, Staff turnover
Staff morale, Access, Waiting time,
Waiting lists, Worker safety
Customer Satisfaction
Product
Output
Impact
Effect
(quality service delivery
& customer benefit)
Benefit value
Cost / Benefit ratio
Return on investment
*
1. Did we treat
you well?
2. Did we help
you with your
problems?
Client results or client outcomes
Effectiveness
Value added
Productivity
* World’s simplest complete
customer satisfaction survey
Effort
The Matter of Control
Quantity
Quality
How much did we do?
How well did we do it?
Most
Control
Effect
Is anyone better off?
Least
Control
PARTNERSHIPS
The Matter of Use
1. The first purpose of performance
measurement is to
IMPROVE PERFORMANCE.
2. Avoid the performance measurement
equals punishment trap.
● Create a healthy organizational environment.
● Start small.
● Build bottom-up and top-down simultaneously.
CHARTS on the WALL
Advanced Baseline Display
Create targets
only when they are:
FAIR & USEFUL
Aspiration
NOT
Punishment
Goal (line)
Target or Standard
●
Avoid publicly declaring
targets by year if possible.
x
Your Baseline
Comparison Baseline
Instead:
Count anything better
than baseline as progress.
Performance Accountability
Types of Measures found in each Quadrant
How much did we do?
How well did we do it?
# Clients/customers
served
% Common measures
# Activities (by type
of activity)
% Activity-specific
measures
e.g. client staff ratio, workload ratio, staff
turnover rate, staff morale, % staff fully
trained, % clients seen in their own language,
worker safety, unit cost
e.g. % timely, % clients completing activity,
% correct and complete, % meeting standard
Is anyone better off?
% Skills / Knowledge
#
#
#
#
Point in Time
vs.
2 Point
Comparison
#
(e.g. parenting skills)
% Attitude / Opinion
%
(e.g. toward drugs)
% Behavior
(e.g.school attendance)
% Circumstance
(e.g. working, in stable housing)
School Hospital Job Training Fire Department
Service: __________________________________
How much did we do?
How well did we do it?
Primary customers
persons trained
patients
# students
Unit cost
Primary activity
hours
diagnostic
alarms
courses
of tests
# job
instruction to
responded
Workload ratio
% of ___x___ that
happen on time
Is anyone better off?
If your service works really well,
how are your customer's better off?
#
students
who
persons
get
patients
fully
keptwho
to room
% fires
graduate
jobs
recover
of
origin
Effort
Choosing Headline Measures and the Data Development Agenda
Quantity
Quality
How much did we do?
How well did we do it?
# Measure 1 ----------------------------
% Measure 8 ----------------------------
# Measure 2 ----------------------------
% Measure 9 -----------------------------
# Measure 3 ----------------------------
% Measure 10 ---------------------------
# Measure 4 ----------------------------
% Measure 11 ---------------------------
# Measure 5 ----------------------------
% Measure 12 ---------------------------
# Measure 6 ----------------------------
% Measure 13 ---------------------------
# Measure 7 ----------------------------
% Measure 14 ---------------------------
#3 DDA
#2 Headline
Effect
Is anyone better off?
# Measure 15 ----------------------------
% Measure 15 ----------------------------
# Measure 16 ----------------------------
% Measure 16 ----------------------------
# Measure 17 ----------------------------
% Measure 17 ----------------------------
# Measure 18 ----------------------------
% Measure 18 ----------------------------
# Measure 19 ----------------------------
% Measure 19 ----------------------------
# Measure 20 ----------------------------
% Measure 20 ----------------------------
# Measure 21 ----------------------------
% Measure 21 ----------------------------
#2 DDA
#3 Headline
#1 Headline
#1 DDA
Primary v.
Secondary
Direct v.
Indirect
Internal v.
External
LR
UR
Baseline & Story
Christus Saint Vincent's Medical Center
Santa Fe, NM
Turned curve on high end ER Use
Etre Kathy <[email protected]>
Watermain Breaks per year
London Ontario
Watermain Breaks per Year
300
35
30
250
25
200
150
100
Start of
Replacement
Program
Trend without
renewal
15
Trend of
Financial Plan
10
50
0
1970
5
1980
1990
Main Breaks
Pipe age (right axis)
Trend (Main Breaks without Renewal)
2000
2010
2020
2030
"Main Breaks without renewal
Trend (Financial Plan forecast)
0
2040
pipe age
20
Select 3 to 5 Performance Measures
ACROSS THE BOTTOM OF THE ORG
CHART
3-5
3-5
3-5
3-5
3-5
3-5
3-5
3-5
3-5
20 – 60 – 20 Rule
3-5
3-5
Next Generation
Contracting
Next Generation Contracting
Contract Provisions
Provision 1. Specify the 3 to 5 most important performance
measures (from the How well did we do it? and Is anyone
better off? categories).
Provision 2. Specify that the contractor will use a continuous
improvement process (the RBA 7 Questions).
Provision 3. Specify how the funder and contractor will work
in partnership to maximize LR customer results (quarterly
meetings using the 7 questions as the agenda).
Provision 4. Specify that the funder will work with the funding
community to simplify and standardize contracting and
performance reporting.
Next Generation Contracting
Contract Provisions
Provision 5: : Clear articulation of role in population/community
well-being using the language of contribution not
attribution.
Provision 6: 10% for quality management and administration.
Provision 7: Multi-year funding using 3 year rolling contracts
Provision 8: Use of targets that are fair and useful.
Provision 9: Fund flexibility and virtual funding pool: transfer
of up to 10% across line items and program lines.
Provision 10: Request for Results: Getting past the sometimes
negative effects of competitive RFP contracting
or tendering.
HOW
Population
and
Performance
FIT TOGETHER
THE LINKAGE Between POPULATION and PERFORMANCE
POPULATION ACCOUNTABILITY
POPULATION
RESULTS
Healthy Births
Rate of low birth-weight babies
Children Ready for School
Percent fully ready per K-entry assessment
Self-sufficient Families
Percent of parents earning a living wage
Alignment
of measures
PERFORMANCE ACCOUNTABILITY
Job Training Program
# persons
receiving
training
Unit cost
per person
trained
# who get
living wage
jobs
% who get
living wage
jobs
Contribution
relationship
Appropriate
responsibility
CUSTOMER
RESULTS
Performance
measure
Population
Indicator
Population Accountability
to which you contribute to most directly.
Result:
Indicators:
Every time
you present
your program,
Use a
two-part
approach.
Story:
Partners:
What would it take?:
Role: as part of a larger strategy.
Your Role
Performance Accountability
Program:
Performance measures:
Story:
Partners:
Action plan to get better:
Population Accountability
to which you contribute to most directly.
Result:
Indicators:
Every time
you present
your program,
Use a
two-part
approach.
Story:
Partners:
What would it take?:
Role: within the larger strategy.
Your Role
Performance Accountability
Program:
Performance measures:
Story:
Partners:
Action plan to get better:
Population Accountability
to which you contribute to most directly.
Result:
Indicators:
Every time
you present
your program,
Use a
two-part
approach.
Shortcut
Story:
Partners:
What would it take?:
Role: as part of a larger strategy.
Your Role
Performance Accountability
Program:
Performance measures:
Story:
Partners:
Action plan to get better:
Different Kinds of Progress
1. Data
a. Population indicators Actual turned curves:
movement for the better away from the baseline.
b. Program performance measures:
customer progress and better service:
How much did we do?
How well did we do it?
Is anyone better off?
2. Accomplishments: Positive activities, not included above.
3. Stories behind the statistics that show how individuals are
better off.
Board of Directors Meeting
AGENDA
1. New data
2. New story behind the curves
3. New partners
4. New information on what works.
5. New information on financing
6. Changes to action plan and budget
7. Adjourn
SUMMARY
RBA in a Nutshell
2–3-7
2 - kinds of accountability plus language discipline
Population accountability
Performance accountability
Results & Indicators
Performance measures
3 - kinds of performance measures.
How much did we do?
How well did we do it?
Is anyone better off?
7 - questions from ends to means in less than
an hour. Baselines & Turning the Curve
THANK YOU !
Book - DVD Orders
amazon.com
resultsleadership.org
“Data is the
unblinking eye
of reform.”
Barbary Curley,
Area Director, Hyde Park Office, Boston
EXERCISES
Creating a Working Baseline
from Group Knowledge
Indicator or Performance
Measure
65%
Now
Not
OK?
Turn the Curve Exercise: Population Well-being
5 min: Starting Points
- timekeeper and reporter
- geographic area
- two hats (yours plus partner’s)
10 min: Baseline
- pick a result and a curve to turn
- forecast (to 2017) – OK or not OK?
15 min: Story behind the baseline
- causes/forces at work
- information & research agenda part 1 - causes
15 min: What works? (What would it take?)
- what could work to do better?
- each partners contribution
- no-cost / low-cost ideas
- information & research agenda part 2 – what works
10 min: Report convert notes to one page
Two
pointers
to action
4. --------- Off the Wall
ONE PAGE Turn the Curve Report: Population
Result: _______________
Indicator
Baseline
Indicator
(Lay Definition)
Story behind the baseline
-----------------------------------------------------
(List as many as needed)
Partners
-----------------------------------------------------
(List as many as needed)
Three Best Ideas – What Works
1. --------------------------2. --------------------------3. ---------No-cost / low-cost
4. --------- Off the Wall
Sharp
Edges
Turn the Curve Exercise: Program Performance
5 min: Starting Points
- timekeeper and reporter
- identify a program to work on
- two hats (yours plus partner’s)
10 min: Performance measure baseline
- choose 1 measure to work on – from the lower right quadrant
- forecast (to 2017) – OK or not OK?
15 min: Story behind the baseline
- causes/forces at work
- information & research agenda part 1 - causes
15 min: What works? (What would it take?)
- what could work to do better?
- each partners contribution
- no-cost / low-cost ideas
- information & research agenda part 2 – what works
10 min: Report convert notes to one page
Two
pointers
to action
4. --------- Off the Wall
ONE PAGE Turn the Curve Report: Performance
Program: _______________
Performance
Measure
Baseline
Performance Measure
(Lay definition)
Story behind the baseline
-----------------------------------------------------
(List as many as needed)
Partners
-----------------------------------------------------
(List as many as needed)
Three Best Ideas – What Works
1. --------------------------2. --------------------------3. ---------No-cost / low-cost
4. --------- Off the Wall
Sharp
Edges
Turn the Curve Exercise – Lessons
Talk to Action in an hour
1. How was this different from other processes?
What worked and what didn’t work?
2. Why did we ask for:
a. Results before indicators?
b. Forecast?
c. Story?
d. No cost / low cost?
e. Two hats?
f. Crazy idea?
g. Only 3 best ideas?
3. Do you think a lay audience could understand
the reports?
4. How many think you could lead this exercise
with a small group? (2+ curves at the same time)
Next Steps:
1. What’s one thing I can personally
do with what I learned today?
2. What’s one thing I would like to ask
someone else to do?
Resources
www.raguide.org
www.resultsaccountability.com
RBA Facebook Group
Book - DVD Orders
amazon.com
resultsleadership.org
THANK YOU !
Book - DVD Orders
amazon.com
resultsleadership.org