Transcript Results and Performance Accountabilty, Decision

The Five W’s and How of
Results-Based Accountability™
Social Outcomes
Measurement Conference
Sydney, AU
Adam Luecking
Results Leadership Group
www.resultsleadership.org
MEASURABLE RESULTS
FOR CLIENTS AND COMMUNITIES
FPSI/RLG
1
Agenda
• Results-Based Accountability:
Who, what, when, where, why & how?
• Questions and Answers
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Why RBA
To implement the 5 conditions of Collective Impact
• Common language
• Shared measurement systems
• Mutually reinforcing activities
• Continuous communication
• Backbone support organization
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Why RBA
To create Measurable Impact
Measurable
Impact
aka
Turning the
Curve
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Why RBA
To know you are making a difference
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Why RBA
To bring precision to your passion
How much did we do?
How well did we do it?
Is anyone better off?
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Where RBA?
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Who RBA?
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
How RBA?
• Introduction to core concepts
• Population Accountability
• Performance Accountability
• Turn the Curve Thinking
• Tie it all Together
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Two Key Principles for
Achieving Measurable
Impact
1. Starting with ends, working
backwards to means
2. Data-driven,
transparent Decision Making
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
RBA in a Nutshell
2-3-7
2 - Kinds of Accountability
• Population accountability
• Performance accountability
3 - Kinds of Performance Measures.
• How much did we do? • How well did we do it? • Is anyone better
off?
7 - Questions from ends to means in less than
an hour (aka. Turn the Curve Thinking)
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Results-Based Accountability
is made up of two parts:
Population Accountability
about the well-being of
WHOLE POPULATIONS
For Communities – Cities – Counties – States - Nations
Performance Accountability
about the well-being of
CLIENT POPULATIONS
For Programs – Agencies – and Service Systems
FPSI/RLG
MEASURABLE
RESULTS FOR CLIENTS AND COMMUNITIES16
PERFORMANCE
ACCOUNTABILITY
POPULATION
ACCOUNTABILITY
DEFINITIONS [
Language
Discipline
]
RESULT
A condition of well-being for children, adults, families or communities.
Children succeeding in school, Safe communities,
Clean environment
INDICATOR
A measure which helps quantify the achievement of a result.
Rate of high school graduation, Crime rate, Air quality
index
PERFORMANCE MEASURE
A measure of how well a program, agency or service system is working.
Three types: 1. How much did we do?
2. How well did we do it?
3. Is anyone better off?
FPSI/RLG
MEASURABLE
RESULTS FOR CLIENTS AND COMMUNITIES17
Why Distinguish Population from
Performance Accountability?
RESULT
&
INDICATORS
A STRATEGY
&
PERFORMANCE MEASURES
Customer
result
Service
delivery
ENDS
MEANS
Ends
Means
FPSI/RLG
MEASURABLE
RESULTS FOR CLIENTS AND COMMUNITIES18
END
Children Live in Safe
And Stable Homes
Indicator
1. Doing the
right things?
Comprehensive Strategy/Partners
MEANS
Program A
Interagency
System
Agency/Program
Performance Measures
2. Doing those things right?
Interagency Service System
Performance Measures
FPSI/RLG
MEASURABLE
RESULTS FOR CLIENTS AND COMMUNITIES19
Population
Accountability
FPSI/RLG
20
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Population
+ Geographic Area
+ Condition of Well Being
= Result
FPSI/RLG
21
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Populations?
• All people in Sydney
• All Australians
• All children prenatal to 5 years old
• People with developmental disabilities
• People that enjoy water sports
FPSI/RLG
22
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Geographic Area?
•
•
•
•
Australia
New South Wales
Sydney
Glebe
FPSI/RLG
23
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Conditions of
Well Being?
•
•
•
•
•
Healthy
Safe
Living with Dignity
Economically Secure
Succeeding in School
FPSI/RLG
24
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Selecting
Indicators
FPSI/RLG
25
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Criteria for
Selecting Indicators
Communication Power
Does the indicator communicate to a broad range of audiences?
Proxy Power
Does the indicator say something of central importance about the result?
Does the indicator bring along the data HERD?
Data Power
Quality data available on a timely basis.
FPSI/RLG
26
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Choosing Indicators
Worksheet
Result_______________________
Candidate Indicators
Measure 1
Measure 2
Measure 3
Measure 4
Measure 5
Measure 6
Measure 7
Measure 8
FPSI/RLG
Communication
Power
Proxy
Power
Data
Power
H M L
H M L
H M L
H
H
H
H
H
L
Data
Development
Agenda
27
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Performance
Accountability
FPSI/RLG
28
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Selecting
Performance
Measures
FPSI/RLG
29
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Quantity
Quality
How
much did
we do?
How well
did we do
it?
Effect
Effort
The Three Kinds of Program Performance Measures
Is anyone
better off?
#
FPSI/RLG
%
30
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Selecting Headline Performance Measures
How much did we do?
# Clients/customers served
# Activities
(by type of activity)
FPSI/RLG
MEASURABLE
RESULTS FOR CLIENTS AND COMMUNITIES31
Selecting Headline Performance Measures
How much did weHow
do?
well did we do it?
% Common measures
e.g. workload ratio, staff
turnover rate, % staff fully
trained, unit cost
% Activity-specific measures
e.g. % timely intakes, %
accreditation standards met
FPSI/RLG
32
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Selecting Headline Performance Measures
How much did we do?
How well did we do it?
Is anyone better off?
#/% Skills / Knowledge
(e.g. cognitive, social, physical)
#/% Attitude
(e.g. toward language, parenting)
#/% Behavior
(e.g. reading to child at home)
#/% Circumstances
(e.g. child care, transportation)
RLG
33
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Effort
Health Plan or Practice
Quantity
Quality
How much did we do?
How well did we do it?
Number of
patients
treated
Percent of
patients treated
in less than
1 hour
Effect
Is anyone better off?
FPSI/RLG
#
children
fully
immunized
%
children
fully
immunized
34
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Effort
Education
Quantity
Quality
How much did we do?
How well did we do it?
Number of
students
Student-teacher
ratio
Effect
Is anyone better off?
FPSI/RLG
Number of
high school
graduates
Percent of
high school
graduates
35
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Effort
Fire Department
Quantity
Quality
How much did we do?
How well did we do it?
Number of
responses
Response
Time
Effect
Is anyone better off?
Number of fire
related deaths
(in catchment)
FPSI/RLG
Rate of
fire deaths
per 100,000
36
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Effort
Not All Performance Measures Are Created Equal
Quantity
Quality
How much did we do?
How well did we do it?
Least
2nd Most
Important
Important
Effect
Is anyone better off?
FPSI/RLG
3rd Most
Important
Most
Important
37
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
The Matter of Control
Quality
Effort
Quantity
How much did we
do?
How well did we do
it?
Most
Control
Effect
Is anyone better off?
Least
Control
PARTNERSHIPS
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Turn the Curve
Thinking™
FPSI/RLG
39
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Turn-the-Curve Thinking™: Talk to Action
Result or Program:
How are
we doing?
Data
Baseline
Why?
Story behind the baseline
Help?
Partners (with a role to play in turning the curve)
Options?
What Works
Propose
to do?
Strategy (w/ Budget)
FPSI/RLG
40
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Turn-the-Curve Thinking™ Talk to Action
Result or Program:____________
How are
we doing?
FPSI/RLG
Data Baseline
41
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
The Matter of Baselines
OK?
Return* on
Investment
Turning the Curve
History
Forecast
Baselines have two parts: history and forecast
* The “ROI” is not financial, it is Results
FPSI/RLG
42
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Turn-the-Curve Thinking™: Talk to Action
Result or Program:_____________
Data
Baseline
Why?
FPSI/RLG
Story behind the baseline
Research Agenda
43
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Force Field Analysis
Factors Restricting?
Factors Contributing?
FPSI/RLG
44
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
The Story Behind the Baseline
 Root Causes (ask “Why?” five
times)
 Positive and negative
 Prioritize – which are the most
important to address to “turn the
curve” of the baseline?
 Research agenda?
FPSI/RLG
45
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Turn-the-Curve Thinking™ Talk to Action
Result or Program:_____________
Data
Baseline
Story behind the baseline
Help?
FPSI/RLG
Research Agenda
Partners (with a role to play in turning the curve)
46
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Partners
 Who are partners who may have
a role to play in turning the
curve?
 Does the story behind the curve
suggest any new partners?
FPSI/RLG
47
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Turn-the-Curve Thinking™: Talk to Action
Result or Program:______________
Data Baseline
Story behind the baseline
Research Agenda
Partners (with a role to play in turning the curve)
Options?
FPSI/RLG
What Works
Research Agenda
48
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
What Works
 Options for actions to “turn the
curve”?
 Research-based?
 Low-cost/no-cost?
 Off-the-wall ideas?
 Research agenda?
FPSI/RLG
49
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Turn-the-Curve Thinking™: Talk to Action
Result or Program: _______________
Data Baseline
Research Agenda
Story behind the baseline
Partners (with a role to play in turning the curve)
What Works
Research Agenda
Criteria: Leverage; Feasible; Specific; Values
Propose
to do?
FPSI/RLG
Strategy
50
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Action Plan
 Leverage: will turn the curve of the
baseline?
 Feasible (a.k.a. “reach”)?
 Specific: who, what, when, where, how?
 Consistent with values?
FPSI/RLG
51
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Population
and
Performance
Accountability
FPSI/RLG
52
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Population Accountability
Result: to which you contribute to most directly.
Indicators:
Results and
Indicators
Story:
Partners:
What would it take?:
Your Role: as part of a larger strategy.
Performance Accountability
Service Systems,
Agencies, Divisions,
Programs and
Performance
Measures
Program:
Performance measures:
Story:
Partners:
Action plan to get better:
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Population and Performance Accountability: Agency
its clients
“System”
with its clients
Community with a
whole population
FPSI/RLG
Agency
E.g., Interagency
Collaboration
State,
Region, City,
Neighborhood …
54
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
A Schematic for Results-Based
Budgeting and Grant Making
1.Community Result
2.Indicators
3.Story Behind the Baselines
4.Strategy
Programs
Program A
Program B
Program C
Local __________
Federal __________
Businesses _________
Civic __________
Non-profits _________
RLG/FPSI
Program A
Performance Measures
Story Behind the
Baselines
• Action Plan & Budget
55
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Management, Budgeting & Strategic Planning
Management
Budgeting
Monthly or quarterly
performance
assessment and action
planning using the
framework./steps.
Use the Performance
Report format for
budget hearings and
budget submissions to
present current
performance and what
will be done next year to
improve.
Use framework / steps
at all levels of
implementation in
the agency.
RLG/FPSI
Budget priorities
informed by the
Strategic Plan
Strategic
Planning
Population Level Results &
Indicators, comprehensive
strategy among and all
stakeholders
Agency Level
Each department’s role in
comprehensive strategy.
Agency’s multi-year
priorities.
56
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Management, Budgeting & Strategic Planning
Management
Budgeting
Monthly or quarterly
performance
assessment and
action planning using
the
framework./steps.
Use the Performance
Report format for
budget hearings and
budget submissions to
present current
performance and what
will be done next year to
improve.
Use framework /
steps at all levels of
implementation in
the agency.
RLG/FPSI
Budget priorities
informed by the
Strategic Plan
Strategic
Planning
Population Level Results &
Indicators, comprehensive
strategy among all
stakeholders
Agency Level
Agency’s role in
comprehensive strategy.
Agency’s multi-year
priorities.
57
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
Agency/Division/Program
AGENDA
1. New data
2. New story behind the curve
3. New partners
4. New information on what works.
5. Changes to action plan/ budget
6. Adjourn
FPSI/RLG
58
MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES
When RBA?
Now!