Transcript Writing Result Measures and Program Evaluation

Writing Result Measures and
Program Evaluation
Kathy Keeley
Northland Foundation
Strengthening Communities
April 14, 2011
Agenda
 Welcome
 Review Technology
 Overview and Definitions
 Writing Result Measures and Examples
 Program Evaluation
 Definitions and Purpose
 Methods
 Using with Results Methods
 Webinar Feedback – Please complete
SurveyMonkey form
2
The Keeley Group
4/14/2011
Technology
 Mute & Unmute Phone *5
We ask that everyone leave their phone on mute until question
and answer time to avoid outside noises.
 Asking Questions
Please use the chat to the right of your screen for any questions.
We hope to have time to respond.
 If Cut Off – what to do
If you have technical difficulties we have loaded this PowerPoint
on our website that you can follow along with.
www.northlandfdn.org/Grants/NonprofitResources.shtml
3
The Keeley Group
4/14/2011
Writing Result Measures
4
The Keeley Group
4/14/2011
Results /Outcomes
 Results are also called Outcomes.
 Impact – Benefit to the client.
 Beyond the numbers who attend your program
or services.
 Answers “what is different because of what you
do?” NOT “what you do
5
The Keeley Group
4/14/2011
Not A Strategy
Outcome is:
 Healthy Babies
 Youth Ready for
College
Strategy is:
 Train mothers in
illness prevention
model
 Educational Reform
in High Schools
 Sense of connection
to social issues in our
community
6
The Keeley Group
 Rewrite all plays to
focus on current
social issues in the
city
4/14/2011
Not Outputs
 Not how many people served
 What is the benefit to the client because they
got help from your organization?
7
The Keeley Group
4/14/2011
Why Important?
 Mission
 Right programs and services
 Demonstrate a return on investment
 Consistent message for donors
8
The Keeley Group
4/14/2011
Definition of Result Measure
 Outcome – Benefit from what you do
 Indicator – Measure of your impact on the
outcome
 Target – Number you are trying to achieve
Example
Public Safety (Outcome)
50% (target) emergency calls will be responded
to within 3 minutes (indicator)
9
The Keeley Group
4/14/2011
Steps to Create
1. Who is the customer?
Decide who you intend to benefit from what you do?
2. How do they benefit?
Identify exactly how you want them to benefit
because of what you do--Outcomes. Examples:
skills, behavior, knowledge, circumstances, attitudes,
enjoyment……
3. Select measurable indicators for each
Outcome.
Review for ability to communicate your story and ease of
data collection
4. Decide 3-5 year targets for each indicator.
10
The Keeley Group
4/14/2011
Results Must
 Communicate Story – Be compelling
 Be affordable to measure
 Have a baseline of data
11
The Keeley Group
4/14/2011
Examples
Healthy Children
80% of children will be immunized before kindergarten.
Employed Individuals with Living Wage
50% of program graduates will obtain a job that pays at
least $12 an hour.
Audience Participation
50% of audience will participate in audience talk back
sessions.
New Audience Development
25% of audiences will be composed of young
professionals
Eliminating Homelessness
50% of homeless individuals will obtain permanent housing
Outcome
12
The Keeley Group
Target
Indicator
4/14/2011
Results - Outcomes
 What to Measure
 Feelings – not very effective
 Learning - Increased knowledge
 Increased skills – new skills to apply
 Effectiveness – change in life circumstances
13
The Keeley Group
4/14/2011
Examples
Improving Parenting Skills
 50% of families receiving case management
achieve goals established in plan.
Children Reading at/or above
 60% of children improve reading as measured by X
14
The Keeley Group
4/14/2011
Program Evaluation
15
The Keeley Group
4/14/2011
Program Evaluation
 Making Decisions
 Funding –Accountable, Cuts, New Funding
 Improve Program
 Respond customer satisfaction
 Measure against Strategic Plan
 Board and Management Dashboard
16
The Keeley Group
4/14/2011
Before
What do you want to know – really know?
2. Who is the audience?
3. What kinds of information is needed?
1.
 Strengths and Weaknesses
 Outcomes vs. Outputs – what happened
 Client experience – customer satisfaction
4. How will you collect?
5. 5. Timeline
6. 6. What can you afford?
4.
17
The Keeley Group
4/14/2011
Types of Evaluation
 Goals – Client, Program, Funding
 Surveys (follow-up)
 Interviews
 Documentation Review
 Outcomes – what has been achieved
 Survey
 Database Records
 Sample Interviews
 Process – how program works
 Focus Groups
 Interviews
18
The Keeley Group
4/14/2011
Data Base Resources
 http://www.communitytech.net/
 Vista Share http://www.vistashare.com/Corporate/Home/
 Sales Forces – www.saleforce.com
 Filemaker Pro
 Access
 Excel Spreadsheet
 Trade Association in Sector
 http://www.socialsolutions.com/
19
The Keeley Group
4/14/2011
Resources
 Program Managers Guide
 Methods Review
 PowerPoint Slides
 www.northlandfdn.org/Grants/NonprofitResour
ces.shtml
20
The Keeley Group
4/14/2011
Wrap-up
Please complete the SurveyMonkey webinar
feedback form. You will be receiving an email
shortly.
 Feedback form
link: www.surveymonkey.com/s/HB2788G
21
The Keeley Group
4/14/2011