Transcript Logic Models

Logic Models for Evaluation:
The Search for Clarity
Gail Johnson
Spring 2008
logic models class.ppt
1
Program Evaluation
Common analytical technique in public
administration and non-profit programs
Purpose: to tell what happened during the
implementation of a program, tell how well a
program is working, and whether it is achieving
the desired results.
Two main types:


Formative: focus on process
Summative: focus on outcomes
logic models class.ppt
2
Problem
For too long, public agencies (and non-profits)
have focused on reporting budgets and activities.
Program managers: get enmeshed in fighting the
alligators that they forget that the purpose of the
organization is to drain the swamp.
Captured by the demands of the job, lose the big
picture
logic models class.ppt
3
Emergence of Logic Modeling
Logic modeling for evaluation grew out of
two other techniques:
 Systems Thinking
 Performance Measurement
logic models class.ppt
4
Systems Model
Environment
Inputs
Black
Box
logic models class.ppt
Outputs
5
Lessons Learned from Systems
Thinking
The importance of the environment—or context

You can have a great program for empowering women
through micro-lending, but if a civil war breaks out, it
chnages everything.
Complexity: Zoom-in and zoom-out:


Logic models within logic models
Challenge: to find the right focus
Non-linear nature: importance of feedback loops
logic models class.ppt
6
Performance Measures
GPRA, GMAP, etc
Move from focus on Process to
focus on Outcomes:
What changes because of the program?

Need to focus on indicators to measure
change

Search for indicators
logic models class.ppt
7
Thinking Differently: Working with
Logic Models
Models help us visualize a program in context
Models help us identify the relationships between
various components:
Models help us think more analytically about
cause and effect:

Describes the theory of change guiding the program
intervention
Models help us identify key assumptions that can
be tested
logic models class.ppt
8
Models: Cause and Effect
Did the program cause something to
happen?
Education
Employment
logic models class.ppt
9
SystemsTheory-based
Evaluation
Its focus is to understand the nature of
the problem and the relationships between
the problem, intervention and expected
outcome.
What is the theory?


Why do we believe that training people will
result in better evaluations?
Why do we believe that funding microbusinesses will reduce poverty?
If an assessment about the relevance is to
be made, the theory has to be examined.
logic models class.ppt
10
Theory and Causal Chains
Training in Evaluation
High Quality Evaluations
Training in Evaluation
High Quality Evaluations 
Useful Information
Training in Evaluation
High Quality Evaluations 
Useful Information 
Better Decisions
logic models class.ppt
11
Logic Model
Impact
Outcomes
Outputs
Activities
Inputs
logic models class.ppt
12
Implementation
Results
Logic Model:
•
Long-term, widespread
improvement in society
Outcomes
•
Intermediate effects of
outputs on clients
Outputs
•
Products and services
produced
•
Tasks personnel
undertake to transform
inputs to outputs
•
Financial, human, and
material resources
Goal
Impacts)
Activities
Inputs
logic models class.ppt
13
Logic Model:
Adult Literacy
Goal
(Impacts)
Outcomes
•
Higher income levels;
increase access to higher
skill jobs
•
Increased literacy skill; more
employment opportunities
Outputs
•
Number of adults completing
literacy courses
Activities
•
Literacy training courses
Inputs
•
Facilities, trainers, materials
logic models class.ppt
14
Logic Model
Outcomes/Impacts:





The changes to expect to see if your theory is correct
and the program has been implemented well
What would be different because of the program?
Observable and Measurable Indicators of Change
Outcomes: Initial, intermediate, and long-term
change
Impact: the big enchilada, long-term, related to
program goal

Reduce Poverty
logic models class.ppt
15
Logic Models
If we train the chronically unemployed, then
their quality of life will be improved, and
the overall poverty rate will be reduced.
What is the underlying theory?
What’s the model?
Unemployed  Training  Employment 
Increased Income  Reduced Poverty
logic models class.ppt
16
Logic Model: Training Program
Inputs
Resources
•money
•staff
•Supplies
•mentors
Activities
•Training
programs
•Dress for
success
coaching
•Interview
coaching
•Resume
assistance
Outputs
Products
•Number
of
graduates
per training
session
•%
graduate
rate
logic models class.ppt
Outcomes
Impacts
Benefits
changes
Goals
•Increased
skills
•% Obtain jobs
•% Obtain high
paying, quality
jobs
•Increased
self-esteem
•Increased
income
•Selfsufficiency
•Family
stability
•Reduction
in poverty
17
Exercise: Women’s Program



Women-owned micro-enterprises
Handout
Put in logical order:
 Sequence
of Inputs, Activities, Outputs and
Outcomes
logic models class.ppt
18
Logic Model: The Process
It brings in all the key players: stakeholders,
staff, clients/customers, political folks, and
researchers
It is a process that helps everyone get clear
about what it is they are trying to
accomplish
It is a living thing: changes

Handout: Compare Figure 6.4 and figure 6.8
logic models class.ppt
19
Complexity
Challenge: level of detail
Handling multiple theories of change
Logic models within logic models
Zoom-in or Zoom out: trying to get the right
focus

Accurate, clear but not overwhelming
logic models class.ppt
20
Logic ModelEvaluation
When done right, sets the focus for the
evaluation (Figure 6.8)

Focus on activities:


Did they do what they said the were going to do? If
not, what barriers were in the way?
Focus on outcomes:
5 critical outcomes
 Do all, some, time-sequence?

logic models class.ppt
21
Exercise
Familiar Program
Goals?
 Objectives?
 Activities?
 Underlying theory

logic models class.ppt
22
Applying the Program Outcome Model
Inputs
Activities
Outputs
logic models class.ppt
Outcomes
Impact
23
Challenges in Developing Logic
Model
Outputs and activities are seen as outcomes
Program managers focus on actions
 Visible and more within program managers’
control
 Easier to confirm task completion than measure
an outcome
 Measures are not easy to come by


Not all the can be counted really matters and all that
matters can not necessarily be counted
logic models class.ppt
24
Logic Model: No Panacea
Can’t fix a situation where there is no real
theory of change
Still need a rigorous research design
Other alternatives might work better

Stuckness
No intervention exists in isolation

Larger events may be uncontrollable.
logic models class.ppt
25
Summary of
Program Outcome Model
Identify components of a program
Identify relationships between the program:
expected and actual
Identify what we want to measure
Focuses our attention on outcomes and
impacts
logic models class.ppt
26
Resources
Joy A. Frechtling, Logic Modeling Methods
in Program Evaluation, Wiley and sons,
2007.
Kellogg Foundation: Logic Model Guide
http://www.wkkf.org/Pubs/Tools/Evaluation/Pub3669.pdf
CCD: Logic Model Resources
http://www.cdc.gov/eval/resources.htm#logic%20model
logic models class.ppt
27