Transcript Impact Monitoring - Engineers Without Borders UK

Theory of Change,
Impact Monitoring, and
Most Significant Change
EWB-UK Away Weekend – March 23, 2013
Theory of Change
Theory of Change is a roadmap of change,
offering pictures of destinations and guiding
you on your journey so you know you’re on
the right path.
This can be done by mapping outcomes,
which in the long-term are known as
impact.
Impact
 Long-term outcomes – the actual
change you want to achieve
 Intended and unintended
consequences
 Can be positive or negative
Outcome Map
One way of laying out this pathway is to create a “so
that” chain:
You start with a powerful strategy, so that a shortterm outcome will occur, so that a long-term
outcome will happen, so that there will be an
impact.
I.e. Link strategies with outcomes, long-term goals,
and vision
ToC Outcome Map
Powerful Strategies
lead to Short-Term Outcomes
which lead to Long-Term Outcomes (Goals)
which lead to the desired Impact (Vision)
EWB-UK Theory of Change Map
Under the new strategy, everything connects
to everything!
Each Portfolio contributes in different ways to the same goals
Vision
First, you want to clarify your goals by identifying the
ultimate impact you aim to achieve – this is
generally an ambitious visionary statement:
“A world where everyone has access to the
engineering they need for a life free from
poverty”
Powerful Strategies
Strategies that are already in place that address the
ultimate impact:
“Create a sense of belonging where one can
identify with the cause, a sense of being
part of a movement”
Short-Term Outcomes
Example from EWB-UK ToC:
“Platform to bring motivated students
from across the UK together”
Long-Term Outcomes (Goals)
1.
Awaken greater attention to global challenges and opportunities
2.
Educate engineers about international development
3.
Excite and inform people about the role and impact of engineering
4.
Empower engineers to respond to global challenges
5.
Enable new paths of development that are appropriate, sustainable, and
inspirational
6.
Transform the engineering profession into an enabling environment for positive
change
7.
Relieve poverty in the communities where our international partners work
8.
Enhance the capabilities of people, communities, and partners
9.
Discover and evolve technologies and approaches that address barriers to
development
10. Unleash passionate, talented, and transformational leaders
Long-Term Outcomes (Goals)
4) Empower engineers to respond
to global challenges
Vision
“A world where everyone has
access to the engineering they
need for a life free from poverty”
Theory of Change
ToC is a strategic picture of the multiple
interventions required to produce the early
and intermediate outcomes that are
preconditions of reaching an ultimate goal
(impact).
How ToC and M&E feed in to:
Impact Evaluation
Monitoring and Evaluation
 Monitoring is:
 continuous collection of data throughout project
cycle
 Internal / for project management
 Evaluation is:
 Carried out at significant stages to assess
progress towards goals
 Can be a self-evaluation or external
Logic Model
Aim (Mission Statement)
The changes or differences that
your organization or project
plans to bring about
Objective
The areas of activity that an
organization or project plans to
bring about
Input
The resources that you put into a
project to enable you to deliver
services
Output
The products or services you
deliver as part of your work
Outcome
The changes, benefits, learning, or
other effects that happen as a
direct result of your work
Impact
The effect of your project at a
broader level, usually measured in
the longer-term
Understanding M&E for Impact
Monitoring: ToC vs. Logic Model
 A logic model is a tactical explanation of the
process of producing a given outcome. It outlines
the program inputs and activities, the outputs they
will produce, and the connections between those
outputs and the desired outcomes.
 The ToC summarizes work at a strategic level,
while a logic model would be used to illustrate the
program-level understanding of the change
process.
ToC and Logic Models
Once a precondition (or outcome) has
been identified through the ToC
process, a logic model can be used to
explain how that outcome will be
produced. Thus, one ToC could
actually be linked to a number of logic
models.
M&E Tools: Indicators
SMART
 Specific
 Measurable
 Achievable
 Relevant
 Time-bound
Most Significant Change
Most Significant Change (MSC)
technique is a means of “monitoring /
evaluating without indicators”
MSC is most useful:
Where it is not possible to
know what the outcome will
be
Where outcomes will vary
widely
What is MSC?
MSC is a form of participatory monitoring and
evaluation. It is participatory because many
project stakeholders are involved both in
deciding the sorts of changes to be recorded
and in analyzing the data collected.
How it links to M&E
 It is a form of monitoring because it occurs throughout
the program cycle and provides information to help
people manage the program.
 It contributes to evaluation because it provides data on
impact and outcomes that can be used to help assess
the performance of the program as a whole.
How it works
 Essentially, the process involves the collection of significant
change (SC) stories from the field level, and the systematic
selection of the most significant of these stories by panels of
designated stakeholders or staff. The designated staff and
stakeholders are initially involved by ‘searching’ for project
impact. Once changes have been captured, selected groups of
people sit down together, read the stories aloud and have
regular and often in-depth discussions about the value of
these reported changes, and which they think is most
significant of all. In large programs there may multiple levels
at which SC stories are pooled and then elected. When the
technique is implemented successfully, whole teams of people
begin to focus their attention on program impact.
Activity
 Split up into groups of 5
 Take turns briefly describing to the other members of
your group what the biggest impact of being a member
of the national executive has had on your life – positive
or negative
 Look for common themes and pick one story to
represent your group as a whole
 Each group shares their story