Transcript Slide 1

Introduction to
Outcome Mapping
Ziad Moussa, Lebanon
Jan Van Ongevalle, Belgium
Heidi Schaefer, Canada
22 September 2014 – Dar Es Salaam – OM Lab
outcomemapping.ca
Workshop Objectives
Clarify what Outcome Mapping is and does
Introduce and apply Outcome Mapping
concepts and tools
Consider if and how Outcome Mapping could
be useful in our work
Agenda Overview
• Key concepts
• Vision
• Mission
• Boundary
Partners
• Progress
Markers
• Strategy Map
• Organisational
Practices
• M&E planning
and journals
Key concepts in
Outcome Mapping
Acknowledgements
This presentation makes use of various materials
that were shared by members of the global OM
community. Without being exhaustive, special
thanks goes to Terry Smutylo, Steff Deprez, Jan
Van Ongevalle, Robert Chipimbi, Daniel Roduner,
Kaia Ambrose and many others.
Source: A guide for project M&E: IFAD
Conventional thinking…
Time
Source: Ricardo Wilson-Grau (inspired by Jeff Conklin)
… clashes with relationships of cause and
effect that are unknown
Time
Source: Ricardo Wilson-Grau (inspired by Jeff Conklin)
Social change can be…
• Complex:
• Unstable:
involve a confluence of actors and factors
independent of project duration
• Non-linear:
• Two-way:
unexpected, emergent, discontinuous
intervention may change
• Beyond control:
but subject to influence
• Incremental, cumulative:
watersheds & tipping points
Source: Terry Smutylo
Time
Source: Ricardo Wilson-Grau
Evaluation challenges
1. Establishing cause & effect in open systems
2. Measuring what did not happen
3. Reporting on emerging objectives
4. Justify continuing “successful” interventions
5. Timing – when to evaluate
6. Encouraging iterative learning among partners
7. Clarifying values
8. Working in ‘insecure’ situations
Source: Terry Smutylo
Brief definition of OM
• A participatory method for
planning, monitoring and
evaluation
• Focused on changes in
behaviour of those with
whom the project or
program works
• Oriented towards social &
organizational learning
OM’s answer
Start from observable
behaviour change
Embrace different
perspectives
Recognise that all
interventions have
limited influence
Support people to build
their own well-being
Enable interventions to
adapt as they engage
Apply a systems
understanding
Source: Terry Smutylo / OM Lab 2012
“Being attentive along he journey is as
important as, and critical to, ariving at the
destination ”
Michael Quinn Patton
Three key concepts in OM:
1. Sphere of influence
2. Boundary Partners
3. Outcomes understood as
changes in behaviour
There is a limit to our influence
Project
Sphere of
control
People the
project works
with/through
Sphere of
influence
People who
benefit from
the project
Sphere of concern
There is a limit to our influence
Inputs,
activities,
outputs
Sphere of
control
Outcomes:
Changes in
behavior
Sphere of
influence
Impact:
Changes in
state
Sphere of concern
Farmers
participate
in field trials
Participating
farmers learn how to
use drip irrigation
equipment
Participatory
research on
demonstration
farms
to develop
approaches
to drip irrigation
Extension workers
visit demonstration
farms
Increased
knowledge of
techniques
Farmers adopting
drip irrigation
methods
Training of
extension
workers
Publication of
performance of
different setups
Reduced
numbers of
new wells
Greater quantities
of groundwater
available
Extension workers
promoting drip
irrigation
Source: Terry Smutylo
Who are your boundary partners?
Programme
Beneficiaries
Stakeholders
Boundary Partners
Focus of Outcome Mapping
Inputs
Activities
Outputs
Outcomes
Outcome Mapping
Impacts
Where is the map?
✓OM is a guide to the journey we take
with our partners. We co-create the map.
✓It focuses on the intention and what
happens along the way
✓The map is not the territory, it shows
the route taken
✓“The only real voyage of discovery
exists, not in seeing new landscapes, but
in having new eyes” (Marcel Proust)
Step 1: Vision
improved human, social, & environmental wellbeing
I have a dream!
Martin Luther King, Jr.
August 28, 1963
A vision statement..
•
Guides
• Motivates and inspires
• Is an ‘accountability-free zone’
Vision in graphic form, Nagaland (India)
37
Vision statement in narrative form
Local authorities, communities, and international organizations in developing
countries in Africa recognize the value of HIV/AIDS intervention as an integral part of
social & economic development. Municipal, regional, and national governments
actively support HIV/AIDS prevention activities by formulating and implementing
effective public health policies. Using research findings, they have developed a
comprehensive public health strategy to slow down the infection rate. Formerly
marginalized groups (e.g. women and youth) are organized into advocacy groups that
can effectively formulate their needs to policy makers. All groups have access to
reliable and relevant technical information about HIV/AIDS prevention and are able to
make informed choices. In essence, there are healthier, happier, and wealthier
communities.
Vision facilitation question
Imagine that, 5-10 years from now, the program has been
extremely successful. Things have improved beyond your most
ambitious dreams.
• What changes have occurred?
• What (& how) are your intended beneficiaries
doing?
• What are your partners doing?
• Describe the better world you are seeking.
Step 2: Mission
The mission is that “bite” of the vision
statement on which the program is
going to focus.
A mission statement describes:
• How the program intends to apply its resources
in support of the vision
• The areas in which the program intends to work
• How the program will support the achievement
of outcomes by its direct partners
Example Mission Statement
In support of this vision and on behalf of its donors, the program will work in the areas of
research, dissemination, capacity building, & coordination. It will contribute to the production,
synthesis, & dissemination of research data, position papers, & other information that will
sensitize local & international actors to HIV/AIDS prevention. The program will seek to expand the
range of disciplines involved in HIV/AIDS research. It will enhance HIV/AIDS research capacity in
order to produce credible information for local, national, & international policy-making &
program development. It will promote an interest in HIV/AIDS research among new researchers
by providing research fellowships, mentorship, & training opportunities. The program will
contribute to the development of linkages between Northern & Southern researchers &
encourage partnerships between research organizations, advocates, & decision makers. It will
increase its visibility & credibility among the donor community & will convince them of the utility
of supporting HIV/AIDS prevention.
Summary
Vision
Mission
✓ About the future
✓
Feasible
✓
Identifies activities and
relationships
✓
About the program
✓ Observable
✓ Idealistic
✓ Not about the program
Exercise :
Correcting Vision and
Mission Statements
1.
Across rural India, women and girls are contributing to and sharing
equitably in the benefits of development.
2.
They utilize and benefit from appropriate health care, education,
food and water security.
3.
They are free from violence in the home and in the community.
4.
The project seeks to improve their well being through a wide range
of interventions.
5.
Women are able to earn their own livelihoods, accessing freely the
markets, credit, banking and municipal services they need to
pursue their economic goals.
6.
Villages are fully served by public transport, are well lit at night and
have police forces that enforce all laws fully and equitably.
7.
The consortium of donors responds favorably to the project’s
progress reports and decides to continue funding for a second
phase
1.
The Swayamsiddha Project works with government, nongovernment and community-based organizations to strengthen
their individual and combined efforts towards improving women’s
health and empowerment.
2.
The development of women’s self help groups is facilitated by the
project, providing women with moral, material and educational
support in identifying and taking action to make community and
government services more responsive to their health and livelihood
needs.
3.
Primary and reproductive health care at the local level is directly
accountable for, and responsive to, the needs of women and girls.
4.
Enrolment of girls in primary schools increases by 35% as a result
of project interventions
5.
Research is undertaken in the application of ecosystem
approaches to human health in agriculture and in the provision of
health and sanitation services.
Step 3:
Boundary Partners
Step 3: Boundary partners
Outcome mapping focuses on outcomes
as changes in behaviour or relationships
of those with whom the program or
project is working directly (= boundary
partners)
Boundary Partners are...
Those individuals, groups, & organizations
with whom a program:
• interacts directly to encourage change as a
contribution to the vision.
• can anticipate some opportunities for
influence
• engages in mutual learning
Who are your boundary partners?
Programme
Beneficiaries
Stakeholders
Boundary Partners
Strategic
partners
Families
PHCs
Banks
SHG
Police
State
NGO
Boundary partners have their own
boundary partners - Often many
actors and results at different levels
Community
Leaders
State
NGO
State
NGO
State
NGO
State
NGO
State
NGO
BAIF
IDRC
CIDA
Swayamsiddha
Facilitation questions
• In which individuals, groups, or organizations is
your program trying to encourage change as a
contribution to the vision?
• With whom will you work directly?
• Are you choosing boundary partners because
you want to influence the ways they help or
influence others?
• On whose actions does your success depend?
Strategic partners
• Selected for their potential to contribute to the
mission
• A person or group with whom the program
works directly to achieve the mission, without
necessarily wanting to change the partner’s
behaviour as part of the mission
• E.g. Donor agency, contracted service providers,
NGOs doing similar work, media agencies
Facilitation questions
• In which individuals, groups, or organizations is
your program trying to encourage change as a
contribution to the vision?
• With whom will you work directly?
• Are you choosing boundary partners because
you want to influence the ways they help or
influence others?
• On whose actions does your success depend?
Screening of boundary partners
A program has normally not more than 4 or 5 types of boundary partners
• What if too many BP’s? What if you are not sure whether it is
really a BP?
– Some boundary partners might be boundary partners of boundary
partners!
– Discuss how the boundary partners are possibly contributing to the
vision
– Can we realistically influence the boundary partners?
– Where will we put most efforts and resources?
– Sometimes similar actors can be clustered as one type of boundary
partners
– Who are your ultimate beneficiaries / target group? There are not your
boundary partners
– Maybe some partners are actually strategic partners?
Exercise :
Boundary partners
Boundary partners: exercise
• Read the handout with the case of the VVOB
teacher education and vulnerability programme
• On the basis of the case of the VVOB
programme (see handout), try to position the
various actors in (or out) of the various
influence spheres on the flip chart provided.
• Solution: see handout!
20-7-2015
59
Developing an actor focused TOC (case of VVOB
Zimbabwe)
Local resource
persons
Boundary partners =
Direct target groups
College admins
Unicef
Staff devpt
committees
VVOB
support
team
DGD
PLAN
International
Sphere of
control
Inputs, activities,
outputs
Indirect target groups
College
Lecturers
Pupils
Earch Childhood
depts
Student peer
educator clubs
College
students
Student support
structures
Sphere of
influence
Outcomes
Sphere of interest
Impact
Final
beneficiaries
Step 4:
Outcome Challenge
Outcome Challenge
• Describes behaviour of a single boundary
partner
• Sets out the ideal actions, relationships
activities
• Describes the boundary partner’s contribution
to the vision
Facilitation questions
• Ideally, in order to contribute to the vision,
what would the boundary partner be doing?
• With whom would they be interacting?
• How could this boundary partner contribute
maximally to the vision?
Women’s self help groups are taking action to make community and
government services more responsive to the health and livelihood needs of
women and girls.
They influence banks, police, health and social service providers, local
officials and state and national government agencies in relationships of
mutual respect and joint action to improve women’s well being.
Women’s self help groups arrange bank loans for members and for life skills
training for girls to be included in the school curriculum.
They influence local, state and national government policies and
expenditures on community improvement and transportation and support
women candidates to run for election to local government office.
Step 5:
Progress Markers
Progress Markers
Love to see
(Deep transformation)
Like to see
(Active engagement)
Expect to see
(Early positive responses)
Love to
see
Like to see
Like to see
Like to see
Like to
see
Like to
see
Expect to
see
Expect to
see
Like to
see
Expect to
see
Why Graduated Progress
Markers?
• Articulate the complexity of the change process
• Allow negotiation of expectations between the program and its
partners
• Permit early assessment of progress
• Encourage the program to seek the most profound
transformation possible
• Help identify mid-course improvements
How can we measure:
Greater awareness…
Empowered women…
Community ownership…
Reduced conflict…
Increased collaboration…
Governmental commitment…
Gender sensitivity…
Equal access…
Budgetary transparency…
Active participation…
Poverty alleviation…
Strengthened capacity…
?
Examples
• Project expects to see Tanzania Forest Service
– Consult with communities on the new forest policy
and the national REDD strategy;
– Develop management plans for all Nature Reserves;
– Continue a policy that is supportive of participatory
forest management.
Examples
• Project would like to see Tanzania Forest Service
– respond promptly to information from communities about illegal
activities and other forest management issues and take action to
resolve the issue.
– incorporate recommendations from communities and civil society
organistions in the revised forest policy and the national REDD
strategy;
– approve voluntary social and environmental timber standards;
– implement and monitor management plans for nature reserves
and other high biodiversity forest reserves with the participation
of communities;
– approve independent forest monitoring and cooperate fully in its
implementation;
– incorporated Biodiversity indicators in NAFORMA
Examples
• Project would love to see Tanzania Forest Service
– successfully advocate for wider application of good
timber standards throughout Tz gov’t systems.
– consistently and effectively apply the Forest Act and
regulations within the timber and charcoal trade
thereby increasing the revenue that they capture and
reducing illegal timber harvesting and charcoal
production;
– incorporated social and environmental standards in
revised harvesting regulations and other documents
related to the national forest programme;
– Applying good social and environmental practices in
respect of reserve management
Facilitation questions
• How can the programme know the boundary
partner is moving toward the outcome?
• What would they be doing?
• What milestones would be reached as the
boundary partner moves towards their
intended role in contributing to the vision?
Progress Marker Checklist
Each Progress Marker:
 Describes a changed behaviour by the boundary
partner
 Can be monitored & observed
As a set, Progress Markers:
 Are graduated from preliminary to more profound
changes in behaviour
 Describe the change process of a single boundary
partner
Exercise :
Arranging Progress
Markers
Step 6:
Strategy Maps
Strategy Maps
For each Boundary partner:
• What will the Project do to SUPPORT desired changes in the
Partner towards the Outcome Challenge?
• Diverse range (an array) of ACTIVITIES:
- Aimed at Partner
- Aimed at Partner’s environment
- Cause, Persuade, Support
Strategy Map
causal
Partner
Environment
persuasive supportive
6 types of support strategies
I
aimed at
individual
boundary
partner
E
aimed at
boundary
partner’s
environment
causal
persuasive
direct
Influence
arouse new
thinking;
build skills,
capacity
(e.g. funding, prepare
a report, …)
alter the
physical or
regulatory
environment
(e.g. technical
transfer, policy
change, internet
access, terms of
reference, rules,
guidelines)
supportive
on-going
support
supporter/mentor who
guides change over
time,involvement is more
frequent and sustained,
nurturing for self(e.g. skill
sufficiency
enhancement,
(e.g. program member
methodological
who provides regular
workshops, training)
guidance and input,
coaching, expert, …)
broad
create /
information
strengthen
dissemination/ peer networks
access to new partners working
together and
info
(e.g. Radio, TV,
website, publications,
conferences,
workshops)
collectively
supporting each
other)
(e.g. research
network, …)
Example strategy map: for extension agents

Causal
Provide research
and training funds
Directed to the
boundary
partner



•
Directed to the
environment of
the boundary
partner
Building
model
storage facility for
grain
and
backstopping


Persuasive
Run technical training workshops
for extension agencies and
activities (e.g. ISFM)
Coordinate training on farmer
participatory action research
approaches to extension agencies,
service providers and research
teams
Conduct training on protocols
development and project monitoring
Develop extension materials and
policy brief on adaptation to climate
change
Facilitate establishment of
exchange forums for sharing of
information, knowledge and
experience among partners
80




Supportive
Contract a farmer-training
specialist to work with
extension on farmer
mobilization, action research,
data collection and
documentation
Conduct quarterly monitoring
and review workshops
Establish a network of
extension agents and
researchers that facilitate
knowledge and information
sharing
Engage researchers and
policy makers as conveners
for discussions on climate
change impact and alternative
measures
Exercise :
Strategy Map
Exercise on strategy maps
Instructions
• Your table will be given a number of cards with activities written on them.
Discuss which type of strategy each of the activities represents. The Outcome
Challenge has been provided so that you are able to think about what the
program is contributing to.
• Select someone from your table to plot the activities in the appropriate section
of the strategy map on the wall. Be prepared to explain your reasoning for
plotting the different strategies as I-1, I-2, I-3, E-1, E-2, or E-3.
• !Remember, a strategy map describes what the program will do to try to
encourage change(s) in a boundary partner. It breaks down depending on
whether they are aimed at the individual boundary partner or the
environment and whether they are causal, persuasive, or supportive.
Time: 30 minutes
vision
mission
Banks
Women’s Self Help
Groups
Girls & Women
Police
BAIF
State NGOs
Families
Community Leaders
Public Health Clinics
Strategies
Strategic
Partners
Boundary
Partners
Project’s
Outcomes
BP’s outcomes
Step 7:
Organisational
Practices
Organizational Practices
How does your team or organization stay
relevant, viable and effective?
8 practices
1. Prospecting for new ideas, opportunities, and
resources
2. Seeking feedback from key informants
3. Obtaining the support of your next highest
power
4. Assessing and (re)designing products, services,
systems, and procedures
8 practices
5. Checking up on those already served to add
value
6. Sharing your best wisdom with the world
7. Experimenting to remain innovative
8. Engaging in organizational reflection
M&E planning and
journals
5 kinds of M&E Information
Contextual InformatIon
Program
Partner
State, status
or situational
data
Strategies
outcomes
relevance & viability
(behaviour changes in the
(actions of the program)
partners)
implementation
(interventions by the program)
M&E Planning Worksheet
What
How will it be
information will collected, from
be collected?
what sources and
when?
Who will
collect it?
Who will make
sense of the info
& how?
Who will
manage the
collection-tousage
processes?
What
resources are
needed &
who
contributes?
Outcome journal
•
•
•
•
•
What, when, who, how much
Contributing factors
Sources of evidence
Unanticipated change
Lessons, required changes, action
Strategy journal
•
•
•
•
•
What did you do, with whom, when
(How) did it influence change
Outputs
Suggested changes/follow up
Lessons
Performance journal
• Progress against internal commitments – e.g.
organisational practices
• Reviewing the logic of the programme
Journal for Monitoring Outcomes
Outcome Monitoring Journal
Period of monitoring and data collection:
Contributors to Monitoring Update:
Name of the person(s) who compiled the journal:
Outcome Challenge:
Based on your analysis of each progress marker, please indicate the number on the scale that best represents the boundary partner's progress in
achieving the progress marker (1 indicates no activity towards the progress marker while 7 represents full achievement of the progress marker)
Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted
Progress markers
What happened (who, how, Date Strategy Used and its
Follow up/corrective measures
what, where)
(when) Efectiveness
1)
Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted
2)
Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted
3)
Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted
4)
Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted
5)
Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted
Evidence
(documents
etc.)
Outcome Journal
Name of the boundary partner
Work dating from/to:
Name(s) of the person(s) who compiled the journal:
Outcome Challenge:.
Progress Markers
Remarkable facts, what
happened
Follow up / corrective
measures
Narrative of a success story:
Unanticipated changes :
Which support strategies from the ICCO alliance where helpful or require further follow-up or action during the next
term?
Contributing or limiting factors and actors towards achievement of progress markers: :
Summary of lessons learned/recommendations :
Thank you!
For more info on OM:
• Visit the OM learning
community:
www.outcomemapping.ca
• Contact:
Simon Hearn
[email protected]
Sharing information to improve evaluation
outcomemapping.ca