Transcript Complexity and Evaluation in International Networks A

Hands-on Outcome
Mapping
GAN-Net Impact Community of Practice
Meeting
The Hague, Netherlands
26-27 August to 2008
Goele Scheers and Ricardo Wilson-Grau
Pre-Exercise
As we wait for people to settle down, please
draw up a list of the internal and external
stakeholders in your work.
Be as specific as possible in identifying the
actors internally or externally who have a
vested interest in the work you do.
Thanks to Enrique Mendizabal
Goele Scheers and Ricardo Wilson-Grau
Purpose
Stimulate discussion amongst GANs, donors
and consultants on the potential of Outcome
Mapping as a planning, monitoring and
evaluation methodology for GANs.
Goele Scheers and Ricardo Wilson-Grau
Intended results
Greater clarity about the strengths and
weaknesses of Outcome Mapping in the light of:
1.
GANs need for a practical planning tool,
performance assessment and impact
measurement
2.
Donors’ reporting and accountability
requirements
3.
Consultants’ interest and ability to use
outcome mapping
Goele Scheers and Ricardo Wilson-Grau
Activities
As a means of sharing with your our
experiences in using Outcome Mapping in
planning, monitoring and evaluating GANs (and
other types of organisations), we will lead you
through three quick exercises, each one
preceded by a brief conceptual introduction.
After the exercises, we will wrap up with a
discussion.
Goele Scheers and Ricardo Wilson-Grau
Goele Scheers and Ricardo Wilson-Grau
Outcome Mapping
A methodology developed by the Evaluation
Unit at the Canadian International
Development Research Center (IDRC)
IDRC grants annually over US$100 million
to research organisations around the world.
They developed Outcome Mapping between
1998 and 2000 as a response to three
difficulties they faced.
Goele Scheers and Ricardo Wilson-Grau
Understand
the impact of
their grantees
Goele Scheers and Ricardo Wilson-Grau
Measure the unmeasurable
Goele Scheers and Ricardo Wilson-Grau
Foster local ownership
Goele Scheers and Ricardo Wilson-Grau
As the solution
Planning: Intentionally design your strategy in
relation to the broader development context
but aiming at achievements within your
sphere of influence: the social actors who are
the key protagonists of development ,
“boundary partners”.
Monitoring: Monitor for “outcomes”: the
changes in boundary partners because you
will only have impact if they change what they
do and how they do it. Also includes
appreciating the significance of the changes
and how you influenced them.
Goele Scheers and Ricardo Wilson-Grau
As the solution
Evaluation: Understand the impact of the
changes in the behaviour, relationships and
actions of boundary partners – in their
policies and practices – and how they
contribute to improving people’s lives or the
state of the environment.
Goele Scheers and Ricardo Wilson-Grau
Exercise Boundary Partners
Who do you aim to support and influence?
Those individuals, groups, and organizations –
from civil society, government or business –
with whom you:
 Interact directly to effect change
 Anticipate opportunities to support or
influence them
 Engage in mutual learning
Goele Scheers and Ricardo Wilson-Grau
Strategic
partners
IMPACT
Boundary
Partners
OUTCOMES
OUTPUTS
You
Strategic
partners
RESOURCES
SPHERE OF
CONTROL
Ultimate
beneficiaries
SPHERE OF
INFLUENCE
SPHERE OF
CONCERN
Thanks to Paul Crawford
Goele Scheers and Ricardo Wilson-Grau
Exercise #1
Select from your list of stakeholders those
you consider to be “boundary partners”.
In pairs with the person sitting next to you:
– Explain to each other why you consider the
individuals, groups or organisations you have
selected are your boundary partners
– Explain why you consider that the other
stakeholders are not boundary partners
In plenary we will discuss a couple of
examples.
Goele Scheers and Ricardo Wilson-Grau
Boundary partners in summary
The essence of the Outcome Mapping PME
methodology is to focus on changes in
these social actors who are protagonists of
impact and are beyond the control but
within the influence of your organisation.
You distinguish boundary partners from all
sorts of strategic partners – individual and
organisational allies – with whom you work.
You do not control strategic partners and do
not intend to influence them to change.
Some actors, however, may be both.
Goele Scheers and Ricardo Wilson-Grau
Exercise on
concept of “outcome”
Outcomes are defined as observable and
significant changes in a boundary partner.
The changes can be internal or external to
your organisation.
Goele Scheers and Ricardo Wilson-Grau
Internal outcome
Observable change (in the behaviour,
relationships, activities, actions) in a
boundary partner that is in line with the
impact envisioned in your vision and
mission.
Internal outcomes are not changes that
represent doing more or better. They are
qualitatively different; they are
developmental.
Goele Scheers and Ricardo Wilson-Grau
Example from GWP
GWP-SEA Steering Committee and
prominent Singapore social actors agreed
to form GWP-Singapore. The active
participation of Singapore not only makes
the geographical coverage of Southeast
Asia almost complete but significanly
strengthens the partnership, in terms of
access to knowledge on water resources
management.
Who?
Did what?
Where? When?
Goele Scheers and Ricardo Wilson-Grau
External outcome
Observable change (in patterns of behaviour,
relationships, activities, actions) in a
boundary partner outside of your
organisation that is in line with the impact
envisioned in your vision and mission.
Often these changes are in the boundary
partner’s policies or practices.
Goele Scheers and Ricardo Wilson-Grau
Example from GWP
At the beginning of 2008, the National Water
Resources Board of the Philippines agreed
to discuss and develop new policies for light
handed regulation for small water providers,
cooperatives and community-based
associations.
The policies include new arrangements for
regulating small water providers, specifically
relating to registration and tariff-setting.
Goele Scheers and Ricardo Wilson-Grau
Exercise #2
Individually, select one boundary partner
from the list you made at the beginning of
the session and in a sentence or two
describe an outcome you have influenced in
the recent past.
The change must be observable and
verifiable and can be in their behaviour,
relationships, policies, practices or other
actions or activities.
Goele Scheers and Ricardo Wilson-Grau
This time, with the person sitting on your
other side, read and comment each other’s
outcomes. Do they describe who changed,
what they changed when and where?
We will then have questions and answers
and a brief summary.
Goele Scheers and Ricardo Wilson-Grau
Outcomes in summary
Outcome Mapping offers the possibility to
assess the results of the interaction between
the network members
Stimulates the learning process within a
network and is highly participatory.
OM is very useful for networks, because it is
a systemic, non-linear approach and
because it is a very flexible tool, that takes
into account unexpected results
Goele Scheers and Ricardo Wilson-Grau
Exercise on
Outcome Challenge
A description of the observable changes in a
boundary partner if you are extremely
successful. The “challenge” is for you to
influence, support, facilitate or in some other
way assist these optimal changes.
What changes are being sought in each one
of your boundary partners?
Goele Scheers and Ricardo Wilson-Grau
Outcome challenge example: GPPAC Network
Members will be implementing activities in
Peace and Conflict Resolution Education
(PCRE) (both formal and informal education)
that will bridge the gap between practice and
theoretical research.
They will also be undertaking PCRE using the
resources and lessons learned through the
Knowledge Generation and Sharing
programme. They will be making the case for
PCRE by demonstrating that PCRE leads to
attitude changes that are essential for the
establishment of a culture of peace.
Goele Scheers and Ricardo Wilson-Grau
An outcome challenge
incorporates multiple
changes within a single
statement.
An outcome challenge
is ambitious. It
describes the deepest
possible transformation
but without losing sight
of the reality of the
context and what can be
achieved in the coming
five years.
Change
A
Change
B
Outcome
Challenge
Change
C
Change
D
Goele Scheers and Ricardo Wilson-Grau
Exercise #3
Individually, select a boundary partner and
formulate in two or three sentences the
observable changes in that social actor’s
behaviour, relationships, activities, or
actions that potentially could be achieved if
you are extremely successful in influencing,
supporting, facilitating or in some other way
assisting the actor over the next 3-5 years.
Goele Scheers and Ricardo Wilson-Grau
Exercise #3… continued
Keep in mind that outcome challenges are
phrased so that they capture what the social
actor would be doing differently if you
achieve your full potential as a facilitator of
change.
In pairs, discuss each other’s outcome
challenges.
Are you both satisfied that there is a solid
balance between realistic and optimistic?
Goele Scheers and Ricardo Wilson-Grau
Outcome challenges summary
Impact – long-term, sustainable improvement
in the conditions of human beings and the
conservation of the environment – is the
result of processes of changes in multiple
social actors intentionally or unintentionally,
directly or indirectly, in a minor or major way
reinforcing each other.
Outcome challenges permit you to visualise
how to influence a processes of change.
Goele Scheers and Ricardo Wilson-Grau
In our experience, we have found that
Outcome Mapping is inappropriate in
situations where relationships of cause and
effect are known.
Outcome Mapping is not especially useful,
for example, if the results are to build a
bridge, equip rural schools or distribute
condoms.
It is most useful in complex, open and
dynamic situations where results are
unpredictable.
Goele Scheers and Ricardo Wilson-Grau
So, when relationships of cause and effect
between activities and results are unknown
or even perhaps unknowable, Outcome
Mapping can be a useful PME tool.
Consequently, Outcome Mapping is an
option for human rights campaigning, policy
advocacy, overcoming gender, racial, ethnic
or other forms of discrimination, or changing
community practices.
Goele Scheers and Ricardo Wilson-Grau
Final discussion
You will remember the results we aimed for.
Who has thoughts on these questions on
Outcome Mapping?
1.
To what extent does OM have the
potential to serve your PME needs as a GAN?
2.
For donor’s, what is the potential of OM to
meet or modify your reporting and
accountability requirements
3.
And for consultants, how interested might
you be in developing the ability to use OM?
Goele Scheers and Ricardo Wilson-Grau
Thank you!
For more information:
IDRC Evaluation Unit :
http://www.idrc.ca/en/ev-26586-201-1-DO_TOPIC.html
Outcome Mapping community of practice:
www.outcomemapping.ca
Us:
[email protected]
[email protected]
Goele Scheers and Ricardo Wilson-Grau