An Introduction to Outcome Mapping

Tools for Planning, Monitoring & Evaluating Development Projects and Programs IPDET June 16, 2011 Terry Smutylo [email protected]

Objectives for this Session

1.

Inspire you to search for and adapt evaluation tools to fit your context & needs 2.

Introduce key concepts & tools in Outcome Mapping 2

Development interventions can be:

 Complex (involve a confluence of actors and factors)  Unstable (independent of project duration)  Non-linear (unexpected, emergent, discontinuous)  Two-way (intervention may change)  Beyond control (but subject to influence)  Incremental, cumulative (watersheds & tipping points) 3

4

Challenges in evaluating development interventions:  establishing cause & effect in open systems  measuring what did not happen  reporting on emerging objectives  timing: success today, failure tomorrow?  encouraging ongoing learning  reconciling differing values, perspectives  measuring ‘sustainable’ results 5

The focus of Outcome Mapping

changed behavior

6

What is Outcome Mapping?

A project/program management tool for:  Clarifying intentions  Developing strategies to achieve results  Identifying information for monitoring and evaluating  Fostering organizational learning  Enhancing other frameworks & methods 7 7

OM: brief history

1990s:

IDRC’s post-Rio search for ‘sustainable’ results

1998:

Barry Kibel and Outcome Engineering

1999:

Collaboration & testing within projects

2000:

Publication of manual in English & French

2002:

Training, facilitation & usage globally

2006:

OM Learning Community: www.outcomemapping.ca

2008:

CLAMA 8

Three key concepts in OM:

 Sphere of influence  Recognizing changes in behaviour as outcomes  Boundary Partners 9

There are limits to our influence

Project or Program Partners Beneficiaries

sphere of

influence

sphere of

concern

10

Changes in behaviour as ‘outcomes’ inputs activities outputs

o u t c o m e s

changes in behaviour changes in conditions, well-being

sphere of

influence

sphere of

concern

11

12

Step 1:

improved human, social, & environmental wellbeing 13

I have a dream!

Martin Luther King, Jr.

August 28, 1963 14

Step 2:

Mission

The mission is that “bite” of the vision statement on which the program is going to focus.

15

A mission statement describes: 

What do you do?



Who are your principle collaborators?



How do you work with them?

16

Vision

✓ about the future ✓ concrete, observable ✓ idealistic ✓ not about the program

Mission

✓ ✓ feasible identifies activities and relationships ✓ about the program 17

Step 3:

Who are our Boundary Partners?

Project or Program Boundary Partners Beneficiaries other stakeholders 18

Farmers participate in field trials using drip irrigation Participatory research on demonstration farms to develop approaches to drip irrigation Extension workers visit demonstration farms Training of extension workers Publication of performance of different set ups

Farmers add to own knowledge of techniques Farmers adopt drip irrigation methods Extension workers explain & promote drip irrigation

Reduced numbers of new wells Greater quantities of groundwater available

Adapted from K. Kelpin, 2009 19

Step 5:

Progress Markers Love to see

(Deep transformation)

Like to see

(Active engagement)

Expect to see

(Early positive responses) 20

Why use Progress Markers?

 Articulate the complexity of the change process  Encourage the program to seek the most profound transformation possible  Facilitate negotiation of expectations with partners  Enable early assessment of progress  Help identify mid-course improvements 21 21

Progress Markers – IMF examples • • • • Expect to see local communities: Participating in regular model forest (MF) meetings Establishing an organizational structure for cooperation Acquiring new skills for managing model forests Articulating a locally relevant vision for the MF Like to see local communities: 5.

Contributing resources to set up their MF 6.

Calling upon external experts for advice 7.

Seeking out new partners for the MF Love to see local communities: 8.

9.

Obtaining funding from different national sources Publishing examples of benefits achieved through MF 10. Helping other communities establish MFs 11. Sharing lessons learned internationally 12. Influencing national policy debates on resource use 22

Sample progress markers Expect to see Women’s Self-Help Groups: 1. Holding regular meetings 2. Discussing a list of shared concerns 3. Contributing to a group bank account Like to see Women’s Self-Help Groups: 4. Soliciting training in maternal & child health for its members 5. Acquiring skills in managing credit programs 6. Lending money to members Love to see Women’s Self-Help Groups: 7. Lobbying local government for expenditures on community improvements 8. Putting forth candidates for election to local government council 23

6 kinds of strategies

Causal Persuasive Supportive I aimed at Individual boundary partner E aimed at boundary partner’s Environment Strong, direct influence Arouse new thinking; build skills, capacity Continuing support Alter the physical, regulatory or information

environment

Broad information dissemination; Access to new info Create / strengthen peer networks

24

Step 6:

Strategy Map

causal persuasive supportive I E

25

Step 7: Organizational Practices

How does your team or organization stay relevant, viable and effective?

26

You:

 keep learning  foster creativity & innovation  seek better ways to assist your partners  maintain your niche  maintain high level support  build relationships 27

Organizational Practices

1.

2.

3.

4.

Prospecting for new ideas, opportunities, and resources Seeking feedback from knowledgeable informants Maintaining the support of your next highest power Assessing and redesigning products, services, systems, and procedures 28

…organizational practices

5.

Getting feedback from those already served 6.

Sharing your learning with the world 7.

Experimenting to remain innovative 8.

Engaging in organizational reflection 29

B O U N D A R Y P A R T N E R 1 O u t c o m e C h a l l e n g e P r o g r e s s M a r k e r s

L o v e t o S e e L i k e t o S e e E x p e c t t o S e e

S t r a t e g y M a p 1 V i s i o n M i s s i o n B O U N D A R Y P A R T N E R 2 O u t c o m e C h a l l e n g e P r o g r e s s M a r k e r s

L o v e t o S e e L i k e t o S e e E x p e c t t o S e e

S t r a t e g y M a p 2 B O U N D A R Y P A R T N E R 3 O u t c o m e C h a l l e n g e P r o g r e s s M a r k e r s

L o v e t o S e e L i k e t o S e e E x p e c t t o S e e

S t r a t e g y M a p 3

30

5 kinds of M&E Information

C o n t e x t u a l I n f o r m a t I o n

Program Strategies Partner State, status or situational data

relevance & viability outcomes

(actions of the program)

implementation

(behaviour changes in the partners) (interventions by the program) 31

Operating Principles of OM:

Changes in well-being Changes in behavior ‘Capacity’ includes the power & responsibility to act Need for diversified strategies Influence not control Contribution not attribution 32

OM is designed to be:

   

Flexible:

modular to be adapted to use & context

Complementary

: can be combined with other methods

Evaluative

: promote a culture of reflection, and social & organizational learning

Participatory

: seek dialogue and collaboration with partners 33

In monitoring & evaluation:

1.

There is no “silver bullet” 2. Seek quality with passion & integrity 3. Recognize & celebrate achievements of your partners 4.

Be “idealistic realists.” 5. Learn, and teach upwards.

34

Inspiration & Information

 Visit the OM learning community:

www.outcomemapping.ca

www.mapeodealcances.net

http://publicwebsite.idrc.ca/EN/Themes/Evaluation/Pages /default.aspx

35