Transcript Monitoring and Evaluation of Knowledge Management

Monitoring and Evaluation of
Knowledge Management
Simon Hearn, ODI, [email protected]
Ewen LeBorgne, IRC International Water and Sanitation Centre, [email protected]
Valerie Brown, Australia National University, [email protected]
Overview
Our journey so far: Definitions,
challenges and signposts
Interlude: Multiple knowledges
Where are we heading next
with this?
“It is, in fact, nothing
short of a miracle that the
modern methods of
instruction have not
entirely strangled the holy
curiosity of inquiry.”
- Albert Einstein
Definitions
“When I use a word, it
means just what I choose it
to mean — neither more
nor less”
Monitoring and evaluation
• OECD definitions:
– Evaluation: The systematic and objective
assessment of an on-going or completed project,
programme or policy, its design, implementation
and results.
– Monitoring: A continuing function that uses
systematic collection of data on specified
indicators to provide indications of the extent of
progress and achievement of objectives (abridged)
Monitoring and evaluation
• OK, but... Any definition must recognise:
– M&E as universal functions, not specialised roles
– Presence of different worldviews
– Validity of evidence from different knowledge
domains*
– The ethical basis for the desired social change
– The importance of the unexpected and the
intangible
Knowledge
•
•
•
•
•
Objective and subjective
Individual and society
Facts and values
Tacit and implcit
E.g. Western scientific conception of
knowledge as ‘justified true belief’ vs African
concept of Ubuntu
Development
• Often conceptualised as a service industry
• Delivery of even basic services (roads,
sanitation..) requires an understanding of the
social, political and economic contexts
• Thus, development is more like a knowledge
industry (Powell 2006)
• But development is more than donor aid and
we must recognise civic-driven change also
Challenges
Challenges in M&E of KM4D
1.
2.
3.
4.
5.
6.
7.
8.
9.
KM4D does not as yet have a well grounded theory
Knowledge for development practice is still young
KM4D goes beyond what is labeled ‘KM’
Competing ontological and epistemological perspectives
(and related knowledge systems)
Existing reporting frameworks are designed for a service
industry rather than a knowledge industry
There can be no simple cause-effect relationship
KM initiatives often lack explicit linkages to individual,
specialist, organisational or social results
Knowledge is not static
Lack of methods for interpreting intangibles
Signposts
1. KM ripple model
Performance improvement
Changed practices
Knowledge capital
Knowledge
processenhancing
activities
Hulsebosch et al (2009)
2. The KM Framework
Talisayon (2009)
Need a better understanding of what
intangibles are
Cognitive Factors
Structural Capital
Relationship Capital
Tangible Assets
Motivational Factors
Human Capital
Value creation
through intangibles
Based on Talisayon (2009)
Need a better understanding of
knowledge transitions
SECI
Nonaka and Takeuchi (1995)
Need a better understanding of
how knowledge is put to use
Knowledge to
action cycle
Graham et al (2001)
Need a better understanding of organisational
factors affecting knowledge use
The RAPID
Framework for
Knowledge
Strategies
Ramalingam (2005)
We need to understand the level of
complexity
Cynefin framework
Snowden (2002)
Summary: a range of perspectives
• Ontological: What world-views are reflected in the
initiative and how do we recognise them?
• Epistemological: What are the knowledge domains
contributing to M&E and how do they relate?
• Socio-political: Who has a stake in the monitoring
process and who has power? How can we monitor
these interdependent relationships?
• Methodological: How to choose tools and approaches
relevant to the parties and processes involved?
• Operational: How do we organise M&E activities
according to each of the knowledge domains?
Your reflections?
• Do you identify with these signposts?
• What signposts do you use?
• How do you see these models supporting
your work?
Multiple knowledges:
M&E as multiple partners
Whole-of-community M&E
 INDIVIDUAL COMMITMENT
 COMMUNITY INTERESTS
 SPECIALISED ADVICE
 ORGANISATIONAL SUPPORT
 HOLISTIC SOLUTIONS
Multiple knowledges
(Brown 2008)
 INDIVIDUAL KNOWLEDGE
Personal lived experience
 LOCAL KNOWLEDGE
Shared community event
 SPECIALISED KNOWLEDGE
Environment, Health, Finance…,
 STRATEGIC KNOWLEDGE
Organisational structure, aims
 HOLISTIC KNOWLEDGE
.
Focus, vision
Rejected knowledges
 INDIVIDUAL KNOWLEDGE
Biased
 LOCAL KNOWLEDGE
Anecdote
 SPECIALISED KNOWLEDGE
Jargon
 STRATEGIC KNOWLEDGE
Self-serving
 HOLISTIC KNOWLEDGE
.
Airy-fairy
Collective knowledge as a nested set
Individual knowledge
Local knowledge
Specialist knowledge
Organisational knowledge
Holistic knowledge
A collaborative system
25
Port Pirie:
small town with the biggest lead smelter in the world
KNOWLEDGES
STRUCTURE
CONDITIONS
INDIVIDUAL
Children diagnosed with lead
COMMUNITY
People long resigned to risk
SPECIALIST
Health Centre stays aloof
ORGANISATION
Mine muzzles council
HOLISTIC FOCUS
Fear for future livelihood
26
New alliances in Port Pirie
INDIVIDUAL
Parent, grandparent
COMMUNITY
Outrage, political action
SPECIALISTS
Technical skills, advocacy
ORGANISATIONAL
HOLISTIC
Public/private good
Children’s well-being
27
M&E as collective learning
- multiple interests
- multiple knowledges
- collaborative action
Next steps:
- The IKM-E approach
- Emergent questions on the horizon
Our approach: Multi-evidence based?
Each knowledge community uses different M&E criteria,
evidence bases, databases for judgments...
• Individuals (experiences)
• Communities (observations)
• Experts (practitioner stories)
• Organisations (monitoring reports as stated)
• Holistic thinkers (ideas, forecasts)
Our approach: Purposes of conducting M&E
•
•
•
•
•
•
•
•
•
Financial accountability
Operational improvement
Strategic readjustment
Capacity strengthening
Contextual understanding
Deepening understanding (research)
Self-auditing
Advocacy
Sensitisation
(From I. Guijt’s PhD thesis ‘seeking surprise’)
Our approach: KM as collective learning
Ideals
Key to nested knowledge cultures:
 Individual
 (Local) Community
 Specialised
 Organisational (strategic)
 Holistic
What should be?
Describe
Develop
Actions
Initiative
What can be?
Facts
What is?
Design
Do
Ideas
What could be?
Our approach: critical questioning
• A series of questions at each step of the way
– Overall, a sound questioning practice
– And specifically, a guideline to tailor one’s
approach:
•
•
•
•
What questions to address?
Who to involve, in what function?
What tools and methods to choose?
What lessons to draw from the approach?
Our approach: A nested iterative inquiry
Emergent questions on the horizon
• How would our approach work in practice?
• Specific methods and metrics to go ‘light’
• Particularly complexity-focused approaches
• Power vs. collective?
What now?
IKM-E + KMIC = IKMEKMIC?
• Avoiding overlaps...
– Connecting KMIC and IKM (blogs...)
– Organising another webinar?
– Identifying different models / approaches?
• Having creative leaps...
– Reviewing the IKM papers?
– Expanding parts of this paper?
– Testing the IKM-E framework (later)?
Additional resources
• IKM-Emergent website:
http://wiki.ikmemergent.net
• The giraffe, Working group 3 blog
• Working paper 3: ‘Monitoring and Evaluation in
Knowledge Management for Development‘
http://su.pr/5rqp8c
• Background paper: ‘Monitoring and evaluating
knowledge management strategies’
http://su.pr/28Q9Yu
Thank you!