Transcript Monitoring and - National Treasury

Monitoring
&
Evaluation
Presentation for
Technical Assistance Unit,
National Treasury
19 August 2004
Fia van Rensburg
Outline





What is monitoring
…and evaluation
Monitoring issues
Evaluation issues
Resources
Monitoring and Evaluation


Monitoring: a systematic
management activity that
involves the analysis of
efficiency and effectiveness
(actual vs. planned) deliverables
Evaluation: analysis of
efficiency, effectiveness, impact
relevance and sustainability
M&E Focus and Context
Determined by…
 Specific project
 M&E policy
Contributes to…
 Annual reports
 Performance
management
 General good
governance
 Future planning
Influenced by …
 Line functional units
 Other projects
 Organisational
strategy, culture and
objectives
 Government
objectives
 External factors
Different levels and
perspectives
What is important for…
 The public or beneficiaries
 Implementing partners
 Project managers
 Line managers
 Top management
 Political leadership
Action Hierarchy (Bennet, 1979)

Inputs: Resources expended, time, money human
resources

Activities: Implementation data, what programme
offers or does

Participation: Characteristics of participants and
clients, nature of involvement, background

Reactions: Client satisfaction, interest in project,
perceptions of strengths and weaknesses



Knowledge, attitude and skills
changes: measures on individual and group levels
Practice and behaviour: measured over time
End results: impact on overall problem, ultimate
goals, side-effects, social and economical consequences.
Monitoring approach








Part of planning or an afterthought?
Does it cost money or save money?
Special responsibility or integrated in
business?
Specific moments or throughout
project?
Different approaches at different
project stages?
Top down or bottom up?
Qualitative or quantitative?
Usefulness of information?
Monitoring issues






Indicators - output, outcome, impact
Situation before and after
intervention - what are we measuring
Variables, control groups,
longitudinal studies
Integration of technical and financial
implementation, monitoring and
reporting
Watch those risks and assumptions!
Do you monitor your monitoring and
evaluate your evaluations?
Monitoring Issues





Annual Operational Plans
Cross-cutting issues - gender
Integration of social
accountability - financial, human
and environmental
Stats SA
Lessons learnt from 10 year
review
Views on Evaluation





Always required
The project is not
“proud” yet
Conclusive change
– quality and
degree
Reasonable
impact
Too many
contributing
variables





Criteria for project
selection
Formative
evaluation
Baseline data
available
Evidence of
improvement and
learning
Reverse logic
Evaluation Issues




Can government evaluate itself?
Role of Public Service
Commission?
Is empowerment evaluation
relevant in SA government
context?
Are external evaluators
objective?
Information systems and
management




Does the pieces of the puzzle
match and do we have a
complete picture?
Integrated databases?
What information is required by
decision makers, project
managers, other stakeholders?
Integrity of data?
Standards
 Utility
– practical information needs of
primary intended users
 Feasibility
– realistic, prudent,
diplomatic and frugal
 Propriety
– conducted legally,
ethically, with due regard for those involved
in evaluation and affected by results
 Accuracy
– technically adequate
information about programme features,
defendable results
Competencies

Systematic inquiry – research and
evaluation

Evaluation competence - info needs
of PIUs, situational analysis, organise and manage
evaluation project

General skills – logical and critical thinking,
communication, interpersonal and computer

Professionalism – self-knowledge,
ethical, standards, professional development
Resources


Websites
Models and checklists