Transcript How to write a good ToR for Evaluation-Pinky

HOW TO WRITE A GOOD
TERMS OF REFERENCE
FOR FOR EVALUATION
Programme Management Interest Group
19 October 2010
Pinky Mashigo
PRESENTATION FORMAT
•
•
•
•
Introduction
Definitions explored
Logframe and Evaluation
Link between Logframe and Evaluation
Criteria
• Components of ToR
• Summary – Reminder for emphasis
INTRODUCTION
• Terms of Reference in general provides a
guide towards results that can be used for
decision making, planning and learning
• The theme today is EVALUATION
• There is a tendency to use certain terms
inter-changeably or linked without
acknowledging that they all mean different
things to different people
• If terms are used in this manner they tend to
confuse outcomes, outputs expected from the
study, project purpose, etc.
INTRODUCTION CONT’D
• In order to avoid the confusion, it best we
define these terms:
• These are Monitoring, Evaluation and
Audits
DEFINITIONS
Monitoring
Evaluation
Audit
Who ?
Responsibility is at all
levels of management
(internal)
Usually incorporates
external inputs
(objectivity)
Incorporate external
inputs
When ?
Ongoing
Periodic – midterm
Completion
Ex –post
Ex – Ante
(systems reviews)
Completion –
Compliance
Why ?
Check progress , take
remedial action, update
plan, control
Learn broad lessons,
application to other
activities, decision
making, input to policy
review,
Provides assurance
and accountability to
stakeholders
Provide
recommendation for
improvements
Links to the Results
Chain
Inputs, Activities, and
Outputs
Depends on focus Across the results
chain
Inputs, Activities and
Outputs
DISTINCTIONS BETWEEN THE THREE
•
Monitoring:
Ongoing analysis of (project) progress towards
achieving planned results, with the purpose of
improving management decision making
•
Evaluation:
systematic collection and analysis of data to ascertain
efficiency, effectiveness, impact, relevance and
sustainability of project, programme, policy, etc.
•
Audit:
i.
ii.
iii.
analysis of the legality and regularity of project expenditure &
income, ie. Compliance with the laws & regulation
analysis whether funds have been used efficiently and
economically in accordance with sound financial management
whether the project funds have been effectively used for the
purpose intended
LOGFRAME AND EVALUATION
• Having understood the difference – focus is how
to write a good Terms of Reference for an
Evaluation exercise
• Key to any evaluation is the Results Chain
(Logic Model) Or Logframe
• Logframe guides in terms of what kind of an
evaluation and what kind of results are expected
LINK BETWEEN EVALUATION CRITERIA
IMPACT
IMPACT
EFFECTIVENESS
OUTPUTS
ACTIVITIES
EFFICIENCY
MEANS
SUSTAINABILITY
OUTCOMES
RELEVANCE
PROBLEMATIC SITUATION
LOGFRAME OBJECTIVE
EVALUATION CRITERIA
EVALUATION CONT’D
• Therefore, Terms of Reference must
reflect what you want to achieve/intent
• The ToR is stage 1 of the evaluation
process, which is very key:
• Who should design the ToR, all
stakeholder involved in the project, policy
to be evaluated
COMPONENTS OF TOR
1. Background and Rationale (Introduction)
2. Purpose/Objective of the study
•
•
•
•
What is the focus of the evaluation
How will the results be used/utilised
Target (primary and secondary users)
Timeframe
3. Issues to be studied – Define scope
•
•
•
Mid-term
End
Ex – Post
•
Guide the evaluator in terms of focus, expectation, target
group, utilisation of results
COMPONENTS OF TOR CONT’D
4. What are the assumptions and risks
5. Plan of Work – Evaluation design
• Methodology of the study (provide guide)
• Secondary or primary
• Survey (what type)
• Quasi experimental
• Importance: guide on costs, implications and link
back to the purpose. Can it be achieved!
(a). Data elements – sources for collection; define
method and tools to collect data
(b). Data analysis - what method will be used, what tools
can be used
(c). Stakeholders – who should be consulted
COMPONENTS OF TOR CONT’D
5. Expertise Required
Team members:
• Define competencies, experience & skills
• If required define qualifications as well
Composition of the team
• Number
• Language where required
COMPONENTS OF TOR CONT’D
6. Reporting
•
•
•
•
Language of the report
Date of delivery
Number of copies required (recipients)
Type of reports (recipients) and timelines
•
•
•
•
•
Inception
Data analysis report
Ist draft
2nd draft
Final report
COMPONENTS OF TOR CONT’D
7. Administration and Logistical Support
- Management of the evaluation, who is
responsible for what
- Define all the support to be provided to the
evaluator (define boundaries)
8. Time Schedule
- Duration of the study
9. Indicative Budget
SUMMARY
• Of Importance:
– Level of evaluation (efficiency, effectiveness,
impact & sustainability)
– Purpose
– ToR
• Issues to be studied
• Team
• Indicative budget
– Respondent to the ToR, should be given a
chance to ask questions for clarification,
correction, etc. Prior to the proposal being
submitted