Transcript Introduction to Evaluation

Introduction to Evaluation
Objectives
•
Introduce the five categories of evaluation that can be
used to plan and assess ACSM activities.
•
Demonstrate how each evaluation category can be used
to answer different evaluation questions.
•
Review examples of each type of evaluation.
Evaluation is Part of the M&E Plan
Data
Quality
Data
Collection
Evaluation
Strategy
Indicators
Framework
Data Use
and
Reporting
M&E
Plan
Budget
Monitoring versus Evaluation
Monitoring
• Data collected on
Evaluation
• Data collected to answer
program activities
•
•
Ongoing, routine
Focus on process and
output, compared to
target
Are we doing the work
we planned?
specific questions
•
•
Periodic
Focus on outcome,
impact
How should we design our
activities?
How effective were they?
Categories of Evaluation
1. Baseline evaluation (needs assessment)
2. Formative evaluation
3. Process evaluation
4. Outcome evaluation
5. Impact evaluation
Baseline Evaluation/Needs Assessment
• Identifies our ACSM starting point
 Who do we need to reach and how?
 What activities are most important or most needed?
 Where do we need to go?
 What is the baseline value for key indicators?
Formative Evaluation
•
•
Helps design activities for the target group
•
Pre-testing
Strengthens interventions by getting the right
message to the right people
 Do we have the right message?
 Is this the right communication method?
 Are we reaching the right people?
 Do they understand the message?
Process Evaluation
•
•
•
Documents what happens in a program
What is going well, what is not, and why
Often overlaps with monitoring
 Did we do what we planned?
 Is the program being implemented effectively?
 What challenges do we need to address?
 What best practices can we share? Lessons learned?
Outcome Evaluation
What was the
quality of our
work – were
we effective?
Did we
contribute to
changes in key
outcomes?
Impact Evaluation
•
Contribution of ACSM interventions to highlevel goals and objectives
•
Generates evidence about effectiveness of
ACSM activities
•
Very rigorous, time consuming, and expensive
 Relies on high-quality baseline data collection.
 MUST be planned from the very beginning.
 In practice, rarely done.
Match the Question and Category!
•
Do focus group members understand this message?
FORMATIVE
•
Did our program contribute to higher case detection in
our country?
IMPACT
•
How many people know the symptoms of TB before we
launch our media campaign?
BASELINE
Match the Question and Category!
•
Did pharmacists refer more customers for DOTS after our
training?
OUTCOME
•
What did we learn about conducting effective advocacy
work this year?
PROCESS
IMPACT
OUTCOME
PROCESS
FORMATIVE
BASELINE
Design
Implementation
When do we evaluate?
End
Creating a Framework
BASELINE
RESULTS
IMPACT
•
•
•
•
•
•
•
•
•
• # of times radio
spots aired
•
• # of subway ads
developed
•
•
•
FORMATIVE
•
PROCESS
OUTCOME
•
Questions?