Transcript M&E timeline

M&E Plan Overview
M&E Capacity Strengthening
Workshop Addis Ababa
4 to 8 June 2012
Arif Rashid, TOPS
What is an M&E Plan?
An M&E plan is:
• A flexible guide to the steps needed to:
– Inform and guide monitoring and evaluation
activities;
– Document, report, and communicate project
activities;
– and illustrate progress toward achievement of
project goals and objectives.
Slide # 1
What is an M&E Plan?
Slide # 2
What are the components of the
M&E Plan?
• M&E goals and objectives
• M&E methodologies & implementation plan
– Data collection procedures
– Data management and analysis procedures
– Reporting and communications plan
• Description of M&E activities
– Output monitoring
– Process monitoring
– Outcome monitoring
– Monitoring of environmental indicators
– Commodity end-use monitoring
Slide # 3
What are the components of the
M&E Plan?
– Calendar of M&E activities
– Data quality assurance plan
– Data collection instruments
– Organizational structure of M&E unit
– Communication strategy and use of data in decision
making
– Budget for M&E activities
Slide # 4
M&E Plan components
Focus of this workshop
M&E
Methods/tools
PIRS /M&E
Matrix
M&E
Activities
- Monitoring
- Surveys
M&E Timeline
Communication
strategy
- Secondary data
IPTT
PMP
- Analysis
- Reporting
M&E Staffing
- Dissemination
Results
Framework
- Review M&E Plan
M&E Budget
General
implementation
plan
Data Quality
DIP
Slide # 5
Unpacking the M&E Plan:
Key elements
Slide # 6
M&E Plan components
M&E
Methods/tools
PIRS /M&E
Matrix
A
M&E
Activities
- Monitoring
- Surveys
M&E Timeline
Communication
strategy
- Secondary data
IPTT
PMP
- Analysis
- Reporting
M&E Staffing
- Dissemination
Results
Framework
- Review M&E Plan
M&E Budget
General
implementation
plan
Data Quality
DIP
Slide # 7
A. Results framework (RF)
“A results framework …[is a]…type of logic
model which represents the food aid program’s
theory of change by laying out the activities and
outputs that will lead to short, medium and long
term outcomes and objectives”
Source: http://www.usaid.gov/our_work/humanitarian_assistance/ffp/ffpib_09.06.pdf
Slide # 8
RF example
GOAL
SO 1
SO2
SO3
IR1.1
IR1.2
IR2.1
IR2.2
IR3.1
IR3.2
Activities
Activities
Activities
Activities
Activities
Activities
Slide # 9
M&E Plan components
M&E
Methods/tools
B
PIRS /M&E
Matrix
M&E
Activities
- Monitoring
- Surveys
M&E Timeline
Communication
strategy
- Secondary data
IPTT
PMP
- Analysis
- Reporting
M&E Staffing
- Dissemination
Results
Framework
- Review M&E Plan
M&E Budget
General
implementation
plan
Data Quality
DIP
Slide # 10
B. IPTT
“The indicator performance tracking table
(IPTT) …includes performance indicators (at the
impact, outcome and output levels) linked to
the food aid program proposal’s objectives, and
baseline and target values for each indicator
(estimated)”
Source: http://www.usaid.gov/our_work/humanitarian_assistance/ffp/ffpib_09.06.pdf Slide # 11
FY2012 IPTT Template
Slide # 12
M&E Plan components
M&E
Methods/tools
C
PIRS /M&E
Matrix
M&E
Activities
- Monitoring
- Surveys
M&E Timeline
Communication
strategy
- Secondary data
IPTT
PMP
- Analysis
- Reporting
M&E Staffing
- Dissemination
Results
Framework
- Review M&E Plan
M&E Budget
General
implementation
plan
Data Quality
DIP
Slide # 13
C. M&E Matrix / PIRS/PMP
A performance management plan (PMP)
includes: “definitions for each indicator listed in
the IPTT, the data collection methodology and
frequency”
Source: http://www.usaid.gov/our_work/humanitarian_assistance/ffp/ffpib_09.06.pdf
Slide # 14
C. M&E Matrix / PIRS/PMP
An M&E matrix (also known as an Indicator
Matrix/ or Performance Indicator Reference
Sheet):
• summarizes of detailed information for each
project indicator.
• serves as a helpful tool for planning and managing
data collection, analysis and use.
• expands on the IPTT/PMP to identify key
information required for each indicator and
summarize the key M&E tasks for the project.
Adapted from: Chaplowe, Scott G. (2008)
Slide # 15
M&E Plan components
M&E
Methods/tools
C
PIRS /M&E
Matrix
M&E
Activities
- Monitoring
- Surveys
M&E Timeline
Communication
strategy
- Secondary data
IPTT
- Analysis
PMP
- Reporting
M&E Staffing
- Dissemination
Results
Framework
- Review M&E Plan
M&E Budget
General
implementation
plan
Data Quality
DIP
Slide # 16
D. Key M&E activities
1. Monitoring
• Output monitoring
• Process monitoring (quality monitoring)
• Outcome monitoring
• Environmental indicator monitoring
• Commodity end-use monitoring
2. Evaluation
3. Reporting and communications
4. Review of M&E process
Slide # 17
Output monitoring
• Reporting and tracking project outputs to
compare with targets and resource used
• Information Source
• Project reporting
• M&E plan needs to develop:
• Reporting forms
• Data base(s)
Slide # 18
Process monitoring
• Monitoring the “quality” of services delivered
to beneficiaries
• Information Source
• Surveys of “final service providers”
• M&E plan needs to develop:
• Sample design
• Instruments
• Analysis plan
Slide # 19
Outcome monitoring
• Monitoring outcome-level changes of project
beneficiaries and institutions
• To what extent have project interventions
altered
• Knowledge
• Perceptions, and
• Behaviors
of beneficiaries
Slide # 20
Outcome monitoring
• To what extent have project interventions
altered
• Institutional competency
• Policy change or
• Quality of service changes.
• Information source
• Surveys of beneficiaries
• M&E plan needs to develop for outcome
monitoring:
• Sample design
• Questionnaire
• Analysis plan
Slide # 21
Environmental indicator monitoring
• Monitoring the indicators identified in the IEE
as negative determination with conditions
• Information Source
• Based on the proposed mitigating measures in
IEE
• M&E plan needs to develop:
• List of indicators and mitigating measures
• Instruments
• Analysis plan
Slide # 22
Commodity end-use monitoring
• Monitoring end-use of the commodities
distributed by the project
• Information Source
• Sample survey of beneficiaries that received
commodities from the project
• M&E plan needs to develop:
• Sample design
• Instruments
• Analysis plan
Slide # 23
Evaluation activities
Evaluating project impacts should be taken into
consideration from the inception of the project.
Project evaluation is composed of three separate
events that should be considered parts of a whole.
[Note: Food for Peace does not require a quantitative survey during the
mid term review]
Baseline
Mid-term
Final
Slide # 24
USAID Evaluation policy
“In most cases, the evaluations will be external (i.e.,
a third-party contractor or grantee, managed directly
by USAID, will evaluate the projects), and the
contract or grant for the evaluation will be managed
by an operating unit’s Program Office.”
Please note that: The Title II Project evaluations are
currently managed by the PVOs not by FFP/USAID
which will continue until FFP decides otherwise.
Slide # 25
Evaluation activities
M&E units are responsible for organizing evaluation
activities:
Steps:
1. Identify study objectives/research questions, target
population(s), coverage and reach of study, timeline
and budget
2. Identify specific indicators to be measured
3. Compile sample frame (from appropriate population)
4. Using information above, develop SOW for external
consultants
5. Organize logistics for evaluation studies to support
external consultants
Slide # 26
Review of M&E plan
M&E plan should be regularly reviewed to assess
whether the current plan is meeting objectives
and needs of project:
• Reporting needs to donor
• Project management needs
Review should assess:
• Indicators
• Data collection and analysis procedures
• Communication strategy
Slide # 27
M&E Plan components
M&E
Methods/tools
PIRS /M&E
Matrix
M&E
Activities
- Monitoring
- Surveys
E
M&E Timeline
Communication
strategy
- Secondary data
IPTT
- Analysis
PMP
- Reporting
M&E Staffing
- Dissemination
Results
Framework
- Review M&E Plan
M&E Budget
General
implementation
plan
Data Quality
DIP
Slide # 28
E. M&E Tools and methods
M&E tools and methods include:
• Project monitoring reporting forms
• Database formats and management
procedures
• Survey sampling strategy and analysis plans
• Reporting processes and procedures
Slide # 29
M&E Plan components
M&E
Methods/tools
PIRS /M&E
Matrix
M&E
Activities
- Monitoring
- Surveys
M&E Timeline
F
Communication
strategy
- Secondary data
IPTT
- Analysis
PMP
- Reporting
M&E Staffing
- Dissemination
Results
Framework
- Review M&E Plan
M&E Budget
General
implementation
plan
Data Quality
DIP
Slide # 30
F. M&E timeline
An M&E timeline is “a schedule of M&E events,
describing procedures and timing for analysis
and use of performance data in order to modify
activities based on lessons learned”
Source: http://www.usaid.gov/our_work/humanitarian_assistance/ffp/ffpib_09.06.pdf Slide # 31
M&E Timeline example
Year 1
Year 2
Year 3
Year 4
Year 5
Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
General activities
M&E Staff hiring
Equipment procurement
Baseline survey
Baseline results reported and disseminated
Mid-term evaluation
MTE results reported and disseminated
Final evaluation
Final evaluation results reported and disseminated
SO 1 activities
Project monitoring/reporting
SO 2 activities
Project monitoring/reporting
SO 3 activities
Project monitoring/reporting
Slide # 32
M&E Plan components
M&E
Methods/tools
PIRS /M&E
Matrix
M&E
Activities
- Monitoring
- Surveys
M&E Timeline
Communication
strategy
G
- Secondary data
IPTT
- Analysis
PMP
- Reporting
M&E Staffing
- Dissemination
Results
Framework
- Review M&E Plan
M&E Budget
General
implementation
plan
Data Quality
DIP
Slide # 33
G. Communication Strategy
An important (and often overlooked) component of
M&E plans is how to disseminate and use the
information obtained by the M&E system
Although the M&E Officer or Unit does not have much control over the
managers or the technicians to ensure that M&E data inform the
programmatic decisions, M&E unit can facilitate the process. M&E
Slide # 34
G. Communication strategy
Careful thought needs to be given to
disseminating and using M&E information
including:
•What information should be distributed?
•Who needs what information?
•Why do they want this information?
•How much information they want to know?
•How does the information get distributed?
Source: Family Health International (2004)
Slide # 35
M&E Plan components
M&E
Methods/tools
PIRS /M&E
Matrix
M&E
Activities
- Monitoring
- Surveys
M&E Timeline
Communication
strategy
- Secondary data
IPTT
- Analysis
PMP
- Reporting
M&E Staffing
H
- Dissemination
Results
Framework
- Review M&E Plan
M&E Budget
General
implementation
plan
Data Quality
DIP
Slide # 36
H. M&E Staffing plan
Based on information needs outlined in earlier steps,
the following capacities may be required for an M&E
unit (Pls refer to core competencies for detailed skills):
•M&E plan design
•Data collection methods (quantitative & qualitative)
•Data management skills
•Data analysis skills
•Data reporting
•Project monitoring
Slide # 37
Sample M&E organizational chart
M&E Manager
(Extensive knowledge of:
M&E plan design, data
collection methods, analysis
and reporting)
M&E Officer
M&E Officer
(Project monitoring &
reporting, data collection,
supervising enumerators,
etc.)
(Project monitoring &
reporting, data collection,
supervising enumerators,
etc.)
M&E Assistant/ Data entry
Clerk
Slide # 38
M&E Plan components
M&E
Methods/tools
PIRS /M&E
Matrix
M&E
Activities
- Monitoring
- Surveys
M&E Timeline
Communication
strategy
- Secondary data
IPTT
- Analysis
PMP
- Reporting
M&E Staffing
- Dissemination
Results
Framework
- Review M&E Plan
M&E Budget
General
implementation
plan
I
Data Quality
DIP
Slide # 39
I. M&E Budget
• One challenge in developing M&E plans for
Development Food Aid Projects is that the M&E
budget is normally established before the detailed
M&E plan is finalized
• As a result, the M&E budget will need to be
revisited after the M&E plan is developed
• Budget must be aligned with activities in M&E
plan
Slide # 40
M&E Plan components
M&E
Methods/tools
PIRS /M&E
Matrix
M&E
Activities
- Monitoring
- Surveys
M&E Timeline
Communication
strategy
- Secondary data
IPTT
- Analysis
PMP
- Reporting
M&E Staffing
- Dissemination
Results
Framework
- Review M&E Plan
M&E Budget
General
implementation
plan
Data Quality
J
DIP
Slide # 41
J. Data Quality: Assurance and
Assessment
About
the Network
It willQuality
also serve
as a place -for
• Data
Assurance
A formalized
organizations
and institutions
to
process
for defining
the appropriate
post relevant meeting, training or
dimensions
and criteria
of data quality, and
other event notices
along with
associated links
to similar
sites
procedures
to ensure
that
data quality criteria
and resources.
are met over time
• Data Quality Assessment –Review of project
M&E system to ensure that quality of data
captured by the M&E system is acceptable.
Slide # 42
What’s a Data Quality Assurance Plan?
About the Network
•
A data quality assurance plan includes
It will also serve as a place for
activities to and
improve
data toquality:
organizations
institutions
relevant
meeting,
training
or tabulation plan and
A.post
Clear
definition
of indicators
including
other
event notices
along with
measurement
protocols
toofsimilar
sites aggregation and analysis
B.associated
Appropriatelinks
design
data collection,
and
resources.
procedures
C.
Designing data base with appropriate checks and bounds to
reduce data entry error
D. Well designed data management procedures
E. Follow-up verification of indicator values
F. Allocate adequate Resources and Skilled Staff
G. Periodically Assess Data Quality using established protocols
Slide # 43
What’s a Data Quality Assessment?
About the Network
•
data
quality
ItAwill
also
serveassessment
as a place foris a periodic review
organizations
and institutions to
that:
post relevant meeting, training or
–other
Helps
thenotices
project to
determine
event
along
with and document “How
good are links
the data?”
associated
to similar sites
and resources.
– Provides an opportunity for capacity-building of
implementing partners.
Slide # 44
We have now identified all the components of an
M&E plan
M&E
Methods/tools
PIRS/ M&E
Matrix
M&E
Activities
- Monitoring
- Surveys
Results
Framework
IPTT
M&E Timeline
Communication
strategy
- Secondary data
A
PMP
- Analysis
- Reporting
M&E Staffing
- Dissemination
- Review M&E Plan
M&E Budget
Data Quality
All these components should be in the M&E plan document
Slide # 45
Outline for M&E plan
1. Objectives of M&E system
2. Description of
–
–
–
–
Results framework
IPTT
PMP/M&E Matrix/PIRS
Project staffing and organizational structure
Slide # 46
Outline for M&E plan
3. Description of Monitoring Activities
– Description of processes, tools and frequency
4. Evaluation
– Description of processes and methodology
5. Reporting and communications strategy
6. Data Quality Assurance Plan
7. M&E review process
8. M&E staffing and organization
9. M&E budget
10.Use of information in Decision Making
Slide # 47
Outline for M&E plan
• Annexes
–
–
–
–
–
–
–
–
Results framework
IPTT
M&E matrix/PIRS/PMP
Reporting formats for output monitoring
Questionnaire forms for process/outcome monitoring
Timeline for M&E activities
M&E unit organizational chart
M&E budget
Slide # 48
The remainder of the workshop will
provide guidance on how to develop these
components of the M&E plan
Slide # 49
This presentation was made
possible by the generous support
of the American people through
the Office of Food for Peace in
United States Agency for
International Development
(USAID). The contents are the
responsibility of Save the Children
and do not necessarily reflect the
views of USAID or the United
States Government.