Transcript Frame work for Monitoring & Evaluation under IWMP Workshop for Effective

Workshop for Effective
Implementation of IWMP
Frame work for Monitoring &
Evaluation under IWMP
DOLR
Govt. of India
7th &8th January 2014
Components to be Monitored under M&E:
1. Concurrent Process Monitoring
2. Input – Output Monitoring
3. Impact Evaluation
Process
Monitoring
4. Thematic/Special Studies
Input-Output
Monitoring
M&E
5. Sustainability Assessment
Impact
Assessment
6. Case Studies/ Success Stories
7. Feedback and Dissemination Mechanism for Learning
8. Learning’s and Adoptability
9. Envisaged Deliverables/Outputs
Learning & Documentation
M&E is an integral part of the project’s day to day operations rather than a periodic offline activity
1. Concurrent Process Monitoring:
Major Processes to be Monitored under IWMP
Entry point activity (EPA)
Sensitization and awareness
CBO formation & functioning
Capacity building
Transparency
Action plan preparation (Net plan & Complete action plan)
Action plan implementation (Activity implementation)
Flow of funds and procurement procedures
Income Generating Activities (Livestock / Skill based & Micro enterprises activities)
Institutional functioning- NGO’s, CBO’s & Government dept. or any other agencies.
Demonstrations if any
Withdrawal strategy
Convergence (with other projects)
Post project O&M strategy
Sustainability
Separate indicators (Both qualitative and quantitative) to be developed for each of above processes
Project Stages and Process Monitoring
Process Monitoring Cont….
A
IV Stage
Indicators
I Stage
Consolidation
D
•
Prioritize
•
Identify new
•
Continue or
discontinue old
capacitation
•
Make changes
(if needed)
II Stage
•
Operationalise
Initiation
B
O
Implementation
III Stage
C
CBO
Ist Six months
IInd six months
IIIrd six months
IVth six months
SHG
Formation
Meetings
Attendance
Savings
Capacitation
Savings
Meetings
Lending
Financial transaction
Book keeping
IGA
Value additions
Profit earned
Sustainability
employment
(Indicators)
“Indicators to be redefined as per the stage of the project”
Weightage to be shifted as per project time chart
Process Monitoring carried out in Five-Stages
Collection of Data via Physical Surveys
Process Data and GIS analysis as applicable
Ex: graphs / tables
Interpret Data using MIS. And other wise
Ex: Trends Graphs / Patterns/ GIS outputs
Use Findings for Learnings and Corrective Measures
Ex: To change practice, inform policy, etc.
Documentation of best practices and strategies for up
scaling
In An Effective M&E the five stages are carried sequentially
Process Monitoring Cont….
A Format for Process Monitoring (Observation) Report - IWMP
Issues observed
CBOs Establishment
& Functioning
IWMP Implementation
Transparency
Capacity Building
IGA & Micro
Enterprises
Situational Status
(What is happening in the
field)
Reasons
(Why it is
happening)
Suggestion / Scope
for improvement
(What is desirable)
Stakehold
ers to be
involved
Eg: Monthly Report Extract
Taluk – Bidar
IWMP – 1/2009-10
Sl. No.
Activities / Processes
01
CBO training
02
Activity wise
Documentation
03
Baseline survey
04
EC functioning & EC
office establishment
05
S & WC Work
BIDAR DISTRICT
Situational Status
Sub-Watershed – Bagdal
NGO – Santeshwari
Brief Analysis of the Status Observed
S1, S2, S3, U1, U2, U3, E1, E2 & E3 All the trainings are completed.
trainings are completed in Bagdal sub Overall participation level in S1, S2 & S3 trainings was about
watershed.
86.45%.
Overall participation level in U1, U2 & U3 trainings was about
69.37%.
Overall participation level in E1, E2 & E3 trainings was about
58.33%.
Details pertaining to the trainings have been provided in the
previous reports.
NGO level documentation is made.
Photo documents of Grama sabha, Jatha & Street play have
been maintained.
S1, S2, S3, U1, U2, U3, E1, E2 & E3 trainings documents are
maintained.
Register book and photo documentation is also maintained.
Baseline survey of 2390 households is Entire 2390 household’s data has been entered into the
completed.
system.
There are 4 ECs in Bagdal watershed.
Office related logistics were provided in the month of August
Offices have been opened for all these to 4 EC offices viz Bagdal, Khashempura, Bairanahalli &
4 ECs in Bagdal sub watershed.
Honnadi in this sub watershed.
Books are not maintained in the EC office.
Documents are not maintained in the EC.
S & WC Work has been carried out in
this Sub-Watershed
Visited Bagdal sub watershed on
19/01/2013
Visited Sy. No. 65 (Lakshmana) where 300 rmt of bunding
work has been done instead of 600 rmt as per net plan.
Bund dressing has not been carried out.
Bund carried out is not as per specification.
Process Monitoring Cont….
Process Monitoring Methods & Reports
 Reports on Monthly/ Quarterly basis
 Reflect the actual ongoing field status.
 All the process and activities will be monitored at random basis (at least 30% of
Project site in a quarter)
 Watershed wise reports will be consolidated at the district level
 Report would also identify the key issues requiring immediate attention
 Flag-off issues to be taken up as special studies or thematic studies.
 Highlight the particular processes observed, the situational status, the reasons
for the same and the institutions or individual associated in it.
 Also suggest field based remedial measures to improve upon.
2.INPUT-OUTPUT MONITORING
Under the concurrent input-output monitoring, the financial and physical progress of the
project would be monitored.
Aspects such as fund release from the PIA to grass root level, actual condition/status of
physical achievements, awareness creation, CBO’s capacity building, etc will be monitored
using specific indicators.
This would be done through the MIS deployed and operationalised by the project.
M&E unit would verify the MIS data integrity through field verification on sample basis
through process monitoring.
Project will ensure flow of MIS data to M&E for analysis.
The MIS data thus received would be analyzed and the report would be made available to
the top management once in six months.
3. Impact Evaluation
Address the overall impact on Natural Resources & Socio -Economic aspects
Impact
Household level
Short term
Community / Village
Long term
Micro / Sub Watershed
APPROACH
•
•
•
•
Before (Baseline)
During (Midterm)
End of the Project
Post Project
Before
&
After
With
&
Without
Continues….
Impact Evaluation Cont….
Continues….
Sampling Approach
State
Sub watershed
Village-Level (Household Survey)
Impact Evaluation Cont….
Micro-Level (Micro watershed)
Impact Evaluation Cont….
Baseline Study:
At the outset, as a part of impact assessment, Baseline Survey would be conducted in the
sampled watersheds.
This analysis will provide a bench mark enabling a comparative analysis at the end of the
project.
The degree of project goals and objectives achieved would be assessed as against this
bench mark.
For a better comparison, ‘control samples’ outside the watershed (i.e. untreated area with
the similar topographic and socio-economic condition) also be analyzed.
Continues….
End Project Impact Study:
Impact Evaluation Cont….
 At the End of the Project, M&E will also carry out the impact assessment to establish the
net impact of the project on the socio-economic and natural resource indicators.
 Data would be collected with respect to the same samples collected during the baseline.
 A comprehensive comparative analysis to the baseline status would be made in this report.
The major perceivable impacts would be:• Hydrological – Ground & Surface water, Drinking water status, etc.
• Soil Status – Runoff, infiltration and moisture content, nutrient capacity, etc.
• Agriculture and allied – Cropping pattern, cropping intensity, land reclamation, productivity
and diversity
• Institutional Aspects – CBOs, PRIs, any other
• Socio-economic – Income level, Employment status, Migration, Women Empowerment,
Educational status, Living condition, Livelihood options, etc.
• Environmental – Land use land cover change, a forestation, biomass and carbon
sequestration and
• Spillovers / Externalities (if any) – Political involvement, linkages with other projects,
financial linkages, etc.
Impact Evaluation Cont….
Sampling Strategy for Impact Assessment
 Sampling will be Based on Ridge, Middle, Valley Concept
 25% of the projects will be studied
 Households will be selected using PPS Method
 Detailed questionnaire schedule covering relevant indicators will be
used for Baseline, Midterm and End of the project
comparative analysis
 Data will be collected by the M&E field staff
 Satellite imageries will be used for spatial analysis
enabling
Impact Evaluation Cont….
Indicators for Impact Assessment
Natural Resources
 Surface Runoff
Socio Economic
 Water Resource Development
 Sustainability of
Structures /CBOs/IGA
 Ground Water level/Yield
 Micro Enterprises
 Change in Irrigated Area
 Employment Opportunity
 Crop Diversity
 Migration Status
 Crop Yield
 Economic potential of
Household Income
 Crop intensity
 Fodder Availability
 Afforestation
 Climate Change & Biodiversity
 Land Use Change
 BPL Family
 Animal Husbandry
 Impact on Milk yield
4. Thematic/Special Studies:
M&E will also conduct certain thematic studies on key issues as identified.
Some of
the thematic areas include Capacity building, Investment pattern; CBO
participation, women empowerment, gender issue, level of participation, Income generating
activities; Run-off and land cover transformation, success stories, etc.
The basic principle governing the thematic studies are that
(i) it should answer specific questions in the short run and help in policy derivations for next
phase.
(ii) should reflect both positive and negative aspects of the project.
(iii) should be demand driven and locally identified (local specific) by the PIA.
It is proposed that about one or two of thematic studies to be conducted per year.
Continues….
Thematic/Special Studies cont…..
A few possible Thematic / Special Studies
 Investment Pattern
• Run off Estimation
 Social Inclusion
• Land use / Land cover changes
 Women Empowerment
• Ground Water recharge
 Capacity Building
• Soil Fertility
 Equity Issues
• Water Quality
 Common Property Resources
 Income Generating Activities
 Live stock
 Short term Impacts
 Sustainability
A Few New Thematic Areas (Suggestive)
Community Based Activities
 Poverty Mapping
 Sustainability Micro Enterprises
 Input & Output estimate for a
Thematic/Special Studies cont….
Soil & Water Conservation
Activities
 Soil erosion and Run-off estimation
 Crop – Water Budgeting
given Micro watershed
Hydrology
Natural Resources
 Groundwater level and quality
 Reclamation of degraded land
 Water Harvesting Structures:
 Agriculture Productivity
Potential of Water availability and
 Biomass Estimation
its utility.
 CDM supportive studies
(Assessment of Carbon Credits)
Environmental Impact Assessment and Climatic Change Studies
 Watershed impact on overall ecosystem and climate
5. Sustainability Assessment:
 Post project sustainability of assets generated and also the institutions created is very
essential.
 The level of involvement various institutions particularly the CBOs are crucial for the
sustainability.
 In the long run the maintenance of the assets has to be taken up by the community on their
own.
 Particularly the Common Property Resources (CPRs) needs to be maintained by the
community.
 It requires a support system built in by the project during the project period.
 The withdrawal strategy needs to be evolved on these lines.
 The M&E studies need to focus on the successful and effective functioning institutional
arrangements for replication at a larger scale over time.
 The withdrawal strategy adopted in the IWMP-I will be assessed for evolving a strategy in the
next phase of the project.
 Selected 10 per cent of the institutions from SHGs, CIGs, and AGs will be studied assessing
their sustainability after completion of IWMP –I projects.
6. Case Studies/ Success Stories:
Case studies to identify the specific project impacts would be carried out from time to time.
These case studies will reflect the immediate outcomes and outputs of the project with
respect to specific area or sector.
The case studies reflect both positive and negative aspects.
This will help to consolidate on the success stories of the project and to improve upon the
weak areas if any.
At least one case study will be prepared per project per year.
7.Feedback and Dissemination Mechanism for Learning:
Along with respective State departments identified M&E agencies evolve suitable
dissemination mechanism for learning at different levels.
It will assist department in reviewing the M&E observations /findings, evolve compliance
mechanism and provide feedback for decision-making.
Dissemination of the findings through learning events is proposed to ensure that available and
accumulated knowledge is fed back into an operational learning process that provides
appropriate and timely information to relevant stakeholders, including management, to allow
informed participation and on-going corrections/adjustments.
8. Learning’s and Adoptability:
The regular feedback of Process Monitoring Reports will provide online corrections in the
course of project implementation itself.
The regular dissemination of information both from Process Monitoring and Impact
Assessment if any would enhance the corrective options.
The thematic studies carried out at a larger scale discretely would support the policy
corrective measures to be adopted as a future strategy.
Annual learning workshop would be organized at each district level and at the project level
(State Level) to exchange the M&E findings arrived at through various monitoring and
assessment mechanisms.
The End of Project reports would lead to a State level Workshop disseminating the major
findings which would guide in developing new policy requirements if any.
Based on the experience of various States a National level workshop (Annual/Once in two
years) can be planned.
9. Envisaged Deliverables/Outputs:
The following outputs / reports will be envisaged during the project period, from the M&E Sector.
• Inception report
• Process monitoring reports (Monthly/Quarterly, Half yearly/ Annually)
• Baseline & Impact report (for the sampled watersheds) - Preparatory phase Evaluation report,
Work phase Evaluation report and Final Evaluation report.
• Input – Output reports (Annually)
• Thematic reports (Discrete)
• Case studies/Success stories
Monitoring & Evaluation (In Nutshell )
Discrete monitoring
Concurrent monitoring
Impact Assessment
Participatory
Monitoring
Independent M&L
(Frequency)
Community and
household level
Pre treatment
Baseline
Input Output
Monitoring
Process
Monitoring
Self Evaluation/
Assessment
Various levels
through MIS Data
Continuous
Monitoring
By CBOs
Yearly
Quarterly
(SWS-EC, AG & SHG)
facilitated by NGO
Mid –term impact
End of project
Post Project
In addition Thematic Studies, Case Studies and
other related studies (Special) if any
will be prepared as directed by VWDA
Conclusion:
 M&E is a part of the project not an outsider
 M&E is a facilitator of learning and is not for policing
 M&E is an effective management tool
 M&E ensures very high level of transparency
MEL &D is the eyes and ears of the project
How an M & E can be successful?
 Good Integration & Co operation between M&E & Project Management
 M&E Information should be unbiased, relevant, timely & accurate
 M&E documentation should address or resolve problems
 M&E staff should have good skills and experience
 Monitoring and Evaluation should be integrated – Single agency
 Project Evaluations and Assessments – at frequent intervals
 Information Dissemination – Upward & Downward
 M & E also plays a facilitator’s role for all stakeholders
 M&E should be an integral part of the project
“M&E is not - & cannot be - a substitute for good management. For M&E to be success –
it needs to be driven by manager’s need for information, their use of information and their
desire to create a learning environment”
Road ahead
• Put in place an independent M&E agency—Feb-2014
(ToR and Bid documents uploaded on DoLR Web site)
• Ensure the critical elements of Concurrent –Process monitoring in
real time
• Arrange M&E workshop at SLNA level (April 2014
• Realise importance of learning and policy linkages
• DoLR to contract directly Regional Independent M&E Agencies
Note the Change in the receptive level and Body language of DoLR“Reciprocate”
Organizational Structure
Proposed MEL&D Structure
HEAD QUARTER
AT PROJECT LEVEL
*A
*Project Director (M&E)
•
•
•
•
•
•
Natural Resource Scientist
Hydrologist
Social Scientist
Statistics Scientist
Remote sensing/GIS/IT Scientist
Documentation Specialist
District Coordinator
(1 for 3 districts)
Natural Sciences / Social Sciences
Field Staff (2 for each District)
Social Sciences
Natural Sciences
senior person among the scientists will be act as Project Director
ANNEXURE
–III
Multi-faceted Indicators of Impact at
Different Spatial Scales
(In Case of Soil Conserving Technologies)
Level
Indicator
Biophysical
Farm
Household

Rate of erosion

Soil fertility status

Food produced

Vegetation cover

Access to water and fuel

Crop yields

Quality of drinking water

Areas abandoned due to 
Quantity of drinking water
high erosion
Social
Economic
--

Fertilizer use

Rate of profits

Level of risk

Level of diversification
Awareness on environmental
aspects
Watershed

Slopes stabilized

Rainfall data

Rate of siltation

Quantity of water in reservoir

Area under tree cover

Rate of immigration

Conflict for access to land and
water

Income redistribution

Access to natural resources

Income level

Infrastructure network

Level of food security

Biodiversity level

Level of assets

Siltation cost
ANNEXURE – IV
A Format for Process Monitoring (Observation) Report - IWMP
Issues observed
CBOs Establishment &
Functioning
IWMP Implementation
Transparency
Capacity Building
IGA & Micro Enterprises
Situational Status
Reasons
(What is happening
in the field)
(Why it is
happening)
Suggestion / Scope
for improvement
(What is desirable)
Stakeholders to be
involved
ANNEXURE – V
EXPECTED / ESTIMATED OUTCOME
List of Expected/Estimated Outcome – to be compiled district wise
Sl. no
Particulars
Unit of measurement
1
Status of water table
2
Ground water structures repaired/ Numbers
rejuvenated
Quality of drinking water
Hardness
Availability of drinking water

Distance from the
house to source
(m)

Duration (months)
Increase in irrigation potential

Pumping hours

Yield (gallon/hr)
3
4
5
Depth in ft

6
Change in cropping/land use
pattern
7
8
9
10
11
Area under agriculture crop
i. Area under single crop
ii. Area under double crop
iii. Area under multiple crop
Net increase in crop production
area
Increase in area under vegetation Ha
Increase in area under
Ha
12
13
Area in ha (Crop
wise Shift from - to

Wasteland
reclaimed
Area in ha
Ha
Ha
Ha
Ha
Pre-project
status
(Baseline)
Expected
Post project status
Increased water table
(Specify in feet)
Increased
Reduced

Reduced

Increased


Increased pumping hrs
Increased yield
(gallon/hr)

Shift to Agroforestry/Horticulture

Wasteland reduced (area
in ha)
Increase (10-15%)
Increase (18-20%)
Increase (10-15%)
Increase (10-15%)
Increase (20-25%)
Increase (10-15%)
Increase (15-20%)
Actual
Outcome
ANNEXURE - VI
LOG MATRIX FOR KEY INDICATORS TO BE ASSESSED UNDER IWMP
This is only an indicative format for operationalization of project assessment through ME&L activities. However, this is not a final document; it can
be used as guide to conduct various activities under ME&L. An improvement on this matrix can be made as per the actual field requirements. (For
Neeranchal project additional indicators will be added and details would be worked out in the format after inputs from various
stakeholders)
Project Goals &
Objectives
Key Performance Indicators
Level of data collection
Methods
Frequency
Changes in household income - Area under Agri. Agri. Income / Ha
Household
Baseline (once) Survey,
Periodic Assessment
Annual
Increment in crop yield
Household/
micro watershed
Baseline (once) Survey,
Periodic Assessment
Crop Seasons
Household/
micro watershed
PRA Survey,
Measurement
Baseline and
End of the Project
Soil Erosion – Soil loss & Run-off
Household/
micro watershed
Field observation
PRA
Annual
Change in Cropping Pattern
Household/
micro watershed
PRA / RRA, Survey
Annual
Changes in milk, fuel wood & fodder production
Household/
micro watershed
Household Survey, FGD,
PRA Survey
Annual
Treatment of non-arable land - changes in forest
cover, forest produce, output from revenue land
Micro watershed
Sample Survey & RS&GIS Annual
Ground water recharges
1. Change in GW level & yield
2. Drinking water availability
Improve the
productive potential 3. Change in cropping intensity
4. Change in irrigated area
of watersheds
When You Have Measurement, You
Have PERFORMANCE
When You Have NO Measurement,
You Have EXCUSES
What Gets Measured, Gets
Managed Well