Transcript Monitoring and Evaluating the Food Security and Nutrition

Monitoring and Evaluating the Food
Security and Nutrition Effects of
Agricultural Projects
Anna Herforth - June 13, 2013 - LCIRAH
Nutrition Indicators in Agriculture
Survey:
Preliminary results
Anna Herforth and Terri Ballard
FAO consultants
Funded by the EU-FAO Improved Global Governance for
Hunger Reduction Programme (2012-2015)
Nutrition indicators in Agriculture
projects - Survey
• Aim: to understand how agriculture projects will
measure impact on nutrition: which indicators
are being used, how, and why
• Why?
– Researchers may be able to connect if desired;
informal technical support possible
– Discussions about the evidence base can be informed
about the current generation of new evidence
• How:
– Follow up on DFID-funded LCIRAH mapping study
• 50% of studies identified were applicable
• Excluded secondary data analysis, unspecified research, and
research with no agricultural intervention
Survey questions based on theory
• Key principles:
– Incorporate explicit nutrition objectives and indicators
into their design
– Maintain or improve the natural resource base
– Empower women
– Facilitate production diversification, and increase
production of nutritious foods
– Expand markets and market access for vulnerable
groups
– Incorporate nutrition promotion and education to
enhance the impact of production and income
– Collaborate and coordinate with other sectors
See: http://unscn.org/files/Agriculture-Nutrition-CoP/Agriculture-Nutrition_Key_recommendations.pdf
Preliminary Results: Response
• 77 project PIs
contacted
• 68 responded (88%)
– 3 of these excluded
– 2 did not complete
82% with complete
data
Respondents' Affiliations
NGO
28%
CGIAR
31%
University
35%
Foundation
UN
2%
4%
Preliminary Results: Indicators
Type of measure
Food consumption or diet
of these, specific varieties?
% measuring
Notes
almost all
Many measuring HDDS, WDDS, and
IDDS for kids; intake of specific foods
half (of above)
Biofortified varieties; natural varieties
Food security
most
HFIAS, HHS, seasonality, coping
strategies
Knowledge or behaviors
most
Specific to project
Women’s empowerment or
labor
half
Indicators somewhat unclear; a
couple using/testing WEAI
Economic outcomes
many
of these, 2/3 disaggregating by
gender
Nutritional status
many
stunting, underweight, BMI, anemia,
indicators of VAS
Link with water, health, or
sanitation
many
Natural resource management
some
Few indictors described; e.g. use of
soil and water conservation practices
Preliminary Results: Design
• Majority are measuring in a comparison
population
• Majority are collecting qualitative data
• Sample sizes range from 120 to 4000 (one
9000)
Early conclusions
• Focus is on nutrition impact among producers
• Total shift from previous generation of research
regarding measurement of diet quality
• Newly developed indicators get used
• High number of studies measure nutritional status,
but available sample sizes suggest low power
• Seems to be attention to program impact pathways
– Indicators chosen mostly because important to project
goals, or evaluates a specific aspect of project
– Interest in support for: “Adapting indicators to fit your
particular study activities and aims”
What these results represent
• This sample describes projects that set out to
affect nutrition
• It does not describe larger agriculture
programs or investments where nutrition is
not necessarily the primary goal
Monitoring and Evaluating the Food
Security and Nutrition Effects of
Agricultural Projects
F. James Levinson and Anna Herforth
Ag2Nut Community of Practice
http://knowledge-gateway.org/ag2nut
Purpose
Monitoring and evaluation of ag projects
seeking to address food security and nutrition
(in addition to their generally primary
production objectives) would be highly
desirable:
• there is, to date, so little empirical data
documenting successes and failures; and
• possible adverse effects need to be identified
and addressed rapidly within programs
Constraints
• Impact on food security on nutrition not
necessarily first priority of agricultural
managers
• Not familiar with how to measure
• M&E in general is often not very strong
– FAO/World Bank study (2010) of M&E in agriculture
projects in Asia, Africa and Latin America
Common shortcomings of Ag M&E
• M&E is often perceived as an externally imposed
obligation with findings seldom integrated into
management and action systems
• Ag managers complain of unmanageable data
collection and reporting demands
• What M&E is carried out gives primary attention to
physical achievements to the neglect of project
outcomes
• Inadequate institutional capacity consistently limit
M&E
(All of these, of course, are problems common to M&E in
development projects more generally.)
A role for external Ag-Nut M&E teams?
• Roles
– identify sensible indicators to measure nutritionrelevant impact based on the type of activities in
the program
– carry out the key M&E necessary for tracking
progress
– feed back to the program management
(monitoring)
– Understand reasons for impact or lack thereof
(eval)
– could support nutrition-sensitive program design
or adjustment
Sentinel Sites
Geographically representative sentinel sites
within the overall project area are one approach
to M&E in ag projects with Ag2Nut interests
Baseline data would be followed by the
collection of quantitative and qualitative data at
6 month intervals from these sites, and from
comparable sites in non-project areas.
Examples of data
• Extent to which households have been
reached/affected by the project
• Relevant outputs specific to project (e.g. certain
crops/foods consumed)
• Household food insecurity levels
• Dietary quality
• Women’s empowerment
• If it makes sense, nutritional status among young
children and women
All complemented by qualitative data to better
understand the dynamics of project effects
Identifying harmful food security or
nutrition effects
• Employment levels have remained static or
deteriorated;
• Small producers have been excluded;
• Household food insecurity has deteriorated (overall or
seasonally);
• Intra-household equity of income has declined;
• The labor burden of women has increased;
• Debt burden has increased;
• In irrigation/water use projects, changes in waterborne diseases;
• In livestock projects, changes in zoonotic disease
• Harm to natural resources (particularly water, soil,
biodiversity)
Data of particular interest to project
managers
Among possible indicators:
• Access, use and satisfaction with services
provided under the project
• Changes in farmer assets
• Percentage of households considering
themselves better off now than 12 months ago
• Percentage of the labor force underemployed
or unemployed
Conditions
Such sentinel site data collection is likely to be
useful if:
•Good quality data can be sensibly aggregated and
presented to project management in timely fashion
– Challenge to collect data that are meaningful enough
to be useful, and brief enough to be usable
•Information indicating harmful effects or
shortcomings in project implementation – will be
addressed by project management
How would it be supported?
Adequate staff and funding:
• There is a need to identify Ag-Nut M&E teams
capable of participating actively in an initial
stream of nutrition-sensitive agriculture
projects.
• External funds probably necessary
In Sum
Through the creative use of separately managed
sentinel site-based M&E, it should be possible
to:
• Generate cooperative efforts
• Generate much needed data
• Document successes in nutrition-sensitive
agriculture
• Develop prototypes, training modules and TA
mechanisms for subsequent use
Building the evidence base
• Need more examples of successful programs
– Not just nutrition outcomes, but win-wins with
other goals
• Ultimately, what is it that we want to scale
up?
– Probably not individual programs
– Rather, principles that explain how individual
programs have positive impact
– Evaluations need to offer generalizable lessons
learned
Discussion
Ag2Nut Community of Practice http://knowledge-gateway.org/ag2nut