Transcript Slide 1
PACO III
Workshop: EPODE Population based interventions
A proposal of framework to evaluate
community-based obesity
prevention interventions in LAC
Marcia Erazo, Margarita Safdie, Mariana Lazo, Angel
Caballero, Richard Visser, Armando Barriguete
Background
• In Latin America and the Caribbean (LAC) the
obesity prevalence has raised at alarmingly
pace.
• Community-based interventions are the most
promising interventions because they can
reach the entire population.
• These interventions are recommended to
address the obesity causes rooted in the
society and special needs of the individuals.
Background
• Ideal community interventions combines
specific health promotion and evaluation
theory.
• In LAC, there is no community interventions
probably due to lack of evaluation
frameworks.
• Limited data are available to guide the
implementation and evaluation of these
interventions in LAC.
Rationale
• Standardized and comprehensive evaluation
frameworks are key for policy-makers,
scientists and stakeholders to evaluate
community-based programs
Objective
To thoroughly examine already available
evaluation frameworks (WHO, EPODE, and
Victoria proposals) and related literature, and to
provide a specific framework to evaluate
community-based interventions in LAC.
Key stakeholders
Policy
Makers
External
Groups
Other
Stakeholders
Practitioners
Public
WHO Steps Evaluation Health
promotion programs
1.
Describing the proposed program, policy or initiative
1.
2.
3.
2.
Identify key issues and concerns
1.
2.
3.
4.
5.
6.
7.
8.
Program logic
Identify group to undertake the evaluation
Baseline data
Key outcomes indicators
Stakeholders interests
Design: study, data collection
Data collection as planned
Data analysis
Recommendations
Dissemination of results
Take action
Page 55 (28)-56(29)
Wimbush and Watson.
Evaluation framework for health promotion
• Common themes in evaluation (Policy and
strategic planning)
– Outcomes and effectiveness
• Other important components (other
stakeholders):
– Improvement in quality of program
– Improvement in program design
Health Education Board Scotland
Tip of Iceberg
Wimbush and Watson.
Evaluation framework for health promotion
• Perspectives:
– Policy Makers and Strategic Planners: (make decisions and are
held accountable)
• What works?
• What are the best buys?
– Program Managers: (role: budget holders, responsible for the
delivery)
• Success, based on achievement of goals, extent of reacheness and
sustainability
– Practicioners: (role: operation and running of programs)
• Feedback from people involved in the collaborative action
– Community/Users:
• Avenue to provide feedback, involvement: express concerns,
satisfaction
• Extent to which program meets their perceived needs
Wimbush and Watson.
Evaluation framework for health promotion
• Stages Evaluation
– Planning
– Design and Pilot
– Implementation –early start up
– Implementation –establishment
– Implementation –fully operational
– Dissemination
Evaluation Framework for Health Promotion and Disease
Prevention Programs. Victoria, Australia
• Describe the program
– Program logic
• Evaluation preview:
– Process
– Outcomes
– Implications for Future programs and policy
• Focus evaluation design:
– Agreed or common indicators
• Collect Data
• Analyse and Interpret Data
• Dissemination of the results
EPODE
• By level:
–
–
–
–
Central
Local
Settings
Child
• To date, all EPODE programs have initiated continuous
monitoring and evaluation practices at central, local and
child levels, as well as in certain settings such as schools.
• Developing consistent practices and collecting comparable
data across communities and programs remains an
important challenge.
Obesity Rev 2011
Proposal of a framework to evaluate community-based
interventions to prevent child obesity LAC
Public Policy
Policy design
Policy process
Conceptual framework
Logic model
Objectives to achieve
Political concern
Focused on the electorate
Objectives not declared
Effectiveness
Rating
HIGH QUALITY
Citizen’s feelings and technically correct
Good governance
Open: Active communication, accessible language, trust between
citizens and the institutions.
Participation: Wide community participation during development,
Implementation and evaluation stages to achieve a public policy with
high quality, pertinence, efficacy.
Efficacy: Actions must achieve results as planned, evaluating their
future impact on the community and considering cumulative
experience.
Coherence: Policies and actions must be coherent and easy to
understand.
We need to evaluate both, Policy
design and its effectiveness and Policy
Process
Policy design and its effectiveness
Policy process
Through rigorous scientific
methods
Through key questions
That reflect the political
Environment and support
And Population satisfaction
Why to evaluate political process?
Politically strategic and democratic: Responsibility and
accountability (social, economic and technical).
Managerial: It helps to improve public management and the
quality of the program
Transparence: Strong mechanism to create “public value”.
Society needs to know the way the money is spent, the results
and the impact of the policies.
Participation: It strengthens the communities and individuals. It
also improves control and efficiency of public administration.
Complexity
Nationwide
Monitoring and
Surveillance
Public Policy
+++
Political:
Decisions
Agreements
Problem definition
Stakeholders
Plans
Programs
Objetives
Logic model
Projects/Interventions
Actions
Investigation
Public services
+
Policy design
WHO
Ireland
Australia
CDC
EPODE
Describe
proposed
program
Common themes
in evaluation
Describe
program
Engage stakeholders
Multilevel
Identify key
issues and
concerns
Engage other
stakeholders
Evaluation
preview
Describe program
Multi
stakeholders
Design study and
data collection
Consider different
perspectives
Evaluation
design
Evaluation design
Continuos
monitoring and
evaluation
Collect data
Consider different
stages of
evaluation
Collect data
Gather credible
evidence
Collecting
comparable data
accroos the
levels, settings
Analyse data
Justify conclusions
Use logic model
Dissemination
Ensure use and
share lessons
learned
Data analysis
Recomendations
Disemination
Take action
Key common issues
Proposal of a Comprehensive framework LAC
Planning
Implementing
Need / Problem
Normative
existance
Diagnose and design
Evaluation
Ex Ante
Population feeling
Community believes
Intervention
Objectives
Results
Impact
Beneficiaries/Society
Stakeholders Participation
Political commitment
Actions
Intermediate results
Implementation
Evaluation
Intermedia
Final results
Results evaluation
Health impact
Impact evaluation
Ex post
Ex Ante
Intermedia
Ex post
• To analyse and determine
if the intervention is
adequate to solve the
problem
• It is a critical analysis
about the
implementation and
early results.
It evaluates:
• It pretends to guarantee
that the intervention will
achieve the goals
• It analyses the
intervention validity, the
pertinence of the
objectives, the quality
of the managerial and
monitoring system.
• Its flexibility and its
capability to adapt
the actions according
to reality.
It must evaluate:
•
•
•
•
Needs
Program theory
Coverage
Potencial effects, results
and impact
• It encourages to
improve the
implementation in
order to achieve the
goals
• The successful of the
strategy.
• Its efficacy and
efficiency
• Managerial
mechanisms
Key questions
1. “The circumstances external to the implementing agency do not impose
crippling constraints”
2.“That adequate time and sufficient resources are made available to the
programme”
3. That the required combination of resources is actually available”
4.”That the policy to be implemented is based upon a valid theory of cause
and effect”
5.“That the relationship between cause and effect is direct and that there are
few if any, intervening links”
6.“That dependency relationships are minimal”
7.“That there is understanding of, and agreement on, objectives”
8.“That tasks are fully specified in correct sequence”
9.“That there is perfect communication and coordination”
10.“That those in authority can demand and obtain perfect compliance”