Transcript Slide 1
Planning Protocols and
Procedures
(Monitoring and Evaluation
Updating Plan)
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Learning objectives
By the end of this session, you should be
able to:
Discuss basic principles of evaluation
Identify key lessons learned for health
emergency managers from recent disasters
Identify areas for action in HEM capacity
development
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
How can we Define Evaluation? (1)
The classic perspective:
o Concerned with the achievement of objectives
The broad perspective:
o Achievement of objectives is a key, but it is
only part of what an evaluation might be
concerned with
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
How can we Define Evaluation? (2)
Key elements are:
The need for systematic collection of
information
The wide range of topics to which evaluation
can be applied
To be effective, the evaluation results has to be
used by someone
The wide variety of purposes of evaluations
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
What’s the Difference between
Research & Evaluation?
Research & evaluations use the same
toolbox
However, for a different purpose
• Research aims to prove…
• Evaluation aims to improve…
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Why do we do evaluations?
It’s all about interventions or programs to:
Inform planning
Define progress
Examine efficiency
Examine effectiveness or achievement
Inform decision-making
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
How Could you Find out the Purpose
of an Evaluation?
Who asked for evaluation?
Who pays for the evaluation?
Why do these people want an evaluation?
What are the decisions that need to be made?
What information is required to facilitate decision
making?
Who is going to be affected by evaluation
outcomes?
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Classifying Evaluation Purposes
Needs assessments
Before planning to understand context & needs
Program monitoring
Compliance with policy/plan
Validity of assumptions & pre-conditions
Formative evaluation
Efficiency
Summative evaluation
Effectiveness
Decision-making
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Evaluation Logic
Purpose
Why do we do this?
Questions
What do we want to know?
Objectives
What information is needed?
Methods
The way
information
Results
Processed data or information
Conclusions
Answers to questions
Recommendations
Suggestions for decision-making
we
collect/analyze
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
What Methods can you Think of to
Collect Information?
Documentation research
Observations
Surveys interviews
Individual interviews
In-depth interviews
Focus group discussions
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Quantitative Data Sources & Methods
Conditions within
samples of a
subgroup
Documented information
on population groups
Indirect measures
Observation
Survey interviews
Characteristics of samples
of a community or
subgroup
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Qualitative Data Sources & Methods
Behaviour
Community perspective
normative view
Observation
Focus groups
In-depth
interviews
Key informants
In-depth knowledge
Individual
interviews
Indirect
measures
Individual perspective Documented
intra-cultural variation information
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
About Methods
Questions will tell you what information you need
Information needed will tell you what methods
could be used
Local context & resource limits will tell you what
methods are feasible
Keep it as simple as possible
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
In Summary
To find out something useful about an intervention or
program, use whatever you have in your toolbox that will
get the job done…
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Small Group Discussions
Based on personal experience, what are the
lessons learned in terms of HEM?
Compare experiences & build consensus on
common lessons learned
List on a flipchart your group results
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
SYNOPSIS OF EVALUATIONS ON
THE HEALTH EMERGENCY RESPONSE TO SEA
REGION TSUNAMI
KEY LESSONS LEARNED FOR HEM
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Sources
The Phuket Conference Papers (2005)
Tsunami Evaluation Coalition (2005)
The WHO-SEARO Tsunami evaluation (2006)
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Lessons Learned in Context of HEM
• Early warning
systems
• Incident command
system
• Health info system
• Resource management
• Capacity assessment
• Risk communication
• Risk reduction
• Donation management
• Policy
• Operations management
• Needs assessment
• Recovery &
reconstruction
• Emergency planning
• Capacity development
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Early Warning Systems
Coordination
developed
structures
need
to
be
Flexible surveillance systems need to be
promoted
Surveillance
systems
need
development in terms of local
building:
further
capacity
Personnel
Infrastructure
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Health Information System
Management
Surveillance systems focused too exclusively on CD
Need to develop surveillance systems that are
capable of covering all aspects of health
Surveillance and monitoring must be strengthened
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Capacity Assessment
Private & other sectors should be involved in
disaster reduction
Preparedness,
mitigation
measures
&
capacity building are more cost-effective than
enhancing damage response
Vulnerability indicators need to be defined
I.e. safe hospitals
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Risk Reduction
The importance of risk reduction need to be
advocated
Health sector & facilities’ ability to recover from
disaster stresses need to be improved
There is a need to develop vulnerability indicators
Integrate mitigation activities into the recovery
process
PHC staff should receive training in disaster risk
reduction
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Policy
Countries that did relatively well in HEM in terms
of national & international coordination had:
Well-developed policies & systems
Strong health systems
Clear lines of authority
Problems encountered post tsunami are not
new:
Policy without enforcement is powerless
Coordination policy-practice too often doesn’t happen
Relevant indicators on effectiveness, efficiency,
costs, & benefits of both preparedness &
response are needed
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Needs Assessment-1
A pragmatic approach to needs assessment
Assessment must be relevant to decision making
Assessments must be timely
Invest in national assessment capacity
Strengthen the capacity of national and local
authorities to carry out cross-sector needs
assessments
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Needs Assessment-2
The need to streamline the international
assessment mechanism
Dysfunctional
Competitive
Not shared
The need for a common information system
of beneficiaries
Mainly statistics
It does not tell who needs what
It does not facilitate matching needs & offers
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Emergency Planning & Response-1
Disaster preparedness is crucial
The quality of early responses is affected by the
degree of preparedness
Lack of effective coordination: most common
complaint
Internal
External
Between policy & practice
Across sectors & agencies
Information management
Reporting formats should facilitate decision-making
Media relationships including pre-disaster
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Emergency Planning & Response-2
Flash appeals: a solution causing problems…
Disaster preparedness must include training on
Flash Appeal
Timing of funding was a problem
Imbalanced demands: curative care & public
health
Funding worsening inequities
Consider psychosocial support mechanisms
Health system deficiencies
Alien expertise
Community-based mechanisms
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Incident Command System Operations
A 24-hour contingency response system should
be in place
WHO internal relationships in response to the
Tsunami raised questions:
Coordination
Sharing of information
Chains of decision making
First teams in the field should always be
composed of experienced staff and not
consultants
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Resource Management-1
Recruitment & briefing of consultants calls
for care
There is scope to strengthen regional &
national financial resource mobilization
capacity including:
Disaster preparedness
Mitigation
Response
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Resource Management-2
WHO should assure its leadership in public health in
its relations with UN agencies and NGOs
MOH should assure its leadership in public health in
its relations with other ministries and international
donors
Collaboration with the military is important and
should be fostered
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Risk Communication
Working with the media is important so that
reporting is:
Technically sound
Epidemiologically sound
We need to enhance the relationship with the
media through:
Training of HEM managers
Regular briefing of media
Sharing some of the lessons learned
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Donation Management
Make international response more relevant
to needs assessment
Humanitarian
industry
response
is
an
unregulated
Setting priorities and distribution of tasks among
actors requires significant improvement
Assessing needs is pointless if decision making is
not reformed
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Operations Management
Standard Operating Procedures (SOPs) are essential
and need to be developed & reviewed
Situation reports need to be standardized and
regular
Logistics, supplies & donations can be troublesome
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Recovery & Reconstruction-1
Planning & preparedness can reduce the impact of
disasters
There is a need for a strong public health
infrastructure
Planning and coordination
reconstruction is essential
of
recovery
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
&
Recovery & Reconstruction-2
Early responses do affect long term system
functioning with consequences for health services
Researchers & implementers need to speak the
same language
Relationships between national & local governments
and international agencies are crucial for the course
of development
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Capacity Development-1
Provide leadership
Strengthening organizational & management
capacities
Human resource development:
International & national level staff are the sole
barrier to improved health sector capacity
Not by quick fix short courses
Incorporate “Health Management in Disasters” in
curricula of all relevant disciplines
Not HEM but HM knowing emergencies
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Capacity Development-2
Develop partnerships & networks
Professional associations
Multi-sectoral
National
Regional
Inter-regional
Develop & share knowledge
Case studies
Research
Continuous learning
HEM integrated in health curricula
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Putting it together-1
Build capacity in risk management & vulnerability
reduction
Policy & legislation development
Develop disaster management lines of authority
& control
Allocate resources for risk mgt & vulnerability
reduction
Address the need for improved needs assessments
Relevant to decision-making
Multi-sectoral
Population-based
Inclusive
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Putting it together-2
Enforce public health practice:
Psycho-social support
Gender equity
Child health
Management of the dead & missing
Involvement of health volunteers
IEC for watsan
Develop benchmarks & standards of practice
Vulnerability indicators
Surveillance systems
Needs assessments
Indicators on effectiveness, efficiency, costs, &
benefits of both preparedness & response
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Putting it together-3
Improve management & coordination of
responses
Internal
External assistance
Policy-practice
Develop logistics of supplies
Legislation
Relevance
Capacity building
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Putting it together-4
Improve donation management
Timely
Sustainable
Relevant
Flexible
Foster contribution of other sectors
Civil-military liaisons
Public-private sector liaisons
NGOs’ inclusion not marginalization
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Putting it together-5
Develop media relations
IEC prior and during disasters
Combat disaster myths
Guidance on media relations
Developing capacity
HEM leadership
HEM capacity
Human resources
Networking & partnerships
Knowledge development
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Capacity Development in HEM
Strengthening
Development
Evaluation
& Research
Capacity
Building
Awareness
Advocacy
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman
Plenary Discussion
At the end of the day…considering:
The roles of HEM in disasters and
The lessons learned from disasters
How would describe the strengths &
weaknesses in terms of HEM for your
country?
First National Course on Public Health Emergency Management
12 – 23 March 2011. Muscat, Oman