Transcript Methodology_Review

METHODOLOGY FOR THE
REVIEW/EVALUATION OF POLICY
DOCUMENTS
By Kwami DADJI, Health Officer HIV/AIDS, TB, Malaria & OID
African Union Commission
Presentation Outline
1. Introduction
2. Objectives of the Evaluation
3. Evaluation questions
4. Methodology
1. Introduction
• The main health policy documents of the African Union are
expiring on 2015.
• The final evaluation of the Abuja Call and for the Maputo Plan of
Action will be conducted to assess their level of implementation
by the UA Member States.
• The outcome will be presented to the Specialized Technical
Committee on Health in April 2015 and to the AU Assembly of
Heads of State and Government in June / July 2015.
2. Objectives
General objective
To document and assess the outcomes according to the set
objectives of the Abuja Call and the Maputo Plan of Action.
Specific objectives
• Assess the relevance (the global context) of the Abuja Call and the
Maputo Plan of Action;
• Assess the coherence of priority areas under the Policy documents
(Abuja Call and the Maputo Plan of Actions) and indicators to
measure them (nature of the indicators, ease of measurement,
tools, methods and means of collection, analysis, etc.);
• Assess the effectiveness of actions implemented in the various
countries in the strategic areas identified by the Policy documents ;
• Determine the level of achievement of key indicators in the context
of monitoring and evaluation;
2. Objectives
Specific objectives
• Assess the adequacy of the main indicators used in the monitoring and
evaluation framework with policy documents and national strategies;
• Analyze the partnership engaged at the country, regional and
continental level (type, functionality, advantages and disadvantage);
• Identify the strengths and weaknesses of the implementation of the
Policy documents ;
• Analyze the implementation of the recommendations of the mid-term
reviews;
• Propose a decision regarding the continuation, reorientation or
termination of implementing the Policy documents , taking in account
the results and recommendations of the review as well as the African
Common position on the Agenda beyond 2015;
• In case of continuation or redirection, provide consistent
recommendations for better implementation of the Abuja Call and the
Maputo Plan of Actions
3. Evaluation questions
• General Question
Abuja Call
• The key question that this final evaluation seeks to answer is:
what changes the implementation of the Abuja Call has
induced in the fight against HIV and AIDS, Tuberculosis,
Malaria and other infectious diseases in the AU Member
States?
Maputo Plan of Action
• The key question that this final evaluation seeks to answer is:
what changes the implementation of the Maputo Plan of
Action has induced in the improvement of the Sexual
Reproductive Health and the MNCH in the AU Member
States?
3. Evaluation questions
• Specific questions
Specific questions that will guide the overall work of the evaluation are as
follows
Abuja Call
- What is the relevance of the Abuja Call in the fight against the priority
diseases?
- What is the efficiency in the implementation of the Abuja Call in the 5
African regions?
- The expected outcomes were they produced? There were any unexpected
outcomes?
- What impact(s) in the medium term can be expected from the
implementation of the Abuja Call at country, regional and continental
levels?
-What is the added value of the Abuja Call in the fight against these priority
diseases, for example on health financing in Africa and the strengthening
of health systems?
- Should we continue with the priority areas proposed by the Abuja Call?
termination or reorientation?
3. Evaluation questions
• Specific questions
Specific questions that will guide the overall work of the evaluation are as
follows
Maputo Plan of Action
- What is the relevance of the Maputo Plan of Action in the improvement of
the Sexual Reproductive Health and the MNCH?
- What is the efficiency in the implementation of Maputo Plan of Action in
the 5 African regions?
- The expected outcomes were they produced? There were any unexpected
outcomes?
- What impact(s) in the medium term can be expected from the
implementation of the Maputo Plan of Action at country, regional and
continental levels?
-What is the added value of the Maputo Plan of Action in the improvement
of the Sexual Reproductive Health and the MNCH and the strengthening
of health systems?
- Should we continue with the priority areas proposed by the Maputo Plan
of Action? termination or reorientation?
4. Methodology
• Conducted by the Department of Social Affairs of the African
Union Commission (AUC)
• The approach to be adopted is based primarily on the analysis
of policy documents and commitment of the Member States,
the monitoring and evaluation Plan, activity reports, the midterm reports and any other documents relating to the
implementation
4. Methodology
The evaluation approach will thus structured around the
following activities:
– Technical meeting at the AUC involving key stakeholders,
to validate the methodology and tools for data and
information collection;
– Send data collection tools to members States and to
partners for filling and collection of data on indicators.
– Invitation of selected Member States to discuss the
relevance of selected indicators, measurement and
difficulties of providing information in real time;
– Analysis of data and information;
– Draft the evaluation report;
– Validation workshop to finalize report;
– Translation and submission of the report to the relevant
stakeholders
4. Methodology
Sampling
• After analyzing the mid-term evaluation reports, it will be
selected in the five African regions, 10 countries including five
which have the highest levels of key indicators and 5 which
have the worst levels of the indicators. The countries
representatives of the selected countries will be invited to
AUC to discuss the main issues related to the implementation
of the policy documents in their respective countries,
strengths and weaknesses.
• Moreover, the discussions will also address the challenges of
availability and sharing of information on key health indicators
to enable monitoring to achieve the Abuja Call.
4. Methodology
Data collection tools
Technique
Mixed: Quanti Qualitative
Tools
Questionnaire for country indicators
Guide for group discussion with partners
Qualitative
Guide for discussion with representatives of
countries invited to the meeting to exchange
with the AUC or by telephone interview,
Skype etc ...
4. Methodology
Data analysis
Data analysis will be done at three levels.
• The first level will be to analyze the relevance of policy
documents from an approach based on the qualitative
analysis of the situational analysis of health in Africa and the
framework of these documents.
• Regarding effectiveness in the implementation of the policy
documents, the analysis will be based on the measurement of
the level reached by the different Member States for the main
indicators used in their monitoring and evaluation plan.
• In a third step, an contained analysis will be made on
qualitative information gathered from key stakeholders
(partners and representatives of the countries)
I thank you