Transcript Monitoring and Evaluation, Poverty Assessment, Targets and

Monitoring and Evaluation,
Poverty Assessment, Targets and
Indicators - An Introduction to the
Challenges and Issues in the PRSP
Chris Pain - June 2002
Consultant, GTZ Africa Department, Policy Section
Joint Donor Staff Training Activity - Partnership for Poverty Reduction
Tanzania, June 17-19, 2002
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
1
Presentation Structure
Monitoring
the PRS
(Selection of
Indicators)
Poverty
Assessment
and Prioritisation
Monitoring and Evaluation Challenges and Issues in the PRS
Capacity
Building for
Monitoring
5/19/2016
Dissemination of
the Monitoring
Results
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
Participation
in PRS
Monitoring
2
Introduction - The PRS Implementation Cycle
PRSP Formulation
Poverty Assessment,
Prioritisation and C osting
E
Implementation
V
- M TEF / Bu dg e t a nd Pub li c Exp en di tu re
A PRS
- Po lic y Re fo rm
(Ma cro co no mic , Stru ctu ral a nd Se ctora l)
L Revisio n
- Pu bl ic Acti on C h oic es
Process
U
-Pu bl ic Se vic es
(q ua li ty , a vai la bi li ty, a cce ss, satis fa ctio n)
A
-Inv estme nt Pro g ram mes
T
- Su b-n ati on al p ol itic s
I
O
N • PRS Policy Review
• Poverty Analysis
• Strategy Adaptation
End Year 3
5/19/2016
M
O
N
I
Progress T
Review
O
End Year 1
R
I
N
G
Progress Review
End Year 2
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
3
Poverty Assessment and
Prioritisation
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
4
Poverty Assessments
 Poverty - Dominant Consensus on what the concept means - not just
in international definitions, but reflected in PRSs.

This prompts the question whether specific national determinants of
poverty are being overlooked
 Poverty Assessments - diagnostic processes that identify the extent,
causes and trends in poverty
 An important mechanism for turning poverty issues into policy


but to do this requires an understanding and acceptance of who the poor
are and what poverty is
the determinants of poverty need to be specified for particular groups
and regions (Disaggregation)
 Contribute to assessing the effectiveness of current policies and
whether the poverty situation is changing
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
5
Poverty Assessments
 Progress towards better poverty data is being made
but PAs need to





establish policy relevant groups of households and be clear in
terms of the actual interventions which would assist these groups
link specific socio-economic groups to the growth and
development process
identify the relationship between poverty and household
demographic characteristics
improve on the ex ante and ex post poverty and social impact
assessment of policy interventions
improve the quality of the data they use
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
6
Poverty Assessments - PSIA
 The increased attention to Poverty Reduction highlights the need for an
understanding of how policies affect the poor
 A PSIA needs to take a multi-disciplinary approach


use economic and social tools
use quantitative and qualitative methods
 Weaknesses in utilising a PSIA in the design of government policy




PSIA is not undertaken early enough to inform the design of policy
The analysis is not made public
The risks of policy implementation are not generally addressed
Potential losers are not explicitly identified
 Challenges in Implementing a PSIA




Data Constraints
Analytical Constraints
Capacity Constraints
Time Constraints
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
7
Poverty Assessment and Prioritisation
 A linear approach to policy making suggests after the Poverty
Assessment priorities can be established - Evidence Suggest this is not
the case
 Prioritisation is a complex issue involving more than just “good
information” - it depends on



The particular version of poverty that is used
Whose voice is heard
There is a strong political element to such negotiations. An important
lesson is that Politics Matters
 Lack of Prioritisation leads to Wish Lists that are expressed in broad
and evasive terms

There is also a need to prioritise within sectors as well as between them
 Participatory Process can help with this - some evidence to suggest that
this is being done (eg Mozambique and Tanzania)
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
8
Poverty Assessment, Consultations and Priority
Setting - Proposed Sequence
 Disaggregation of “the poor” into specific categories
 Analysis of information based on the identified categories
 Analysis of past and present poverty reduction programmes with
regard to who has been reached
 Give Priority to specific categories of households
 Develop Specific Strategies for each prioritised category of
household
 Reconcile priority setting with available resources (cost the
options)
 Establish targeting mechanisms to ensure the prioritised
categories of households are reached
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
9
Monitoring the
Poverty Reduction Strategy
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
10
Monitoring the PRS
 Why Monitor
Track Progress in achieving the poverty reduction goals
Helps to validate the choices made in the first place, justifying government action
to the public
Reveal reasons for success or failure, allowing effective management of the
strategy and improvements to be made
Mobilises and Sustains Public Support for the Targets
Offers the opportunity for greater involvement of Civil Society in the process
Acts as a means of accountability in the use of resources, generating transparency
 In general, monitoring allows for the opportunity to have quality management
within the PRS
 To Date Monitoring is the Poor Relation in the PRS - it is generally the last
thing to be addressed
 Poverty Monitoring is not the same as PRS Monitoring
Poverty Monitoring Generally Refers to final indicators
PRS Monitoring is much broader - including expenditure and outputs
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
11
Monitoring the PRS - Some General Concepts



Goals the objectives a country or a society want to achieve
Indicators the variables used to measure progress towards the
goals
Targets the quantified level of the indicators that a country wants
to achieve in a given time frame.
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
12
Monitoring the PRS - Long Term Indicators - The MDGs
Goals
Targets
Indicator
Eradicate Ext reme Poverty
and Hunger
1) Halve, between 1990and 2015, the proportion of
people whose income is less than one dollar a day
2) Halve, between 1990 and 2015, the proportion of
people who suffer fro m hunger
1)
2)
3)
4)
5)
Achieve universal primary
education
3) Ensure that by 2015, children everywhere, boys and
girls alike will be able to co mplete a full course of
primary schooling
6)
7)
8)
Pro mote Gender Equality
and Empowerment
Women
4)
9)
Reduce Ch ild Mortality
5)
Reduce by 2/3, between 1990 and 015, the under-five
mortality rate
12)
13)
14)
15)
Improve Maternal Health
6)
The rate of maternal mortality should be reduced by
¾ between 1990 and 2015
16) Maternal Mortality Ratio
17) Births Attended by Skilled Health Personnel
Co mbat HIV/Aids,
Malaria and other
Diseases
7)
Have halted by 2015, and begun to reverse, the spread
of HIV/Aids
Have halted by 2015 and begun to reverse, the
incidence of malaria and other major diseases
18)
19)
20)
21)
22)
5/19/2016
8)
Eliminate gender Disparity in primary and secondary
education, preferably by 2005 and to all other levels
of education no later than 2015
10)
11)
Population living below $1 a day
Poverty Gap Rat io
Poorest 20% share of National Consumption
Prevalence of Underweight Under 5s
Population below min imu m level of dietary energy
consumption
Net enrolment in Primary Education
Co mplet ion of 4th Grade of Primary Education
Literacy Rate of 15 to 24 year o lds
Ratio of girls to boys in Primary, Secondary and Tertiary
Education
Ratio of literate females to males (15 to 24 year olds)
Share of wo men in wage-emp loyment in the non-agricultural
sector
Proportion of seats held by women in national parliament
Infant Mortality Rate
Under 5 Mortality Rate
Proportion of one year old children immun ised against measles
Contraceptive Prevalence Rate
HIV prevalence rate in 15 to 24 year old pregnant wo men
Nu mber of ch ild ren orphaned by HIV/Aids
Prevalence and death rates associated with malaria
Proportion of population in malaria risk areas using effective
prevention and treatment measures
23) Prevalence and death rates associate with TB
24) Proportion of TB cases detected and cured under DOTS
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
13
Monitoring the PRS - Long Term Indicators - The MDGs
Ensure Environ mental
Sustainability
9)
Integrate the principles of sustainable development
into country policies and programmes and reverse the
loss of environmental resources
10) Halve, by 2015, the proportion of people without
sustainable access to safe drinking water
11) By 2020, to have achieved a significant improvement
in the lives of at least 100 million slum dwellers
25)
26)
27)
28)
29)
Proportion of land area covered by forest
Land area protected to maintain bio logical diversity
GDP per unit o f energy use
Carbon Dio xide emissions per capita
Proportion of Population with sustained access to an
improved water source
30) Proportion of people with access to improved
sanitation
31) Proportion of people with access to secure tenure
Develop a Global Partnership for
Develop ment
12) Develop further an open, rule based, predictable, nondiscriminatory trading and financial system
13) Address the special needs of the least developed
countries
14) Address the special needs to landlocked countries and
small island developing states
15) Deal comp rehensively with the debt problems of
developing countries through national and
international measures in order to make debts
sustainable in the long run
16) In cooperation with developing countries develop and
implement strategies for decent and productive work
for youth
17) In cooperation with pharmaceutical co mpanies
provide access to affordable, essential drugs in
developing countries
18) In cooperation with the private sector, make availab le
the benefits if new technologies, especially
informat ion and communicat ions
32)
33)
34)
35)
36)
37)
38)
39)
40)
41)
42)
43)
44)
45)
46)
47)
48)
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
Net ODA as % of DAC Donors GNI
Proportion of ODA to basic social services
Proportion of ODA that is untied
Proportion of ODA for environ ment in s mall island
developing states
Proportion of ODA for transport sector in land-locked
countries
Proportion of exports admitted free of duties and
quotas
Average tariffs and quotas on agricultural products
and textiles and clothing
Do mestic and export agricu ltural subsidies in OECD
countries
Proportion of ODA provided to help build trade
capacity
Proportion of bilateral HIPC debt cancelled
Debt service as a percentage of exports of goods and
services
Proportion of ODA provided as debt relief
Nu mber of countries receiving HIPC debt relief
Unemploy ment rates of 15-24 year olds
Proportion of population with access to affordable
essential drugs on a sustainable basis
Telephone lines per 1000 people
Personal Co mputers per 1000 people
14
Monitoring the PRS - Types of Indicators
Intermediate
Indicators
Final
Indicators
Input
Indicators
Financial and Physical Indicators
(Including Personnel) of
Resources Used
Output
Indicators
The Intermediate Goods
and Services Generated
Outcome
Indicators
Impact
Indicators
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
Access and Use of
Goods and Services
and Satisfaction of Beneficiaries
Effect on Key Dimensions
of Well-Being
(Improvements in
Living Standards)
15
Monitoring the PRS - When is it Useful to
Use These Indicators
Intermediate
Indicators
Final
Indicators
5/19/2016
Input
Indicators
Useful in the Short Run Source of Information Budget Documents and
Actual Expenditure Data
Output
Indicators
Useful in Short and
Medium Term
Source of Info - Admin
Systems, MIS
Money Allocated to
Primary Education
Number of
Schools Built
Outcome
Indicators
Useful in Medium to
Long Terms
Source - Priority and Quick
Surveys,PPAs
Enrolment Rate
of Girls
Impact
Indicators
Useful in Long Run Source - Household Income
Surveys
Literacy Level
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
16
Monitoring PRS Implementation - A Framework
Policy Reforms
Public Action
Choices
Public
Services
Investment
Programs
Institutional
Capacity
Building
Inputs
Outputs
Outcome
Poverty Monitoring
Impact
5/19/2016
(MDGs, other sectoral outcomes)
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
17
Monitoring the PRS - Experience to Date in
Establishing Indicators
 Long Term Targets Have Been Adopted
 But the inclusion of Good Intermediate Indicators has been
neglected
 Why has this been the Case



Agreeing what happens in the middle is always the hardest
They are less contentious
The emphasis placed on them in the documentation may have encouraged this
 It is now very important to address the issue of the Missing Middle
to assure the quality management of the PRS


Clear link between agreed targets and activities - what is needed is not a
greater general emphasis on some particular point on the chain from inputs to
outputs but greater linkages along the chain
Make it useful and worthwhile to produce and use the information - provide
incentives to use the information, eg better access to resources
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
18
Monitoring the PRS - Other Issues Identified to Date
 In general the PRSPs contain too many indicators



This means it would be difficult to monitor.
PRSPs need to contain a small number of realistic core indicators and
targets
Prioritisation of Interventions will lead to prioritisation of indicators - needs
to involve stakeholders
 The Indicators developed to date are linked to the MDGs


They need to be disaggregated to take account of regional aspects
They also need to be more realistic in view of prior achievements and
available resources
 Over Reliance in the Indicators of monetary poverty reduction and
education and health

Now Need to incorporate issues of governance, natural resource
management, security and empowerment
 Emphasis on technical process of poverty data collection and analysis
focus on poverty monitoring outcome and impact level

Statistical information gathering and analysis - risk of not tying into policy /
decision making
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
19
Monitoring the PRS - How is the Information Collected
 Traditional Reliance on IHS and WMS supplemented with PPAs

These are bulky, take time and tend to focus on final poverty outcomes
 Fortunately there is a large range of survey and non-survey instruments
available 


Budget Monitoring includes improving the quality of expenditure tracking,
developing medium term perspectives to budget making (such as MTEFs),
focusing on performance and promoting greater participation
Management Information Systems offer a huge potential but have an
incentive to distort the results and are not trained data collectors
Alternative Data Collection Methods particularly relevant to get a quick
feedback
 CWIQ - Mali, Mozambique, Malawi
 Report Cards - Phillipines and India
 In the PRS, each have their own strengths and weaknesses. They need
to be integrated in a manner that allows them draw on each other.
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
20
Monitoring the PRS - Impact Evaluation
 Evaluation is complementary to Monitoring - coming at the end of the life
of a PRS it is important to judge which public actions have been effective
and which have not worked
 Used to Inform decisions on expansion, modification or elimination of a
particular policy or programme.
 Key Questions in an Impact Evaluation






Have the interventions achieved their goals
Can changes in the Poverty Situation be attributed to the interventions or have
other factors caused the changes
What have been the impacts on different groups - men, women, various
ethnic groups
Have there been unintended side effects
How effective are the programmes when compared to alternatives
Are the interventions worth the cost
 Data Requirements

5/19/2016
Different Sources - Qualitative and Quantitative
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
21
Participation in
PRS Monitoring
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
22
Participation in Monitoring - Experience of
Participation to date in the PRS Framework
 so far efforts focussed on participation in the formulation of
the strategies
 ahead: how to


make it more organic
maintain participation in implementation and in M&E?
Participation in PRS Monitoring


a building block to institutionalize participation
… how stakeholders share influence and control
over the monitoring process, its results and the
subsequent policy and decision making
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
23
Participation in PRS Monitoring - Why?





moving out of the exclusive circle of MoF and some sector Ministries
bring PRS and its implementation into public domain/ public debate
amplify voice and agency of the weak and usually unheard
increase accountability and transparency of public actions
Increased public awareness by demystifying policies, budgets

contribution to more inclusive public policy debate
 Better and more complete information for decision making



direct feedback from citizen
consultation with multiple perspectives
representation of interests (winners/ losers)
 Greater transparency and public accountability
 Contribute to performance and client orientation of public sector
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
24
Participation in PRS Monitoring - Where Does it Fit?
Policy Reforms
Public Action
Choices
Public
Services
Investment
Programs
Institutional
Capacity
Building
Inputs
Participatory Expenditure Tracking
Outputs
Citizen Report Cards
Outcome
Impact
5/19/2016
Qualitative Policy
Impact Monitoring,
PPA’s
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
25
Disseminating the Results
of PRS Monitoring
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
26
Disseminating the Results of Monitoring - Why?
 Providing Information on the Results of Exercises and Progress
towards achieving Goals and Targets can mobilise and sustain
public support for the targets
 Results that are not disseminated and targeted at specific groups
in civil society will not be used.

Resources used in generating information that is not used are wasted.
 For Civil Society to Participate Effectively in Issues of the PRSP
they must have access to information

Provision of the results to Civil Society improves the general feeling of
Transparency and Accountability
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
27
Disseminating the Results - How?
 Have Mechanisms tailored to different PRS Stakeholders






policy makers
parliamentarians
programme managers
civil society organisations
general public
Donors
 Utilise the Media to Stimulate Public Debate and enhance
accountability of development agencies
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
28
Disseminating the Results - Generating Demand
 Common Complaint in a Number of Countries that the
Demand for Information is Generated by Donor Agencies
 Need to generate domestic demand for the results


Amongst CSOs - to create pressure for policy change and
enhanced transparency
Amongst Government Ministries - Tie Budget Allocation to
Performance by Adopting Target Oriented MTEFs
 It is only once the results of monitoring initiatives are converted
into the public domain can it be expected that action will be
taken and therefore can it be said that monitoring has influenced
decision making.
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
29
Capacity Building for
PRS Monitoring
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
30
Capacity Building for PRS Monitoring
The quality of country systems for measuring and monitoring results is
important for the challenge before us. This puts a premium on our
capacity building support for public sector management, statistical
development, and monitoring and evaluation systems, which are
important in their own right for underpinning countries accountabilities
to their people and results agreements and compacts with donors
Joint Statement on “Measuring, Monitoring and Managing for Development Results”
made by the heads of five development banks at the
UN International Conference on Financing for Development
The Key Questions
 What is the Current Capacity to Monitor the PRS (Where are we?)
 What is to be achieved through “Capacity Building” (Where do we
want to be?)
 What is to be done to improve the capacity (How do we get there?)
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
31
Capacity Building for PRS Monitoring Experience to Date
 PRSs generally identify the need for capacity development to assist in the
Implementation of the M&E System.



It is not clear to what extent assessment of the capacity to monitor the PRS
have been carried out
Nor is it made particularly clear where we want to be with this capacity.
They do not assess why capacity previously developed is no longer
available.
 Further, Capacity for Whom needs to be addressed - must include
elements of the public sector, civil society and the private sector have to
be included.

Monitoring of the PRS involves the participation of a number of agencies
inside and outside government.
 PARIS-21 initiative established in 1999 to promote a culture of
“evidence based policy-making and monitoring in all countries”
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
32
PRS monitoring - Implications for Donors
 Objective of the OECD Task Force on Donor Practices Alleviate the cost for partner countries of meeting multiple donor reporting
and monitoring requirements
 The Monitoring System and Review Process should meet the
monitoring and reporting needs of both the Government and
Donors

A strong Monitoring System will help to create transparency
 Some movement towards this, but there are constraints still
Fear attached to subordinating individual reporting requirements to a
common framework
The need to reduce their own reporting and monitoring requirements to the
necessary minimum
National Monitoring Systems often lack capacity - this needs to be built

5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
33
Thank you for Your Time and Attention
Any Questions?
5/19/2016
M&E, Poverty Assessment, Targets
and Indicators in the PRSP
34