Transcript Development Monitoring Data base

Results Based Monitoring
and Evaluation System
T.M.Jayasekera
B Sc. Eng, C Eng,
MBA,FIE,FIM,FCIWEM,MICE,MSLIM,MSLITAD
National Consultant
UNDP TA on Strengthening Plan Implementation
Capability
Ministry of Plan Implementation .
Challenges Confronting Society
in 21st Century
Development
Challenges
Social
Challenges
Poverty, Hunger, Malnutrition, Child Mortality,
Maternal Mortality,
Degrading Environment
Braking families increasing
crimes Drug abuse,
child and spouse abuse,
teenage pregnancy
Alternatives Programmes designed
What is
Working
Poorly
What is
more
effective
to attack the problems
Scarce Resource –
Tough
Choices :- Cancel some
programmes to continue others
Programmes Evaluation
What is
Working
Well
What is
less
effective
Traditional Project Monitoring System to speedup Implementation
Implementation Failures

Low Foreign Aid Disbursement
Disbursement Ratio
 Sri Lanka
 China
 Thailand
 Vietnam





13.12% (Target 20%)
29.82%
37.14%
17.87%
(ADB 2004)
Sick Projects/ Problems Projects
Time Overrun
Loan Cancellations
Monitorable Work Plan with Financial / Physical target
Monitor Financial and Physical Progress (formats)
Identify short falls / Gaps – issues – Remedial Action.
Early warning Signal – Alerts
M&E Mechanism – LM Meetings – Back to Office Report – PMC
Foreign Aid Utilization – 2003 – 14%
2004 – 20%
Implementation Progress
alone not sufficient:
Project Failures and Declining
“Results”
US
9%
GS
44%
PS
47%
US
PS
GS
ADB Projects 2004
Concept of RBME

Tracks performance of a projects
based on
 Implementation Progress (Physical
/ financial)
 Development Objectives (Benefits
/ effects and Impacts)
Input
Outputs
Effect
Impact
Implementation Progress may not
necessarily lead to satisfactory
achievement of development
objectives
 Results Management Culture

The Logical Framework
LFA is an analytical tool for objective oriented project
planning and management including Monitoring and
Evaluation. It is a way of structuring the main
elements of a project, highlighting logical linkages
between inputs, planned activities and expected
results. In the development process
If the inputs are available then activities will take
place.
If activities take place then the outputs would be
produced.
If the outputs are produced then the purpose will be
achieved.
If purpose is achieved, it will contribute to the
fulfillment of goal.
Input
Activities
Output
Purpose
Goal
Project Logic
Implementation
Input
Men
Maternal
Money
Management
Results
Activities
Outputs
Purpose
Rehabilitation
of Tanks, canals,
Sluice gates and
distribution
channels
Functional
Irrigation
Scheme
Increased
Acreage
Cropping
Intensify
Access to
water
Goal
Increased
Production
Increased
Employment
Improved
Living standard
Programme Theory
The model describes as to how these
activities could conceivably achieve the
stated goals and objectives (Results
Chain)
 A specification of what must be done to
achieve the desired goals.
 Programme model conveys the linkages
between program goals and objectives
and progrmme action. i.e. establishes the
link between the problem and the
programme.

Programme Theory
Water Supply Project
Input
Water pipes
Taps
Advice etc.
Activities
Outputs
Outcome
Impacts
Installation of
pipes.
 training,
involving
women
community
management
20 water taps
installed;
10000 safe
gallons p.a.
Water borne
diseases
reduced
Less diarrhea
More time
available to
women
Improved
living
conditions to
women
Improved
health situation
Log Frame Matrix Introduction
The Log Frame provides a one-page summary of:
WHY
WHAT
HOW
WHICH
HOW
WHERE
WHAT
a project is carried out
the project is expected to achieve
the project is going to achieve these results
external factors are crucial for the success of the
project
we can assess the success of the project
we find the data required to assess the success
of the project
the project will cost
THE LOGICAL FRAMEWORK
- Tool for better Project Cycle Management
IDENTIFICATION
Goal
Purpose
EVALUATION
LOG
FRAME
Goal
Purpose
DESIGN/
APPRAISAL
Outputs
Activities
IMPLEMENTATION AND
MONITORING
Outputs
Activities
Project Planning Matrix / Logical Framework Matrix
Hierarchy of
Verifiable
Objective
Indicators
Measuring the achievement
GOAL
PURPOSE
OUTPUTS
INPUTS
Means of
Verification
Important
Assumptions
of the GOAL
Measuring the achievement
of the PURPOSE
Measuring the achievement
of the OUTPUTS
Summary of the project
inputs: COSTS AND
RESOURCES

Note 1 : Emphasis on developing performance indicators

Note 2 : The Project Management should be able to guarantee the
project outputs. The immediate objective is beyond their direct
control
Definition and Terminology
Input
-
Resources (Men, Material, Money, Management)
Output -
Physical accomplishment that the project management
should be able to guarantee.
Purpose -
The immediate objective or effect which is expected to
be achieved as the result of the project
Goal
-
The ultimate (long term) overall objective of the project
OVI
-
Set of criteria's, indicators or measures to verify
achievement of objectives at different levels. This
includes direct or proxy (indirect) indicators.
Objectively verifiable means, different independent
observers would come to the same conclusion as to
the status of achievement.
LFA Contd..
MOV
-
Assumption-
The means, methods and sources which permit OVI to
be measured in precise term.
These are important events, conditions decisions that
must exist if the project is to succeed but which are
outside the direct control of the project management.
These are linked to different levels in the project
planning matrix
Log Frame : Vertical Logic
Overall
Goal
Assumptions
Project
Purpose
Assumptions
Results/
Outputs
Assumptions
Activities
/ Inputs
Assumptions
Vertical
Logic
Log Frame : Horizontal Logic
Overall
Goal
Indicator
Means of
verification
Project
Purpose
Indicator
Means of
verification
Results/
Outputs
Indicator
Activities/
Inputs
Means of
verification
Specification of inputs
Horizontal
Logic
Basic Structure
a.
b.
Vertical Logic
Horizontal Logic
Vertical Logic
This identifies what project intends to do, clarifies the logical
linkages between means and ends, and specifies uncertainties
concerning the project. The elements of vertical logic are
1.
Hierarchy of project objectives:
Input, output and purpose are specific to project, while
higher level-Goal-relates the project to the programme
of which project is a part
2.
Casual linkages within the hierarchy:
Means-end relationship.
IF inputs are provided THEN output will be
produced……..
3.
Important Assumptions:
- Assumption at each level are linked to different levels in the
LFM.
“IF the planed outputs are proceed and IF the
important assumptions are valid THEN the project
purpose will be achieved”
- Assumptions are weighted according to importance and
probability.
4.
The notion of hypothesis:
- Success at each lower level of vertical logic does not
guarantee success at ultimate Goal level.
THE LOGICAL FRAMEWORK
Important Assumptions:
Describe the external risks to the project
Narrative
Summary
Verifiable
Indicators
Means of
Verification
Important Assumptions
GOAL
Assumptions about external factors for
sustaining the benefits of the project.
PURPOSE
Assumptions about external factors for
achieving the GOAL.
OUTPUTS
Assumptions about external factors for
achieving the PURPOSE.
ACTIVITIES
Assumptions about external factors for
achieving the OUTPUTS
Key points:


Both the criticality and probability of assumptions should be
assessed.
Risks should, as far as possible be ‘internalized’ by
redesigning the project. This leads to stronger projects.
The Logical Framework
CAUSE AND EFFECT RELATIONSHIPS
1
4
Goal
Purpose
Assumptions
Outputs
.
Activities
Assumptions
.
.
Assumptions
.
Assessment of Assumptions
1st Questions: “Is the assumption important
No
Yes
Don’t include this one
2nd Question: “How likely will it occur”
Almost
certain
Quite
likely
Not likely
Don’t include this one either
Put this assumption into your Project
Planning Matrix
Project Management must
monitor it and
if possible influence it
3rd Questions: “Can the project be re-designed”
No
Yes
Killer assumption!
Warn decision makers!
Stop!
Redesign project!
Plan activities to influence assumptions
Horizontal Logic
“ The objective of the horizontal logic is the
MEASUREMENT of the resources and results of a project,
through OVI and MOV” (CIDA)
1.
Objectively Verifiable Indicator
- Measures (direct or Indirect) to verify to what extent
Input, Output, Purpose and Goal is achieved. (Input
indicators, Output indicators, Purpose indicators, Goal
indicators)
- Measurement can be
Quantitative
(e.g. Km of road rehabilitation)
Qualitative (e.g. Farmers cooperative functioning
effectively)
Behavioral (e.g Increased use of sanitary facilities)
Qualitative indicators should be made measurable.
- Direct indicators should be supplemented by
indirect (proxy) indicators (e.g. indirect wealth
indicator may be quality of house construction
materials, ownership of bicycles/radio)
- Properties of Goal indicator
Specific
Measurable
Achievable
Realistic
Time-bound
2. Means of Verification (MOV)
Secondary Education Improvement Project
Goal
(Impact)
• Improved literacy rate
• Employable workforce - Absorption
rate by Industries
Purpose
(Effect)
• Increased enrolment rate
• Less Dropout rate
• Improved quality of teaching –
students pass rate
• No of teachers trained
• No of text books supplied
• Curriculum Development – No of
modules completed
• No. of school buildings constructed
• Facilities (computers, drinking
water, toilets, electricity)
Outputs
.
Inputs
.
• Funds spent
Key Performance Indicators

Qualities






S – Specific
M – Measurable
A – Achievable
R – Realistic
T – Time bound
Setting KPIs
Baseline
Indicator
Level
Desired Level of
Improvement
(Intervention)
=
Target
Performance
Benchmark
Issues and Challenges

LFA be incorporated into design stage of the Project

Unintended effects and impacts not covered in the LFA

Highly optimistic performance target setting is dangerous as
it would finally lead to rating of a good project as a failure

Indicators developed may necessitate the conduct of field
surveys which are time consuming and costly.
Logical Framework Analysis
Hierarchy of
Objectives
Objectively
Verifiable Indicators
Key
Assumptions
Goals

Reduce poverty in
rural area
 Reduce pressure on
coastal fisheries
Resources

8000 beneficiaries should increase
income above poverty line.
 Percentage of borrowers below
poverty line declined from 30% to
19%
 No of new off-shore employment
Supportive
Government
policy
Environment
Purpose
 Number of viable micro-enterprises
Fishermen are
willing to give
up traditional
fishing activities
Availability of
extension
services

Promote non-fishing
income generating
activities and
establishment of
entrepreneurial culture
established
 Recovery Rate
 Number of additional workers
employed per enterprises
 Cost per job generated
Output


Provision of credit of
Rs.400 ml to 8000
micro enterprises
through PCIs and
accredited NGOs
Number of micro enterprises financed
 Sector- wise and PCI wise loan
disbursement
 Average sub-loan size per PCI and
male female participation
 Percentage of borrowers without
previous borrowing experience
Banks are
flexible to grant
loans on
character
reference rather
than collateral’s