Transcript Slide 1

Client Readiness in Project Evaluation
J.K. Ssegawa, PhD
Introduction
1. For any process where there is interaction between
two parties the adage ‘it takes two tango’ must apply.
2. In end of project evaluation, both the evaluator and
client must show and display readiness for the process,
in a number of ways including:




Decisions made
Actions taken
Practices used
Behaviour/attitude displayed
Is true for the two parties?
Evaluators
1. Literature is full of prescriptions of what evalutors
should be, what decision they should make and what
behaviour they should display during evaluation.
2. Scholars suggest that evaluators should (i) identify a
problem, (ii) recommend a solution, and (iii) help with
implementation.
3. In doing so they may behave like a teacher, student,
detective, barbarian, clock, monitor, talisman,
advocate, and ritual pig (Steele,1975) others have said
they could be consultants, educators, facilitators and
counsellors (Morabito, 2002).
Research Question
How ready are clients for the end of project evaluations, in
terms of decisions, actions, practices, and behaviour
/actions (DCPB) they display during the process?
THREE Cases
Project A
Project B
Project C
To evaluate a project meant
to facilitate the teaching,
research and retention
capability in a medical
school. Funds were obtained
from an international donor
were to be used to create a
conducive environment for
both faculty and teachers o
carry out their functions.
This included capacity
building, procurement of
equipment, computers,
software and hiring of staff.
It was a four-year project
with a budget of US7.2
million.
To evaluate a project that
involved capacity building in
project management and
Monitoring &Evaluation
(M&E) to 67 NGOs
implementing HIV and AIDS
and affiliated to a mother
body which acted as a hub
for the NGO network and
which disbursed project
funds it obtained from an
international donor on
application by NGOs. The
training project lasted eleven
months and amounted to
US$1.7 million.
To evaluate the assistance
provided by an international
bank to an economic sector
of a country. In the sector,
six projects were provided
with assistance ranging from
a grant to a loan and
included a feasibility, study,
sector review, capacity
building and infrastructure
development. The projects
spanned a period of eight
years (2004-2012) and their
total value amounted to
US$120 million.
Approach
1. To reflect on the decisions, actions, practices, and
behaviour /actions (DCPB) in the five phases of the three
project evaluations .
2. Note there is little contact in pre-contract phase but the
implication are very visible in the downstream phases
Pre-contract
Kick-off
meeting
Inception
Draft report
Final report
Results-Pre-contract phase
Construct Project A
The
timing
had
been
scheduled to start during the
mid-November. On advise of
Timing
the consultant it was changed
as it was going to include
Christmas holidays
Client was flexible with
Duration
timelines
A provision for the evaluation
was made but client did not
have an idea about the going
rates
Budget
Lump
sum
fee
after
completion.
Expenses
reimbursed as incurred
Project B
The timing of the evaluation
was bad as it happened three
months after the end and
some NGOs had closed
Project C
The timing was problematic.
Some projects has closed (3),
some had just ended (2) while
some were still going on (1)
Strict adherence to the
schedule but time allowed
was not adequate
Budget admitted that no
budget was made for the
evaluation but money was
scavenged from other budget
heads
Two tier payment system draft (75%) and final (25%)
report. Expenses reimbursed
during the stage payments
Strict adherence to the
schedule but time allowed
was enough
Client made a sufficient
allowance for the evaluation
fees and expenses
Each deliverable was paid
upon
delivery
–inception
(25%), draft (50%) and final
(25%). Expenses reimbursed
during the stage payments
Results – Kick off meeting
Construct
Project A
M&E design was not
properly formulated – many
Quality
of indicators were not relevant
documentatio (CREAM) to the results
n
Client specified the list of
stakeholders to interview but
the list kept on growing as
the evaluation proceeded.
Investigation
approach
Issues
There was
constructed
Existence
Baseline
of
no
Project B
Project C
Documentation
provided
was poor. It only contained
project
proposal
and
minutes.
Client was clear about the
evaluation process and the
documentation was quite
adequate incorporating M&E
reports
Client constructed the list of Evaluator was given free
stakeholders despite the reign to construct a sample.
records showing those who
were involved and hence an
opportune moment for the
valuator to construct a
representative sample
baseline There was
constructed
no
baseline Baselines were constructed
but were not accurate for
majority (5) of the projects
Results – Inception Report
Construct
Project A
Project B
Client was not clear
the
M&E
Clarity
of about
concepts but evaluation
evaluation
part
of
the
purpose and was
sponsorship contract
question
Nature
contact
person
Client confused outcome Client was clear about the
and impact as he valuation process
insisted on measuring
the latter just three
months after the project
elapsed
of The contact person was The contact person was
enthusiastic, committed aloof to the exercise.
and
facilitated
the
exercise.
Availability
of
documentati
on
Client
had
documentation
Project C
the Client did not provide
documentation on time
to the extent of providing
some
documents
towards the end.
Client
provided
six
contacted persons for
each project and as
expected they behaved
differently (i.e. committed
4 and Aloof -2)
Client
provided
all
documentation on time
through an electronic drop
box
Results – Draft Report
Construct Project A
Sample
Reporting
performan
ce
Project B
Project C
There was an argument No problem with the No problem with the
about
why
some sample
sample
stakeholders were left
out in the sample
M&E design was not
properly formulated –
many indicators were
not relevant (CREAM)
to the results
There was no M&E Client was clear about
designed apart from a the valuation process
simple statement of
how many people
were to be trained.
Results – Draft Report
Construct
Project A
Evaluator was unduly
pressured
to
write
Request to favourable comments.
tone down The performance was
the report
poor
Payment
Project B
Project C
Evaluator given free
reign to write report as
they saw fit. The
performance
was
moderately good
Evaluator given free
reign to write report as
they
saw
fit.
The
performance was good
(2), moderately good (3,
poor (1)
Moderately good payer On time payer
but was found to be
mean in the payment
terms – lump sum at
end was found not
conducive.
Very poor payer who
demands
on-time
deliverables but yet slow
to reciprocate when it
came to payment. One
payment came after
three months after due
date.
Conclusion
1. As we forge ahead, we need to develop standards and
guidelines for both the evaluator and client so that they
appreciate the role of each other.
2. It takes two – a good and husband - for marriage to
last’
Thank you