Transcript Effective IFMIS Procurement and Management of Consultants Eduardo Talero Nairobi, November 11, 2004

IFMIS Implementation Workshop
Effective IFMIS Procurement and
Management of Consultants
Eduardo Talero
Senior E-Government Consultant - The World Bank (Retired)
Nairobi, November 11, 2004
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
1
IFMIS PROCUREMENT IN AFRICA
Develop Functional/Technical Specifications
Prepare Bidding documents
Advertise and receive bids
Evaluate bids
Award contract
Best case
9-15 months
4-5 months
2 months
6 months
21-28 months
Possible?
3-5 months
3-4 months
2 months
3 months
11-14 months
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
2
OVERVIEW



How to be effective in IS procurement
(and minimize headaches)
How to get maximum value from
consultants
How to deal with the World Bank to
prevent delays
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
3
Effective Procurement (or how to
minimize headaches)






Acknowledge complexity
Find the right skills & organize the work
Develop a sound strategy
Manage timetable effectively
Follow sound bid evaluation method
Develop a smart, fair contract
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
4
The procurement process is difficult
because





It is a heavily regulated, competitive
process
It has antagonistic stakeholders
It is complex and deals with high
stakes.
It is corruption prone.
It often has conflicting objectives.
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
5
Potentially conflictive objectives in
competitive procurement
TRANSPARENCY
ECONOMY
Competitive
Procurement
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
6
Potentially conflictive objectives in
competitive procurement
TRANSPARENCY
EFFICIENCY
Competitive
Procurement
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
7
IS Procurement is even more difficult
because of:





Diverse, complex technical content
Interdependency with organizational &
social dynamics
Value from the whole, not the parts
Mix of intellectual services & supply of
goods.
High failure rate of IS Projects
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
8
Procurement profiles
Technology.
(Goods)
Intellectual Services
(Consultants)
Working Systems
(IFMIS)
Supplier
Client
Shared
“DESIGN” RISK
CSFs
QUALFICATION
CRITERIA
EVALUATION
CRITERIA
¨ Tech. Specs
¨ Logistics
¨ Ability to
supply
¨ Fin. strength
¨ Prof. Expertise
¨ Communications
¨ Credentials
¨ Track record
¨ Price
¨ Quality of
M&S
¨ Quality of
Proposal
¨ Expertise
¨ Price
Work done
PAYMENT
TRIGGER
E. Talero - 11-11-04
Delivery
IFMIS Workshop - Nairobi
¨ Prof. Expertise
¨ Process Mgmt.
¨ Financial /operational
capabilities
¨ Track record
¨ Credentials
¨ Quality of Proposal
¨ Expertise
¨ Price
Acceptance of Deliveries
9
Effective procurement (or how to minimize
headaches






Acknowledge complexity
Find the right skills & organize the work
Design a sound strategy
Manage timetable effectively
Follow sound bid evaluation method
Develop a smart, fair contract
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
10
Find the right skills & organize the work

Skill requirements:






Domain knowledge: public accounting, budgeting
and treasury management.
Expertise in management of IS projects
Information technology knowledge
Contracting knowledge
Procurement knowledge
Effective procurement requires organized
teamwork: authorities, responsibilities,
schedule, resources.
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
11
Effective Procurement (or how to
minimize headaches)






Acknowledge complexity
Find the right skills & organize the work
Develop a sound strategy
Establish feasible timetable
Follow sound bid evaluation method
Develop a smart, fair contract
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
12
Develop a sound strategy





Understand the solution market
Consult with peer network
Choose a packaging option
Choose a process option
Set a timetable
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
13
Ponder packaging options carefully (1)
Phase I
PROJECT PHASE
PROCUR. TRACK
Project Strategy &
System
Requirements
Phase II
RFI &
Pre-Qualification
Phase III
Phase IV
Phase V
System Engineering Implementation Maintenance
System
Specification &
Project Mgmt.
System
Engineering/
Integration
Supply &
Maintenance
of
Technology
Independent
Audits
Key:
Competitive Bidding
Packaging Option
Conflict of Interest
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
Consulting Services
Supply &Maintenance
System Engineering
14
Ponder packaging options carefully (2)
Phase I
PROJECT PHASE
PROCUR. TRACK
Project Strategy &
System
Requirements
Phase II
RFI &
Pre-Qualification
Phase III
Phase IV
Phase V
System Engineering Implementation Maintenance
System
Specification &
Project Mgmt.
System
Engineering/
Integration
Supply &
Maintenance
of
Technology
Independent
Audits
Key:
Competitive Bidding
Packaging Option
Integration
Conflict of Interest
E. Talero - 11-11-04
Consulting Services
IFMIS Workshop - Nairobi
15
Choose between single and 2-stage
process

Single stage when:




Possible solutions are well known
Outsourcing is not an option
Procurement complexity and risk are moderate
Two stage when:



Possible solutions are poorly known
Outsourcing of operation or ownership is an option
Large scope or complexity pose excessive risks
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
16
Effective Procurement (or how to
minimize headaches)






Acknowledge complexity
Find the right skills & organize the work
Develop a sound strategy
Establish feasible timetable
Follow sound bid evaluation method
Develop a smart, fair contract
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
17
Establish feasible timetable
Begin Acceptance End
Pilot
Testing
End rollout
Engineer system
Document System
Deliver test Hw/OS
Train users
Run Acceptance Tests
Deliver pilot Hw/OS
Conduct pilot
Deliver rollout Hw/OS
Rollout system
Time
At least a month
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
18
Effective Procurement (or how to
minimize headaches)






Acknowledge complexity
Find the right skills & organize the work
Develop a sound strategy
Manage timetable effectively
Follow sound bid evaluation method
Develop a smart, fair contract
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
19
Follow sound bid evaluation method (1)





Assemble a team beyond reproach
Decide what features to evaluate
Beware of evaluation tradeoffs
Focus on features with high discerning
value
Leave room for exercise of judgment
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
20
Follow sound bid evaluation method (2)





Define & test the point/weight scheme
Plan effective system demonstrations
Carry out & document evaluation
Keep evaluators to what they know
Iterate evaluation rounds to achieve
consistency
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
21
Decide what features to evaluate
Known
Unknown

Needed
Unaccep
table
Desirable
Undesira
ble


M


M


S +E






E. Talero - 11-11-04
E


S +E
E
E

IFMIS Workshop - Nairobi
E
22
Beware of tradeoffs in quality evaluation
Precision of qualification requirements
Precision of Technical Specifications
Importance of Quality Evaluation
Level of competition
Range of solutions considered
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
23
Illustrative Top Level Scoring Table
Criterion
Criterion1 Criterion 2 Criterion 3 Criterion 4 Total (Maximum)
40
20
20
20
[1]
-10
-5
-5
-5
[2]
0
0
0
0
[3]
20
10
10
10
[4]
40
20
20
20
Weight
Scores and Weight points
100
100
Note: See IT Procurement Technical Note 10 at www.worldbank.org,
procurement
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
24
Illustrative Evaluation Criteria for IFMIS
Contract
Evaluation Criteria
Weight
1: Quality of proposed solution
20%
2: Bidder's expertise, performance record, business risk profile.
30%
3: Quality of implementation plan and knowledge transfer
strategy
40%
4: Quality of proposed technical environment. - Technical Risk
Profile
10%
Note: See IT Procurement technical note 10 at www.worldbank.org,
procurement
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
25
Follow sound bid evaluation method (2)






Define & test the point/weight scheme
Plan effective system demonstrations
Carry out & document evaluation
Keep evaluators to what they know
Iterate evaluation rounds to achieve
consistency
Evaluate lifecycle costs
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
26
Plan effective system demonstrations

Choose one of two objectives:







Evaluate attributes of proposed solution
Build and evaluate a prototype solution
Predefine standard protocol
Require PM to lead demonstration
Watch for presentation order bias
Iterate to ensure evaluation consistency
Lower compliance cost to bidders
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
27
Follow sound bid evaluation method (2)






Define & test the point/weight scheme
Plan effective system demonstrations
Carry out & document evaluation
Keep evaluators to what they know
Iterate evaluation rounds to achieve
consistency
Evaluate lifecycle costs
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
28
Effective Procurement (or how to
minimize headaches)






Acknowledge complexity
Find the right skills & organize the work
Develop a sound strategy
Manage timetable effectively
Follow sound bid evaluation method
Develop a smart, fair contract
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
29
Develop a smart, fair contract (1)




Make contractor responsible for solving
your business problem despite
specification error or omission
Spell out client’s obligations and fulfill
them.
Require formal work methodology
Pay against intermediate deliverables
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
30
Develop a smart, fair contract (2)





Define the acceptance test protocol
Test all the human/data/process aspects
before acceptance
Do system stress tests
Define the rollout protocol
Define rollout readiness indicators
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
31
Develop a smart, fair contract (3)


Leave flexibility in rollout schedule
Demand industry-standard
documentation
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
32
Develop a smart, fair contract (4)
Software License Agreements (1)





Retain host computer flexibility
Guard against default expiration of licenses
Consider concurrent user licenses
Forbid cross-licensing conditions
Disallow unilateral revocation of SLA,
disabling or impairment of system operation.
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
33
Develop a smart, fair contract (5)
Software License Agreements (2)



Require manufacturer’s authorization and
concurrence on supplier’s SW licensing
obligations.
Supplier must support an old version  24
months after new one is received by client.
If a new version causes problems, supplier
must support the prior version until problems
are solved.
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
34
Develop a smart, fair contract (6)
Develop good M & S contract:





Ask for proof of local capability
Commit M & S supplier for 5+ years
Keep client’s right to exit after 1 year
Block effectiveness of supply contract
until M&S contract is signed
Retain performance security until
delivery capacity is verified.
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
35
OVERVIEW



How to be effective in IS procurement
(and minimize headaches)
How to get maximum value from
consultants
How to deal with the World Bank to
prevent delays
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
36
Consultants: How to get maximum value
(1)

Rely on peer network to:





Identify skill / experience requirements
Develop TOR.
Develop candidate shortlist
Expect bias and neutralize through TOR
Watch out for conflict of interest rules
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
37
Complex IS Procurement – Specify and
Engineer Option
Phase I
PROJECT PHASE
PROCUR. TRACK
Project Strategy &
System
Requirements
Phase II
RFI &
Pre-Qualification
Phase III
Phase IV
Phase V
System Engineering Implementation Maintenance
System
Specification &
Project Mgmt.
System
Engineering/
Integration
Supply &
Maintenance
of
Technology
Independent
Audits
Key:
Competitive Bidding
Conflict of Interest
E. Talero - 11-11-04
Packaging Option
Possible Contractor
Continuation
IFMIS Workshop - Nairobi
Consulting Services
Supply &Maintenance
System Engineering
38
Consultants: How to get maximum value
(2)





Define time and residence requirements
for key personnel
Define knowledge transfer indicators
Evaluate communication, social skills
Use contract management instruments
consistently and forcefully
Control staff replacements
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
39
Consultants: How to get maximum value
(3)






Brief/induct consultants thoroughly
Control incidental costs separately
Pay attention to fairness
Do not expect fire fighting miracles
Pay on time
Manage client-side obligations
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
40
Dealing with the World Bank (1)





WB not staffed for major technical reviews
Fiduciary responsibility imposes constraints.
Key recommendation: avoid triggering
reviews.
Consequently, engage expert procurement
advice during process.
You are the client. Do not hesitate to
complain.
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
41
Dealing with the World Bank (2)

Do not hesitate to ask for reasonable,
well justified exceptions.



Better evaluation formula
Relief from COI provisions for pilot
consultants
Justified sole sourcing decisions
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
42
Possible World Bank support



To develop and nurture IFMIS network
To develop baseline system requirement
specifications
To analyze and compare technology
licensing terms
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
43
IFMIS PROCUREMENT IN AFRICA
Develop Functional/Technical Specifications
Prepare Bidding documents
Advertise and receive bids
Evaluate bids
Award contract
Best case
9-15 months
4-5 months
2 months
6 months
21-28 months
Possible?
3-5 months
3-4 months
2 months
3 months
11-14 months
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
44
THANK YOU
E. Talero - 11-11-04
IFMIS Workshop - Nairobi
45