1. E. Talero - 11-11-04IFMIS Workshop - Nairobi 1 IFMIS Implementation Workshop Effective IFMIS Procurement and Management of Consultants Eduardo Talero Senior E-Government Consultant - The World Bank (Retired) Nairobi, November 11, 2004

2. E. Talero - 11-11-04IFMIS Workshop - Nairobi 2 9-15 months6 months Develop Functional/Technical Specifications Advertise and receive bids Prepare Bidding documents Evaluate bids Award contract 4-5 months2 months21-28 months 3-5 months 3 months3-4 months2 months11-14 months Best case Possible? IFMIS PROCUREMENT IN AFRICA

3. E. Talero - 11-11-04IFMIS Workshop - Nairobi 3 OVERVIEW How to be effective in IS procurement (and minimize headaches) How to get maximum value from consultants How to deal with the World Bank to prevent delays

4. E. Talero - 11-11-04IFMIS Workshop - Nairobi 4 Effective Procurement (or how to minimize headaches) Acknowledge complexity Find the right skills & organize the work Develop a sound strategy Manage timetable effectively Follow sound bid evaluation method Develop a smart, fair contract

5. E. Talero - 11-11-04IFMIS Workshop - Nairobi 5 The procurement process is difficult because It is a heavily regulated, competitive process It has antagonistic stakeholders It is complex and deals with high stakes. It is corruption prone. It often has conflicting objectives.

6. E. Talero - 11-11-04IFMIS Workshop - Nairobi 6 Competitive Procurement TRANSPA- RENCY Potentially conflictive objectives in competitive procurement ECONOMY

7. E. Talero - 11-11-04IFMIS Workshop - Nairobi 7 Competitive Procurement TRANSPA- RENCY Potentially conflictive objectives in competitive procurement EFFICIENCY

8. E. Talero - 11-11-04IFMIS Workshop - Nairobi 8 IS Procurement is even more difficult because of: Diverse, complex technical content Interdependency with organizational & social dynamics Value from the whole, not the parts Mix of intellectual services & supply of goods. High failure rate of IS Projects

9. E. Talero - 11-11-04IFMIS Workshop - Nairobi 9 EVALUATION CRITERIA  Price  Quality of M&S  Quality of Proposal  Expertise  Price  Quality of Proposal  Expertise  Price PAYMENT TRIGGER Delivery Work done Acceptance of Deliveries Procurement profiles

10. E. Talero - 11-11-04IFMIS Workshop - Nairobi 10 Acknowledge complexity Find the right skills & organize the work Design a sound strategy Manage timetable effectively Follow sound bid evaluation method Develop a smart, fair contract Effective procurement (or how to minimize headaches

11. E. Talero - 11-11-04IFMIS Workshop - Nairobi 11 Find the right skills & organize the work Skill requirements: Domain knowledge: public accounting, budgeting and treasury management. Expertise in management of IS projects Information technology knowledge Contracting knowledge Procurement knowledge Effective procurement requires organized teamwork: authorities, responsibilities, schedule, resources.

12. E. Talero - 11-11-04IFMIS Workshop - Nairobi 12 Effective Procurement (or how to minimize headaches) Acknowledge complexity Find the right skills & organize the work Develop a sound strategy Establish feasible timetable Follow sound bid evaluation method Develop a smart, fair contract

13. E. Talero - 11-11-04IFMIS Workshop - Nairobi 13 Develop a sound strategy Understand the solution market Consult with peer network Choose a packaging option Choose a process option Set a timetable

14. E. Talero - 11-11-04IFMIS Workshop - Nairobi 14 Supply & Maintenance of Technology System Engineering/ Integration Independent Audits RFI & Pre-Qualification Phase II Project Strategy & System Requirements Phase I System Engineering Phase III Implementation Phase IV Maintenance Phase V System Specification & Project Mgmt. PROJECT PHASE PROCUR. TRACK Key: Competitive Bidding Packaging Option Conflict of Interest Consulting Services Supply &Maintenance System Engineering Ponder packaging options carefully (1)

15. E. Talero - 11-11-04IFMIS Workshop - Nairobi 15 Supply & Maintenance of Technology System Engineering/ Integration Independent Audits RFI & Pre-Qualification Phase II Project Strategy & System Requirements Phase I System Engineering Phase III Implementation Phase IV Maintenance Phase V System Specification & Project Mgmt. PROJECT PHASE PROCUR. TRACK Key: Competitive Bidding Packaging Option Conflict of Interest Consulting Services Integration Ponder packaging options carefully (2)

16. E. Talero - 11-11-04IFMIS Workshop - Nairobi 16 Choose between single and 2-stage process Single stage when: Possible solutions are well known Outsourcing is not an option Procurement complexity and risk are moderate Two stage when: Possible solutions are poorly known Outsourcing of operation or ownership is an option Large scope or complexity pose excessive risks

17. E. Talero - 11-11-04IFMIS Workshop - Nairobi 17 Effective Procurement (or how to minimize headaches) Acknowledge complexity Find the right skills & organize the work Develop a sound strategy Establish feasible timetable Follow sound bid evaluation method Develop a smart, fair contract

18. E. Talero - 11-11-04IFMIS Workshop - Nairobi 18 At least a month Time Engineer system Document System Deliver test Hw/OS Acceptance End rollout Train users Run Acceptance Tests Deliver rollout Hw/OS Begin Testing Establish feasible timetable Deliver pilot Hw/OS Conduct pilot Rollout system End Pilot

19. E. Talero - 11-11-04IFMIS Workshop - Nairobi 19 Effective Procurement (or how to minimize headaches) Acknowledge complexity Find the right skills & organize the work Develop a sound strategy Manage timetable effectively Follow sound bid evaluation method Develop a smart, fair contract

20. E. Talero - 11-11-04IFMIS Workshop - Nairobi 20 Follow sound bid evaluation method (1) Assemble a team beyond reproach Decide what features to evaluate Beware of evaluation tradeoffs Focus on features with high discerning value Leave room for exercise of judgment

21. E. Talero - 11-11-04IFMIS Workshop - Nairobi 21 Follow sound bid evaluation method (2) Define & test the point/weight scheme Plan effective system demonstrations Carry out & document evaluation Keep evaluators to what they know Iterate evaluation rounds to achieve consistency

22. E. Talero - 11-11-04IFMIS Workshop - Nairobi 22 Decide what features to evaluate Known Un- known Needed Unaccep table Desirable Undesira ble  M  M  S +E   E  E  E  E

23. E. Talero - 11-11-04IFMIS Workshop - Nairobi 23 Beware of tradeoffs in quality evaluation  Importance of Quality Evaluation  Level of competition  Range of solutions considered  Precision of qualification requirements  Precision of Technical Specifications

24. E. Talero - 11-11-04IFMIS Workshop - Nairobi 24 Illustrative Top Level Scoring Table Note: See IT Procurement Technical Note 10 at www.worldbank.org, procurement

25. E. Talero - 11-11-04IFMIS Workshop - Nairobi 25 Evaluation Criteria1: Quality of proposed solution2: Bidder's expertise, performance record, business risk profile.3: Quality of implementation plan and knowledge transfer strategy 40%4: Quality of proposed technical environment. - Technical Risk Profile Weight 20% 30% 10% Illustrative Evaluation Criteria for IFMIS Contract Note: See IT Procurement technical note 10 at www.worldbank.org, procurement

26. E. Talero - 11-11-04IFMIS Workshop - Nairobi 26 Follow sound bid evaluation method (2) Define & test the point/weight scheme Plan effective system demonstrations Carry out & document evaluation Keep evaluators to what they know Iterate evaluation rounds to achieve consistency Evaluate lifecycle costs

27. E. Talero - 11-11-04IFMIS Workshop - Nairobi 27 Plan effective system demonstrations Choose one of two objectives: Evaluate attributes of proposed solution Build and evaluate a prototype solution Predefine standard protocol Require PM to lead demonstration Watch for presentation order bias Iterate to ensure evaluation consistency Lower compliance cost to bidders

28. E. Talero - 11-11-04IFMIS Workshop - Nairobi 28 Follow sound bid evaluation method (2) Define & test the point/weight scheme Plan effective system demonstrations Carry out & document evaluation Keep evaluators to what they know Iterate evaluation rounds to achieve consistency Evaluate lifecycle costs

29. E. Talero - 11-11-04IFMIS Workshop - Nairobi 29 Effective Procurement (or how to minimize headaches) Acknowledge complexity Find the right skills & organize the work Develop a sound strategy Manage timetable effectively Follow sound bid evaluation method Develop a smart, fair contract

30. E. Talero - 11-11-04IFMIS Workshop - Nairobi 30 Develop a smart, fair contract (1) Make contractor responsible for solving your business problem despite specification error or omission Spell out client ’ s obligations and fulfill them. Require formal work methodology Pay against intermediate deliverables

31. E. Talero - 11-11-04IFMIS Workshop - Nairobi 31 Develop a smart, fair contract (2) Define the acceptance test protocol Test all the human/data/process aspects before acceptance Do system stress tests Define the rollout protocol Define rollout readiness indicators

32. E. Talero - 11-11-04IFMIS Workshop - Nairobi 32 Develop a smart, fair contract (3) Leave flexibility in rollout schedule Demand industry-standard documentation

33. E. Talero - 11-11-04IFMIS Workshop - Nairobi 33 Develop a smart, fair contract (4) Retain host computer flexibility Guard against default expiration of licenses Consider concurrent user licenses Forbid cross-licensing conditions Disallow unilateral revocation of SLA, disabling or impairment of system operation. Software License Agreements (1)

34. E. Talero - 11-11-04IFMIS Workshop - Nairobi 34 Develop a smart, fair contract (5) Require manufacturer ’ s authorization and concurrence on supplier ’ s SW licensing obligations. Supplier must support an old version  24 months after new one is received by client. If a new version causes problems, supplier must support the prior version until problems are solved. Software License Agreements (2)

35. E. Talero - 11-11-04IFMIS Workshop - Nairobi 35 Develop a smart, fair contract (6) Develop good M & S contract: Ask for proof of local capability Commit M & S supplier for 5+ years Keep client ’ s right to exit after 1 year Block effectiveness of supply contract until M&S contract is signed Retain performance security until delivery capacity is verified.

36. E. Talero - 11-11-04IFMIS Workshop - Nairobi 36 OVERVIEW How to be effective in IS procurement (and minimize headaches) How to get maximum value from consultants How to deal with the World Bank to prevent delays

37. E. Talero - 11-11-04IFMIS Workshop - Nairobi 37 Consultants: How to get maximum value (1) Rely on peer network to: Identify skill / experience requirements Develop TOR. Develop candidate shortlist Expect bias and neutralize through TOR Watch out for conflict of interest rules

38. E. Talero - 11-11-04IFMIS Workshop - Nairobi 38 Supply & Maintenance of Technology System Engineering/ Integration Independent Audits RFI & Pre-Qualification Phase II Project Strategy & System Requirements Phase I System Engineering Phase III Implementation Phase IV Maintenance Phase V System Specification & Project Mgmt. PROJECT PHASE PROCUR. TRACK Key: Competitive Bidding Packaging Option Conflict of Interest Possible Contractor Continuation Consulting Services Supply &Maintenance System Engineering Complex IS Procurement – Specify and Engineer Option

39. E. Talero - 11-11-04IFMIS Workshop - Nairobi 39 Consultants: How to get maximum value (2) Define time and residence requirements for key personnel Define knowledge transfer indicators Evaluate communication, social skills Use contract management instruments consistently and forcefully Control staff replacements

40. E. Talero - 11-11-04IFMIS Workshop - Nairobi 40 Consultants: How to get maximum value (3) Brief/induct consultants thoroughly Control incidental costs separately Pay attention to fairness Do not expect fire fighting miracles Pay on time Manage client-side obligations

41. E. Talero - 11-11-04IFMIS Workshop - Nairobi 41 Dealing with the World Bank (1) WB not staffed for major technical reviews Fiduciary responsibility imposes constraints. Key recommendation: avoid triggering reviews. Consequently, engage expert procurement advice during process. You are the client. Do not hesitate to complain.

42. E. Talero - 11-11-04IFMIS Workshop - Nairobi 42 Dealing with the World Bank (2) Do not hesitate to ask for reasonable, well justified exceptions. Better evaluation formula Relief from COI provisions for pilot consultants Justified sole sourcing decisions

43. E. Talero - 11-11-04IFMIS Workshop - Nairobi 43 Possible World Bank support To develop and nurture IFMIS network To develop baseline system requirement specifications To analyze and compare technology licensing terms

44. E. Talero - 11-11-04IFMIS Workshop - Nairobi 44 9-15 months6 months Develop Functional/Technical Specifications Advertise and receive bids Prepare Bidding documents Evaluate bids Award contract 4-5 months2 months21-28 months 3-5 months 3 months3-4 months2 months11-14 months Best case Possible? IFMIS PROCUREMENT IN AFRICA

45. E. Talero - 11-11-04IFMIS Workshop - Nairobi 45 THANK YOU