1. PIA 2734 Privatization and Contracting Out: The Knowledge and Skills Base

2. Assessing Performance in Contract Relationships Contracts, Foreign Aid and International Development

4. The Policy Process Department of State U.S. Agency for International Development Office of Management and Budget- Executive Office of President Congress

5. Policy Making and the Interaction of Major Agency Processes Office of Management And Budget (OMB) Evaluation Ex-Post Facto Evaluation Implementa- tion Pre- Implementa- tion Project Paper (PP) Project Review Paper (PRP) Project Identification Document (PID) Field of Concentration Strategy (DAPII) Country Program Strategy (DAPI) Prior Evaluation Operational Year Budget (OYB) Appropriation Congressional Presentation (CP) Budget Submissions Ongoing Projects Host Country Legis- lation Foreign Policy LDC Needs Agency Policy Global Sector Strategies Regional Strategies Research Strategy Management Objectives Financial MANAGEMENT Programming INFORMATION Management Reports Implementation SYSTEM External Needs Program SupportData Bank (CPDB, PAIS, DIS, ESDB) Personnel Administration SupportDatabase for Future Decisions, Policy Lessons Learned Evaluation Criteria Project Reporting Project Performance Tracking (PPT); Financial Reporting Planning Budgeting Design Approval Implementation Evaluation Reporting

6. Foreign Aid?

7. From Policy to Project Grants vs. Contracts: Assessing Sub- Grants RFA’s and Implementation Project is the Common Denominator for the International Donor

8. Resourcing Projects

9. Project Identification, Formulation, Preparation & Design Problems in project identification  In developing countries: Lack effective procedures for project identification within national planning agencies and operating ministries Weak conceptual and operational links exist between various national, regional, local, and special interest constituencies National plans often fail to provide a strategy for development Allocation of resource issues Issues of priority

10. Project Identification, Formulation, Preparation & Design Problems in project identification  In developing countries, cont.: Influence of interest groups Limited international assistance agency staff time to help government planners Excessive turnover and rotation of field representatives of assistance agencies Weaknesses in the overall planning system Design Primarily focuses on Contracting Process

11. The Name of the Game

12. Blueprint Approach to Development Planning Pilot Project Researchers Before-After Surveys Planner Administrators Target Population Evaluation Researchers Actions Project Blueprints Actual Change Versus Targeted Change Tested Models

13. The Blueprint

14. Contract Analysis Assessment vs. Evaluation Impact Assessment The Need for Quantitative Data The Reason for Blueprints

15. The Project Cycle and the Contract Analysis--collection of:  Social Analysis targeted groups: women, minorities, indigenous peoples  Economic Analysis--Cost Benefit  Institutional Analysis Sustainability Organizational Requirements Recurrent Cost Implications Human Skills Needed Social Acceptance

16. The Project Cycle & Contracts Design  Identifying nature of problem and possible solutions--specific needs and desired changes Appraisal  (Mandatory) data needed to prepare project plan and measure completion

17. C OMMUNITY P ROJECT D ESIGN

18. The Project Cycle Analysis--collection of information  Prediction  Selection of preferred alternatives  Measurement of Impact to determine contract fulfillment

19. Source: Project Management System, Practical Concepts, Inc., Washington, DC 1979. Project Objectives Achieved 3. Evaluation2. Execution 1. Design The Project Cycle

20. The Project Cycle Analysis  The Logical Framework: (LOGFRAME) If-then conditions Some donors have moved away from Log- frame Was replaced by a system based on identifying Strategic Objectives, Intermediate Results, Measurable Indicators, etc. That system was recently "de-emphasized." AID mission requests for funds were tied to promises of specific results Results Framework system is "under review."

21. The Log Frame

22. Logical Framework Performance Networks Practical Concepts, Incorporated Project Objectives Achieved 3. Evaluation2. Execution 1. Design Evaluation SystemReporting System Evaluations assess performance against plans and analyze causal linkages Progress indicators and formats for communicating project information Networks display performance plans over time ACHIEVEMENT EXCEPTION Project Management System Provides Management Tools to Support all Stages of the Project Cycle

23. Preparation of Documents: Donor – USAID Development Assistance Programs (DAPs) Country Strategy Paper Concept Paper Project Identification Document (PID)

24. Policy Making and the Interaction of Major Agency Processes Office of Management And Budget (OMB) Evaluation Ex-Post Facto Evaluation Implementa- tion Pre- Implementa- tion Project Paper (PP) Project Review Paper (PRP) Project Identification Document (PID) Field of Concentration Strategy (DAPII) Country Program Strategy (DAPI) Prior Evaluation Operational Year Budget (OYB) Appropriation Congressional Presentation (CP) Budget Submissions Ongoing Projects Host Country Legis- lation Foreign Policy LDC Needs Agency Policy Global Sector Strategies Regional Strategies Research Strategy Management Objectives Financial MANAGEMENT Programming INFORMATION Management Reports Implementation SYSTEM External Needs Program SupportData Bank (CPDB, PAIS, DIS, ESDB) Personnel Administration SupportDatabase for Future Decisions, Policy Lessons Learned Evaluation Criteria Project Reporting Project Performance Tracking (PPT); Financial Reporting Planning Budgeting Design Approval Implementation Evaluation Reporting

25. PP (USAID) (PP = Project Paper) Program Agreement (Donor) Technical Proposal (Contractor to Donor) Country Context (Contractor to Country) Implementation Documents

26. Characteristics Affecting Project Implementation Less Problematic 1. Simple technical features 2. Marginal change from status quo 3. One-actor target 4. One-goal objective 5. Clearly stated goals 6. Short duration More Problematic 1. Complex technical features 2. Comprehensive change from status quo 3. Multi-actor targets 4. Multi-goal objectives 5. Ambiguous or unclear goals 6. Long duration

27. The Project Cycle Implementation (Data- Country Program Data Bank, Economic and Social Data Bank, Project Accounting Information System, Development Information System)  Carrying out actions planned  Personnel- local (and foreign)  Budget and Accounting Information

28. The Project Cycle Monitoring and Evaluation: Focus on Verification of Completion of Contract Linked to End of Contract and Verification of Objective indicators  Understanding what has happened and assessing changes and quality of change  Issue: sustainability regarding follow-on within the country and replicability from one country to another

29. Monitoring and Evaluation Nature of Data:  Interview vs. survey  Seat of the pants observation "the old quick and dirty"  The problem of project goals: Goals are to be limited and bounded Specific activities are to be clearly defined and achieved Short run success leads to successful evaluation Short-term loop is five years

30. Monitoring and Evaluation Nature of Data:  Judgment: Evaluation vs. Assessment Two views:  a. Learn from experience  b. Judge performance Problem: judgment requires clear goals, in contradiction with learning Problem: power of the expert Problem: Contract limits judgement

31. Monitoring and Evaluation Nature of Data:  Evaluation is a donor requirement External activity Targets blueprint activity (CPA) Critical path analysis (Time based action) PERT chart (Project Evaluation Review Technique) very technical, programmed Evaluation often the need for more action Contracts Lack Flexibility

32. Monitoring and Evaluation Nature of Data:  Evaluation as an end product: Separate from implementation Action pre-determined in design prior to evaluation  Separates evaluation from the on-going activity  Evaluation Used to Determine Certification of Completion

33. Monitoring and Evaluation Issues  Problem with Evaluation concept Implementation suggests a finished product Bureaucratic action is ongoing Part of larger system with ambiguous boundaries  Assessment Ongoing, part of implementation process Not conducive to Contracts Management

34. End of Project Status Central to the Contracting Out Process

35. End of Project Status (EOPS) Are of great importance and are primary target of project efforts and discussion Projects are usually very complex  It is common to find that no single indicator is sufficient to describe the project achievement completely

36. End of Project Status (EOPS) In determining EOPS we apply following principle:  If all EOPS conditions are satisfied, then there would be no credible alternative explanation Except the purpose of the project (and the contract) has been achieved  Good project design will include the conditions that demonstrate successful achievement of the Project Purpose

37. End of Project Status (EOPS) Example  PROJECT PURPOSE IN THE DEVELOPMENT OF AN OIL FIELD: Export oil  EOPS 50,000 Barrels of crude/day transferred to tankers at nearest port; Quality of crude produced is competitive with that currently sold on world market. To verify, one needs a) the purity of oil, b) the world price, c) price sold, d) amount sold

39. Group Presentations