1. Auditing a project or using audits to improve project performance? The development of the Performance Assessment Tool (PAT) for the South West Road Project, Kazakhstan Romain Pison, Transport Specialist

3. Anti-competitive practices Low fiduciary capacity Lack of internal audits Weak institutional capacity for oversight of contracts Low civil society involvement

4. How do you monitor and control the risks? How do you make sure that the project delivers? How do you use Bank resources efficiently?

5. Project Management Consultant Civil Society Oversight Users complaint handling mechanism OP BP above and beyond

6. 6 Technical compliance Performance on criteria Evolution of progress Flagging of critical issues Evolution of disbursements Focus on project’s objectives How are we going to do that?

7. Concise and complete Quantitative and qualitative Structured and replicable Dedramatizing and factualizing Measurable and comparable Simple and reportable All that in one tool?

8. Payments efficiency Rate of Progress Quality of Works Working Environment

9. Compliance with Safeguards Minimizing CW impacts Interactions with PAPs

10. 2 ratings went from Moderately Satisfactory to Satisfactory Environment Supervision of Civil Works 10 ratings are Satisfactory 70 % disbursement after 4 years

13. More risks than ever… but more audit techniques, design tools and users’ feedback available A replicable and motivation-based approach to benchmark performance and empower the Client and stakeholders Necessity and capacity to adjust tool and audit techniques to improve future delivery

14. https://spark.worldbank.org/groups/pat-cop

15. Thank you ! The development of the Performance Assessment Tool (PAT) for the South West Road Project, Kazakhstan Romain Pison, Transport Specialist

16. Technical annexes

18. 2. PAT Methodology (11/13) Facts Findings Evaluate facts against criteria Is audit evidence reliable, sufficient and relevant? Audit procedures Criteria YES NO Preliminary Findings Reporting

19. 2. PAT Methodology (10/13) R ELIABILITY Consistent when used by another auditor in the same circumstances O BJECTIVITY Free from any bias of the auditor or management U NDERSTANDABILITY Clearly stated and not subject to significantly different interpretations C OMPARABILITY Consistent with those used in other or previous performance audits C OMPLETENESS Inclusive of all significant criteria to assess performance A CCEPTABILITY Agreeable to experts in the field, audited entities, general public, etc.

20. 2. PAT Methodology (12/13) TypesConsiderationsAudit proceduresDocumentation PHYSICAL Usually the most persuasive evidence, presence may distort what would occur Direct inspection of observation Notes, photographs, charts, maps, samples DOCUMENTARYElectronic or hard-copy format Review of documents, reports, manuals,… Performance management reports, policies, procedures…. ORAL Usually important, as information is up-to-date and maybe not obtained elsewhere. However requires corroboration Enquiry or interviews of auditee staff Summary of information obtained through these methods ANALYTICAL Using professional judgment to evaluate all kinds of evidence Analysis through computation, comparisons… Summary of analytical data, ratio analysis, regression, benchmarking….

21. CW SOENV… EMP………………