1. 1 “Good Practices in Managing for Results” Workshop Santiago, Chile October 27 th and 28 th, 2010 Benjamin Nelson Managing Director for Quality Office of Quality and Continuous Improvement Managing for Results At GAO

2. 2 Outline The GAO Context Managing for Results Strategy –Strategic Planning –Integrated Quality Assurance Framework –Measures and Results –Monitoring

3. 3 The GAO Context –Broad mandate –Large organization –Large volume and different product types –80 or more senior executives can sign products –Active involvement in the legislative process –All performance audits follow the Yellow Book: Generally Accepted Government Auditing Standards (GAGAS) –Well developed and transparent quality and risk management approach –14 Teams carry out the work related to GAO’s strategic goals –Matrix management of engagements –Wide range of expertise among GAO staff

4. 4 The GAO Context –Employed about 3,300 people, three-fourths in HQ, with a budget over $0.5 billion –Received over 950 Requests/Mandates 95% of resources are spent on requests and mandates 5% are spent on CGA work –Produced over 960 products 192 testimonies 600 reports and correspondence 42 oral presentations 127 other (guides, referrals, statements for the record, other correspondences) –Made 2,005 Recommendations Historically about 80-85% implemented

5. 5

6. 6 Products with Recommendations /implemented Products with Recommendations /implemented Financial and non-financial benefits Financial and non-financial benefits Leading, recognizing, and listening to staff Leading, recognizing, and listening to staff Attracting and retaining staff Attracting and retaining staff Getting work done Getting work done Quality of work life Quality of work life Testimonies Timeliness Developing, supporting, and using staff Developing, supporting, and using staff Note: Internal business process measures support the categories listed above. Targets Set For Performance Measurement Clients People Internal Operations Results

7. 7 Specific Categories of Work –Performance audits, which include program effectiveness and results audits, economy and efficiency audits, internal control assessments, and prospective analyses to assess program or policy alternatives advantages or disadvantages of proposals best practices –Financial audits and attestation engagements –Non-audit services, which include objective, rigorously researched information on a program’s condition, without the application of criteria legal opinions bid protest adjudications technical assistance investigations

8. 8 One Quality Assurance System While GAO undertakes a wide variety of engagements—using many different approaches and methodologies and a wide range of specialized skills—we have one quality assurance system. The quality assurance system Is comprised of a comprehensive set of clearly defined, well-documented, and transparent standards and control procedures, and Assures compliance with Government Auditing Standards

9. 9 Quality Assurance System GAO’s Quality Assurance System provides reasonable assurance that GAO’s Work is professional, independent, and objectively designed Evidence is complete and reliable Conclusions and products are fair and balanced Recommendations are sound

10. 10 Overview of Quality Assurance Framework Our approach to quality is comprehensive and transparent Designed around people, processes, and technology Broader than the management of individual engagements Core controls focus on engagement performance Value- and risk-based and includes an appropriate degree of transparency about key policies, procedures, criteria, and access to the results of our work

11. 11 Key Components of GAO’s Quality Assurance Framework Quality Assurance Framework encompasses GAO’s Core Values of Accountability, Integrity, and Reliability Government Auditing Standards Policy Manual Professional Practices Advisory Committee EAGLE: Electronic Assistance Guide for Leading Engagements

12. 12 Quality Assurance Framework Structured summary of the key elements of GAO’s quality control system. Helps the staff understand what they are to do and why

13. GAO’s Quality Assurance Framework for Assuring Compliance With Government Auditing Standards Strategic Planning Strategic Planning Leadership Human Capital Engagement Performance Monitoring/Policy Review Professional Development Professional DevelopmentAssigning Staff Agency Protocols Agency Protocols Tone at the Top Tone at the Top Mission, Standards and Core Values Mission, Standards and Core Values Recruiting/ Hiring Recruiting/ Hiring Independence Portfolio & Risk Management Portfolio & Risk Management Advancement Performance Management Performance Management Congressional Protocols Congressional Protocols Evidence Supervision and Review Supervision and Review Stakeholder Involvement Stakeholder Involvement Consultations with Experts and Specialists Consultations with Experts and Specialists Referencing Engagement Planning Engagement Planning Public Reporting Public ReportingAudit Documentation Agency Comments Agency Comments Partner Concurrence Partner Concurrence Message Agreement Message Agreement Professional Practices Advisory Committee Professional Practices Advisory Committee Quality and Continuous Improvement Assessments Quality and Continuous Improvement Assessments Internal Audit Internal Audit Peer Review Peer Review External Audit/ Audit Committee External Audit/ Audit Committee Annual Inspections Annual Inspections GAO’s Intranet QA Framework

14. 14 Risk Based Engagement Management Process Requests Mandates CGA Starts ERM Design Commitments to Clients Draft Reports and Agency/3 rd Party Views (all risk levels) Other issues (i.e., access, independence) Bi-weekly Meeting EAM Authority Independence Risk Level Cost Complexity Controversy Priority Internal Controls Stakeholders Weekly Meeting HIGH MEDIUM

15. GAO’s Quality Assurance Framework for Assuring Compliance With Government Auditing Standards Engagement Performance Evidence Supervision and Review Supervision and Review Stakeholder Involvement Stakeholder Involvement Consultations with Experts and Specialists Consultations with Experts and Specialists Referencing Engagement Planning Engagement Planning Public Reporting Public Reporting Engagement Documentation Engagement Documentation Agency Comments and Third-Party Views Agency Comments and Third-Party Views Partner Concurrence Partner Concurrence Message Agreement Message Agreement

16. 16 Roles and Responsibilities Second Partners Analysts Analysts in Charge Assistant Directors First Partners Stakeholders Experts and Specialists

17. 17 Electronic Assistance Guide for Leading Engagements (EAGLE) Standardizes engagement management process Assures compliance with GAGAS, GAO Policy Manual, and Quality Assurance Framework Provides “one-stop shopping” for auditors to access guidance, forms, templates, and points of contact for conducting GAO work

18. 18 Electronic Assistance Guide for Leading Engagements (EAGLE)

19. 19 Measures and Results – Performance and Accountability Report 15 measures Recommendation follow-up Accomplishment Reporting

20. 20 Annual Performance Plan

21. 21

22. 22 Results: Financial Benefits

23. 23 Results: Nonfinancial Benefits

24. 24 Results: New Products with Recommendations

25. 25 Results: Past Recommendations Implemented

26. 26 Clients: Testimonies

27. 27 Clients: Timeliness 1 1 Since fiscal year 2004 we have collected data from our client feedback survey on the quality and timeliness of our products, and in fiscal year 2006 we began to use the independent feedback from this survey as a basis for determining our timeliness.

28. 28 Monitoring and Review Annual internal inspection Peer review every 3 years Annual audit by external auditors Audit Advisory Committee GAO’s Inspector General reviews Annual internal assessments of internal controls

29. 29 Annual Internal Inspection  Reports whether GAO’s quality control system is suitably designed and operating effectively  Conducted for products issued in a given calendar year  Deals with substance as well as process  Recognizes good practices  Based on criteria (GAGAS or policy)

30. 30 Continuous Assessment Loop Assess Quality Assess Policies & Guidance Modify/ Clarify Policies Provide Training Communicate Through Various Means Plan Next Assessment