2. 2 Breakout Session # 504 Michael P. Fischetti, Director, Office of Procurement and Assistance Policy Department of Energy Date April 15, 2008 Time10:45 am How DOE Assesses its Acquisition System under OMB Circular A-123 Using the GAO Framework

3. 3 Agenda Introduction Background DOE’s Balanced Scorecard Approach What is the GAO Framework? The Cornerstones The Evaluation Team Performing the Assessment OMB Circular A-123: DOE’s Approach and Results An Example Final Outcome Summary Compliance Model

4. 4 Introduction DOE is striving to be removed from the “High Risk List” for contract management DOE addressing GAO agency criteria for removal –Demonstrate strong commitment and leadership –Demonstrate progress in implementing corrective measures –Develop a corrective action plan that identifies Root causes Effective solutions Near-term plan for implementing the solutions

5. 5 Background DOE learned of the “Framework” in September 2003 –Obtained a draft for agency comment Monitored development with GAO through FY2004 until formal publication in September 2005 Developed evaluation criteria in the form of questions Completed an internal assessment during FY2005 and FY2006 Monitored use of “Framework” criteria as part of GAO audits of TSA, INS, and DHS procurement functions

6. 6 DOE’s Balanced Scorecard Approach Maintaining BSC performance measures in four strategic assessment areas: 1)Customer Perspective 2)Internal Business Perspective 3)Learning and Growth Perspective 4)Financial Perspective

7. 7 DOE’s Balanced Scorecard Approach DOE initiated an internal BSC project to assess its procurement system using the “Framework” developed by the GAO to assess the health of its procurement system –Identified areas requiring management attention or early mitigation Followed up with an action plan for achievement Incorporated into existing FY-2006 Balanced Scorecard (BSC), self-assessment procurement performance reviews, and management system

8. 8 DOE’s Balanced Scorecard Approach Objective –Using “Framework” developed a DOE Risk evaluation model for internally assessing DOE’s procurement system Approach for implementation Approach –Review the DOE BSC Self Assessment Guide against the “Framework” –Coordinate with the BSC Program Manager and Senior Procurement Management –Assess current procurement functions –Establish permanent process for proactive, continuous evaluation of procurement areas

9. 9 DOE’s Balanced Scorecard Approach Documentation was: –Collected –Independent evaluation performed –Final report completed –Remediation conducted providing iterative improvement Follow-up actions: –Internal management control assessments under OMB Circular A-123 guidelines –Development of follow-on BSC initiatives addressing lower-rated areas

10. 10 What is the GAO Framework? Cornerstones –Organizational Alignment and Leadership –Policies and Processes –Human Capital –Knowledge and Information Management Elements Critical Success Factors Key Questions, Things to look for, Cautions

11. 11 The Cornerstones Organizational Alignment and Leadership –Aligning Acquisition with Agency Mission and Needs –Commitment from Leadership

12. 12 The Cornerstones Plans and Processes –Planning Strategically –Effectively Managing the Acquisition Process –Promoting Successful Outcomes of Major Projects

13. 13 The Cornerstones Human Capital –Valuing and Investing in the Acquisition Workforce –Strategic Human Capital Planning –Acquiring, Developing, and Retaining Talent –Creating Results-Oriented Organizational Cultures

14. 14 The Cornerstones Knowledge and Information Management –Identifying Data and Technology to Support Acquisition Decisions –Safeguarding the Integrity of Operations and Data –Not part of review

15. 15 The Evaluation Team Selected senior procurements analysts new to the Department –Why? Not familiar with existing processes Significant diversity of procurement knowledge and experience –This is not just a “file review” –Benchmarking against other organizational experience Objectivity Dedicated resources

16. 16 Performing the Assessment Understanding the “Framework” Collecting documentation (biggest challenge) Benchmarking other organizations Document Adhere to principles in OMB A-123 Stay on schedule

17. 17 OMB Circular A-123: DOE’s Approach and Results “Agencies should strive to integrate control related activities within the control framework outlined in A-123. In particular, management should identify opportunities to integrate and coordinate in order to leverage the internal reviews already being performed” (Implementation Guide dated July 2005) A-123 provides a vehicle to assess and evaluate internal controls –Assurance they are well designed and operated –Appropriately updated to meet changing conditions A-123 assesses internal controls for: –Effectiveness and efficiency of operations –Reliability of financial reporting –Compliance with laws and regulations OFPP circulated draft Guide for using the Framework for review –DOE provided comments to the draft based on its experience

18. 18 OMB Circular A-123: DOE’s Approach and Results DOE developed: –Test plan –Tested pre-defined areas for impact, likelihood of occurrence, and design effectiveness Test sub-categories included: –Management’s commitment to competence –Management of Human Capital –Proper execution of transactions and events –External relevant, reliable, and timely communications –Risk analysis and actions

19. 19 OMB Circular A-123: DOE’s Approach and Results The Office of Procurement and Assistance Management provided reasonable assurance that: –Internal controls for financial reporting were working effectively –There were no material weaknesses identified in the design or operation of the specific Headquarters controls over financial reporting evaluated

20. 20 An Example Human Capital: Having the capacity (people and resources) to resolve the problems –Framework Assessment Demographic trends analysis Educational Levels Internal training database (now migrated to ACMIS) Funding levels Position Descriptions –Risks identified Recruitment, retention, and training of the workforce

21. 21 An Example Responses to identified risks translated to BSC initiatives –“Talent Management Plan” –Human Capital Survey Survey Conducted demographically representative focus groups with a summary report to the Director Leadership Interviews Documentation review

22. 22 An Example Responses to identified risks translated to BSC initiatives –Department-wide Acquisition Career Development (ACD) Policy and Program Handbook (revised and issued) Outlines certification requirements for the acquisition workforce –“Acquisition Letters” issued Strengthened organizational relationships between Department Headquarters and field operational contracting elements, including re-emphasizing ACD policy –Performed Skill-Gap Analysis of 1102 workforce –Met with DOE IG in drafting an Advisory Memo to the Deputy Secretary * See the follow-on briefing on DOE’s ACD program

23. 23 Final Outcome Continuous Monitoring of Procurement System Effectiveness “Monitor and independently validate the effectiveness and sustainability of corrective measures” BSC continuous review and identification of system improvements –Using the “Framework” tool –Conducting periodic reviews and monitor field operations self- assessments of contracting office operations –Implementing field Procurement Management Review –Review Contract Management Plans –Review contractor purchasing systems

24. 24 Final Outcome Support interagency working groups in procurement and acquisition regulatory development Monitor procurement risks, issues, or vulnerabilities of other organizations Government Accountability Office (GAO) reports Agency Inspector General (IG) reports Internal financial and management audits and reviews Professional and trade association periodicals Procurement news sources or other documents Benchmarking/surveying other agencies Validation under A-123 Review

25. 25 Summary Corrective Feedback using BSC Use of the GAO Framework Reporting and compliance under OMB Circular A-123, as implemented by the DOE Chief Financial Officer

26. 26 Compliance Model Contracting Activities complete Compliance Self Assessment and PMR Checklist Compliance Self Assessment team and PMR review Compliance Self Assessment and PMR - Assessment Team recommendations Compliance Self Assessment and PMRs summary and trend data compiled Compliance Self Assessment and PMR Checklist revised as necessary Headquarters Strategic Direction * Balanced Scorecard Procurement Management Reviews Best Practices * * * Policy Development PMRs

27. 27 OMB Circular A-123 DOE Next Steps Implementing the “Framework” tool and A-123 in the field –Establishing policy/oversight/review functions in field offices –Procurement Management Review Program