Risk Assessment for Efficiency and Impact Niki Raggi and Corrie Stokes, Austin, Texas

Session Objectives 0 Cover Austin’s current approach (and recent changes) to using risk assessment to maximize audit efficiency and impact 0 Discuss real-life examples of project risk assessment in the City of Austin 0 Share the templates used to document this work 2

Our Definition Risk assessment is a process used to determine the most significant and vulnerable aspects of the audited area, both for the annual plan and within an audit project 3

Risk Assessment in GAGAS 0 In planning the audit, auditors should assess significance and audit risk and apply these assessments in defining the audit objectives... (6.07) 0 Auditors should obtain an understanding of the …visibility, sensitivity, and relevant risks associated with the program under audit (6.13) 0 Review should determine if the audit plan adequately addresses relevant risks (6.52) 4 Lady GAGAS

Background: About Austin’s Office of the City Auditor 0 City Auditor appointed by Council for a 5-year term 0 26 permanent staff, divided into two units: Audit Services (4 managers, 14 auditors): 0 Conduct planned performance audits and respond to special requests from Council (~30 per year) Integrity Services (1 manager, 3 investigators) 0 Conduct investigations of allegations of fraud, waste, and abuse by City employees or contractors (~60 cases each year) 0 Conduct risk response and other integrity projects as time permits 5

Background: OCA Evolution 1990Existing internal audit department transitioned to a performance audit shop reporting to Council 2009Turnover of 80% of the management team Council hires new City Auditor who brings a different approach/perspective, for example: 0 “We did 80 audits per year where I came from!” 0 “Audit reports should be no more than 5 pages” 2010New City Auditor forms new management team and conducts an “Initial Assessment” which resulted in identifying several areas for improvement: 0 Projects could be managed better 0 Reports not always timely 0 Reports often lengthy and not reader-friendly 6

Background: OCA Planning and Audit Process AT THE PROJECT LEVEL Planning Phase: Focus on key processes and related key risks and perform a formal Risk Assessment, to identify focus for fieldwork Fieldwork Phase: Continue to focus on what really matters What’s the “so what”? Are we adding value? Reporting Phase: Articulate essential messages to convey high risks and defer unaddressed risks for further study ANNUALLY Strategic Audit Plan: Environmental scan and review other sources of risks to identify risks that may affect the City ONGOING RISK ASSESSMENT 7

Changes to OCA’s Risk Assessment Process Change #2: Give Management credit for managing high risks Change #1: Start audits with targeted risks from the Annual Planning Process Change #4: Focus on key risks in the key processes only Change #3: Defer unaudited high risks for future work/consideration Change #5: Standardize the planning process through templates and steps AT THE PROJECT LEVEL ANNUALLY ONGOING RISK ASSESSMENT 8

Change 1: start audits with targeted risk from Annual Planning process Pre-2010: 0 Developed an annual audit plan with general audit topic areas and broad objectives 0 Used significant resources to conduct a 3-year comprehensive risk assessment of all City activities 9

Change 1: Start audits with targeted risk from Annual Planning process 10 Post-2010: Audits are identified annually through the Strategic Audit Plan

Change 2: Give management credit for managing high risks 0 Pre-2010 Example: 0 Despite management managing high risks, we continued to review all aspects of the remittance process 0 Result: 2800 hours spent 10 mostly wimpy recommendations aggravated management 11

Change 2: give management credit for managing high risks 0 Post-2010 Example: 0 Recognized that high risks were being management in alignment with best practices and ended our work 0 Result: 360 hours spent 0 recommendations credit to management/goodwill 12

Change 3: Defer unaudited high risks for future work/consideration 0 Pre-2010: 0 Did not have a formal process for disposing of risks 0 Tended to try to cover any and all risks identified (concerned that we wouldn’t be back to an area for a long time) 0 Post-2010: 0 Use an issues log on each project 0 Incorporate “referrals” into integrity work and next audit plan 13

Change 4: focus on the key risks in the key processes only 0 Pre-2010: 0 Trained and skilled in risk assessment 0 Started with very broad objectives 0 Did not limit risk assessment to key processes 0 Did not always limit fieldwork to a subset of risks Benchmarks Best Practices Interviews Reported Performance Prior Audits/ Evaluations Similar Audits by Other Entities Contracts/ Agreements Budget & Financial Information Laws/ Regulations Organizational Charts Data from Available Systems Policies & Procedures RISK & VULNERABILITY ASSESSMENT OBJECTIVE(S), SCOPE, & METHODOLOGIES FOR FIELDWORK 14

2009 One Stop Shop Audit 0 Monster Risk/ Vulnerability Matrix 15 Change 4: Focus on the key risks in the key processes only

Post-2010: 0 Start with a more focused objective/issue 0 Approach planning by identifying the key processes related to the audit objective then focusing on the key risks within those processes 0 Ongoing risk assessment in addition to a formal risk assessment at the end of planning 16 Change 4: focus on the key risks in the key processes only

Affordable Housing Audit 0 Broad preliminary objective 0 Planning phase of 1,400 hours 0 Fieldwork objective still broad 0 77 pages reports 0 12 recommendations 0 Total project took 3,000 hours Affordable Housing Audit 0 More focused preliminary objective 0 Planning phase of 600 hours 0 Identified two highest risk areas 0 14 pages report 0 2 recommendations 0 Total project took 1,150 hours Exercise: find the finding! What support or assistance is provided to organizations developing affordable housing to increase probability of success? How well has rental housing development assistance performed in the last 6 years? Determine if key performance and financial controls are in place for bond and grant funded housing projects Evaluate whether A&D and RHDA programs had procedures in place to ensure that:  HUD and City program guidelines for long-term monitoring are complied with and;  GO Bond goals are being met. 17 Change 4: Focus on the key risks in the key processes only

Change 5: Standardize the planning process through planning steps and templates Pre-2010: Spending too much time on: 0 Reinventing how to perform planning steps for every audit 0 Reinventing how to document every step each time it was performed 18

Change 5: Standardize the planning process through planning steps and templates Planning Step Examples of Planning Procedures Examples of Planning Tasks 1Why are we doing this audit? Gain a general understanding of the audit’s purpose Review annual audit plan and meet with others who surfaced the risk 2 What do we already know about the audited entity? Identify, gather, and review prior work related to the audit objectives Review prior audits on topic areas; identify prior recommendations 3 What are the available criteria we could use? Research and identify criteria related to the audit objectives Review relevant laws, regulations, contracts 4 What do we know about the area that we are auditing? Gather information about the topic area to identify relevant key processes Review business plans, org charts, etc.; identity key data sources and key IT systems 5 What are the key risks related to the audit objective? Gather information about key risks associated with the topic area; evaluate potential sources of evidence Consider risk of fraud, waste, and abuse; analyze documentation and conduct interviews 6 What are the key controls over the key risks identified above? Using the key risks as a framework, gather information about key controls Perform walkthroughs and observations of relevant processes 7 How can we add value?Conduct an overall assessment of risks and controls Summarize and rank information gathered on risks and controls; identify fieldwork objectives 19

Change 5: Standardize the planning process through planning steps and templates Planning Step QuestionExamples of Planning TasksExamples of Templates 1Why are we doing this audit? Review annual audit plan and meet with others who surfaced the risk Kick-off meeting 2 What do we already know about the audited entity? Review prior audits on topic areas; identify prior recommendations Prior audits form 3 What are the available criteria we could use? Review relevant laws, regulations, contractsCriteria matrix 4 What do we know about the area that we are auditing? Review business plans, org charts, etc.; identity key data sources and key IT systems Data reliability form 5 What are the key risks related to the audit objective? Consider risk of fraud, waste, and abuse; analyze documentation and conduct interviews Fraud brainstorming Interview matrix 6 What are the key controls over the key risks identified above? Perform walkthroughs and observations of relevant processes R/V Matrix 7 How can we add value?Summarize and rank information gathered on risks and controls; identify fieldwork objectives End of Planning Memo 20

OCA’s Risk Assessment Process in Action Customer Care & Billing Audit II: Background 0 In 2009, City contracted with IBM for $52 M 0 Billing system collects payments for all City utilities 0 Payments collected are approximately $2 B per year Planning June-August 2009 Assessment August January 2010 Design/Build/Test January 2010-June 2011 Acceptance June- August 2011 Deployment August - October 2011 CC&B Audit II CC&B Audit I 21

22

CC & B II – Risk and Vulnerability Assessment 23

24

Recap/Lessons Learned Change 1: start with targeted risks in your annual plan identifying audits is an art not a science Change 2: give management credit for managing high risks its okay to walk away Change 3: defer unaudited high risks for future consideration you don’t have to audit everything at once Change 4: focus on key risks in the key processes only focus on what really matters (“where’s the beef?”) Change 5: standardize the planning process don’t reinvent the wheel; save your creativity for fieldwork 25