Making Risk Based Auditing Practical for Staff Application Ruthe Holden Chief Auditor Los Angeles County Metropolitan Transportation Authority

Agenda Overview Enterprise-wide Risk Assessment Project Based Risk Assessment Wrap Up

Overview

Enterprise-wide Risk Assessment Focus Understanding key objectives and identifying / prioritizing potential risks Balanced risk and opportunities with effective resource utilization Approach Reviewed existing documentation Conducted key interviews Standardized risk ranking criteria Prioritized risks Debriefed with executives

Enterprise-wide Risk Assessment Core Business Processes Planning transportation for the region Constructing new public transportation assets and routes Operating, managing, and maintaining public transportation services Maintaining compliance with legislative / regulatory requirements Resource Management Processes Financial Management Information Technology Human Resources Labor Relations Almost Certain Likely H G E B C A L i k e l i h o o d O f O c c u r r e n c e Possible F A D Unlikely Rare Insignificant Minor Moderate Major M a g n i t u d e O f I m p a c t

Project Based Risk Assessment The Problem Focusing on controls instead of risks Controls are not a guarantee The Concept Audit only high risk areas Consistent Process & Tools The Process Risk Matrix Heat Map The Potential High Impact Findings Making a Difference

Overview Do you see a young or an old woman? IIA Tone at the Top (July 2007) “The Yin and Yang of Risk” Goal: Audit Reports with impact Report on what Senior Management believes is important Audit Standards Require It Yellow Book - 7.05 –Fieldwork standards for performance audits Red Book – 2110 Risk Management

The Problem Which combination poses the highest risk? Likelihood 4 Likely 2 Unlikely Impact 3 Moderate 5 Catastrophic

The Concept Focus on risk rather than on controls Identify & focus resources on highest risk Understand Management’s Risk Appetite Prioritize Audit Findings based on Risk

Tools Needed Likelihood of Occurrence Table Magnitude of Impact Table Heat Map Risk Assessment Matrix

The Tools Likelihood of Occurrence Table Level Description Almost Certain Event is expected to occur in most circumstances Likely Event will probably occur in most circumstances Possible Event should occur at some time Unlikely Event could occur at some time Rare Event may occur in exceptional circumstances 11

The Tools Likelihood of Occurrence Table Level Description Risk Description 1 Low Less than 1 in 1,000 2 Moderate Greater than 1 in 1,000, but less than 1 in 100 3 High Greater than 1 in 100 12

The Tools Magnitude of Impact Table Level Description Risk Description 1 Insignificant < $500,000 impact on profitability No potential impact on market share No impact on brand value Issues would be delegated to junior management and staff to resolve 2 Minor $500,000 to $2.5 million impact on profitability Consequences can be absorbed under normal operating conditions Potential impact on market share and brand value Cash flow impact will be absorbed under normal operating conditions Issues will be delegated to middle management for resolution 3 Moderate $2.5 - $10 million impact on profitability Market share and/or brand value will be affected in the short term Cash flow may be affected The event will require senior and middle management intervention 13

The Tools Magnitude of Impact Table Level Description Risk Description 4 Major $10 million to $25 million impact on profitability Cash flow may be seriously affected Short term liquidity issues Serious diminution in market share and reputation with adverse publicity Key alliances are threatened Serious legal/regulatory issues (government action, removal of officers, significant law suits) Events and problems requires Board and executive management attention 5 Catastrophic > $25 million impact on profitability Imminent cash flow problems Sustained, serious loss in market share and reputation Sustained decline in stock price Loss of key alliances 14

L i k e l i h o o d O f O c c u r r e n c e The Tools Heat Map Almost Certain Likely Possible Unlikely Rare Insignificant Minor Moderate Major Catastrophic L i k e l i h o o d O f O c c u r r e n c e M a g n i t u d e O f I m p a c t 15

L i k e l i h o o d O f O c c u r r e n c e The Tools Heat Map Almost Certain Likely Possible Unlikely Rare Insignificant Minor Moderate Major Catastrophic L i k e l i h o o d O f O c c u r r e n c e M a g n i t u d e O f I m p a c t 16

The Tools Risk Assessment Matrix

The Process – Phase 1 Identify the audit objective Link audit objective to Strategic Goals/Objectives Document inherent risk for each audit objective What impacts department/program from meeting goals & objectives Rank Inherent Risk use heat map 18

The Process Phase 1: 1 (1) Agency Objective (2) Audit (3) Inherent Risk (4) IR Ranking (5) Current Control Activities (6) Residual Risk Score (7) Stop/ Go (8) Steps 1 Exercise Fiscal Responsibility/ Maximize Agency Resources To verify that warranty claims have been processed for all item pulled from stores that M3 identifies as under warranty Operating Divisions are not properly processing warranty parts resulting in loss of agency monies because claims are not filed LoO = Possible MoI =Moderate IRR = High

The Process – Phase 2 Identify Related Control Activities Rank Residual Risk Impact Controls have on lowering Inherent Risk Score Summarize Issues that Impact Residual Risk Score 20

The Process Assessment of Controls – Impact on Inherent Risk Level Description Risk Description 1 Strong The risk management processes are very strong for the level of risk identified Control/Response is very strong. Lowers Inherent Risk Score by 3 levels 2 Adequate The risk management processes are appropriate for the level of risk identified Control/Response is appropriate Lowers Inherent Risk Score by 2 levels 3 Moderate (acceptable) The risk management processes need to be strengthened Control/Response is not appropriate, but does not significantly expose the organization to risk Lowers Inherent Risk Score by 1 level 4 Weak or None Risk management processes needs to be strengthened Control/Response is not appropriate and leaves the organization significantly exposed to risk. Does not lower Inherent Risk Score 21

The Process - Phase 2 1 22 (1) Agency Objective (2) Audit (3) Inherent Risk (4) IR Ranking (5) Current Control Activities (6) Residual Risk Score (7) Stop/ Go (8) Steps 1 Exercise Fiscal Responsibility/Maximize Agency Resources To verify that warranty claims have been processed for all item pulled from stores that M3 identifies as under warranty Operating Divisions are not properly processing warranty parts resulting in loss of agency monies because claims are not filed LoO = Likely MoI = Moderate IRR = High Warranty process is manual – each operating division is required to put the “warranty” part in a bin located in middle of division floor – after new part is picked up in stock room (w/p ref C.17-5) No reconciliation between parts pulled from stock room and parts put in bin for warranty processing (w/p ref C.19-1) Warranty group does track parts identified by stock room as pulled for warranty, however difficult to reconcile to warranty bin (w/p ref C.20-3) High - Score unchanged because weak or no controls (w/p ref D.2) -Major critical system implemented last year (M3) (w/p ref D.2-1) -processes have not changed to reflect new system (w/p ref D.2-4) -store keeper control of warranty parts diffused-bin moved to floor for convenience (w/p ref E.2-9) 22

The Process – Phase 3 Stop/Go Analysis Fraud Brainstorming Audit Procedures Test and/or validate whether risk is adequately mitigated 23

The Process – Stop/Go Decision Tree A “GO” IS: Residual Risk is Critical or High OR Residual Risk is less than high, and Audit Objective is linked to Key Strategic Objective AND Inherent Risk is Critical or High, AND Audit has not recently validated controls that reduce IR, OR Significant changes in this area subsequent to last audit 24

The Process - Phase 3 (7) Stop/ Go (8) Steps GO (1) Objective (2) Audit (3) Inherent Risk (4) IR Ranking (5) Current Control Activities (6) Residual Risk Score (7) Stop/ Go (8) Steps 1 Exercise Fiscal Responsibility/Maximize Agency Resources To verify that warranty claims have been processed for all item pulled from stores that M3 identifies as under warranty Operating Divisions are not properly processing warranty parts resulting in loss of agency monies because claims are not filed LoO= Likely MoI =Major IRR = High Warranty process is manual – each operating division is required to put the “warranty” part in a bin located in middle of division floor – after new part is picked up in stock room (w/p ref C.17-5) No reconciliation between parts pulled from stock room and parts put in bin for warranty processing (w/p ref C.19-1) Warranty group does track parts identified by stock room as pulled for warranty, however difficult to reconcile to warranty bin (w/p ref C.20-3) High - Score unchanged because weak or no controls (w/p ref D.2) -Major critical system implemented last year (w/p ref D.2-1) -processes have not changed to reflect new system (w/p ref D.2-4) -store keeper control of warranty parts diffused-bin moved to floor for convenience (w/p ref E.2-9) GO Focus of audit is on efficiency & effectiveness of processes to implement new M3 system Fraud Brainstorming: Review for parts replaced by Stores that are not in bin – tie to mechanic pulling parts (ref A/S 6) - Review storekeeper records associated with most parts pulled for warranty (ref A/S 2) Compare 25 parts in the warranty bin on shop floor to M3 Compare 25 warranty items in M3 trace to warranted parts turned in Confirm findings w/ store room clerk for validation. Analyze 25 samples of w/o that require new parts “under warranty” Analyze trends in parts pulled by stockkeepers to w/o’s in M3 Document issues identified 25

Summary Art rather than science Don’t split hairs – difference of one rating should still be in ball park Keep Focus on Big Picture Customize the Tools so they work for you Likelihood of Occurrence Table Magnitude of Impact Table Key is to build this using Senior Management input Tie the ranking of risk to Management’s Risk Appetite Impact of Controls on Residual Risk 26

Good Books on Subject Audit Planning – A Risk Based Approach K.H. Spencer Pickett Auditing the Risk Management Process Control Self Assessment CD Published by Pleier Corporation Assessing Risk – 2nd Edition David McNamee 27

Thank You Questions 28

All information provided is of a general nature and is not intended to address the circumstances of any particular individual or entity. Although we endeavor to provide accurate and timely information, there can be no guarantee that such information is accurate as of the date it is received or that it will continue to be accurate in the future. No one should act upon such information without appropriate professional advice after a thorough examination of the particular situation.