1. Making Risk Based Auditing Practical for Staff Application
Ruthe Holden
Chief Auditor
Los Angeles County Metropolitan
Transportation Authority

2. Agenda Overview Enterprise-wide Risk Assessment
Project Based Risk Assessment
Wrap Up

3. Overview

4. Enterprise-wide Risk Assessment
Focus
Understanding key objectives and identifying / prioritizing potential risks
Balanced risk and opportunities with effective resource utilization
Approach
Reviewed existing documentation
Conducted key interviews
Standardized risk ranking criteria
Prioritized risks
Debriefed with executives

5. Enterprise-wide Risk Assessment
Core Business Processes
Planning transportation for the region
Constructing new public transportation assets and routes
Operating, managing, and maintaining public transportation services
Maintaining compliance with legislative / regulatory requirements
Resource Management Processes
Financial Management
Information Technology
Human Resources
Labor Relations
Almost Certain
Likely H G E B C A
L i k e l i h o o d O f O c c u r r e n c e
Possible F A D
Unlikely
Rare
Insignificant
Minor
Moderate
Major
M a g n i t u d e O f I m p a c t

6. Project Based Risk Assessment
The Problem
Focusing on controls instead of risks
Controls are not a guarantee
The Concept
Audit only high risk areas
Consistent Process & Tools
The Process
Risk Matrix
Heat Map
The Potential
High Impact Findings
Making a Difference

7. Overview Do you see a young or an old woman?
IIA Tone at the Top (July 2007)
“The Yin and Yang of Risk”
Goal: Audit Reports with impact
Report on what Senior Management believes is important
Audit Standards Require It
Yellow Book –Fieldwork standards for performance audits
Red Book – 2110 Risk Management

8. The Problem Which combination poses the highest risk? Likelihood 4
Likely 2
Unlikely
Impact 3
Moderate 5
Catastrophic

9. The Concept Focus on risk rather than on controls
Identify & focus resources on highest risk
Understand Management’s Risk Appetite
Prioritize Audit Findings based on Risk

10. Tools Needed Likelihood of Occurrence Table Magnitude of Impact Table
Heat Map
Risk Assessment Matrix

11. The Tools Likelihood of Occurrence Table Level Description
Almost Certain
Event is expected to occur in most circumstances
Likely
Event will probably occur in most circumstances
Possible
Event should occur at some time
Unlikely
Event could occur at some time
Rare
Event may occur in exceptional circumstances 11

12. The Tools Likelihood of Occurrence Table Level Description
Risk Description 1
Low
Less than 1 in 1,000 2
Moderate
Greater than 1 in 1,000, but less than 1 in 100 3
High
Greater than 1 in 100 12

13. The Tools Magnitude of Impact Table Level Description Risk Description1
Insignificant
< $500,000 impact on profitability
No potential impact on market share
No impact on brand value
Issues would be delegated to junior management and staff to resolve 2
Minor
$500,000 to $2.5 million impact on profitability
Consequences can be absorbed under normal operating conditions
Potential impact on market share and brand value
Cash flow impact will be absorbed under normal operating conditions
Issues will be delegated to middle management for resolution 3
Moderate
$2.5 - $10 million impact on profitability
Market share and/or brand value will be affected in the short term
Cash flow may be affected
The event will require senior and middle management intervention 13

14. The Tools Magnitude of Impact Table Level Description Risk Description4
Major
$10 million to $25 million impact on profitability
Cash flow may be seriously affected
Short term liquidity issues
Serious diminution in market share and reputation with adverse publicity
Key alliances are threatened
Serious legal/regulatory issues (government action, removal of officers, significant law suits)
Events and problems requires Board and executive management attention 5
Catastrophic
> $25 million impact on profitability
Imminent cash flow problems
Sustained, serious loss in market share and reputation
Sustained decline in stock price
Loss of key alliances 14

15. L i k e l i h o o d O f O c c u r r e n c e
The Tools
Heat Map
Almost Certain
Likely
Possible
Unlikely
Rare
Insignificant
Minor
Moderate
Major
Catastrophic
L i k e l i h o o d O f O c c u r r e n c e
M a g n i t u d e O f I m p a c t 15

16. L i k e l i h o o d O f O c c u r r e n c e
The Tools
Heat Map
Almost Certain
Likely
Possible
Unlikely
Rare
Insignificant
Minor
Moderate
Major
Catastrophic
L i k e l i h o o d O f O c c u r r e n c e
M a g n i t u d e O f I m p a c t 16

17. The Tools
Risk Assessment Matrix

18. The Process – Phase 1 Identify the audit objective
Link audit objective to Strategic Goals/Objectives
Document inherent risk for each audit objective
What impacts department/program from meeting goals & objectives
Rank Inherent Risk
use heat map 18

19. The Process Phase 1: 1 (1) Agency Objective (2) Audit (3)
Inherent Risk
(4) IR
Ranking
(5)
Current
Control
Activities
(6)
Residual Risk
Score
(7)
Stop/ Go
(8)
Steps 1
Exercise Fiscal Responsibility/ Maximize Agency Resources
To verify that warranty claims have been processed for all item pulled from stores that M3 identifies as under warranty
Operating Divisions are not properly processing warranty parts resulting in loss of agency monies because claims are not filed
LoO = Possible
MoI =Moderate
IRR = High

20. The Process – Phase 2 Identify Related Control Activities
Rank Residual Risk
Impact Controls have on lowering Inherent Risk Score
Summarize Issues that Impact Residual Risk Score 20

21. The Process Assessment of Controls – Impact on Inherent Risk
Level
Description
Risk Description 1
Strong
The risk management processes are very strong for the level of risk identified Control/Response is very strong.
Lowers Inherent Risk Score by 3 levels 2
Adequate
The risk management processes are appropriate for the level of risk identified Control/Response is appropriate
Lowers Inherent Risk Score by 2 levels 3
Moderate (acceptable)
The risk management processes need to be strengthened
Control/Response is not appropriate, but does not significantly expose the organization to risk
Lowers Inherent Risk Score by 1 level 4
Weak or None
Risk management processes needs to be strengthened
Control/Response is not appropriate and leaves the organization significantly exposed to risk.
Does not lower Inherent Risk Score 21

22. The Process - Phase 2 1 22 (1) Agency Objective (2) Audit (3)
Inherent Risk
(4) IR
Ranking
(5)
Current
Control
Activities
(6)
Residual Risk
Score
(7)
Stop/ Go
(8)
Steps 1
Exercise Fiscal Responsibility/Maximize Agency Resources
To verify that warranty claims have been processed for all item pulled from stores that M3 identifies as under warranty
Operating Divisions are not properly processing warranty parts resulting in loss of agency monies because claims are not filed
LoO = Likely
MoI = Moderate
IRR = High
Warranty process is manual – each operating division is required to put the “warranty” part in a bin located in middle of division floor – after new part is picked up in stock room (w/p ref C.17-5)
No reconciliation between parts pulled from stock room and parts put in bin for warranty processing
(w/p ref C.19-1)
Warranty group does track parts identified by stock room as pulled for warranty, however difficult to reconcile to warranty bin (w/p ref C.20-3)
High
- Score unchanged because weak or no controls (w/p ref D.2)
-Major critical system implemented last year (M3) (w/p ref D.2-1)
-processes have not changed to reflect new system (w/p ref D.2-4)
-store keeper control of warranty parts diffused-bin moved to floor for convenience (w/p ref E.2-9) 22

23. The Process – Phase 3 Stop/Go Analysis Fraud Brainstorming
Audit Procedures
Test and/or validate whether risk is adequately mitigated 23

24. The Process – Stop/Go Decision Tree
A “GO” IS:
Residual Risk is Critical or High OR
Residual Risk is less than high, and
Audit Objective is linked to Key Strategic Objective AND
Inherent Risk is Critical or High, AND
Audit has not recently validated controls that reduce IR, OR
Significant changes in this area subsequent to last audit 24

25. The Process - Phase 3 (7) Stop/ Go (8) Steps GO
(1)
Objective
(2)
Audit
(3)
Inherent Risk
(4) IR
Ranking
(5)
Current
Control
Activities
(6)
Residual Risk
Score
(7)
Stop/ Go
(8)
Steps 1
Exercise Fiscal Responsibility/Maximize Agency Resources
To verify that warranty claims have been processed for all item pulled from stores that M3 identifies as under warranty
Operating Divisions are not properly processing warranty parts resulting in loss of agency monies because claims are not filed
LoO= Likely
MoI =Major
IRR = High
Warranty process is manual – each operating division is required to put the “warranty” part in a bin located in middle of division floor – after new part is picked up in stock room (w/p ref C.17-5)
No reconciliation between parts pulled from stock room and parts put in bin for warranty processing
(w/p ref C.19-1)
Warranty group does track parts identified by stock room as pulled for warranty, however difficult to reconcile to warranty bin (w/p ref C.20-3)
High
- Score unchanged because weak or no controls (w/p ref D.2)
-Major critical system implemented last year (w/p ref D.2-1)
-processes have not changed to reflect new system (w/p ref D.2-4)
-store keeper control of warranty parts diffused-bin moved to floor for convenience (w/p ref E.2-9) GO
Focus of audit is on efficiency & effectiveness of processes to implement new M3 system
Fraud Brainstorming:
Review for parts replaced by Stores that are not in bin – tie to mechanic pulling parts (ref A/S 6)
- Review storekeeper records associated with most parts pulled for warranty
(ref A/S 2)
Compare 25 parts in the warranty bin on shop floor to M3
Compare 25 warranty items in M3 trace to warranted parts turned in
Confirm findings w/ store room clerk for validation.
Analyze 25 samples of w/o that require new parts “under warranty”
Analyze trends in parts pulled by stockkeepers to w/o’s in M3
Document issues identified 25

26. Summary Art rather than science
Don’t split hairs – difference of one rating should still be in ball park
Keep Focus on Big Picture
Customize the Tools so they work for you
Likelihood of Occurrence Table
Magnitude of Impact Table
Key is to build this using Senior Management input
Tie the ranking of risk to Management’s Risk Appetite
Impact of Controls on Residual Risk 26

27. Good Books on Subject Audit Planning – A Risk Based Approach
K.H. Spencer Pickett
Auditing the Risk Management Process
Control Self Assessment
CD Published by Pleier Corporation
Assessing Risk – 2nd Edition
David McNamee 27

28. Thank You Questions 28

29. All information provided is of a general nature and is not intended to address the circumstances of any particular individual or entity. Although we endeavor to provide accurate and timely information, there can be no guarantee that such information is accurate as of the date it is received or that it will continue to be accurate in the future. No one should act upon such information without appropriate professional advice after a thorough examination of the particular situation.