1 First External Quality Assessment by January 1, 2007: Are You Ready? The Institute of Internal Auditors October 12, 2004 Ed Dudley, CIA, CPA Retired Vice-President & General Auditor-ABB Americas

2 Introduction & Key Issues for Today Ed Dudley Quality Assurance & Improvement Program Carl Balderson Lessons Learned From 3 Quality Assessments Rick Wardrip So, You’ve Decided to Have an External QA – What’s the Process? David Walsh Break Q & A Summary of Main Points Ed Dudley Agenda

3 Key External Quality Assessment Issues for Today Key Elements of a Quality Assurance & Improvement Program Elements of Internal Quality Assessments Elements of External Quality Assessments External Quality Assessment Scopes Self Assessment & Independent Validation

4 Key External Quality Assessment Issues for Today Reasons for needing a Quality Assessment Lessons Learned From the Quality Assessment Process Communicating Results From the Quality Assessment A 12 point Quality Assurance Process Common Findings From External Quality Assessments

5 Quality Assurance & Improvement Programs Carl Balderson, CIA, CPA, CFE Director of Audit Services Pinnacle West Capital Corporation Chair, The IIA International Committee on Quality

6 Not Just External QA’s The International Standards for the Professional Practice of Internal Auditing (Standards) 1300 Et. Seq. Describes a Full “Program” Which Should Be Designed to Provide Assurance of Compliance With: –The Standards As a Whole –The IIA Code of Ethics –The Activities Described in the Definition of Internal Auditing

7 Reasonable Assurance A QA & IP Should Provide Reasonable Assurance of: –Performance in Accordance With Our Charter, Which Should Be Consistent With the Standards and Code of Ethics –Effective & Efficient Performance –Adding Value & Improving Performance –Employing Best Practices

8 Key Elements of a QA & IP Infrastructure Planning Execution Reporting & Follow-up Metrics & Measures Continuous Improvement

9 Infrastructure –Charter –Policies, Procedures, Audit Manual –Budgeting & Financial Administration –Staff Recruiting, Retention & Rotation –Staff Training and Development

10 Planning –Comprehensive Audit Universe –Risk Evaluation & Long-Range Planning –Scheduling & Time Tracking

11 Execution –Scheduling & Time Tracking –Engagement Planning –Tools & Technology –Sufficient & Competent Evidential Matter

12 Reporting & Follow-up –Reports to Management & Audit Committee –Follow-Up on Open Issues

13 Metrics & Measures –Statistics –Metrics –Customer Surveys

14 Continuous Improvement –Current Standards –Emerging Issues –Best Practices

15 Quality Program Assessments Rigorous, comprehensive process to monitor the overall effectiveness: –Routine, continuous supervision & testing of performance. –Periodic validation of compliance with the Standards. –Ongoing measurements & analyses of metrics. –Implement improvements as indicated.

16 Internal Assessments - Ongoing Engagement Supervision (PA ) Policy & Procedure Checklists Feedback (Surveys) Project Budgets, Timekeeping, etc. Analyses of Metrics Conclusions Developed & Follow-Up Action Taken

17 Internal Assessments - Periodic Non-Routine – Special Purpose Designed to Assess Compliance: –Charter –The Standards & Code of Ethics –Meeting Stakeholder Needs –Legal or Regulatory Requirements Communicate results to senior management, audit committee, and external auditors.

18 External Assessments At Least Once Every Five Years Qualified Reviewer or Review Team Independent Outside the Organization

19 External Assessors Involve senior management and the audit committee in the selection process and obtain their approval Qualifications: –Competent, Certified Audit Professionals –Current, In-Depth Knowledge of the Standards –Well versed in Best Practices –Recent Management Level Internal Audit Experience

20 External Assessment Scope Compliance Expectations of Board & Management Integration Into Governance Processes Tools & Techniques Employed Staff Knowledge, Experience and Disciplines Adding Value & Improving Operations

21 Self Assessment With Independent Validation Rigorous, Comprehensive Self Assessment Emulating the External Assessment Process Independent, On-Site Validation by a Qualified Reviewer Report Conclusions & Action Plans Concur or Dissent or Separate Report

22 Conducted in Accordance With the Standards We are encouraged to state that our work is being done in accordance with the Standards An external assessment finding such is needed to use the phrase

23 Summary Lessons Learned Making Your QA & IP Valuable Managing an External Quality Assessment Reporting Results.

24 Lessons Learned From Having Three Quality Assessments Rick Wardrip, CPA Corporate Auditor SRP (Major Water and Electric Provider in the Phoenix Metropolitan Area)

25 Overview Some Very Important Reasons Why You Need & Want to Have a Quality Assessment (QA) Lessons Learned From the QA Process How Do You Communicate the Results of the QA? Closing Thoughts

26 Reasons Why You Need A QA Are You on the Ball? Answers, “Who audits the auditors?” –Better to Have a “Peer Review” Answer the Question Being Proactive –Or We Can Have a Board Member or the External Auditor Ask the Question Being Reactive –Where Do We Prefer to Be as a CAE (Chief Audit Executive)?

27 Reasons Why You Need A QA Brings Credibility to the Department Because it Shows the Confidence of the CAE –Demonstrates to the Audit Committee and Management That the CAE Is Interested in Having the Best Operation Possible and Is Interested in Their Input to Improve

28 Reasons Why You Need A QA Brings Credibility to the Department Because it Shows the Confidence of the CAE –Having Such Reviews Is a “Best Practice” and Sends the Message That Our Goal Is to Be the Best –Provides Emphasis to All (Board, Management and Staff) That We Are a Profession and That We Take Our Profession Seriously by Abiding with the Standards

29 Reasons Why You Need A QA Brings Credibility to the Department Because It Shows the Confidence of the CAE –As a New CAE, It Is Probably the Best Thing You Can Do –It Provides a Baseline by Which Progress May Be Measured Can Help Provide a Road Map That May Help Get Resources Needed to Improve Audit Services

30 Reasons Why You Need A QA What Is the Biggest Room in the World? –Insight and Ideas From Peer Review Members Could Be the Key to Unlocking the Door to Making Your Department the Best It Can Be –Serving on an External Peer Review Team Is the Best Training Any Audit Professional Can Get. If You Serve on an IIA Team, You Receive CPE Credits

31 Reasons Why You Need A QA Are You a Smart Lumberjack? –If a CAE Has Been on the Job for Some Time, Arrange for an Internal Review –For Shortcomings Found, Implement Corrective Action by Sharpening Your Axe –Then Follow Through With the External Peer Review –Are You a Member of GAIN?

32 Lessons Learned From the QA Process Is Your Parachute Open? –Remember the Boy Scout Motto, “Be Prepared” –Recognize Your “Audit Zone” –Having a Credible and Competent Team Is the Key –Interviews Will Be Held With Your Chairman of the Board, Chairman of the Audit Committee, CEO, CFO and Other Key Senior Executives by the QA Team

33 Lessons Learned From the QA Process Progress Through Sharing –When on a QA Team the Sharing of Information Enables You to Gain So Many Ideas to Take Back and Share With Your Own Internal Audit Department Some Recent Examples Include: –Use of the “Best Practices” Document and Risk Planning Document –The Contacts You Make From a Professional Standpoint Become Invaluable

34 Lessons Learned From the QA Process Are You Ready to Take Your Eyes Off Your Own Department and Look At Another Internal Audit Department? –When on a QA Team This Enables You to “Unfocus” on Your Own Shop –Enables You to Broaden Your Horizons on How You Can Improve Your Own Internal Audit Department

35 Lessons Learned From the QA Process You Are Not an Expert Unless You Are From Out of Town –Our Own Experience Shows That the QA Helped Us Be Prepared for Future Events We Could Not Even Imagine –When Enron, WorldCom and Sarbanes-Oxley Occurred We Were Prepared to Proactively Discuss With the Audit Committee and Management How We Had Current and Best Practices in Place Two Examples Include: –Audit Committee Charter –Company Wide Statement of Policy on Internal Control

36 Lessons Learned From the QA Process Client Survey Feedback Has Been Invaluable –Evaluation Criteria Relationships With Management Audit Staff Scope of Audit Work Audit Process and Report Management of the Internal Audit Activity Value Added –80% of Managers Polled Responded –Written Comments Provide Valuable Insight

37 How Do You Communicate the Results of the QA? Have You Remembered the “No Surprise” Rule? What Is Your Corporate Culture? Be Open With the Results and Illustrate Areas Where the Department Is Strong, With the Corresponding Areas for Improvement Do You Have an Action Plan With a Timeline? Use the Results to Move Forward on Audit Initiatives –IT (Information Technology) Governance –Audit Committee Charter –Statement of Policy on Internal Control

38 How Do You Communicate the Results of the QA? Standard 1320 Requires a CAE to Report External Peer Review Results to Senior Management and the Board Reports Can Be Given at Senior Executive Meetings and at Audit Committee Meetings

39 Closing Thoughts Do All You Can to Prepare and Communicate –Then Just Be Willing to Go Through the Process –Remember, the Peer Review Team Members Are “Peers” Many of Whom Have Had Peer Reviews Done of Them, Thus They Are Generally Reasonable and Objective What Is the Biggest Room in the World?

40 Closing Thoughts What Is the Vision Statement of Your Internal Audit Department? SRP Corporate Audit Services vision is to provide the best independent and objective audit services that will meet or exceed the needs of our customers and the Standards for the Professional Practice of Internal Auditing of The Institute of Internal Auditors.

41 So, You’ve Decided to Have an External QA – What’s the Process? David Walsh, CIA, CPA Consultant – QA Project Leader

42 A 12-Point Process 1.Select a QA team 2.Prepare the self study for the team 3.Team leader makes a preliminary visit to the organization 4.Send out customer and staff surveys 5.Evaluate the internal audit activity’s effectiveness at remaining current and adding value through interviews with:  Audit Committee Chairperson  Executive management  Operating managers  Internal audit staff Forestry

43 A 12-Point Process 6.QA team performs the on-site work including:  Review of administrative policies and procedures  Consideration of enterprise risk  Evaluation of risk assessment in audit planning  Review of working papers and final reports for selected engagements  Review of number and skills of internal audit staff  Evaluate adequacy of IT audit coverage 7.Evaluate coordination of internal audit work with that of independent auditors and other monitoring functions. 8.Evaluate the internal audit activity’s conformance with IIA Standards and other relevant standards.

44 A 12-Point Process 9.Review quality/process improvement actions currently underway 10.Provide a summary of issues and recommendations, and hold a closing conference with the CAE and/or other requestors. 11.Draft a report, obtain comments, and issue a final report 12.Hold a follow-up executive conference (optional) Forestry

45 Common QA Findings Inappropriate CAE reporting relationships Out-of-date charters for the internal audit activity and/or audit committee Lack of board approved policy on internal control responsibility Client perception of inadequate audit staff knowledge Lack of a formalized risk assessment process Lack of understanding regarding: –Internal audit activity’s consulting responsibilities –Reflection of consulting in the mission and charter Inadequate IT coverage or technical skills Forestry

46 Common QA Findings Insufficient Training Risk Assessment Deficiencies Audit Universe Deficiencies Lack of Performance Measurements Failure to Track Auditors’ Time Poor Cycle Time

47 Common QA Findings Inconsistent Workpapers Non-electronic Workpapers Audit Report Deficiencies, e.g., lack of Executive Summary, too long Customers Saying Auditors Don’t Know Their Business No overall report conclusion Findings aren’t ranked according to risk

48 Q & A your questions to or use the link on the webcast view page

49 Summary Of Main Points Infrastructure, Planning, Execution, Reporting & Follow-up, Metrics and Continuous Improvement Are Essential Internal Assessments Should Be Designed to Assess Both Compliance and Stakeholder Needs External Assessments Must Be Done at Least Once Every Five Years External Assessors Must Be Fully Qualified

50 Summary Of Main Points External Assessment Scope Should Be Integrated Into the Governance Processes Comprehensive Self Assessment Should Mirror the External Assessment Process & Be in Accordance With The IIA Standards Quality Assessment Brings Credibility to the Organization, Showing Confidence of the CAE

51 Summary Of Main Points Quality Assessments of Others Allow You to Improve Your Own Internal Audit Organization Standard 1320 Requires a CAE to Report External Peer Results to Both Senior Management and Board Many Common Findings Relate to Both Internal Audit and Audit Committees

52 Summary Of Main Points Prepare for an External Quality Assessment to Enhance Your Organization and Show That Professionally We Are in Compliance With One Set of Standards Worldwide

53 Get Your CPE Certificate: If you are a primary Webcast participant: If you view the live Webcast, you should be receiving your CPE certificate via today. You can also view the certificate in your account. Just log in and hit the “CPE” button. If you are viewing the archived Webcast, you will have to take the corresponding quiz which you will find in your webcast account. If you are not the primary participant but will be viewing the Webcast: Additional viewers may obtain CPE for a $15 administrative fee per additional viewer per Webcast. Register online at

54 November 16, 2004 “ Business Conduct & Ethics ”

55 Webcast Evaluation Visit the Login Page