Preparing for an External Quality Assessment of your Quality Assurance and Improvement Program Institute of Internal Auditors El Paso Chapter August 29, 2013

Preparing for an External Quality Assessment of your Quality Assurance and Improvement Program The International Professional Practices Framework (IPPF) Standard 1300 series requires a Chief Audit Executive (CAE) to develop and maintain a Quality Assurance and Improvement Program (QAIP). A QAIP enables an evaluation of the internal audit activity’s conformance with the Definition of Internal Auditing and the Standards and an evaluation of whether internal auditors apply the Code of Ethics. The program also assesses the efficiency and effectiveness of the internal audit activity and identifies opportunities for improvement. (Source: Professional Practices Framework (IPPF)Definition of Internal AuditingCode of Ethics 8/29/20132IIA El Paso Chapter

Preparing for an External Quality Assessment of your Quality Assurance and Improvement Program 1300 – Quality Assurance and Improvement Program The chief audit executive must develop and maintain a quality assurance and improvement program that covers all aspects of the internal audit activity. Interpretation A quality assurance and improvement program is designed to enable an evaluation of the internal audit activity's conformance with the Definition of Internal Auditing and the Standards and an evaluation of whether internal auditors apply the Code of Ethics. The program also assesses the efficiency and effectiveness of the internal audit activity and identifies opportunities for improvement.​ 8/29/20133IIA El Paso Chapter

Preparing for an External Quality Assessment of your Quality Assurance and Improvement Program 1310 – Requirements of the Quality Assurance and Improvement Program The quality assurance and improvement program must include both internal and external assessments. 8/29/20134IIA El Paso Chapter

Preparing for an External Quality Assessment of your Quality Assurance and Improvement Program 1311 – Internal Assessments Internal assessments must include: 1. Ongoing monitoring of the performance of the internal audit activity; and 2. Periodic self-assessments or assessments by other persons within the organization with sufficient knowledge of internal audit practices. Interpretation Ongoing monitoring is an integral part of the day-to-day supervision, review, and measurement of the internal audit activity. Ongoing monitoring is incorporated into the routine policies and practices used to manage the internal audit activity and uses processes, tools, and information considered necessary to evaluate conformance with the Definition of Internal Auditing, the Code of Ethics, and the Standards. Periodic assessments are conducted to evaluate conformance with the Definition of Internal Auditing, the Code of Ethics, and the Standards. Sufficient knowledge of internal audit practices requires at least an understanding of all elements of the International Professional Practices Framework.​ 8/29/20135IIA El Paso Chapter

Preparing for an External Quality Assessment of your Quality Assurance and Improvement Program 1312 – External Assessments External assessments must be conducted at least once every five years by a qualified, independent assessor or assessment team from outside the organization. The chief audit executive must discuss with the board: 1. The form and frequency of external assessments; and 2. The qualifications and independence of the external assessor or assessment team, including any potential conflict of interest. Interpretation External assessments can be in the form of a full external assessment, or a self-assessment with independent external validation. A qualified assessor or assessment team demonstrates competence in two areas: the professional practice of internal auditing and the external assessment process. Competence can be demonstrated through a mixture of experience and theoretical learning. Experience gained in organizations of similar size, complexity, sector or industry, and technical issues is more valuable than less relevant experience. In the case of an assessment team, not all members of the team need to have all the competencies; it is the team as a whole that is qualified. The chief audit executive uses professional judgment when assessing whether an assessor or assessment team demonstrates sufficient competence to be qualified. An independent assessor or assessment team means not having either a real or an apparent conflict of interest and not being a part of, or under the control of, the organization to which the internal audit activity belongs. 8/29/20136IIA El Paso Chapter

Preparing for an External Quality Assessment of your Quality Assurance and Improvement Program 1320 – Reporting on the Quality Assurance and Improvement Program The chief audit executive must communicate the results of the quality assurance and improvement program to senior management and the board. Interpretation The form, content, and frequency of communicating the results of the quality assurance and improvement program is established through discussions with senior management and the board and considers the responsibilities of the internal audit activity and chief audit executive as contained in the internal audit charter. To demonstrate conformance with the Definition of Internal Auditing, the Code of Ethics, and the Standards, the results of external and periodic internal assessments are communicated upon completion of such assessments and the results of ongoing monitoring are communicated at least annually. The results include the assessor's or assessment team's evaluation with respect to the degree of conformance. 8/29/20137IIA El Paso Chapter

Preparing for an External Quality Assessment of your Quality Assurance and Improvement Program 1321 – Use of "Conforms with the International Standards for the Professional Practice of Internal Auditing" The chief audit executive may state that the internal audit activity conforms with the International Standards for the Professional Practice of Internal Auditing only if the results of the quality assurance and improvement program support this statement. Interpretation The internal audit activity conforms with the Standards when it achieves the outcomes described in the Definition of Internal Auditing, Code of Ethics, and Standards. The results of the quality assurance and improvement program include the results of both internal and external assessments. All internal audit activities will have the results of internal assessments. Internal audit activities in existence for at least five years will also have the results of external assessments. 8/29/20138IIA El Paso Chapter

Preparing for an External Quality Assessment of your Quality Assurance and Improvement Program 1322 – Disclosure of Nonconformance When nonconformance with the Definition of Internal Auditing, the Code of Ethics, or the Standards impacts the overall scope or operation of the internal audit activity, the chief audit executive must disclose the nonconformance and the impact to senior management and the board. 8/29/20139IIA El Paso Chapter

Preparing for an External Quality Assessment of your Quality Assurance and Improvement Program External Quality Assessment will review: I.Organization & Environment A. Background of Organization B. Risk Management, Governance, Accountability & Oversight II.The Internal Audit Activity A. Background of Internal Audit B. Internal Audit Practice Environment C. Relationship of IA w/Executive Mgt & Board D. Management of Internal Audit Activity E. Information Technology 8/29/201310IIA El Paso Chapter

Preparing for an External Quality Assessment of your Quality Assurance and Improvement Program External Assessments enhance value: Enable the IA Activity to evaluate conformance with the Standards. Review of IA & Audit Committee Charters. Organization’s risk & control assessments Effective use of resources Use of successful practices 8/29/201311IIA El Paso Chapter

Preparing for an External Quality Assessment of your Quality Assurance and Improvement Program Review Team Review Team is lead by a qualified audit professional. Team members should be competent professionals well versed in the Standards, assessing methodology, and IA practices. Team works on-site at the IA Activity. Use a Quality Assessment Manual Prepare a final report stating conformance or non- conformance with the standards. Along with recommendations. 8/29/201312IIA El Paso Chapter

Preparing for an External Quality Assessment of your Quality Assurance and Improvement Program Item that need to be up to date: 1)Policies & Procedures 2)Organizational Charts 3)Internal Audit Charter 4)Audit Committee Charter 5)Audit Plan & Risk Assessment 6)Audit Plan Updates 7.List of Audit Reports for the review period 8.Previous External Quality Assessment Report 9.Annual Budget 10.Staff Training Records 11.Conflict of Interest Statements 12.Objectivity Statements 13.Annual Report to Audit Committee on QIAP 8/29/201313IIA El Paso Chapter

Preparing for an External Quality Assessment of your Quality Assurance and Improvement Program First step to prepare for an External Quality Assessment is to completed a Self Assessment Review. Several reference material is available from numerous professional groups – IIA, ACUA, ACUIA, ALGA. This presentation is based on the IIA Quality Assessment Manual. 8/29/201314IIA El Paso Chapter

Preparing for an External Quality Assessment of your Quality Assurance and Improvement Program Key Aspects of a Self Assessment are: I.Positioning Is IA strategically positioned within the organization to enable it to contribute to the organization? II.People Does the IA Activity have the right people to deliver the approved Annual Audit Plan & objectives? III.Process Does the IA Activity processes enable achievement of their objectives and Annual Audit Plan? Also allow the IA Activity to be responsive to the changing needs of the organization. 8/29/201315IIA El Paso Chapter

Preparing for an External Quality Assessment of your Quality Assurance and Improvement Program Secrets to a successful Quality Assessment 1.P&P up to date 2.Audit Files in perfect condition 3.Well document QAIP 4.Independent audit function 5.Certified and well trained staff 6.Well supervised shop 7.Audit Committee 8.Process to conduct Follow-Up Audits 9.Audit Plan with good Risk Assessment tool 10.Have an independent budget 11.Great Audit Report format 12.Have a good reputation 8/29/201316IIA El Paso Chapter

Preparing for an External Quality Assessment of your Quality Assurance and Improvement Program Common Problems found during a External Assessment 1.Inappropriate CAE reporting relationships 2.Out of date charters 3.Lack of Board approved policy on internal control responsibility 4.Inadequate audit staff knowledge 5.Inadequate Quality Assurance and Improvement Program 6.Lack of understanding IA consulting responsibilities 7.Consulting omitted from the mission and charter 8.Inadequate IT coverage or technical skills 9.Lack of performance measures 8/29/201317IIA El Paso Chapter

Preparing for an External Quality Assessment of your Quality Assurance and Improvement Program Possible determination of the External Quality Assessment 1.Full Conformance with the International Professional Practices Framework. 2.Partial Conformance with exceptions with the IPPF. 3.Non-Conformance with the IPPF. 8/29/201318IIA El Paso Chapter

Preparing for an External Quality Assessment of your Quality Assurance and Improvement Program Self Assessment with Independent Validation by S/A Team 1.The S/A Team will prepare relevant portions of Self Assessment Guide and CAE Questionnaire, considering all portions but completing only those necessary to meet the scope of objective of S/A. 2.Send out Audit Client Surveys and Internal Audit Activity Staff Surveys to clients and staff. 3.Summarize & analyze the survey responses including comments for subsequent use in enhancing interviews. 4.Review a representative sample of audit & consulting engagements for review of planning, work papers, supervision, results, and communication issued. 8/29/201319IIA El Paso Chapter

Preparing for an External Quality Assessment of your Quality Assurance and Improvement Program Independent Validator’s Work Program 1.Review the Self Assessment and other advance materials. Identify items for follow-up during on-site visit. 2.On-site = review documentation of the S/A paying attention for areas of opportunities of improvement. Perform limited testing of referenced materials. Review summary of conformity with the Standards. Discuss the basis of conclusion w/CAE. Review CAE Report draft with Board/Audit Committee. 3.Conduct interviews with CEO, Audit Committee, Board Members and Clients. Gain an indication of effectiveness & credibility of IA Activity. Provides a check & balance to S/A process. 4.Prepare a Memo to serve as the basis for the Exit Conference w/CAE & S/A Team. Info will be transferred to a formal Validation Report. Contains a summary of procedures performed. Outline opportunities for improvement. Conclusion on conformity with the Standards. CAE S/A Report and Independent Validator’s Report go to Management & Board. 8/29/201320IIA El Paso Chapter

Preparing for an External Quality Assessment of your Quality Assurance and Improvement Program CONTACT INFO Edmundo S. Calderon Chief Internal Auditor City of El Paso /29/201321IIA El Paso Chapter