PROGRESS REPORT ON THE TASK FORCE ON AUDITS QUALITY CONTROL Agenda Item #03 PROGRESS REPORT ON THE TASK FORCE ON AUDITS QUALITY CONTROL Mrs. Mirtha Piscoya Supervisor at SAI of Peru Mr. Hasan Masud WGVBS Liaison Official at the SAI of Pakistan

Quality Control Background Relevance for the SAIs to count on a suitable quality control Ensuring an adequate level of reliability on the SAIs work

Quality Control Background Preventive actions Control implementation Risk assessments Integrity policies against impunity Accuracy in SAIs reports

Quality Control Origins of the Task Team on Audits Quality Control Reflections on “Audits Quality Control” (2016 WGVBS Meeting, Arusha) Different institutional arrangements regarding the implementation of a quality control policy Importance of the tone of the top for not limiting quality control to products, but also consider outcomes Relevance of institutional principles, guiding the quality management Reference on SAI PMF (among other references) for SAIs to verify quality assurance

Quality Control Origin of the Task Team on Audits Quality Control Implementation of Quality Control in the Audit Process (Paper draft by a voluntary team) beyond the process defined in ISSAI 40 level of independence, mandate and position of the responsible unit within the institution guiding principles of the unit responsible for audit quality assurance Consideration of IDI´s information on SAIs experience after implementing SAI PMF

Quality Control Origin of the Task Team on Audits Quality Control Call to form the Task Team Group (Task Team) formation Perspectives on how to proceed forward Diagnosis 2017 – 2018 XXIII INCOSAI (2019) January – March 2017 Paper – Guideline? January – March 2017 December 2016 – January 2017

Quality Control Origin of the Task Team on Audits Quality Control Austria China Iraq Pakistan Peru United States of America Task Team Members

Quality Control Perspectives of the Task Team on Audits Quality Control Use of INTOSAI Community Portal Coordination with the IDI (Global Survey, February 2017) Maintaining flexible / opened to outputs Potential future guidance (coordination with FIPP) Establishment of requirements and methodology for quality reviewers

SAI of Peru SAI of Pakistan Quality Control Perspectives of the Task Team on Audits Quality Control SAI of Peru SAI of Pakistan

Quality Control SAI of Pakistan’s perspectives Developing Audit Quality Control Mechanism within INTOSAI Member States Hasan Masud DIRECTOR SAI PAKISTAN

Establishing such a mechanism at the INTOSAI level… Quality Control Why Review Quality? serves as critical external validation function for any organization Auditing Independent & objective examination of financial and non financial data Governments work, not for profit, but basically for the welfare of its people Public sector Not only enhance the quality of public sector reports, but also promote free flow of ideas between various SAIs Establishing such a mechanism at the INTOSAI level…

Reasons for having in place an audit quality control mechanism Quality Control Why Review Quality? Reasons for having in place an audit quality control mechanism Quality reviews aim at assuring that internal quality systems are in place Established policies and procedures are being followed Applicable audit standards are being followed

Selection of quality reviewer Quality Control Characteristics of a Quality Reviewer Review can be developed as a multilateral exercise Auditors from various SAIs may work as members of a joint team to review auditing standards and practices in a member country Selection of quality reviewer Adequately qualified & and have current knowledge of the type of work to be reviewed and the applicable auditing standards Independent of the audit organization being reviewed Proper supervision must be in place to ensure an adequate review work

Public sector Audit Reports Quality Control Areas to be Reviewed in a Quality Review Public sector Audit Reports Go beyond the usual Help SAIs to work as custodians of public money Quality Assurance Most significant tool for managing the audit function in any organization Key features Analyzing a reasonable cross section of audit reports. Examining Audit working papers and planning documents. Focusing on the Audit charter, Code of ethics, Policies and procedures. Training plans of the organization. Job descriptions and job requirements

Quality Review process Quality Control The Methodology Quality Review process Quantitative and qualitative techniques Key methods while developing quality review teams A quality review team has to be an adequately qualified set of auditors, independent from the national process being reviewed. Organization’s charter, policies, procedures, training records and annual planning processes may be examined. Audit reports, working papers and other related documentation may be examined. Written report may be submitted to the head of the team.

Quality Control The Methodology (Continued) It needs to be understood that all reports generated, any supporting documents or working papers prepared will be property of the INTOSAI Secretariat. It is important to note that this will be a quality review only of the Audit Reports as per the laid down national and INTOSAI standards and not an audit exercise in itself. Following are the steps which may be taken as part of the process: The review team may select a completed audit or group of audits to review. The audit report and supporting working papers are examined to assess overall quality and compliance with national and INTOSAI standards. Results of interviews and surveys of stakeholders. A written report is prepared for the head of the audit unit.

Quality Control The Results of Quality Reviews Quality review is not an audit but a review of the process of auditing. Quality reviews should give the idea how the work compares to the applicable standards. The performance of process reviewed in comparison to similar audit organizations in fellow member organizations in the INTOSAI framework. Quality Review process is key factor in ensuring uniformity in practice and exchange of latest ideas in the field throughout the INTOSAI member states. These reviews should provide an opportunity to make improvement of the auditing process and the results. It is expected that the users of audit information will have greater confidence in the process as a result of the quality review.

Quality Control SAI Peru’s perspectives An approach to SAIs audit quality control in OLACEFS

Quality Control SAI Peru’s perspectives «Striving for service excellence and quality» (ISSAI 12 - principle 11)

BACKGROUND PERU Audit standards establish quality controls both during the audit process and the issuance of reports. 2012: Quality policy and objectives are endorsed 2012 – 2016: Implementation of the quality management and quality assurance systems ISO 9001:2008 certification in 5 process (audit excluded) 2015 – 2016: Application of SAI PMF and opportunity to improve the coverage and effectiveness of the control system and quality assurance.

SAIs QUALITY CONTROL IN OLACEFS Activity: a 17 questions survey Objective: Identify models for the audit quality control in the region. References: ISSAI 40, ISSAI 12, SAI PMF Results: Response of 12 SAIs out of 22 in the regional organization. Participants: Bolivia, Chile, Colombia, Costa Rica, Cuba, Ecuador, El Salvador, Honduras, Mexico, Panama, Paraguay, and Uruguay.

1. QUALITY CONTROL SYSTEM 2 SAIs reported having a implemented quality system under ISO 9001: 2008. 1 SAI responded “no” stating that its quality and processes handbook is in process, based on the ISO 9001:2008.

2. IMPLEMENTATION OF ISSAI 40 66% of the surveyed SAIs have implemented methodologies and procedures to respond to the new mandates on quality control. 2 of those SAIs referred to have considered ISSAI 40. 2 SAIs have implemented quality control in accordance with ISO 9001:2008. Such fact reveals that there is a high level of consistency between ISSAI 40 and the ISO Standard, in compliance with ISSAI 40 requirements.

3. REGULATIONS GOVERNING QUALITY CONTROL IN SAIs Constitution Laws Government audit standards Internal control standards Procedures and rules ISO 9001: 2008 75 % of SAIs stated to have the governing regulations on quality control

4. POLICIES AND PROCEDURES TO PROMOTE AN INTERNAL QUALITY CULTURE IN WORKS 75% of SAIs stated to have applied policies, gathered in the plans and codes of good governance. PROCEDURES Socialization campaign and awareness-raising on line. Working tables, Monthly training, Individual assessment at the end of every audit. Proficiency Knowledge test on audit and code of ethics Communiques, bulletins, email and intranet.

5. CODE OF ETHICS (Drafting, dissemination, implementation, and update) 92% of SAIs reported to have a code of ethics or an integrity policy. Dissemination of the Code: Training session, awareness-raising, group discussion, appointing officials promoting a culture of values and the code of ethics, intranet. Code update: mostly in the Organization Committee of Ethics A SAI has implemented a ethical line of complaints to detect transgressions either inside the SAI or by the citizens.

6. IDENTIFICATION OF ETHICAL RISKS 58% of SAIs stated have identified ethical risks. 2 SAIs stated that the identification of the ethical risks was favored by the application of tools such as IntoSAINT and SAI PMF. On the other hand, a SAI stated that the Committee of Ethics carried out the integrity risk evaluation based on an anonymous survey. 2 SAIs referred to have identified the risks based on complaints Mitigation of ethical risks is performed through training, advice by the officer responsible for promoting values in their SAI, disciplinary procedure.

7. CLEAR APPOINTMENT OF AUTHORITY, AND ALLOCATION OF RESPONSIBILITIES TO COLLABORATORS 92 % of SAIs have appointed an authority, in charge of the operation manual or operation regulations, description manuals of profiles and positions.

8. POLICIES AND PROCEDURES TO ADDRESS ETHICAL NON-COMPLIANCE Only 2 SAIs referred to have an established procedure to notify the Head of the institution on ethical non-compliance. The remaining SAIs address ethical non-compliance according to the Code of Ethics and regulations on conduct through the Committee of Ethics or the Human Resources dept.

9. INDICATORS TO MEASURE QUALITY MANAGEMENT 75 % of surveyed SAIs have indicators to measure their performance (efficiency, effectiveness). 2 SAIs stated that indicators are part of the ISO 9001:2008. 1 SAI referred to have conducted satisfaction Surveys to customers and control subjects.

2 SAIs have risk prevention plans in their audit. 10. INSTITUTIONAL SYSTEMS TO IDENTIFY RISKS THAT MAY AFFECT QUALITY THE PERFORMANCE OF SAI’s WORk 67% of surveyed SAIs have procedures in place to identify the risks that may affect the quality of SAI’s work, mainly during the audit planning. 2 SAIs have risk prevention plans in their audit. 1 SAI has implemented an automatized system for the administration of strategic risks COSO 2013, which will progressively include the identification and management of all operational risks.

11 RESULTS ON THE IMPLEMENTATION OF QUALITY MANAGEMENT. BENEFITS SAIs consider to have contributed in the strengthening of audit process and to align international good practices. Relevance of SAI and their role. For a SAI to make a difference in the lives of citizens, SAIs should be able to: 5.1 Strengthen accountability, transparency and integrity of the government and public entities; 5.2 Demonstrate the constant relevance for citizens, the parliament and other stakeholders, and 5.3 Being a model organization, leading by example.

SOME THOUGHTS Quality control have acquired a higher relevance. In a context where SAIs are expected to obtain the results valued by the citizens, they should strive for service excellence and quality (ISSAI 12 principle 11) There are tools allowing us to measure the performance of SAIs, in terms of quality control within the SAI and regarding the audit process, such as SAI PMF (dimension C), IntoSAINT (integrity), and iCATs (3i Programme). It would be interesting to know whether SAIs are using them. Need to identify the aspects of quality control where SAIs do the specifications (e.g. notice to the Head regarding ethical risks). There is an opportunity to learn from good practices and lessons learned to supplement ISSAI 40. Evaluate the relationship between ISO and ISSAI 40, as some SAIs are implementing them (6).

Motions (paper beyond and supplementing ISSAI 40) Extend the diagnosis made in the OLACEFS region to the INTOSAI community Considering the use of tools to measure the performance of SAIs in terms of quality control Results to be presented in 2018 (11th WGVBS meeting) Draft a paper on the implementation of Quality Control in the Audit Process level of independence, mandate and position of the responsible unit within the institution guiding principles of the unit responsible for audit quality assurance (requirements and methodology for quality reviewers) Potential guideline to be presented by the WGVBS before the XXIII INCOSAI, 2019 Coordination with FIPP and KSC Chair

PROGRESS REPORT ON THE TASK FORCE ON AUDITS QUALITY CONTROL Agenda Item #03 PROGRESS REPORT ON THE TASK FORCE ON AUDITS QUALITY CONTROL Mrs. Mirtha Piscoya Supervisor at SAI of Peru Mr. Hasan Masud WGVBS Liaison Official at the SAI of Pakistan