1. PROGRESS REPORT ON THE TASK FORCE ON AUDITS QUALITY CONTROL
Agenda Item #03
PROGRESS REPORT ON THE TASK FORCE ON AUDITS QUALITY CONTROL
Mrs. Mirtha Piscoya Supervisor at SAI of Peru
Mr. Hasan Masud
WGVBS Liaison Official at the SAI of Pakistan

2. Quality Control Background
Relevance for the SAIs to count on a suitable quality control
Ensuring an adequate level of reliability on the SAIs work

3. Quality Control Background
Preventive actions
Control implementation
Risk assessments
Integrity policies against impunity
Accuracy in SAIs reports

4. Quality Control Origins of the Task Team on Audits Quality Control
Reflections on “Audits Quality Control” (2016 WGVBS Meeting, Arusha)
Different institutional arrangements regarding the implementation of a quality control policy
Importance of the tone of the top for not limiting quality control to products, but also consider outcomes
Relevance of institutional principles, guiding the quality management
Reference on SAI PMF (among other references) for SAIs to verify quality assurance

5. Quality Control Origin of the Task Team on Audits Quality Control
Implementation of Quality Control in the Audit Process (Paper draft by a voluntary team)
beyond the process defined in ISSAI 40
level of independence, mandate and position of the responsible unit within the institution
guiding principles of the unit responsible for audit quality assurance
Consideration of IDI´s information on SAIs experience after implementing SAI PMF

6. Quality Control Origin of the Task Team on Audits Quality Control
Call to form the Task Team
Group (Task Team) formation
Perspectives on how to proceed forward
Diagnosis
2017 – 2018
XXIII INCOSAI (2019)
January – March 2017
Paper – Guideline?
January – March 2017
December 2016 – January 2017

7. Quality Control Origin of the Task Team on Audits Quality Control
Austria
China
Iraq
Pakistan
Peru
United States of America
Task Team Members

8. Quality Control Perspectives of the Task Team on Audits Quality Control
Use of INTOSAI Community Portal
Coordination with the IDI (Global Survey, February 2017)
Maintaining flexible / opened to outputs
Potential future guidance (coordination with FIPP)
Establishment of requirements and methodology for quality reviewers

9. SAI of Peru SAI of Pakistan
Quality Control Perspectives of the Task Team on Audits Quality Control
SAI of Peru
SAI of Pakistan

10. Quality Control SAI of Pakistan’s perspectives
Developing Audit Quality Control Mechanism within INTOSAI Member States
Hasan Masud
DIRECTOR
SAI PAKISTAN

11. Establishing such a mechanism at the INTOSAI level…
Quality Control Why Review Quality?
serves as critical external validation function for any organization
Auditing Independent & objective examination of financial and non financial data
Governments work, not for profit, but basically for the welfare of its people
Public sector
Not only enhance the quality of public sector reports, but also promote free flow of ideas between various SAIs
Establishing such a mechanism at the INTOSAI level…

12. Reasons for having in place an audit quality control mechanism
Quality Control Why Review Quality?
Reasons for having in place an audit quality control mechanism
Quality reviews aim at assuring that internal quality systems are in place
Established policies and procedures are being followed
Applicable audit standards are being followed

13. Selection of quality reviewer
Quality Control Characteristics of a Quality Reviewer
Review can be developed as a multilateral exercise
Auditors from various SAIs may work as members of a joint team to review auditing standards and practices in a member country
Selection of quality reviewer
Adequately qualified & and have current knowledge of the type of work to be reviewed and the applicable auditing standards
Independent of the audit organization being reviewed
Proper supervision must be in place to ensure an adequate review work

14. Public sector Audit Reports
Quality Control Areas to be Reviewed in a Quality Review
Public sector Audit Reports
Go beyond the usual
Help SAIs to work as custodians of public money
Quality Assurance
Most significant tool for managing the audit function in any organization
Key features
Analyzing a reasonable cross section of audit reports.
Examining Audit working papers and planning documents.
Focusing on the Audit charter, Code of ethics, Policies and procedures.
Training plans of the organization.
Job descriptions and job requirements

15. Quality Review process
Quality Control The Methodology
Quality Review process
Quantitative and qualitative techniques
Key methods while developing quality review teams
A quality review team has to be an adequately qualified set of auditors, independent from the national process being reviewed.
Organization’s charter, policies, procedures, training records and annual planning processes may be examined.
Audit reports, working papers and other related documentation may be examined.
Written report may be submitted to the head of the team.

16. Quality Control The Methodology (Continued)
It needs to be understood that all reports generated, any supporting documents or working papers prepared will be property of the INTOSAI Secretariat. It is important to note that this will be a quality review only of the Audit Reports as per the laid down national and INTOSAI standards and not an audit exercise in itself. Following are the steps which may be taken as part of the process:
The review team may select a completed audit or group of audits to review.
The audit report and supporting working papers are examined to assess overall quality and compliance with national and INTOSAI standards.
Results of interviews and surveys of stakeholders.
A written report is prepared for the head of the audit unit.

17. Quality Control The Results of Quality Reviews
Quality review is not an audit but a review of the process of auditing.
Quality reviews should give the idea how the work compares to the applicable standards.
The performance of process reviewed in comparison to similar audit organizations in fellow member organizations in the INTOSAI framework.
Quality Review process is key factor in ensuring uniformity in practice and exchange of latest ideas in the field throughout the INTOSAI member states.
These reviews should provide an opportunity to make improvement of the auditing process and the results.
It is expected that the users of audit information will have greater confidence in the process as a result of the quality review.

18. Quality Control SAI Peru’s perspectives
An approach to SAIs audit quality control in OLACEFS

19. Quality Control SAI Peru’s perspectives
«Striving for service excellence and quality»
(ISSAI 12 - principle 11)

20. BACKGROUND PERU
Audit standards establish quality controls both during the audit process and the issuance of reports.
2012: Quality policy and objectives are endorsed
2012 – 2016: Implementation of the quality management and quality assurance systems
ISO 9001:2008 certification in 5 process (audit excluded)
2015 – 2016: Application of SAI PMF and opportunity to improve the coverage and effectiveness of the control system and quality assurance.

21. SAIs QUALITY CONTROL IN OLACEFS
Activity: a 17 questions survey
Objective: Identify models for the audit quality control in the region.
References: ISSAI 40, ISSAI 12, SAI PMF
Results: Response of 12 SAIs out of 22 in the regional organization.
Participants: Bolivia, Chile, Colombia, Costa Rica, Cuba, Ecuador, El Salvador, Honduras, Mexico, Panama, Paraguay, and Uruguay.

22. 1. QUALITY CONTROL SYSTEM
2 SAIs reported having a implemented quality system under ISO 9001: 2008.
1 SAI responded “no” stating that its quality and processes handbook is in process, based on the ISO 9001:2008.

23. 2. IMPLEMENTATION OF ISSAI 40
66% of the surveyed SAIs have implemented methodologies and procedures to respond to the new mandates on quality control.
2 of those SAIs referred to have considered ISSAI 40.
2 SAIs have implemented quality control in accordance with ISO 9001:2008. Such fact reveals that there is a high level of consistency between ISSAI 40 and the ISO Standard, in compliance with ISSAI 40 requirements.

24. 3. REGULATIONS GOVERNING QUALITY CONTROL IN SAIs
Constitution
Laws
Government audit standards
Internal control standards
Procedures and rules
ISO 9001: 2008
75 % of SAIs stated to have the governing regulations on quality control

25. 4. POLICIES AND PROCEDURES TO PROMOTE AN INTERNAL QUALITY CULTURE IN WORKS
75% of SAIs stated to have applied policies, gathered in the plans and codes of good governance.
PROCEDURES
Socialization campaign and awareness-raising on line.
Working tables,
Monthly training,
Individual assessment at the end of every audit.
Proficiency Knowledge test on audit and code of ethics
Communiques, bulletins, and intranet.

26. 5. CODE OF ETHICS (Drafting, dissemination, implementation, and update)
92% of SAIs reported to have a code of ethics or an integrity policy.
Dissemination of the Code: Training session, awareness-raising, group discussion, appointing officials promoting a culture of values and the code of ethics, intranet.
Code update: mostly in the Organization Committee of Ethics
A SAI has implemented a ethical line of complaints to detect transgressions either inside the SAI or by the citizens.

27. 6. IDENTIFICATION OF ETHICAL RISKS
58% of SAIs stated have identified ethical risks.
2 SAIs stated that the identification of the ethical risks was favored by the application of tools such as IntoSAINT and SAI PMF. On the other hand, a SAI stated that the Committee of Ethics carried out the integrity risk evaluation based on an anonymous survey.
2 SAIs referred to have identified the risks based on complaints
Mitigation of ethical risks is performed through training, advice by the officer responsible for promoting values in their SAI, disciplinary procedure.

28. 7. CLEAR APPOINTMENT OF AUTHORITY, AND ALLOCATION OF RESPONSIBILITIES TO COLLABORATORS
92 % of SAIs have appointed an authority, in charge of the operation manual or operation regulations, description manuals of profiles and positions.

29. 8. POLICIES AND PROCEDURES TO ADDRESS ETHICAL NON-COMPLIANCE
Only 2 SAIs referred to have an established procedure to notify the Head of the institution on ethical non-compliance.
The remaining SAIs address ethical non-compliance according to the Code of Ethics and regulations on conduct through the Committee of Ethics or the Human Resources dept.

30. 9. INDICATORS TO MEASURE QUALITY MANAGEMENT
75 % of surveyed SAIs have indicators to measure their performance (efficiency, effectiveness).
2 SAIs stated that indicators are part of the ISO 9001:2008.
1 SAI referred to have conducted satisfaction Surveys to customers and control subjects.

31. 2 SAIs have risk prevention plans in their audit.
10. INSTITUTIONAL SYSTEMS TO IDENTIFY RISKS THAT MAY AFFECT QUALITY THE PERFORMANCE OF SAI’s WORk
67% of surveyed SAIs have procedures in place to identify the risks that may affect the quality of SAI’s work, mainly during the audit planning.
2 SAIs have risk prevention plans in their audit.
1 SAI has implemented an automatized system for the administration of strategic risks COSO 2013, which will progressively include the identification and management of all operational risks.

32. 11 RESULTS ON THE IMPLEMENTATION OF QUALITY MANAGEMENT.
BENEFITS
SAIs consider to have contributed in the strengthening of audit process and to align international good practices.
Relevance of SAI and their role.
For a SAI to make a difference in the lives of citizens, SAIs should be able to:
5.1 Strengthen accountability, transparency and integrity of the government and public entities;
5.2 Demonstrate the constant relevance for citizens, the parliament and other stakeholders, and
5.3 Being a model organization, leading by example.

33. SOME THOUGHTS
Quality control have acquired a higher relevance. In a context where SAIs are expected to obtain the results valued by the citizens, they should strive for service excellence and quality (ISSAI 12 principle 11)
There are tools allowing us to measure the performance of SAIs, in terms of quality control within the SAI and regarding the audit process, such as SAI PMF (dimension C), IntoSAINT (integrity), and iCATs (3i Programme). It would be interesting to know whether SAIs are using them.
Need to identify the aspects of quality control where SAIs do the specifications (e.g. notice to the Head regarding ethical risks).
There is an opportunity to learn from good practices and lessons learned to supplement ISSAI 40.
Evaluate the relationship between ISO and ISSAI 40, as some SAIs are implementing them (6).

34. Motions (paper beyond and supplementing ISSAI 40)
Extend the diagnosis made in the OLACEFS region to the INTOSAI community
Considering the use of tools to measure the performance of SAIs in terms of quality control
Results to be presented in 2018 (11th WGVBS meeting)
Draft a paper on the implementation of Quality Control in the Audit Process
level of independence, mandate and position of the responsible unit within the institution
guiding principles of the unit responsible for audit quality assurance (requirements and methodology for quality reviewers)
Potential guideline to be presented by the WGVBS before the XXIII INCOSAI, 2019
Coordination with FIPP and KSC Chair

35. PROGRESS REPORT ON THE TASK FORCE ON AUDITS QUALITY CONTROL
Agenda Item #03
PROGRESS REPORT ON THE TASK FORCE ON AUDITS QUALITY CONTROL
Mrs. Mirtha Piscoya Supervisor at SAI of Peru
Mr. Hasan Masud
WGVBS Liaison Official at the SAI of Pakistan