1. SAI Performance Measurement Framework SAI PMF - development of a global performance measurement tool for Supreme Audit Institutions Presented by: Camilla C. Fredriksen, IDI Place and Date: Recife, 3 December 2015 XXIVIII Congresso dos Tribunais de Contas do Brasil

2. SAI Performance Measurement Framework 2 Presentation Outline 1.The development process 2.What is SAI PMF? 3.Roll-out strategy 4.Experience from pilot assessments 5.Further development of SAI PMF

3. SAI Performance Measurement Framework 3 1. The development of SAI PMF

4. SAI Performance Measurement Framework 4 The starting point: An International Agreement to Develop a Measurement Framework for SAIs

5. SAI Performance Measurement Framework 5 International Declarations Lima Declaration, October 1977 Mexico Declaration on SAI Independence, 2007 United Nations, resolution A/66/209 – Promoting the efficiency, accountability, effectiveness and transparency of public administrations by strengthening Supreme audit Institutions (22 December 2011)

6. SAI Performance Measurement Framework 6 How to ensure implementation of the UN declaration: Full set of International Standards of Supreme Audit Institutions (ISSAIs) endorsed by INTOSAI in 2010 INTOSAI Framework on the Value and Benefits of SAIs adopted 2010 Increased focus on strengthening of SAIs within the international donor community (INTOSAI-Donor MoU signed 2009) Decision of 2010 INTOSAI Congress to Develop Performance Measurement Tool

7. SAI Performance Measurement Framework 7 Mandate: Johannesburg Accords (2010) Working Group on the Value and Benefits of SAI Responsibility Develop further credible measurement tools to support evaluations in terms of the framework for Communicating and Promoting the Value and Benefits of SAIs Task INTOSAI Capacity Building Committee INTOSAI-Donor Steering Committee Collaborating with Tools that are already available at the level of… international organizations Cognizance of First pilot version for consideration at XXI INCOSAI in 2013 Timetable

8. SAI Performance Measurement Framework 8 Responsibilities and Reporting Lines INTOSAI Working Group on the Value and Benefits of SAIs Oversight and Direction WGVBS SAI PMF Task Team Development of SAI Performance Measurement Framework Coordinator: INTOSAI-Donor Secretariat @ IDI SAI PMF Reference Group Technical Advice, Review and Quality Assurance INTOSAI Congress Approval INTOSAI Capacity Building Committee Members INTOSAI-Donor Steering Committee Members Other Groups: PEFA Secretariat, International Budget Partnership INTOSAI Capacity Building Committee Members INTOSAI-Donor Steering Committee Members In Consultation with: INTOSAI Capacity Building Committee INTOSAI-Donor Steering Committee Other Relevant INTOSAI Committees

9. SAI Performance Measurement Framework 9 Development of SAI PMF WGVBS endorsement of Concept Note, project plan (2012) Mandate from INCOSAI (2010) 1st round of consultations and pilots - SAI PMF v 2.0 (2012) SAI PMF Pilot Version endorsed by WGVBS (2013) 2nd pilot phase (2013-2015) Consultations on Pilot Version (Dec 2014- March 2015) Development and approval of Endorsement Version (2015-2016) Endorsement at INCOSAI 2016

10. SAI Performance Measurement Framework 10 2. What is SAI PMF?

11. SAI Performance Measurement Framework 11 What is the SAI Performance Measurement Framework? A Performance Measurement Framework tailored to SAIs Provides a holistic, high level overview of SAI performance against established INTOSAI good practice: – ISSAIs – The Framework on the Value and Benefits of SAIs; – Other INTOSAI guidance material and international good practices. Provides a detailed assessment of strengths, weaknesses and capacity development needs Contributes to identifying the value and benefits of the SAI to citizens Facilitate managing, measuring and monitoring of SAI performance

12. SAI Performance Measurement Framework 12 Purposes 1.Enhance own performance 2.Internal performance measurement / annual reporting 3.Inform and secure support for capacity development initiatives 4.To demonstrate progress and value and benefits to society 5.As a step towards ISSAI implementation

13. B. Independence and Legal Framework A.SAI Reporting Value and Benefits of Supreme Audit Institutions SAI Environment SAI Capability Country Context, Governance and Public Financial Management D. Audit Standards and Methodology SAI SUPPORT SERVICES E. Management & Support Structures F. Human Resources and Leadership G. Communication and Stakeholder Management C. Strategy for Organizational Development SAI STRATEGY SAI CORE BUSINESS SAI Performance Measurement Framework SAI Reporting SAI Impact

14. SAI Performance Measurement Framework 14 SAI PMF: Evidence Based Assessment of Performance Qualitative assessment (Performance Report) Measurable indicators Objective measurement to inform qualitative assessment and track progress over time Holistic assessment of the SAI and its environment – including factors which are not covered by the indicators

15. SAI Performance Measurement Framework 15 3. Roll-out strategy

16. SAI PMF Piloting Activities Piloting SAI PMF - 20+ countries, all INTOSAI regions - Guidance material developed - Quality assurance of draft reports - Assessments supported by network of peers, donors, and IDI/IDS Training on SAI PMF - 800 trained assessors - Training for users and facilitators - English, French, Spanish, Arabic, Portuguese Community of Practice - SAI PMF Virtual Community (English, Spanish, French coming) - Knowledge sharing and quality assurance workshops for experienced assessors Improvement of SAI PMF - Consultation phase - Experiences from pilots - Task team meetings and work

17. SAI Performance Measurement Framework 17 4. Experience from pilot assessments

18. SAI Performance Measurement Framework 18 Progress on SAI PMF Pilot phase 2 Target in Roll-out strategy: test SAI PMF Pilot version in about 20 SAIs with different characteristics Target reached

19. SAI Performance Measurement Framework 19 Bhutan Cuba El Salvador Guatemala Czech Rep Vietnam Ireland Palestine Mexico Afghanistan Oman Argentina Dominican Republic Costa Rica Nicaragua Ecuador Brazil Burkina Faso Barbados Colombia Mozambique Somalia Bangladesh Nepal SAI PMF Pilot Phase 1 & 2 Final reportDraft report Assessment decided by head of SAI Honduras Peru Cyprus Trinidad and Tobago Cook Islands Paraguay Sierra Leone Norway Djibouti Slovak Republic Latvia Full color: based on SAI PMF Pilot Version Gradient color: based on earlier versions of SAI PMF Swaziland Saudi Arabia Guam Fiji

20. SAI Performance Measurement Framework 20 Quality AssuranceAssessment Approaches Crucial for quality and credibility of reports – IDS published QA guidance 2014 IDS offer to do/arrange IR of all reports IDS works to establish pool of SAI PMF experts (potential QA reviewers)

21. SAI Performance Measurement Framework 21 SAI PMF - Meets its Purpose …as basis for strategic planning..for identification of capacity development needs Strong tool for measuring SAI performance

22. SAI Performance Measurement Framework 22 5. Further development of SAI PMF

23. SAI Performance Measurement Framework 23 Recommendations 4th Task Team meeting Based on -feedback from pilots -analysis of scores -discussions at the four SAI PMF quality assurance and knowledge sharing workshops held in 2014 and 2015 - consultation comments

24. SAI Performance Measurement Framework 24 High level Recommendations Task Team Both high level and detailed assessment – but avoid too much details Professional Judgement – Flexibility in useScores have a fair distribution overall

25. SAI Performance Measurement Framework 25 Development of SAI PMF Endorsement Version Direction approved by the INTOSAI Working Group on Values and Benefits of SAIs (WGVBS) in September 2015 The WGVBS endorsed the SAI PMF Task Team’s considerations regarding consultation comments and experiences from pilots, including its recommendations for the development of the SAI PMF Endorsement Version

26. SAI Performance Measurement Framework Country governance, political, social, cultural + public financial management environment SAI contribution to Impact SAI outcomes SAI outputs SAI organisation systems + professional staff capacity SAI institutional capacity Independence + legal framework Internal governance and ethics + SAI core services FA standards, process, quality + results PA standards, process, quality + results CA standards, process, quality + results other core services + results Judgement process + results Corporate services HRM + Professional staff development A A C C accountability reporting D D E E Audits recommendations implemented Improved PMF systems Public confidence in government financial systems Cost savings + efficiency gains Sanctions + surcharges Stakeholders engagement in accountability Value and benefits of SAIs Accountability Transparency Reduced corruption Better services for citizens SAI capacity Public confidence in the SAI Improve compliance with laws and regulations SAI culture + leadership B B Communication and Stakeholder management F F Ruling / Decision

27. SAI Performance Measurement Framework 27 Thank you for your attention! www.idi.no E-mail: INTOSAI.Donor.Secretariat@idi.noINTOSAI.Donor.Secretariat@idi.no