1. SAI Performance Measurement Framework The SAI Performance Measurement Framework Presented by: Marcela Mora, IDI Young EUROSAI Jerusalem, 10 November 2015

2. SAI Performance Measurement Framework 2 Agenda 1 Introduction 2 Group exercise 3 Group presentations 4 Sum up

3. SAI Performance Measurement Framework 3 Agenda 1 Introduction 2 Group exercise 3 Group presentations 4 Sum up

4. SAI Performance Measurement Framework 4 Workshop objectives 1.Understand what SAI PMF is 1.Purpose of undertaking a SAI PMF assessment 2.Be able to apply the SAI PMF methodology 3.How the report can be used by the SAI and external stakeholders

5. SAI Performance Measurement Framework 5 What is SAI PMF ? SAI PMF is a tool for measuring monitoring managing reporting the performance of a Supreme Audit Institution.

6. SAI Performance Measurement Framework 6 Why SAIs use SAI PMF? SAI PMF Purpose Banglade sh Barbad os Burkina Faso Brazi l Cypru s Irelan d MexicoPalestine Identify SAI strengths, weaknesses and areas for improvement XXXXXXXX Improve audit quality through supporting ISSAI implementation X XX X Strengthen internal performance management and accountability X X XX Needs assessment to assist strategic planning XXX A baseline to demonstrate future progress in performance XXX Lead by example in the public sector - assess & report on performance X XX

7. SAI Performance Measurement Framework 7 Foundation for a global measurement framework ISSAI 20 calls for SAIs to assess and report publically on their performance No internationally agreed common framework for measuring SAI’s performance The INTOSAI Congress (2010), decided to develop a framework for measuring SAI performance (PMF)

8. SAI Performance Measurement Framework 8 Six Key Decisions to be Made by the SAI Undertake SAI PMF? Assessment owner? Purpose(s) of the Assessment? Assessment Approach? 5. Timing? 6. Use of Results? 1 1 2 2 3 3 4 4 5 5 6 6

9. SAI Performance Measurement Framework 9 Defining the assessment approach 1-SAI Self Assessment 2-Peer Assessment 3-External Assessment Team from within the SAI Peer SAI INTOSAI Regional Body External auditor Development partner Consultant Hybrid

10. SAI Performance Measurement Framework 10 SAI PMF Structure Qualitative assessment (Performance Report) Measurable indicators Objective measurement to inform qualitative assessment and track progress over time Holistic assessment of the SAI and its environment – including factors which are not covered by the indicators

11. SAI Performance Measurement Framework B. Independence and Legal Framework A.SAI Reporting Value and Benefits of Supreme Audit Institutions SAI Capability Country Context, Governance and Public Financial Management D. Audit Standards and Methodology E. Management & Support Structures F. Human Resources and Leadership G. Communication and Stakeholder Management C. Strategy for Organizational Development 7 Domains Structure

12. SAI Performance Measurement Framework 12 DomainIndicator 1 Dimension 1 Criteria 1Criteria 2Criteria 3 Dimension 2Dimension 3Dimension 4 Indicator 2 Dimension 1 Criteria 1Criteria 2 Dimension 2 Criteria are taken directly from the ISSAIs or other international good practices SAI PMF Indicator Framework

13. SAI Performance Measurement Framework 13 Scoring example Case: The SAI engages actively with the media to communicate its audit reports and findings, though does not yet hold press conferences when launching its performance audit reports. Communications to civil society and citizens are through the SAI’s website and heavy use of radio, being the predominant form of communication in the country. The SAI has not yet developed mechanisms for filing complaints or whistle blowing by the public. SAI – 24 Communication with the Media, citizens and Civil Society organisations Dimension (i) Good practices regarding the communication with the media Regarding the communication with the media: a)During the period under review, the SAI held press conferences to launch its annual report and other major reports. INTOSAI Guideline “Communicating and Promoting the Value and Benefits of SAIs”:2.2 b)During the period under review, the SAI issued press releases with major reports. INTOSAI Guideline “Communicating and Promoting the Value and Benefits of SAIs”:2.2 c)During the period under review, the SAI used appropriate media to disseminate audit reports and summaries of audit reports. INTOSAI Guideline “Communicating and Promoting the Value and Benefits of SAIs”:2.2 - Dimension (i) : 2 Apply scoring table (2 dimension indicator) - Dimension (ii): 4 Overall score: 3

14. SAI Performance Measurement Framework 14 Independent Review Process Independent Review of Terms of Reference Independent Review of Draft Report Independent Review of Final Report CREDIBILITY!

15. SAI Performance Measurement Framework 15 SAI PMF Pilot Assessments (total:32, as per October 2015)

16. SAI Performance Measurement Framework 16 Agenda 1 Introduction 2 Group exercise 3 Group presentations 4 Sum up

17. SAI Performance Measurement Framework 17 Scoring Team work: 40min Presentation of Scoring by 1 group: 5min Group discussion - Q&A: 15min 1.Domain B: SAI Independence

18. SAI Performance Measurement Framework 18 Agenda 1 Introduction 2 Group exercise 3 Group presentations 4 Sum up

19. SAI Performance Measurement Framework 19 Scores: 1. 2. 3. 4. … 1.Domain B: SAI Independence ? Challenges affecting SAI Performance

20. SAI Performance Measurement Framework 20 Agenda 1 Introduction 2 Group exercise 3 Group presentations 4 Sum up

21. SAI Performance Measurement Framework 21 Conclusion SAI PMF main purpose Importance of measuring, reporting and monitoring Importance of assessing not only the normative foundations but also the actual state of affairs Challenges affecting SAI performance How to improve

22. SAI Performance Measurement Framework 22 Learning More SAI PMF: (http://www.idi.no/artikkel.aspx?MId1=102&AId=704) SAI PMF Training Course Part A – for assessors, 2.5 days Part B – deeper understanding and future trainers, 2 days Example of published SAI PMF report: 1.Bhutan SAI: http://www.bhutanaudit.gov.bt/publi%20cation/S AI%20PMF%20Report%20Bhutan.pdf http://www.bhutanaudit.gov.bt/publi%20cation/S AI%20PMF%20Report%20Bhutan.pdf 2.Nepal SAI: http://oagnep.gov.np/uploads/022- 207172-004_Peer%20Review%20by%20SAI- India%20SAI-PMF%202014(1).pdf 3.Costa Rica: https://cgrfiles.cgr.go.cr/publico/docsweb/rev _dig/inf_olacefs_2015/index.html#/1 Request a course: INTOSAI.Donor.Secretariat@idi.no