Approaches to PFM reform Module 1.3 The starting point: assessing PFM systems

Day 1: Approaches to PFM reform Module 1.1. PFM objectives and budgetary approaches Module 1.2. Why reform PFM systems? Why establish a sequence? Module 1.3. The starting point: assessing PFM systems Module 1.4. Conditions for successful reform

Module 1.3. The starting point: assessing PFM systems - Objectives of the module Before reforming a budgetary system, it is necessary to identify weaknesses and their causes This module examines the instruments to assess PFM, including PEFA, looking at both their advantages and their shortcomings

Module 1.3 Outline Presentation of PFM assessment tools The PEFA performance indicators Advantages and shortcomings of PEFA Issues linked to corruption

PFM diagnostic tools Public Expenditure & Financial Accountability Assessment (PEFA) – 7 partners Fiscal Transparency Assessment (FTA) – IMF Operational Assessment (OA) for Macro-Financial Assistance – DG ECFIN Public Expenditure Review (PER) – World Bank Public Expenditure Tracking Surveys (PETS) – WB Methodology for Assessing Procurement Systems (MAPS) – OECD/DAC Debt Management Performance Assessment (DeMPA) – IMF & WB Tax Administration Diagnostic Assessment Tool (TADAT) - IMF

Other PFM tools

SIGMA – Principles of Public Admin   Qualitative indicators (56) Quantitative indicators (102) Strategic framework of PAR* 3 10 Policy development & co-ordination 12 13 Public service and HRM* 11 16 Accountability 4 17 Service delivery Public financial management 23 33 Around ½ are similar to PEFA dimensions 7

Public Expenditure Reviews (PER) To strengthen budgetary analysis & processes in country concerned, so as to achieve a better focus on growth & poverty reduction To assess a country’s public expenditure program, to meet accountability requirements & provide government with external view of budget

Fiscal Transparency Evaluation - IMF The Code of Good Practices on Fiscal Transparency sets out principles & practices that government should follow The code based on 4 key principles (1) Clarity of roles & objectives (2) Public availability of information (3) Open budget preparation, execution & reporting (4) Independent assurances of integrity Independent assessment of where the country stands in relation to the Code

TADAT – Tax Administration Diagnostic Assessment Tool TADAT is supported by international development partners & institutions, including EC, Germany, IMF, Japan, Netherlands, Norway, Switzerland, UK & WB http://www.tadat.org/ Piloted in several countries from late 2013 – now in use

TADAT The tax administration is transparent in the conduct of its activities and accountable to the government and the community. All businesses, individuals, and other entities that are required to register are included in a taxpayer registration database. Information held in the database is complete and accurate. Tax administration operations are efficient and effective in performing key functions and achieving expected outcomes. The tax administration’s management of compliance risks results in higher levels of voluntary compliance and community confidence in the tax administration. The tax dispute resolution process is independent, accessible to taxpayers, and effective in resolving disputed matters in a timely manner. Taxpayers have the necessary information and support to voluntarily comply at a reasonable cost to themselves. Taxpayers report complete and accurate information in their tax returns. Taxpayers file returns on time. Taxpayers pay their taxes in full on time.

INTEGRATED ASSESSMENT MODEL FOR TAX ADMINISTRATION (IAMTA) The model recently developed by WB is made available to tax administrations: assesses & monitors performance of tax administration over time The measurement framework of IAMTAX can be used both as a benchmarking tool and a monitoring and diagnostic tool http://www.iamtax.org/IAMTAX/

Why is procurement important? Public Procurement systems at centre of spending public money: 30% to 70% of budgets are translated into services by government purchasing goods & services: e.g. - Azerbaijan: 58% Angola: 30% Bulgaria: 34% Malawi & Vietnam: 40% Key area for countries to increase efficiency of expenditure & improve service delivery Key area for donors: to help manage risk of misuse of funds & achieve development objectives

Measuring Procurement Performance OECD/DAC & World Bank Joint Procurement Roundtable developed detailed procurement benchmarking & measurement tools: WHY? To: provide international standards & stop imposing individual donor standards facilitate integration of CPAR with strengthened approach to PFM OECD/DAC Procurement Indicators: 4 Pillars, 12 Indicators, 55 sub-Indicators Legislative & Regulatory Framework Institutional Framework & Management Capacity Procurement Operations & Market Practices Integrity of Public Procurement System

Procurement in PEFA Indicators Dedicated indicator PI-24 focuses on unique aspects of procurement system not captured in other indicators Degree of competition in system Dispute resolution to augment controls Related indicators PI- 9, 13, 16, 21, 25, 26 & 30 capture aspects of procurement performance

Debt Management Performance Assessment Tool (DeMPA) Modeled on PEFA: 15 indicators, 34 dimensions, Governance & Strategy Development Coordination with Macroeconomic Policies Borrowing & Related Financing Activities Cash Flow Forecasting & Cash Balance Management Operational Risk Management Debt Records and Reporting

PEFA Instrument to measure performance & progress according to the three budgetary objectives “High level” indicators Part of the “strengthened approach” to PFM reform A reform programme led by government A coordinated programme of support by donors A common assessment framework, to monitor results

PEFA report in the PFM reform cycle Carry out PFM reforms Formulate a reform programme Recommend measures for PFM reform High level assessment of performance PFM performance report Indentifying the main weaknesses of PFM Le programme de réforme PFM doit tenir compte des recommandations issues d’une étude effectuée pour identifier les points faibles dans les systèmes, procédures et institutions PFM. Cet examen de haut niveau de la performance du système PFM peut être effectué en utilisant le cadre PEFA. Le diagramme illustre la place du PFM-PR dans le cycle de réforme de la gestion des finances publiques et les éléments couverts. Le PFM-PR tient compte de l’examen de haut niveau et identifie les principaux points faibles dans la gestion des finances publiques. Recommend PFM reform measures Identify main PFM weaknesses Examine root causes

PFM diagnostic tools ECFIN/ TADAT DeMPA PETS PEFA MAPS PER FTA OA X   PEFA FTA ECFIN/ OA PER PETS MAPS DeMPA TADAT Identification of PFM strengths & weaknesses X Focused on part of the Budget Cycle Integrated Focus In-depth analysis of capacity factors Recommendations for reform Assess fiduciary risk to public/external funds Track progress over time 19 Public Expenditure Analysis and Management Course Zimbabwe 2014

Why do donors require FRA? “Giving aid is a risky business”! Aspects of Fiduciary risk Funds not used for intended purposes Funds not properly accounted for Funds do not achieve ‘value for money’ Principles of assessing risk Structured process: likelihood & impact Record risk in order to monitor Differentiate inherent & residual

PEFA & typical FRA requirements FRA Issues PEFA Analysis of historical, governance & institutional context Helpful How well is PFM system currently performing? Yes What are the key risks? What is the overall level of fiduciary risk? What is the overall risk of corruption? No Is there a credible programme to reform? What risks are not addressed by existing programmes?

PEFA/FTA PER FTA PETS, TADAT MAPS, DeMPA 1 policy design & review 2 strategic Planning 3 budget Preparation 4 budget Execution 5 accounting & Reporting 6 external audit & scrutiny PEFA/FTA PER FTA PETS, TADAT MAPS, DeMPA

The socio-political background The importance of external factors (cultural, political, etc.) led to developing studies of the socio-political background Drivers of change (DFID) Public Policy Analysis Etc.

Module 1.3 Outline Presentation of PFM assessment tools The PEFA performance indicators Advantages and shortcomings of PEFA Issues linked to corruption

The PEFA Assessment Tool Analytical Framework Fiscal discipline, strategic allocation of resources, efficient service delivery 7 pillars; 31 high level performance indicators Legal and Regulatory Framework Institutional Arrangements Reform Process Progress on Government PFM reforms Institutional arrangements for planning and implementing PFM reform

PEFA: seven performance pillars PEFA indicators are grouped around 7 PFM pillars I Budget reliability (1 – 3) II Transparency of Public Finances (4 – 9) III Management of Assets & Liabilities (10 – 13) IV Policy-based fiscal strategy & budgeting (14 – 18) V Predictability & Control in Budget Execution (19 – 26) VI Accounting & Reporting (27 – 29) VII External Scrutiny & Audit (30 – 31)

IV. Policy-based fiscal Overview II. Transparency of public finances (PI:4-9) III. Management of Assets & Liabilities (PI:10-13) IV. Policy-based fiscal strategy & budgeting (PI:14-18) VI. Accounting, Recording and Reporting (PI:27-29) VII. External scrutiny and audit (PI:30-31) V. Predictability and control in budget execution (PI:19-26) I. Budget reliability (PI:1-3)

Calibrated on 4 Point Cardinal Scale (A, B, C, D) Calibration & scoring Calibrated on 4 Point Cardinal Scale (A, B, C, D) Evidence-based – use ‘Guidance’ issued by Secretariat Do not score above ‘D’ if evidence is insufficient ‘A’ reflects internationally accepted ‘good practice’ Determine score by starting from ‘C’, go upwards The scoring system is based on a four point ordinal scale (A, B, C, D). Performance is measured against a calibration based on ‘internationally accepted good practice’ representing a score ‘A’. This is not necessarily the same as international ‘best practice’. Sophisticated reforms such as performance based budgeting are not incorporated in the Framework, since the salient features of such reforms often lack international consensus. So ‘A’ ratings do not mean New Zealand standard of system. Performance is not measured against the local legislation, because this legislation may not always represent internationally accepted good practice. E.g. PI-11(iii) on approval of the budget before the start of the fiscal year. A numerical scale has deliberately been avoided in order to discourage simplistic numerical aggregation, which may serve only the purpose of cross-country comparison. Such cross-country comparison was not the purpose of developing the PEFA framework and therefore not the basis for its design. Look at an indicator example in the blue booklet e.g. PI-19 pages 38-39. Note the 4 parts for each indicator: general description; definition of dimensions; calibration table with specific requirements for each score; scoring aggregation method. It is useful to start scoring by considering the description of the D rating and whether the PFM system meets the requirements of a C rating. If so, would it also meet the criteria for a B rating etc. In this way, none of the requirements will be overlooked. Do not score if evidence is insufficient. A ‘no score’ indicates a serious gap in information which the government should normally possess. This information gap should be a focus for improvement before the next assessment takes place. Do not score ‘D’ if there is no information available Use of Arrow: (i) a performance change from last assessment , not reflected in the score (ii) a recent change in systems which is certain to have brought an improvement for which hard evidence is not yet available due to the time lag of obtaining such data – the use of arrows will increase as more repeat assessments come forward.

Calibration & scoring Most indicators have 2, 3 or 4 dimensions (total: 94) Each dimension must be rated separately Aggregate dimension scores for indicator; two methods WL or AV, SPECIFIED for each indicator* Intermediate scores (B+, C+) for multi-dimensional indicators, where dimensions score differently *For example: for PI with 3 dims rated A; B; C If WL (e.g. PI-20): then score = C+ but if AV (e.g PI-25): then score = B (from table) The scoring system is base don a four point ordinal scale (A, B, C, D). Performance is measured against a calibration based on ‘internationally accepted good practice’ representing a score ‘A’. This is not necessarily the same as international ‘best practice’. Sophisticated reforms such as performance based budgeting are not incorporated in the Framework, since the salient features of such reforms often lack international consensus. So ‘A’ ratings do not mean New Zealand standard of system. ‘D’ represents poor/unsatisfactory performance. ‘C’ a rudimentary but crudely functioning system. Performance is not measured against the local legislation, because this legislation may not always represent internationally accepted good practice. E.g. PI-11(iii) on approval of the budget before the start of the fiscal year. A numerical scale has deliberately been avoided in order to discourage simplistic numerical aggregation, which may serve only the purpose of cross-country comparison. Such cross-country comparison was not the purpose of developing the PEFA framework and therefore not the basis for its design. Look at an indicator example in the blue booklet e.g. PI-19 pages 38-39. Note the 4 parts for each indicator: general description; definition of dimensions; calibration table with specific requirements for each score; scoring aggregation method. It is useful to start scoring by considering the description of the D rating and whether the PFM system meets the requirements of a C rating. If so, would it also meet the criteria for a B rating etc. In this way, none of the requirements will be overlooked. Do not score if evidence is insufficient. A ‘no score’ indicates a serious gap in information which the government should normally possess. This information gap should be a focus for improvement before the next assessment takes place. Do not score ‘D’ if there is no information available Use of Arrow: (i) a performance change from last assessment , not reflected in the score (ii) a recent change in systems which is certain to have brought an improvement for which hard evidence is not yet available due to the time lag of obtaining such data – the use of arrows will increase as more repeat assessments come forward.

Module 1.3 Outline Presentation of PFM assessment tools The PEFA performance indicators Advantages and shortcomings of PEFA Issues linked to corruption

Advantages of using PEFA in preparing reforms PEFA is a very useful tool to provide a global view of a PFM system, & to identify its strengths & weaknesses PEFA covers the core functions of a PFM system Remember the ‘mirror’!

Short-comings of PEFA Some aspects of PFM not or insufficiently covered: e.g. Legislative & regulatory framework; IT; capacity; sectoral management; Accounting methods; decentralisation PEFA usually concerns processes, but does not necessarily provide information on their results PEFA does not take into account external factors (e.g. Role of political environment; corruption)

Avoiding the ‘cult of grading’ “A” rating reveals a good practice, but does not necessarily guarantee good result: there may be factors not captured in rating criteria “D” rating reveals lack of a some function(s), but priority should not necessarily go to focusing reform efforts on the processes graded with a D Ratings plus PEFA report provide good knowledge of a PFM system, but preparing a reform programme must also take into account external factors & priorities assigned to different PFM objectives

Placing PEFA indicators in ‘results chain’

Module 1.3 Outline Presentation of PFM assessment tools The PEFA performance indicators Advantages and shortcomings of PEFA Issues linked to corruption

Concerning corruption Reinforcing budgetary system is only one aspect of fight against corruption Often necessary, never sufficient Beware of “easy solutions”: accumulation of controls can have perverse effects High priority must be granted to improving transparency & reinforcing external surveys/reviews of the executive A few restrictions must be kept in mind Does this action contribute to fight against corruption? What else must be done? How to make actions more effective to fight corruption?

Examples of corruption in the PFM cycle Budget preparation Classic example: the white elephant Actors: powerful decision-makers How to fight it? Transparency? Checks and balances from administrative technicians? …

Examples of corruption in the PFM cycle Budget execution 1: Off-budget expenditure Actors: MoF, powerful decision-makers, & voluntarily blind Tax Office How to fight it? Transparency of Tax Office accounts; external audit? 2: (bought off) Delays in the processing of operations Actors: financial control, Tax Office, sectorial ministries How to fight it? Impose operations deadlines; Establish an “audit trail” 3: Services not provided; missing stocks Actors: managers, & sometimes financial control unit How to fight it? Reinforce internal control, inspections & audit

Examples of corruption in the PFM cycle Procurement .. Personnel administration … Revenue management …. Etc. What are your observations? Any suggestions?

Sanctions Adopted sanction systems are crucial Sanctions must be applied Warning Administrative mistakes must not be confused for either fraud nor corruption The fight against corruption can be misguided to serve political ambitions

Key messages Prior to the launch of PFM reform programme, an overview of the situation is necessary PEFA is essential instrument to establish overview of situation, but other analyses must be added to it, specifically analyses concerning external factors & capacity issues Specific diagnostic tools – relatives of PEFA – for tax administration (IAMTAX; TADAT), for debt management (DEMPA) & for public procurement systems (OECD DAC)