1. Approaches to PFM reform Module 1.3
The starting point:
assessing PFM systems

2. Day 1: Approaches to PFM reform
Module 1.1. PFM objectives and budgetary approaches
Module 1.2. Why reform PFM systems? Why establish a sequence?
Module 1.3. The starting point: assessing PFM systems
Module 1.4. Conditions for successful reform

3. Module 1.3. The starting point: assessing PFM systems - Objectives of the module
Before reforming a budgetary system, it is necessary to identify weaknesses and their causes
This module examines the instruments to assess PFM, including PEFA, looking at both their advantages and their shortcomings

4. Module 1.3 Outline Presentation of PFM assessment tools
The PEFA performance indicators
Advantages and shortcomings of PEFA
Issues linked to corruption

5. PFM diagnostic tools
Public Expenditure & Financial Accountability Assessment (PEFA) – 7 partners
Fiscal Transparency Assessment (FTA) – IMF
Operational Assessment (OA) for Macro-Financial Assistance – DG ECFIN
Public Expenditure Review (PER) – World Bank
Public Expenditure Tracking Surveys (PETS) – WB
Methodology for Assessing Procurement Systems (MAPS) – OECD/DAC
Debt Management Performance Assessment (DeMPA) – IMF & WB
Tax Administration Diagnostic Assessment Tool (TADAT) - IMF

6. Other PFM tools

7. SIGMA – Principles of Public Admin
Qualitative indicators
(56)
Quantitative indicators
(102)
Strategic framework of PAR* 3 10
Policy development & co-ordination 12 13
Public service and HRM* 11 16
Accountability 4 17
Service delivery
Public financial management 23 33
Around ½ are similar to PEFA dimensions 7

8. Public Expenditure Reviews (PER)
To strengthen budgetary analysis & processes in country concerned, so as to achieve a better focus on growth & poverty reduction
To assess a country’s public expenditure program, to meet accountability requirements & provide government with external view of budget

9. Fiscal Transparency Evaluation - IMF
The Code of Good Practices on Fiscal Transparency sets out principles & practices that government should follow
The code based on 4 key principles
(1) Clarity of roles & objectives
(2) Public availability of information
(3) Open budget preparation, execution & reporting
(4) Independent assurances of integrity
Independent assessment of where the country stands in relation to the Code

10. TADAT – Tax Administration Diagnostic Assessment Tool
TADAT is supported by international development partners & institutions, including EC, Germany, IMF, Japan, Netherlands, Norway, Switzerland, UK & WB
Piloted in several countries from late 2013 – now in use

11. TADAT
The tax administration is transparent in the conduct of its activities and accountable to the government and the community.
All businesses, individuals, and other entities that are required to register are included in a taxpayer registration database. Information held in the database is complete and accurate.
Tax administration operations are efficient and effective in performing key functions and achieving expected outcomes.
The tax administration’s management of compliance risks results in higher levels of voluntary compliance and community confidence in the tax administration.
The tax dispute resolution process is independent, accessible to taxpayers, and effective in resolving disputed matters in a timely manner.
Taxpayers have the necessary information and support to voluntarily comply at a reasonable cost to themselves.
Taxpayers report complete and accurate information in their tax returns.
Taxpayers file returns on time.
Taxpayers pay their taxes in full on time.

12. INTEGRATED ASSESSMENT MODEL FOR TAX ADMINISTRATION (IAMTA)
The model recently developed by WB is made available to tax administrations: assesses & monitors performance of tax administration over time
The measurement framework of IAMTAX can be used both as a benchmarking tool and a monitoring and diagnostic tool

13. Why is procurement important?
Public Procurement systems at centre of spending public money: 30% to 70% of budgets are translated into services by government purchasing goods & services: e.g. -
Azerbaijan: 58%
Angola: 30%
Bulgaria: 34%
Malawi & Vietnam: 40%
Key area for countries to increase efficiency of expenditure & improve service delivery
Key area for donors: to help manage risk of misuse of funds & achieve development objectives

14. Measuring Procurement Performance
OECD/DAC & World Bank Joint Procurement Roundtable developed detailed procurement benchmarking & measurement tools: WHY? To:
provide international standards & stop imposing individual donor standards
facilitate integration of CPAR with strengthened approach to PFM
OECD/DAC Procurement Indicators: 4 Pillars, 12 Indicators, 55 sub-Indicators
Legislative & Regulatory Framework
Institutional Framework & Management Capacity
Procurement Operations & Market Practices
Integrity of Public Procurement System

15. Procurement in PEFA Indicators
Dedicated indicator PI-24 focuses on unique aspects of procurement system not captured in other indicators
Degree of competition in system
Dispute resolution to augment controls
Related indicators PI- 9, 13, 16, 21, 25, 26 & 30 capture aspects of procurement performance

16. Debt Management Performance Assessment Tool (DeMPA)
Modeled on PEFA: 15 indicators, 34 dimensions,
Governance & Strategy Development
Coordination with Macroeconomic Policies
Borrowing & Related Financing Activities
Cash Flow Forecasting & Cash Balance Management
Operational Risk Management
Debt Records and Reporting

17. PEFA
Instrument to measure performance & progress according to the three budgetary objectives
“High level” indicators
Part of the “strengthened approach” to PFM reform
A reform programme led by government
A coordinated programme of support by donors
A common assessment framework, to monitor results

18. PEFA report in the PFM reform cycle
Carry out PFM reforms
Formulate a reform programme
Recommend measures for PFM reform
High level assessment of performance
PFM performance report
Indentifying the main weaknesses of PFM
Le programme de réforme PFM doit tenir compte des recommandations issues d’une étude effectuée pour identifier les points faibles dans les systèmes, procédures et institutions PFM. Cet examen de haut niveau de la performance du système PFM peut être effectué en utilisant le cadre PEFA. Le diagramme illustre la place du PFM-PR dans le cycle de réforme de la gestion des finances publiques et les éléments couverts. Le PFM-PR tient compte de l’examen de haut niveau et identifie les principaux points faibles dans la gestion des finances publiques.
Recommend PFM reform measures
Identify main PFM weaknesses
Examine root causes

19. PFM diagnostic tools ECFIN/ TADAT DeMPA PETS PEFA MAPS PER FTA OA X
PEFA
FTA
ECFIN/ OA
PER
PETS
MAPS
DeMPA
TADAT
Identification of PFM strengths & weaknesses X
Focused on part of the Budget Cycle
Integrated Focus
In-depth analysis of capacity factors
Recommendations for reform
Assess fiduciary risk to public/external funds
Track progress over time 19
Public Expenditure Analysis and Management Course Zimbabwe 2014

20. Why do donors require FRA?
“Giving aid is a risky business”!
Aspects of Fiduciary risk
Funds not used for intended purposes
Funds not properly accounted for
Funds do not achieve ‘value for money’
Principles of assessing risk
Structured process: likelihood & impact
Record risk in order to monitor
Differentiate inherent & residual

21. PEFA & typical FRA requirements
FRA Issues
PEFA
Analysis of historical, governance & institutional context
Helpful
How well is PFM system currently performing?
Yes
What are the key risks?
What is the overall level of fiduciary risk?
What is the overall risk of corruption? No
Is there a credible programme to reform?
What risks are not addressed by existing programmes?

22. PEFA/FTA PER FTA PETS, TADAT MAPS, DeMPA 1 policy design & review
2 strategic Planning
3 budget Preparation
4 budget Execution
5 accounting & Reporting
6 external audit & scrutiny
PEFA/FTA
PER
FTA
PETS, TADAT
MAPS, DeMPA

23. The socio-political background
The importance of external factors (cultural, political, etc.) led to developing studies of the socio-political background
Drivers of change (DFID)
Public Policy Analysis
Etc.

24. Module 1.3 Outline Presentation of PFM assessment tools
The PEFA performance indicators
Advantages and shortcomings of PEFA
Issues linked to corruption

25. The PEFA Assessment Tool
Analytical Framework
Fiscal discipline, strategic allocation of resources, efficient service delivery
7 pillars; 31 high level performance indicators
Legal and Regulatory Framework
Institutional Arrangements
Reform Process
Progress on Government PFM reforms
Institutional arrangements for planning and implementing PFM reform

26. PEFA: seven performance pillars
PEFA indicators are grouped around 7 PFM pillars
I Budget reliability (1 – 3)
II Transparency of Public Finances (4 – 9)
III Management of Assets & Liabilities (10 – 13)
IV Policy-based fiscal strategy & budgeting (14 – 18)
V Predictability & Control in Budget Execution (19 – 26)
VI Accounting & Reporting (27 – 29)
VII External Scrutiny & Audit (30 – 31)

27. IV. Policy-based fiscal
Overview
II. Transparency of
public finances
(PI:4-9)
III. Management of
Assets & Liabilities
(PI:10-13)
IV. Policy-based fiscal
strategy & budgeting
(PI:14-18)
VI.
Accounting, Recording
and Reporting
(PI:27-29)
VII.
External
scrutiny
and audit
(PI:30-31) V.
Predictability
and control
in budget
execution
(PI:19-26) I.
Budget
reliability
(PI:1-3)

28. Calibrated on 4 Point Cardinal Scale (A, B, C, D)
Calibration & scoring
Calibrated on 4 Point Cardinal Scale (A, B, C, D)
Evidence-based – use ‘Guidance’ issued by Secretariat
Do not score above ‘D’ if evidence is insufficient
‘A’ reflects internationally accepted ‘good practice’
Determine score by starting from ‘C’, go upwards
The scoring system is based on a four point ordinal scale (A, B, C, D).
Performance is measured against a calibration based on ‘internationally accepted good practice’ representing a score ‘A’. This is not necessarily the same as international ‘best practice’.
Sophisticated reforms such as performance based budgeting are not incorporated in the Framework, since the salient features of such reforms often lack international consensus. So ‘A’ ratings do not mean New Zealand standard of system.
Performance is not measured against the local legislation, because this legislation may not always represent internationally accepted good practice. E.g. PI-11(iii) on approval of the budget before the start of the fiscal year.
A numerical scale has deliberately been avoided in order to discourage simplistic numerical aggregation, which may serve only the purpose of cross-country comparison. Such cross-country comparison was not the purpose of developing the PEFA framework and therefore not the basis for its design.
Look at an indicator example in the blue booklet e.g. PI-19 pages Note the 4 parts for each indicator: general description; definition of dimensions; calibration table with specific requirements for each score; scoring aggregation method.
It is useful to start scoring by considering the description of the D rating and whether the PFM system meets the requirements of a C rating. If so, would it also meet the criteria for a B rating etc. In this way, none of the requirements will be overlooked.
Do not score if evidence is insufficient. A ‘no score’ indicates a serious gap in information which the government should normally possess. This information gap should be a focus for improvement before the next assessment takes place. Do not score ‘D’ if there is no information available
Use of Arrow: (i) a performance change from last assessment , not reflected in the score (ii) a recent change in systems which is certain to have brought an improvement for which hard evidence is not yet available due to the time lag of obtaining such data – the use of arrows will increase as more repeat assessments come forward.

29. Calibration & scoring
Most indicators have 2, 3 or 4 dimensions (total: 94)
Each dimension must be rated separately
Aggregate dimension scores for indicator; two methods WL or AV, SPECIFIED for each indicator*
Intermediate scores (B+, C+) for multi-dimensional indicators, where dimensions score differently
*For example: for PI with 3 dims rated A; B; C
If WL (e.g. PI-20): then score = C+
but if AV (e.g PI-25): then score = B (from table)
The scoring system is base don a four point ordinal scale (A, B, C, D).
Performance is measured against a calibration based on ‘internationally accepted good practice’ representing a score ‘A’. This is not necessarily the same as international ‘best practice’.
Sophisticated reforms such as performance based budgeting are not incorporated in the Framework, since the salient features of such reforms often lack international consensus. So ‘A’ ratings do not mean New Zealand standard of system.
‘D’ represents poor/unsatisfactory performance. ‘C’ a rudimentary but crudely functioning system.
Performance is not measured against the local legislation, because this legislation may not always represent internationally accepted good practice. E.g. PI-11(iii) on approval of the budget before the start of the fiscal year.
A numerical scale has deliberately been avoided in order to discourage simplistic numerical aggregation, which may serve only the purpose of cross-country comparison. Such cross-country comparison was not the purpose of developing the PEFA framework and therefore not the basis for its design.
Look at an indicator example in the blue booklet e.g. PI-19 pages Note the 4 parts for each indicator: general description; definition of dimensions; calibration table with specific requirements for each score; scoring aggregation method.
It is useful to start scoring by considering the description of the D rating and whether the PFM system meets the requirements of a C rating. If so, would it also meet the criteria for a B rating etc. In this way, none of the requirements will be overlooked.
Do not score if evidence is insufficient. A ‘no score’ indicates a serious gap in information which the government should normally possess. This information gap should be a focus for improvement before the next assessment takes place. Do not score ‘D’ if there is no information available
Use of Arrow: (i) a performance change from last assessment , not reflected in the score (ii) a recent change in systems which is certain to have brought an improvement for which hard evidence is not yet available due to the time lag of obtaining such data – the use of arrows will increase as more repeat assessments come forward.

30. Module 1.3 Outline Presentation of PFM assessment tools
The PEFA performance indicators
Advantages and shortcomings of PEFA
Issues linked to corruption

31. Advantages of using PEFA in preparing reforms
PEFA is a very useful tool to provide a global view of a PFM system, & to identify its strengths & weaknesses
PEFA covers the core functions of a PFM system
Remember the ‘mirror’!

32. Short-comings of PEFA
Some aspects of PFM not or insufficiently covered: e.g. Legislative & regulatory framework; IT; capacity; sectoral management; Accounting methods; decentralisation
PEFA usually concerns processes, but does not necessarily provide information on their results
PEFA does not take into account external factors (e.g. Role of political environment; corruption)

33. Avoiding the ‘cult of grading’
“A” rating reveals a good practice, but does not necessarily guarantee good result: there may be factors not captured in rating criteria
“D” rating reveals lack of a some function(s), but priority should not necessarily go to focusing reform efforts on the processes graded with a D
Ratings plus PEFA report provide good knowledge of a PFM system, but preparing a reform programme must also take into account external factors & priorities assigned to different PFM objectives

34. Placing PEFA indicators in ‘results chain’

35. Module 1.3 Outline Presentation of PFM assessment tools
The PEFA performance indicators
Advantages and shortcomings of PEFA
Issues linked to corruption

36. Concerning corruption
Reinforcing budgetary system is only one aspect of fight against corruption
Often necessary, never sufficient
Beware of “easy solutions”: accumulation of controls can have perverse effects
High priority must be granted to improving transparency & reinforcing external surveys/reviews of the executive
A few restrictions must be kept in mind
Does this action contribute to fight against corruption?
What else must be done? How to make actions more effective to fight corruption?

37. Examples of corruption in the PFM cycle
Budget preparation
Classic example: the white elephant
Actors: powerful decision-makers
How to fight it? Transparency? Checks and balances from administrative technicians? …

38. Examples of corruption in the PFM cycle
Budget execution
1: Off-budget expenditure
Actors: MoF, powerful decision-makers, & voluntarily blind Tax Office
How to fight it? Transparency of Tax Office accounts; external audit?
2: (bought off) Delays in the processing of operations
Actors: financial control, Tax Office, sectorial ministries
How to fight it? Impose operations deadlines; Establish an “audit trail”
3: Services not provided; missing stocks
Actors: managers, & sometimes financial control unit
How to fight it? Reinforce internal control, inspections & audit

39. Examples of corruption in the PFM cycle
Procurement ..
Personnel administration …
Revenue management ….
Etc.
What are your observations? Any suggestions?

40. Sanctions Adopted sanction systems are crucial
Sanctions must be applied
Warning
Administrative mistakes must not be confused for either fraud nor corruption
The fight against corruption can be misguided to serve political ambitions

41. Key messages
Prior to the launch of PFM reform programme, an overview of the situation is necessary
PEFA is essential instrument to establish overview of situation, but other analyses must be added to it, specifically analyses concerning external factors & capacity issues
Specific diagnostic tools – relatives of PEFA – for tax administration (IAMTAX; TADAT), for debt management (DEMPA) & for public procurement systems (OECD DAC)