1. Public Financial Management Performance Measurement Framework
The primary objective of the strengthened approach is improved PFM performance in partner countries, which then contributes to achievement of development outcomes.
June 5, 2007
Presentation by Nicola Smithers, AFTPR
Slides provided by PEFA Secretariat

2. WHAT IS THE PFM PERFORMANCE MEASUREMENT FRAMEWORK ? AN EXAMPLE
WHAT WE’LL COVER
WHAT IS THE PFM PERFORMANCE MEASUREMENT FRAMEWORK ?
AN EXAMPLE
WHERE AND HOW IS IT APPLIED?
CASE OF ZAMBIA
The PFM work was used mainly for two purposes:
To guide PFM reform preparation
To feed into the fiduciary risk assessments of individual donor/financing agencies

3. CONTEXT - The Strengthened Approach to Supporting PFM Reform
A country-led PFM reform program
including a strategy and action plan reflecting country priorities; implemented through government structures
A donor coordinated program of support
covering analytical, technical and financial support
A common assessment and monitoring framework
the PEFA* PFM Performance Measurement Framework
*Public Expenditure and Financial Accountability Program
This approach was adopted by the Bank through the memorandum issued to all sectors jointly by the Vice Presidents of PREM and OPCS in July 2005

4. The link between the different elements of the Strengthened Approach to supporting PFM reforms
Government PFM Reform Strategy, Action Plan and Results
Coordinated program of analytical/TA/ funding support
PFM Performance report
PFM indicators
Government/Donor
Policy Dialogue
Donor Country Assistance Strategy and knowledge requirement

5. PFM Performance Measurement Framework
A standard set of high level PFM indicators to assess performance
28 government performance indicators
3 donor indicators, reflecting donor practices influencing the government’s PFM
A concise, integrated report – the PFM Performance Report
Standard content and format
provides the narrative to support the indicator assessments (the evidence)
draws a summary from the analysis

6. COVERAGE OF PFM INDICATORS
C. Budget Cycle
Policy Based budgeting
D. Donor Practices
A. PFM Out-turns
B. Cross-cutting features
External scrutiny and audit
Predictability and control in Budget Execution
Budget credibility
Budget credibility indicators reflect outputs of the system as a result of the institutional and more qualitative assessment covered by the remainder of the indicator set. Understanding links between the institutional strengths/weaknesses and budget credibility is important for summarizing the assessment.
Similar for donor indicator ratings’ relations to government performance and budget credibility.
Comprehensiveness and Transparency
Accounting, Recording, Reporting

7. Content of Indicator Set
A. PFM Out-turns
Credibility of the budget Indicators
Deviations from aggregate budgeted expenditure and revenue as well as expenditure composition. Level of expenditure arrears.
B. Key Cross-cutting issues
Comprehensiveness and transparency Indicators
Coverage of budget classification, budget documentation, reporting on extra-budgetary operations, inter-governmental fiscal relations, fiscal risk oversight and public access to information.
C. Budget Cycle
i. Policy-based budgeting Indicators 11-12
Annual budget preparation process, multi-year perspective in fiscal planning, expenditure policy and budgeting
Examples of tricky indicators:
PI-4 : definition of arrears from an economic rather than accounting point of view.
PI-7 : How to measure what is not included in fiscal reports. Triangulation from many sources.

8. Content of Indicator Set (cont’d)
C. Budget Cycle
ii. Predictability & control in budget execution Indicators 13-21
Revenue administration, predictability in availability of funds, cash balances, debt & guarantee management, payroll controls, procurement, internal controls and internal audit
iii. Accounting, recording and reporting Indicators 22-25
Accounts reconciliation, reporting on resources at service outlet level, in-year budget execution reports, financial statements
iv. External scrutiny and audit Indicators 26-28
Scope, nature and follow-up on external audit; legislative scrutiny of annual budget law and external audit reports
D. Donor Practices Indicators D1- D3
Predictability of direct budget support; donor information for budgeting and reporting; use of national procedures
Procurement incorporated in many indicators, not just covered by PI-19 (e.g. PI-4, PI-12, PI-16, PI-20, PI-26)

9. Calibration and Scoring
Calibrated on four point ordinal scale (A, B, C, D)
Requirements for each score explicitly specified
Scoring based on internationally recognized ‘Good Practice’
Indicators have 1, 2, 3 or 4 dimensions
in total 74 dimensions
to provide detailed information & transparency of score
each dimension must be rated separately
Aggregation only from dimensions to indicator
Two scoring methodologies
Aggregation of indicator scores is not encouraged because:
An overall score (number) does not assist in identifying strengths and weaknesses to guide reform dialogue on priorities and more detailed analytical work needed;
such an aggregate mainly serves country comparison which is not the intention of the Framework;
standard aggregation assumes that all indicators should be given the same weight in all countries which is not the case (e.g. importance of SN government, SOEs and external aid)
aggregates mainly serve for individual donor decisions on allocation of aid and modalities of aid implementation. Different agencies assign different weight to the individual parts of the PFM system and therefore use different weights.

10. Example

11. Applying the Framework

12. Roll-out of PFM Assessments
PFM Performance Measurement Framework launched June 2005
Assessment Status as at March 2007
45 substantially completed i.e. draft/final report
24 on-going but report not yet issued
27 agreed with government but not started
Roll-out rate: a steady 2-3 new assessments per month
Outlook for mid 2008
75-80 countries covered
8-10 repeat assessments

13. Geographical distribution

14. Decentralized Process
Application of the PEFA Framework to be decided at country level. Decisions to be made:
If and Why ?
When ?
How ?
Recommended by international organizations as good practice (e.g. OECD-DAC, ComSec)
No supra-agency mandates or responsibilities. Each country and organization decides its interest in a PEFA assessment and ability to contribute.

15. Government Involvement
Government’s role
Self-assessment (with external validation)
Joint assessment (joint team)
Collaboration with donor-led assessment
Determined by interest and capacity
What are the benefits to government?
Government staff may need training

16. Donor Collaboration A donor reference group is essential
to ensure that needs of all parties are addressed
to ensure common acceptance of findings
The reference group to agree internally and with the government on:
Diagnostic packaging
Resources for assessment work
Stages and timing of the assessment work
Quality assurance arrangements

17. Diagnostic Packaging – Coverage of PFM Performance Report
Implement PFM reforms
High level performance overview
Formulate PFM reform program
PFM-PR
Before embarking on the formulation of a reform program, it is necessary to obtain a high level overview of the system performance and aclear understanding of the main PFM weaknesses.
The causes of these weaknesses need to be investigated and this may entail the application of further diagnostic work to drill down into the areas of concern. The recommendations arising from these investigations will then be used to formulate a PFM reform program.
Recommend PFM reform measures
Identify main PFM weaknesses
Recommend PFM reform measures
Investigate underlying causes

18. Quality Assurance Terms of Reference & Draft Report to be Q.A.’d
Government and donor reference group
Should ensure that information is used correctly and reflects the situation in the country
Donor specific arrangements to be respected
e.g. World Bank peer review mechanism
The PEFA Secretariat recommended as a peer reviewer
Will assess whether the product respects the Framework’s standards and methodology
Can also provide advice and guidance
In-country assessment reference groups (typically including the government and donors beyond those directly participating in the assessment team) provide an important review mechanism in that they should have the best impression of the quality and completeness of the information provided in the reports and the extent to which appropriate professional judgment has been exercised in areas of limited data availability. Such arrangements have been made in Kyrgyz, Moldova, Uganda and Mozambique. A weakness of a very comprehensive team of donor agency staff is that few, if any, informed donor representatives will be left in-country to take an arms-length view of the quality of the assessment (Tanzania).
Some donors have specific quality assurance arrangements which their teams will have to also comply with. For example the World Bank’s institutionalized peer review system.
The PEFA Secretariat’s contribution to the peer review process is mainly to offer an opinion on the extent to which the Framework’s assessment scope, methods and report provisions have been met. The Secretariat usually cannot judge the accuracy and completeness of the data used for the exercise, but does assess adequacy of data presented to justify the scoring.

19. Zambia case