Evaluating the Performance of PBOs Helaina Gaspard Institute of Fiscal Studies and Democracy (IFSD) June 6-10, 2016 World Bank GN-PBO Annual Meeting Washington,

Slides:



Advertisements
Similar presentations
The Hague, The Netherlands November 2004 Budget reform in OECD countries -Common themes-
Advertisements

EuropeAid Pre-Assessment and Assessment for Parliamentary Development Promoting domestic accountability: engaging with parliaments EC support to governance.
Linkages Between NPoA and MTEF
The Role and Management o of the Scrutiny Unit Jessica Mulley, Head of the Scrutiny Unit June 2013.
Brussels, Lessons from NL Independent forecasts and the budgetary process Lessons from the Netherlands by Henk Don former director of CPB (Netherlands.
Measuring National Well-being – Measuring What Matters John C Hughes Measuring National Well-being Programme Manager (Scottish Universities Insight Institute,
How can Parliamentarians contribute to a Positive Investment Climate? by Rainer Geiger Senior Regional Advisor, OECD 3rd Global Conference of Parliamentarians.
Ray C. Rist The World Bank Washington, D.C.
The Medium Term Expenditure Framework
The Role of Parliament in approving the budget World Bank Institute’s Parliamentary Staff Training Program.
Rocky Harris Department for Environment, UK Ecosystems accounting in the UK UNCEEA Rio de Janeiro June 2012.
Key National Indicators and Supreme Audit Institutions: U.S. and INTOSAI Perspectives Bernice Steinhardt Director, Strategic Issues U.S. Government Accountability.
Office of the Auditor General of Canada The State of Program Evaluation in the Canadian Federal Government Glenn Wheeler Director, Results Measurement.
Accounts Chamber of the Russian Federation Global imbalances analysis: implications for KNI selection Anton Kosyanenko.
EVALUATION IN THE GEF Juha Uitto Director
PARLIAMENTARY OVERSIGHT Niall Johnston Parliamentary Consultant.
Critical Role of ICT in Parliament Fulfill legislative, oversight, and representative responsibilities Achieve the goals of transparency, openness, accessibility,
Strengthening the quality of research for policy engagement in the African context – achievements and aspirations Tebogo B. Seleka Botswana Institute for.
PBO: Role and Mandate FMI: How Government Works. June 11 th, 2015.
Ethics for the Prevention of Corruption February 16-19, 2009 – Ankara, Turkey.
Evaluation methods and tools (Focus on delivery mechanism) Jela Tvrdonova, 2014.
Recap and Synthesis of National and Regional Research MK21 Inception workshop for local research projects Yangon, June 2015.
1 UNDECLARED WORK IN CROATIA Executive Capacity of Governance and Underground Economy: The Case of Croatia Zagrebl, September 1, 2015.
The Exchange, May 2013 Einar Gorrissen, INTOSAI Development Initiative (IDI) Capacity Development to Strengthen Value and Benefits of SAIs.
Establishment of the Parliamentary Budget Office.
The Role of Parliament in the budget process. Overview Actors in the budget process Stages in the budget process Budgeting for the medium term.
Implementing Brunei Darussalam Long Term Development Plan
Evaluating the impact of health research: Revisiting the Canadian Institutes of Health Research Impact Assessment Framework Nicola Lauzon, Marc Turcotte.
The Budget Process A simplified and generalized summary of budgeting in the public sector. Political Dynamics Actors in the budget process Stages in the.
The Next Stage for Results in Africa. Context 2005 Paris Declaration on Aid Effectiveness 2006 Mutual Learning Events Uganda & Burkina Faso 2007 Hanoi.
Money Bills Amendment Procedure and Related Matters Act: Summary by: Mkhethwa MKHIZE Committee Section.
Reform Trends in OECD Member countries Jón R. Blöndal Deputy Head of Division Budgeting and Management Division Warsaw, 8 February 2005.
Recent Developments of the PEFA Program Video-conference of the PEMPAL BCOP PEFA Working Group February 20, 2009 Frans Ronsholt Head of PEFA Secretariat.
Civil society and public expenditure What can parliament gain from opening the budget process to public participation? Rick Stapenhurst Governance Team.
Kathy Corbiere Service Delivery and Performance Commission
PUBLIC FINANCIAL ACCOUNTABILITY – MULTI STAKEHOLDER APPROACHES
Policy, Budgeting and Oversight: The Role of the Legislature in Uganda Ishmael M Magona Ministry of Finance, Planning and Economic Development.
World Bank Institute 2 nd Annual GNPBO Conference Case Study: Forecasting Commodity Revenues Sahir Khan Senior Fellow, University of Ottawa Jason Jacques.
Developing an Operational Plan for PBOs Kevin Page Jean-Luc Pépin Chair in Canadian Government June 12, 2014 Global Network of PBOs Community Meeting.
GN-PBO Ottawa, Canada June 11, 2015 Kevin Page & Sahir Khan Jean-Luc Pepin Research Chair University of Ottawa An Approach to Budget Analysis for PBOs:
NATIONAL ASSEMBLY BUDGET AND RESEARCH OFFICE (NABRO) NIGERIA A PRESENTATION AT THE GLOBAL NETWORK OF PARLIAMENTARY BUDGET OFFICES COMMUNITY MEETING “SEMMINAR.
OECD NETWORK MODEL: OECD Network of Parliamentary Budget Officials and Independent Fiscal Institutions Lisa von Trapp Association of Parliamentary Budget.
AGENDA ACTIVISTS Mr. Enock Jarumi Mr.Bisase Moses Mr.Romulo Emmanuel Ms.Sorangel Diaz Ms. Mmapula Sekatane Ms.Orathai Kokpol Ms.Busagwa Prossy.
SESSION ON LEARNING FROM OTHER NETWORKS Lisa von Trapp Global Network of Parliamentary Budget Offices Meeting Seminar on Independent Budget Analysis and.
World Bank GN-PBO Community Meeting, Ottawa How to write MP friendly reports Sahir Khan Visiting Senior Fellow Jean-Luc Pépin Chair on Canadian Government.
1 Governance strategy for the Washington City Group Comments on draft proposal by Howard Meltzer (UK) February Brussels.
GN-PBO Ottawa, Canada June 11, 2015 Kevin Page Jean-Luc Pepin Research Chair University of Ottawa Considerations on Forecasting for PBOs.
Selected Issues in PRSP Implementation The World Bank and The IMF Thessaloniki May 28, 2005.
INCREASING PUBLIC VALUE THROUGH OPEN GOVERNMENT AND OPEN DATA Barbara-Chiara Ubaldi Project Manager, E-government Public Sector Reform Directorate for.
Performance Budgeting Global Network of Parliamentary Budget Officers (GN-PBO) Assembly Ivor Beazley, Washington DC, June 8 th,
Introduction and Overview
U4 – who we are Operational since 2003 as a web-based resource centre funded by:
The Ugandan Budget Process A Presentation to the Association of Budget Offices Conference Montreal, Canada 17th -19th June 2013 by Hon. Okot.
Introduction to Program Budgeting
Our role: fiscal analysis
Country Level Programs
GEF Familiarization Seminar
Standing Committee on Finance
Reform Trends in OECD Member countries
Parliament and the National Budget Process
Budget Formulation: good practices
Presentation on Expenditure Management By Team GVF
Audit & Risk Management
Medium-Term Expenditure Framework: Lessons
Introduction and Overview
2018/19 draft APP & Budget June /20/2018.
Global Trends in Budgeting Reform
Philippe Lagassé Associate Professor and Barton Chair
Gauteng Provincial Legislature Money Bills Act Discussion
11 July 2019 APP Presentation to the Joint Standing Committee on Financial Management of Parliament June 2019.
Presentation transcript:

Evaluating the Performance of PBOs Helaina Gaspard Institute of Fiscal Studies and Democracy (IFSD) June 6-10, 2016 World Bank GN-PBO Annual Meeting Washington, DC USA

Why evaluate PBOs? 1.Support to parliament  Parliament’s scrutiny function derives from the Magna Carta Legitimacy and survivability of the office  Credibility of outputs; perceptions of stakeholders; and organizational factors can influence a PBO’s operation

Why a common evaluation framework?  PBOs are emerging globally – What works? What doesn’t?  An evaluation framework can work in two ways: 1.To assess the work of an existing PBO 2.To establish the parameters for a new PBO  A common evaluation framework can be applied based on context, inputs, outputs and outcomes.

An Evaluation Framework for PBOs CONTEXTINPUTSOUTPUTSOUTCOMES  Legislation  Sources of accountability  Defining stakeholders  Budget  Human resources  Access to data  Independence  Peer review of reports (scientific and methodological soundness)  Transparency gains  Assessment against counterfactuals  Assessment against peers  Stakeholder perceptions (via surveys)  Media impact analysis  Expert interviews

Case study - the 2014 UK OBR review  First review of the UK OBR  Context, input, output, outcome frame  Defined performance – better decision support and enhanced transparency, rather than being “right”  Provided advice to support organizational sustainability  Helped to build legitimacy or the fledgling institution

Context  Constitutional and political ecosystem.  Founding legislation; parameters of the PBO.  Legislation guarantees a minimum.

Points of Departure Civil Society Media Legislators Parliament Cabinet Civil Service Legislation Implementation PBO Sustainability Imperative (t 0)

Inputs  Resources: both human and monetary that enable a PBO to fulfill their mandate.  Access to data: can a PBO get what it needs from the executive branch to do its work?  Independence: is a PBO free from political influences to select its staff, head and fulfill its mandate?  Don’t let the numbers fool you – you can do a lot with a little.

Inputs of OECD IFIs. Source: Review of the United Kingdom’s Office for Budget Responsibility (2014), p. 32.

Independence of the OBR. Source: Data sourced from the Review of the United Kingdom’s Office for Budget Responsibility (2014), p. 34.

Outputs  The products or reports prepared and released by the PBO.  Evaluating outputs is a technical assessment of a PBO’s products.  Are they credible?  Are they methodologically sound?  Are they transparent?  Are they peer-reviewed?  Are they comparable to those of peers?

Process Defined Transparent Disciplined Economy Indicators assessed over multiple years Errors analysed by level, growth rate, contributions to growth and probabilistic estimates Public finances Details of receipts and spending assessed over 3 year period Changes due to economic and fiscal forecast, policy and classification assessed on annual basis Lessons learned Transparency drives agenda for forecast improvement Forecast evaluation transparency gains. Source: Review of the United Kingdom’s Office for Budget Responsibility (2014), p. 61.

Outcomes  Perceptions and impacts of a PBO matter for sustainability  Identify a PBO’s key stakeholders (from legislation and convention)  Survey them, interview them and track media results  A PBO can exist on paper but if it cannot communicate with stakeholders, it may become irrelevant  Assessing outcomes:  How do parliamentarians perceive the PBO? Do they leverage their outputs?  Does the PBO resonate in the media?

Factiva analysis of OBR media citations and mentions ( ). Source: Review of the United Kingdom’s Office for Budget Responsibility (2014), p. 72.

 Note: Parliamentary survey results from the OBR evaluation were unusable due to a lack of response rates  Only 52 responses out of a potential 1,427 (650 Members of Parliament and 777 Peers (House of Lords))  5 from MPs, 47 responses from Peers  A survey of Canadian parliamentarians (February 2015-August 2015) yielded a 20% response rate (considered statistically relevant)

Establishing a PBO with the evaluation framework  “Flip the framework on its head” and build evaluation into your design  It can imbue your office with credibility, motivation and institutional sustainability  Identify your desired outcomes; consider your context:  What inputs and outputs are required to fulfill your goals? ContextInputsOutputsOutcomes

Why a common framework?  Baselines  Define a basic common standard of institutional organization and products  What are the fundamental common attributes of PBOs?  Benchmarks  Establish peer groups for assessment and comparison  You know you have succeeded when…  Aspirational institutions  Models of successful PBOs can be useful examples for new institutions

Conclusion  Cement dries quickly – you must act to set precedents of practices  No legislation is perfect. Use your inputs and outputs to maximize your mandate and your outcomes for stakeholders  With strong leadership, you can maximize mandate by ensuring that the office’s products are relevant, their content authoritative and their release timely and foster sustainability

Thank you Institute of Fiscal Studies and Democracy University of Ottawa