1 INDICATORS OF REGULATORY QUALITY Project website Claudio M. Radaelli Brussels, 25 January 2005

2 Description of the project Regulatory quality is a complex concept. In our project we acknowledge the role of different stakeholders and institutional contexts Quality measured in terms of design, activity-output, and real-world outcome Assessment of indicators and indexes proposed in the literature Analysis of better regulation programmes: focus on four tools (IA, Simplif., Access, Consultation) Questionnaire sent to support network in May 2004 The project is about the DESIGN of indicators. We have not calculated indicators, neither have we scored Member States

3 Why quality? Quality versus quantity Quality and efficiency of regulations Better regulation as a public policy (actors, resources, instruments, and decision- making structure) Quality and the Lisbon strategy Quality and indicators

4 Measuring regulatory quality in EU Member States (questionnaire) Diffusion of the Mandelkern report principles Most countries have opted for an experimental approach to indicators – one based on the gradual introduction of simple measures Setting quantitative targets is an effective way to encourage public administration to implement measurement systems (as shown by the case of B, DK, NL and S) No systematic use of cost-benefit analysis in the EU Member States. Rather, the emphasis is on how stakeholders are affected by proposed regs

5 Diffusion of principles but different practice: three clusters 1. Tools of better regulation are still at the stage of pilot project. Measurement of quality has been considered, but limited activity. 2. Basic systems of quality assurance and focus on administrative burdens. 3. Sophisticated quality assurance, well-embedded impact assessment, initiatives for the measurement of quality which target the whole regulatory process

6 Combining diversity of approaches and common progress in the EU Different practice is compatible with common progress A common approach, shared beliefs about regulatory quality and its components, and sense of direction towards the Lisbon goals Three specific sets of indicators Three specific sets of indicators

7 The hard questions FROM INSTRUMENTS TO POLICY: How do we know that a good IA produces better regulation? FROM POLICY TO REGULATORY OUTPUT: How do we know that ‘better regulation policy’ produces better regulations? FROM REGULATORY OUTPUT TO FINAL ECONOMIC OUTCOME: What is the causal chain going from good regulation to real-world economic outcomes? How robust is it?

8 How we answer these questions Focus on how regulation changes the behaviour or stakeholders Selection of five indicators that measure the quality of better regulation policy without going too far in the causal chain We use Lisbon indicators, but we do not test the quality of better regulation policy directly on economic indicators like the growth of the knowledge-based economy

9 The three systems 1. Design of better regulation policy and tools. Macro and ex-ante 2. Ex-post and micro 3. Bridge between measurement of regulatory quality and the systematic evaluation of better regulation as public policy

10 Where do the indicators come from? Operationalisation of the principles of better regulation Questionnaire and consultation with support network Experience of MS and other countries Criteria of “usable knowledge” = indicators that can be embedded in policy processes

11 Indicators – characteristics of system 1: quality of the process an ex-ante macro-system - does not go down to the level of individual impact assessments or consultations based on objective indicators, it covers the design of better regulation policies, instruments (contents of formal guidance), and supporting and quality assurance activities can be managed by coordinating units in MSs and the SG in the Commission

12 Quality assurance Indicator What is measured Interpretation Data collection Quantitative targets Existence or absence Targets facilitate implem. of BR programmes IRQ dataset Reporting on progress Existence or absence Accountability and transparency IRQ dataset Performance of quality assurance unit Existence or absence Establishing quantitative criteria to assess success or failure IRQ dataset Civil society – public official document Existence or absence Legitimacy of better regulation programmes Official guidance Monitoring and evaluation Existence or absence Culture of monitoring and evaluation IRQ database plus official guidance

13 Indicators – characteristics of system 2: internal evaluation An ex-post micro-level system focused on what the tools of better regulation deliver in a given time-period We look at guidance as designed in system 1 and measure what has been achieved

14 System 2 contains the following: 1. A simple checklist on the quality of individual impact assessments and major consultations 2. Real-world measures of the impact of better regulation policies (subjective: surveys over time) 3. Five real-world measures of the impact of better regulation policies (objective) 4. Limited number of important indicators of economic outcome – role of causal chains

15 Survey of regulators Percentage of regulators who think that IA has the following effect: [1] Makes policy formulation more transparent [2] Speeds up / delays policy implementation [3] Reduces conflict at the stage of policy formulation [4] Reduces conflict at the stage of policy implementation [5] Be more likely to represent the preferences of citizens [6] Final policy decision will be more likely to achieve goals [7] Final policy decision be more likely to represent a consensus among affected entities [8] Final policy decision be more likely to create open markets

16 Survey of citizens and firms [1] % of citizens who think that their views are taken on board in the development of single market policy in the EU [2] % of firms who think that the quality of regulation in the EU (including both EU and domestic rules) has improved over the last three years [3] % of firms who think that participation in EU-level consultation makes an impact on the final decision [4] % of firms who think that EU policy has an impact in opening markets, making it easier to compete, and generating competition in the professions [5] % of firms who think that in last three years restrictions and obstacles to their business have disappeared altogether or significantly reduced [6] % of firms who think that regulatory changes at the EU level are predictable

17 The Five Real-World measures of better regulation policy [1] Estimate of total regulatory costs delivered by regulations for which impact assessments (IAs) were prepared in year xxxx / Estimate of total regulatory benefits delivered by regulations for which IAs were prepared in year xxxx [2] Net benefits delivered by regulations for which IAs were prepared in year t / Net benefits delivered by regulations for which IAs were prepared in year t-1 [3] Total cost reduction resulting from simplification [4] Annual rate of reduction in the total administrative burdens stemming from EU legislation [5] Cost of administrative procedures eliminated in year t / Cost of administrative procedures eliminated in year t-1

18 System 3: external evaluation “All policies can be evaluated, and better regulation is no exception” no matter how sophisticated, indicators are just a component of quality assurance the latter requires strategic and operational management, specific structures, and dedicated tools one important element is external evaluation indicators must be complemented by interviews, case studies, etc. external evaluation should be done by think tanks, academics, and stakeholders - notion of pluralistic evaluation

19 Indicators – which system is suitable for whom? system 1 could be used by the cluster of Member States with basic approaches to regulatory quality and impact assessment in experimental-pilot stage. They should move as soon as possible to the second system system 2 could be used by the cluster in which consultation, simplification, and the assessment of administrative burdens are already well embedded - however, some simple indicators of this system can be adopted by all EU governments now, no need to wait

20 Indicators – which system is suitable for whom? finally, system 3 is applicable to countries with sophisticated quality assurance mechanisms (as well as to the Commission)

21 How to use indicators Open method of coordination High-level group on Competitiveness – Competitiveness Council, this can be a body that can adopt indicators DBER can be the forum where data are collected and technical discussions on indicators take place

22 Why use open method of coordination? Better regulation policy is an essential component of the Lisbon agenda. It involves different levels of governance. A responsibility for the EU and the Member States

23 There is an OMC in the making: 1. Principles (Mandelkern) 2. National programmes of regulatory reform 3. Peer review of reforms (within DBER) 4. Timetable (Lisbon) 5. Need to complete OMC with Indicators and iterative processes