1. “General RIA Training” 6–8 July 2009 EuropeAid/125317/D/SER/TR Session 12 Data Collection and Stakeholder Consultation in the RIA process

2.  Data collection quality is the main indicator of a good RIA  You will usually need much highly specific data that is tailored to the questions raised by the specific regulation Data Gathering

3.  Plan in advance and develop an inventory of data sources  Much data needed is held by the regulated community. Data collection is a public-private task. You must develop working relations with private sector partners IN ADVANCE  Define standards of data acceptability IN ADVANCE, as well as the quality control process for data use Plan ahead for data collection

4. TechniqueProsCons Literature searches and existing databases Fast and cheap. Credible because it draws on scientific expertise Unfocussed. Cannot usually answer questions specific to the regulation. Gaps in the literature increase uncertainties Meta-studies (combining several studies into one big study) Can compensate for weaknesses in individual studies Limited in the range of information (cost data but little benefits data) Experts from science committees, academia and industry (e.g. interviews) Fast and cheap, can provide good quality cost and benefit data in response to specific questions. Can explore alternative approaches Vulnerable to bias and to existing knowledge which is not specific to the regulation. Must diversify sources Common data collection techniques (1)

5. Technique ProsCons SurveysFast data collection, can be carried out for targeted groups, can collect information otherwise unavailable Limited in the range of information (cost data but little benefits data), reliability is suspect, costs can be high. Difficult to explore options Focus groupsBusiness Test Panels can be fast and provide a wider range of cost and benefits data. Can explore alternatives and options easily Data quality can be a problem. Requires upfront preparation and good relations with stakeholders Common data collection techniques (2)

6. Public consultation - 2295 replies to a questionnaire and 103 written submissions received 6 major studies between 2004 and 2006 about future postal policy orientations, in addition to 14 studies since first postal Directive 10 public workshops in Brussels related to studies since 2004 25 focus groups on postal services in EU countries with a particular aim of understanding needs and satisfaction of individual postal users 'Eurobarometer' survey on consumer satisfaction Discussions with Member States, social partners, operators and all other stakeholders including consumers Participation in several consumer organization, economic and social partner conferences focusing on postal issues Example: Data Collection for EU Postal Services IA

7.  In the absence of adequate valid data:  clearly identified assumptions are necessary for conducting the RIA  Transparency is even more essential to test the assumptions  Analysis of the risks, benefits, and costs associated with regulation must be guided by the principles of full disclosure and transparency  Shift data costs to the stakeholders by asking questions in the consultation period (see next)  Those affected by regulations have the incentives to provide the data necessary to complete RIA How can you compensate if you cannot find enough data?

8.  Some stakeholders will provide needed data ‘according to them’  => Risk is high that the RIA will be biased.  This risk can be managed by diversifying data sources, a check and balance approach  Data biases can also be detected by being completely transparent  When data are weak, the more external review and validation through consultation the better it is likely to be Risk of “data capture”

9. Supplement and/or complement data gathering –Gather data effectively –Measure actual consequences and assess impacts Public interest and involvement –Highlight potential problems and oppositions –Raise awareness of future regulation –Increase legitimacy of final proposals –Build trust Create ownership and boost compliance Why consult?

10. Australia Targeted Timeliness Accessible Transparent Consistent and flexible Evaluated and reviewed European Commission  Clear content  Target groups  Publication  Time limits for participation  Acknowledgement and feedback Consultation principles and standards

11. RIA stepsObjectivesExamples Problem Definition and Risk Profiling Evaluate baseline Scope key topics Identify key stakeholders early on Informal consultation Experts advice Surveys Options Selection and Impact Assessment Assess alternatives Collect data Test panels Official consultation document Notice and Comments Prepare analysis and refine proposal Minimize burdensHearings Monitor outcomesCost of implementation Focus groups Consultation is different for each RIA step

12. 1.Plan the consultation in advance 2.Identify stakeholders 3.Prepare consultation document 4.Consult 5.Analyze and validate responses 6.Integrate analysis into the RIA 7.Provide feedback Practical Steps for Better Consultation

13. The effort should be proportionate Set clear objectives Manage the timing and the timeliness Budget the consultation process Adapt and mix consultation mechanisms according to the RIA process Step 1: Plan the consultation

14. Regulated entities –Businesses (local, foreigners, large & small) –Workers –Consumers Stakeholders –NGOs –Academics, think tanks –Other users Public administration (supra & subnational) Political representatives Step 2: Stakeholder Analysis

15. ‘Insiders’ Vs ‘Outsiders’ –Consulting consumers –Future regulated entities Capture through consultation –Lobbying and consultation –Manage vocal groups Do consulted parties represent everyone?

16.  RIA Document  Outline 1.Executive summary 2.How to respond 3.Why is the proposal needed? (problem definition) 1.Options considered 1.Description of the options 2.Type of impacts 4.Questionnaire Step 3: Prepare a Consultation Document (Preliminary RIA)

17. “Garbage in, garbage out” –Partial consultation reduces credibility Broad Vs Target consultation Active Vs Passive consultation e-consultation –Internet publication and feedback Step 4: Consult

18. Targeted Stakeholder and Representative groups Focus groups/Test Panels Public meetings and Conference events Practitioner Panels and Industry-Government Forums Surveys / questionnaires Broad (often mandatory) Written consultation Notice and Comments (pre-publication) Hearings Mandatory advice Broad Vs Targeted Consultation

19. Active Advisory groups, committees, public hearings, Informal consultation Panel & focus tests Surveys Passive Circulation for comment, Notice and Comments (Prepublication) Passive Vs Active Consultation

20. Analysis –Follow up questions, in case –Double check ‘irritants’, opinions and facts –Investigate ‘outstanding’ responses Validate with experts –Consultation is not a public vote –Avoid ‘data’ capture Monitoring results for representativeness Step 5: Analyze & Validate Responses

21. Where evidence is used in RIA report, provide sources Dedicate a separate chapter on consultation: –Who, when and how was consulted –How representativeness and credibility was ensured –What are the results of consultation Provide additional information in annexes: models of questionnaires, tables and graphs related to consultation Step 6: Integrate analysis into the RIA

22. Accountability element Crucial for –credibility, –building trust and –sustainability of future consultation Use information technology Step 7: Provide Feedback