Planning Evaluation 1 Planning evaluation An example - ERDF Berlin Dr. Oliver Schwab IfS Institut für Stadtforschung und Strukturpolitik

Planning Evaluation Overview ERDF OP of Berlin - the Evaluation Subject Evaluation System - the Structure Planning Evaluation - the Process Conclusions 2

Planning Evaluation The subject ERDF-OP Berlin Some 635 Mio. € ERDF Four Priorities –Innovation -Specific objective: Strengthening R&D in enterprises -Result indicator: R&D-personnel in Berlin –Investment -Specific objective: Increasing productivity -Result indicator: Growth rate of productivity (compared to the national average) –Climate protection –Integrated urban development OP will be formally submitted by end of this month 3

Planning Evaluation Structure: Monitoring and evaluation system - Overview 4 Implementation Outputs Results Projects System of objectives Expected results Specific objective Actions Indicator System Monitoring Common output indicators Result indicators Financial indicators Output indicators (OP-specific) External factors Annual reports Internal monitoring reports Evaluation

Planning Evaluation Structure: Evaluations One Evaluation study per priority axis Scheduled from 2017 to 2019 Possibly covering pre-2014 data and results for instruments that have not been changed Predominantly theory-based Checking if comparison-group approach is feasible for parts of PA 1 (innovation) Additional ad-hoc-studies Focused on implementation-related issues 5

Planning Evaluation Process – First teps 2012 Call for tender for external evaluation and strategic support –Defining the basic pattern of the evaluation system –First decisions on the approach in the offer Monitoring and evaluation system profits from the experiences of the 2007 to 2013 period 6

Planning Evaluation Process – Strategy Development 2012/2013 (More) strategic programming –Draft Strategy Outline (MA, June 2012) –Expert Workshop per Priority (Autumn 2012) –Development of concrete proposals to implement the strategy –Consultation on a draft programme structure (Summer 2013) Crucial for future evaluation –Clear definition of objectives –Good result indicators –Selection of instruments as focused as possible -Basis for future intervention models 7

Planning Evaluation Process – Planning Evaluation 2014 Literature Review –Reviewing academic studies plus available relevant evaluations (some 20 pages per priority) -(Re-)Construction of the intervention model for each priority -Identification of the relevant external factors 8

Planning Evaluation Process – Evaluation Plan 2014 The actual evaluation plan, per priority: –Defining the concrete evaluation questions –Selecting the focus for the studies –Defining data sources -Relation to monitoring? -Additional own data collection? Methods and Design? –Methods for evaluation –Processes of evaluation and discussion of results 9

Planning Evaluation Conclusions Intensive communication is essential, linking competences of Managing Authorities Bodies responsible for the delivery of single instruments Evaluation Team Potential users of the evaluation results (Monitoring Committee) Coordination of crucial aspects Developing a common understanding of the intervention logic Coordination of data collection (indicator definition!) Link to Monitoring processes (using existing reports and analysis) Coordination with additional evaluation activities Question and purpose of the evaluation 10

Planning Evaluation Conclusions Internal evaluation plan More comprehensive and detailed than the „official“ one Training? Specific situation Early involvment (and long-term contract) Good relation to most actors Possibility to coordinate with the development of the monitoring system and additional evaluations Most actors are interested in evaluation (or at least open for) 11