Counterfactual impact evaluations in Poland Expert hearing on MS experiences on using control groups in ESF evaluations Stanislaw Bienias National Evaluation Unit Ministry of Regional Development Poland Ministry of Regional Development - May 8, 2019

The impact of ESF interventions on the human capital development. Counterfactual impact evaluations commissioned by the National Evaluation Unit The impact of cohesion policy on the level and quality of employment in Poland. The study was supposed to reveal the impact of the co-financed structural interventions on creating and maintaining jobs, including also the evaluation of their quality and durability and allowing to formulate conclusions and recommendations in this field. The impact of ESF interventions on the human capital development. Evaluation of the impact of Cohesion Policy on growth of competitiveness and innovativeness of Polish enterprises and economy. There are also a few counterfactual evaluations commissioned by the Polish Agency for Enterprise Development Ministry of Regional Development - May 8, 2019

The impact of cohesion policy on the level and quality of employment in Poland - Reasons for choosing this particular topic to be evaluated Poland has decided to conduct an ex-post evaluation of 2004-2006 period. The evaluation study The impact of cohesion policy on the level and quality of employment in Poland was a part of this ex-post evaluation process. This employment study was at the same time a part of common international project of V4 countries, which was based on the similar Terms of Reference. The employment topic has seemed extremely interesting and important to be evaluated also due to the Lisbon Strategy objectives. Ministry of Regional Development - May 8, 2019

Challenges in preparing and managing CIEs One of the biggest challenge while using control groups in the researches has been to complete all the necessary data from the available databases: It was necessary to contact about 100 Labour Offices in the regions which administrate PULS databases for each district - Accessibility of the dispersed data Databases are different in each region and people who administrated them no longer work in the offices and they lack important data Personal data protection – special contracts had to be signed between the evaluator and Labour Offices, which took a lot of time There are few research/consulting companies on the market, who are able to conduct the counterfactual evaluation – it is a new trend/method. It is sometimes difficult to explain the findings – we receive the answer to the „how much” question but we do not always get the right explanation for the „why” question. Ministry of Regional Development - May 8, 2019

Use and dissemination of results The report and brochure have been printed and an article presenting the findings has occurred in a newspaper. The report has been placed on the evaluation website in order to make it available for all interested readers. The employment evaluation has been used in order to prepare an ex-post evaluation of 2004-2006 and it was presented on the Evaluation Conference in Warsaw. The National Evaluation Unit has prepared a special document including conclusions from this report, which constitutes a basis for discussion on the future shape of cohesion policy. Ministry of Regional Development - May 8, 2019

Possible corrective actions taken to improve the systems/procedures to undertake CIEs; Improving accessibility and quality of the data seems essential – it should be planned from the beginning what we want to achieve and which data are going to be needed. It is important to strengthen the potential and the awareness of evaluation units and institutions involved in the system as well as research companies. At the beginning of each programming period it should be clear what changes are expected. Under that condition it will be easier to conduct such counterfactual evaluations. Ministry of Regional Development - May 8, 2019

Thank you very much for your attention! Stanislaw Bienias National Evaluation Unit Ministry of Regional Development Poland stanislaw.bienias@mrr.gov.pl Thank you very much for your attention! Ministry of Regional Development - May 8, 2019