26 June 2008 DG REGIO Evaluation Network Meeting Ex-post Evaluation of Cohesion Policy Programmes co-financed by the European Fund for Regional Development (Objectives 1 and 2) Work package 2: Data feasibility study This document is designed as support to the oral presentation and is not intended to be used separately

2 Agenda 1.Main objectives of WP 2 2.Architecture of the DB and methodological approach 3.Basic statistics 4.How to use the Database ?

3 Main objectives of WP 2 To assemble information on indicators of O1 and O2 operational programmes single programming documents and Community Support Frameworks co- financed by the ERDF in the programming period To verify the availability of data To provide an input for other work packages of the ex-post evaluation Task 1: Assembling information on indicators in a database  Interventions co-financed by ERDF (233 programmes covered)  Definition of indicators (in language of MS and English), Baselines, Target values, Values achieved, type of indicators Task 2: Singling out core indicators  Task 3: Green house gas emission 

4 Tasks 1 & 2: DB Architecture (1/2) Architecture related to the programme structure Priority 1 Priority 2 … Measure 2.2 Priority 1 Measure 1.2 Measure 1.1 Measure 2.1 Programme Member State … -O1 / O2 -Region -Period -Language …

5 Tasks 1 & 2: DB Architecture (2/2) Architecture related to the indicator - Link to categories ( ) - Link to core indicators ( ) Transversal issues -Gender related -Environment related Type of indicator: -Output -Result -Impact Values: -Baseline -Target -Achieved Link to programme, priority(ies) or measure(s) Indicator: - Title - Definition

6 Methodological approach – Tasks 1&2 Desk research using in particular the latest annual reports available Database design and implementation Test phase for data entry using data from five annual reports Guidelines for data entry Helpdesk for people in charge of encoding Quantitative analysis focusing on the indicators’ characteristics Identification of the linkages with the core indicators during data entry Specific quality control on the indicators entered in the database

7 Quality control process for the indicators entered A cross check of all the data entered by an internal expert to systematically review the following points: The types of indicators, especially when the type is NOT mentioned in the report The unit mentioned and its consistency with the indicator’s title The core indicators possibly related to it The linkage of the indicator to the programme structure Consideration of the implicit baseline value of zero Checking gender- and environment-related marks English translation and spelling mistakes

8 Basic statistics Average number of indicators per Member State More than 21,250 indicators for 233 programmes Overall average of 91 indicators per programme Variation of this figure across the programmes

9 Types of indicators Share of each type of indicator per Member State Specified in 72% of the reviewed programmes Greece, Sweden, Finland and Cyprus do not specify the types Germany and France annual reports provide the type for about a half of the reports

10 Baseline values There are a limited number of non null baseline values Most of the baseline values are « implicit » null values Share of indicators having a baseline value

11 Actual values Share of indicator having an actual value per Member State

12 Target values Share of indicators having a target value per Member State

13 Main limitations It is not a finished product. Must be maintained (correct mistakes, add lacking information) and fine-tuned (add queries and features) Main shortcomings: –Imperfect and heterogeneous nature of the information provided on the indicators ( unit, definition of coverage, year of reference) Problems of non cumulative values and double counts Need for a careful interpretation: –Regarding the type of indicators: difficult to specify and analyse it in the right way –Indicators must be analysed in the context of the programme, priority or measure they are related to –References are rarely available (benchmark)

14 Core indicators: 2867 out of 21250

15 Database features Software requirements: –MS Access 2003® or later version –MS Excel 2003® or later version Simple and detailed search with the search results provided under MS Excel® Input interface at programme and indicator levels (Version A) Report for printing layout (for each programme) (Version A)

16 Features of the detailed search of indicators Search keywords, several keywords are searched with a « OR » logical operator. Search in : –the English title and definition fields –the measures and priorities the indicator is relating to (De-)Select the objective, the type of indicator and the level of indicator for the search

17 Features of the detailed search of indicators Structured criteria include the selection of MS, Region, Programme, Prority and measure Categories and themes can also be used in comniation with the selection of MS

18 Features of the detailed search of indicators Search indicators related to any of the 38 core indicators Search indicators related to one specific indicator All different search criteria are combined using the logical operator « AND »

19 Search results provided in a Excel file

20 Important information to use the results Search results presented according to the programme structure Functionalities of MS Excel are fully available The values of the baseline, achieved and target values have to be converted to MS Excel number format before making calculations The dot is used as the decimal separator (see your Excel configuration) Modifying the content of the table will no affect the data in the database Calculations should be done carefully in regards to the limitations mentioned Please refer to the Quick User Guide on your CD