A Fresh Look at the Intervention Logic of Structural Funds Veronica Gaffey, Head of Evaluation, DG Regional Policy, European Commission Helsinki, 4th October, 2012 RegionalPolicy

Content of Paper Background – 1990 to 2000 2000-2006 : Impact indicators used across 270 regional policy programmes – what do they tell us? 2010/2011: a new interest in "outcomes" & Proposals for 2014 – 2020 Results in current 2007-2013 programmes Impact Evaluation – not Impact Indicators Regional Policy 2

Background Reform of Structural Funds in 1988 & 1993 led to multi-annual programmes and a stronger role for monitoring and evaluation 1995: the MEANS programme (Means for Evaluating Actions of a Structural Nature) 6 volumes of guidance, succeeded by EVALSED in 2004 RegionalPolicy 3

Logic of Intervention 1995-2010 Linearity? Link between bottom up & top down? Can we observe an impact? Other contributing factors? Inputs Outputs Results Impacts Global objectives Specific Operational Programme objectives Programme operations

What has been reported 2000-2006? 22,600 indicators in 227 programmes Average 106 per programme (25 in Denmark to 192 in Italy) Classification: 51% outputs 30% results 19% impacts Regional Policy 5

Impacts? With no Baselines – how to judge progress? 5,000 impact indicators – 17% with no values 4,100 with values: 94% had final achievements 58% had targets 6% had baselines 55% had targets and achievements 5% had baselines, targets and achievement With no Baselines – how to judge progress? Contribution of other factors? Regional Policy 6

Monitoring & Evaluation New Intervention Logic? NEEDS Specific Objectives Intended Result selecting indicators of change Monitoring & Evaluation Programming Strategy Other Factors Actual Results change in value of indicators Contribution (Impact) Policy Allocated INPUTS (defining amounts by categories of expenditure) Targeted OUTPUTS (including common indicators) (reporting on expenditure by categories) Achieved

Results Orientation for Future Objectives must have a corresponding result indicator and baseline Result indicator is a proxy for intended change Policy monitoring reports on evolution of results indicator & feeds debate Evolution of result indicator is a consequence of policy and other factors Evaluation to disentangle effects of policy from other factors & unintended effects RegionalPolicy 8

Programmes with Clear Objectives Pilots with 14 regions in 9 Member States, exploring the intervention logic in current programmes: New approach feasible, but not without significant change in mindsets & practice Current indicators mostly do not reflect the intended effects of the priorities Results focus must be part of programme design Concentration a pre-condition Decisions on what to fund and what not to fund – political choice Regional Policy 9

Impact Evaluation not Impact Indicators! Traditional notion of impact – an aspiration Impact should be what can credibly be attributed to a policy intervention We cannot observe impact Focus on impact evaluation not impact indicators (rigorous methods – both quantitative and qualitative, preferably combined) Regional Policy 10

Final Message Evaluators: Reclaim impact! Regional Policy 11

Reference documents Concepts & Recommendations on Monitoring and Evaluation http://ec.europa.eu/regional_policy/information/evaluations/guidance_en.cfm#2 Report on Pilot Tests on Result Indicators http://ec.europa.eu/regional_policy/impact/evaluation/tech_en.cfm#1 Ex ante evaluation (ERDF, ESF, CF) RegionalPolicy