1. A Fresh Look at the Intervention Logic of Structural Funds
Veronica Gaffey,
Head of Evaluation, DG Regional Policy, European Commission
Helsinki, 4th October, 2012
RegionalPolicy

2. Content of Paper Background – 1990 to 2000
: Impact indicators used across 270 regional policy programmes – what do they tell us?
2010/2011: a new interest in "outcomes" & Proposals for 2014 – 2020
Results in current programmes
Impact Evaluation – not Impact Indicators
Regional Policy 2

3. Background
Reform of Structural Funds in 1988 & led to multi-annual programmes and a stronger role for monitoring and evaluation
1995: the MEANS programme (Means for Evaluating Actions of a Structural Nature)
6 volumes of guidance, succeeded by EVALSED in 2004
RegionalPolicy 3

4. Logic of Intervention 1995-2010
Linearity?
Link between bottom up & top down?
Can we observe an impact?
Other contributing factors?
Inputs
Outputs
Results
Impacts
Global
objectives
Specific
Operational
Programme objectives
Programme operations

5. What has been reported 2000-2006?
22,600 indicators in 227 programmes
Average 106 per programme (25 in Denmark to 192 in Italy)
Classification:
51% outputs
30% results
19% impacts
Regional Policy 5

6. Impacts? With no Baselines – how to judge progress?
5,000 impact indicators – 17% with no values
4,100 with values:
94% had final achievements
58% had targets
6% had baselines
55% had targets and achievements
5% had baselines, targets and achievement
With no Baselines – how to judge progress?
Contribution of other factors?
Regional Policy 6

7. Monitoring & Evaluation
New Intervention Logic?
NEEDS
Specific
Objectives
Intended Result
selecting indicators of change
Monitoring & Evaluation
Programming
Strategy
Other Factors
Actual
Results
change in value of indicators
Contribution (Impact)
Policy
Allocated
INPUTS
(defining amounts by categories of expenditure)
Targeted
OUTPUTS
(including common indicators)
(reporting on expenditure by categories)
Achieved

8. Results Orientation for Future
Objectives must have a corresponding result indicator and baseline
Result indicator is a proxy for intended change
Policy monitoring reports on evolution of results indicator & feeds debate
Evolution of result indicator is a consequence of policy and other factors
Evaluation to disentangle effects of policy from other factors & unintended effects
RegionalPolicy 8

9. Programmes with Clear Objectives
Pilots with 14 regions in 9 Member States, exploring the intervention logic in current programmes:
New approach feasible, but not without significant change in mindsets & practice
Current indicators mostly do not reflect the intended effects of the priorities
Results focus must be part of programme design
Concentration a pre-condition
Decisions on what to fund and what not to fund – political choice
Regional Policy 9

10. Impact Evaluation not Impact Indicators!
Traditional notion of impact – an aspiration
Impact should be what can credibly be attributed to a policy intervention
We cannot observe impact
Focus on impact evaluation not impact indicators (rigorous methods – both quantitative and qualitative, preferably combined)
Regional Policy 10

11. Final Message
Evaluators:
Reclaim impact!
Regional Policy 11

12. Reference documents
Concepts & Recommendations on Monitoring and Evaluation
Report on Pilot Tests on Result Indicators
Ex ante evaluation (ERDF, ESF, CF)
RegionalPolicy