1 EU Strategy for the Baltic Sea Region Evaluation: Setting Outcome Indicators and Targets Seminar: 15 March 2011, La Hulpe Veronica Gaffey Acting Director and Head of Evaluation DG for Regional Policy European Commission

2 Objectives and Structure of the Strategy 4 Cornerstones: to make this part of the EU:environmentally sustainable, prosperous accessible and attractive safe and secure 15 Priority Areas about 80 Flagship Projects wide range of Actions

3 Specific Features of the Strategy integrated framework (many related policies) wide range of Objectives at the level of the Strategy and Priority Areas many Actions contribute to more than one Objective concerns over 100 million people in 8 Member States different Member States responsible for co-ordinating various Priority Areas and Actions absence of own funding source - variety of other funding instruments (major source: Structural Funds)

4 Added Value of the Strategy Strategy: co-ordination platform of various policies being implemented in the Baltic Sea Region Goal: co-ordinating an integrated use of existing co- operation patterns, financial resources and know-how of various actors to achieve objectives of several policies in the Baltic Sea Region (Strategys Objectives) in a more effective and efficient way Added Value no one Member State can address this goal alone

5 Terminology Objective: something you plan to achieve/change; expressed as intended Outcome Inputs: are the resources Outputs: describe physical effect of spending resources - what the money purchased Outcomes: reflect targeted change influenced by intervention and external factors; evaluation can help identify Impact (the contribution of the Inputs and Outputs to actual Outcomes) Indicator: variable providing quantitative or qualitative (development trend) information on a phenomenon (Output or Outcome); it can include Target (value + measurement unit + timelines) Baseline: value of Indicator before intervention

6 Theory of Change and Intervention Logic What does the Strategy aim to influence/change (Objectives expressed as Outcomes)? How can we be sure that Flagship Projects and Actions will contribute to achieving the Objectives of the Strategy? How the Strategy contributes to intended or observed Outcomes? How will we know when we have been successful?

7 Challenges: Monitoring and Evaluation (I) complexity of Objectives cross-sectoral and cross-border framework absence of own direct funding shared co-ordination arrangements What do we try to capture? 1.It would be difficult to use Output Indicators (the Strategy does not have its own funding source) although we could refer to Outputs identified under other funding instruments which support the Strategy (it would give us some idea of various interventions)

8 Challenges: Monitoring and Evaluation (II) What do we try to capture? 2.Capturing Outcomes should be possible: they should express changes, taking place in the Baltic Sea Region, which are influenced by the Strategy and other external factors – challenge would be to identify relevant statistics or launch appropriate surveys 3.We should be able to provide evidence that the Strategy has an Impact But, how should we understand this Impact?

9 Challenges: Monitoring and Evaluation (III) Impact: not as a contribution of the Inputs and Outputs to actual Outcomes - establishing clear Cause-Effect Relations between them would be difficult (the Strategy does not have its own source of financing) But, Impact as a contribution of the Strategy - through its Co-ordination Processes - to achieving various policies objectives in the Baltic Sea Region (Strategys Objectives: Outcomes) In other words, Impact = Added Value of the Strategy It would not be possible to quantify this kind of Impact although we should be able to provide a qualitative evidence on it by using evaluation techniques

10 Providing Evidence on Impact: Practical Steps Defining Outcomes Indicators at the level of the Strategy and for each Priority Area and demonstrating their inter-relationships Clarifying Baseline situation in relation to these outcomes indicators and monitoring annually their development Identifying Flagship Projects and Actions which should contribute to actual Outcomes under each Priority Area Evaluating how and to what extent the Strategy (through its Co- ordination Processes - dependencies between mechanisms, e.g., tasks, peoples behaviour, and context) has influenced observed Outcomes: evaluating Added Value of the Strategy

11 The process… How to proceed? –Some external expertise can be built in (e.g. to clarify the logic of intervention, outcomes and their baseline situation, to review the current monitoring system and to accompany preparations of an evaluation exercise) –Who should ensure the annual monitoring of outcome indicators at the level of the Strategy and Priority Areas? Should Member States take ownership of the process under their respective Priority Areas? –Who should be in charge of evaluating the impact of the Strategy and when?

12 Thank you for your attention