1. Evaluation Network Meeting
Guidance document on monitoring and evaluation Update November 2012 (ERDF + Cohesion Fund)
Evaluation Network Meeting
23 November 2012
Kai Stryczynski

2. Changes Clarification on use of result indicators
Independence of evaluators
Adaptation to compromise text of Presidency (GAC June 2012)
Adaptation of annex on common indicators (GAC October 2012)

3. Result indicators in OP and their monitoring: not just beneficiaries
Baseline and targets in OP should relate to the same population / region / sector,…
Coverage: future (still unknown!) beneficiaries plus non-beneficiaries
Why? The situation of the total of the population / region / sector motivates the decision to run an intervention
Monitoring in AIR: same coverage as for baseline and target
One chain: Baseline – monitoring - target

4. Evaluation 1: values for result indicators on beneficiaries and more needed
Evaluations need certain data depending on method applied. Typically this is different from monitoring data!
Example: a counterfactual analysis might need:
Financial data: who received how much?
Observed change of result indicator for beneficiaries and a (not supported) comparison group
Additional data for matching (subsector, enterprise size, age,…)
Cohesion Policy 4

5. Evaluation 2: values for result indicators on beneficiaries and more needed
Source: Enterprise panel, tax administration,… but most of it not information that is in AIR
The evaluation plan needs to spell out which data is needed depending on method
These are the headline findings from the study so far.
Jaspers has provided assistance to projects worth about 64 billion Euros, between 2006 and 2011.
There has been a measurable effect on the amount of time which the DG for Regional Policy took to make a decision on
Jaspers supported projects – those projects took on average 86 days less to get approval, than similar projects which had not received JASPERS assistance.
Some MS used Jaspers assistance for all their major projects, others such as Poland and Estonia, used Jaspers for only about 50% of their major projects.
The most popular topic on which Jaspers gave advice, was cost benefit analysis, followed by Funding and Financing issues, project concept and programming
and then environmental issues.

6. Example: Result indicators in M+E for counterfactual evaluation (diff in diff)
 Selected result indicator
 Data on whom / what
2013
During period
2022
 Use for
For whole population
Baseline
AIR
Target
Programming / monitoring
For beneficiaries
Initial situation
Situation after support
 Evaluation
Comparison case(s)

7. Example: policy monitoring and evaluation for transport infrastructure
Task becomes simpler if the sector / region found to have a problem at the beginning of a period and the benefitting sector / region are identical
E.g.: A new highway will cross a region and improve its access to the next metropolitan region, measured in average travelling time (result indicator)
Still a task for evaluation: is the observed change in travelling time only due to the new highway?

8. Closing the policy cycle
Policy monitoring + impact evaluation `=
full picture:
How has a specific dimension of wellbeing that motivates policy action changed during the period – and what was the contribution of the policy to this.

9. Independence of evaluators (art. 47, section 3.4)
Distinction of
best practice: external experts; different organisation
good practice: different department
other cases: clear arrangements ensuring independence

10. Adaptations to Presidency Compromise June 2012
Evaluation plan: submission to monitoring committee no later than a year after programme adoption (CPR, art. 104)
Evaluation plan can cover several OP
Summary of evaluations by MS in 2021 (CPR, art. 104)

11. Next steps Revision after this meeting
(Possible) revision after adoption of regulations