1. Twinning Project RO/06/IB/SPP/01 Support to Authority for Coordination of Structural Instruments (ACIS) to ensure a sound and efficient management of the EU Structural Funds savioli@feedback-bremen.de National Evaluation Conference Bucharest, 18.02.2009 Evaluation of SF Some German experience Benno Savioli FEEDBACK-Bremen

2. Twinning Project RO/06/IB/SPP/01 Support to Authority for Coordination of Structural Instruments (ACIS) to ensure a sound and efficient management of the EU Structural Funds savioli@feedback-bremen.de 2 Decentralised System of Implementation  Federal State and 12 / 16 federal Länder  Structural Funds since 1989 – now 5th funding period  Mixing & switching implementation between centralisation & decentralisation according to:  Funds, Funding Period,  Objective 1 (former GDR) and Objective 2/3 areas,  Single Programming Document (e.g. ESF 2000-2006) or multiple OPs approach (current ESF) on side of COM  Management of evaluation reflected this variety of implementation settings

3. Twinning Project RO/06/IB/SPP/01 Support to Authority for Coordination of Structural Instruments (ACIS) to ensure a sound and efficient management of the EU Structural Funds savioli@feedback-bremen.de 3 History of shifts in approach to Evaluation I  Ideal function of evaluation  Instrument for critical reflection regarding to adequacy, efficacy and efficiency of public policy interventions  Introducing abstract public interest (general stakeholder) by objective methodology  creating legitimacy  Complement to NPM discourse Active policy /conditional programmation Commodification of public policy implementation Functional equivalent to market prices  neutral assessment / judgment  Actual function: Support to management  Monitoring substitute  Monitoring complement  Publicity & policy justification  Administrative attitudes  Impingement and reluctance  Benchmarking /QMS mainly focused to providers /administrative performance not that much  More active use as tool for publicity and as a rationale for action

4. Twinning Project RO/06/IB/SPP/01 Support to Authority for Coordination of Structural Instruments (ACIS) to ensure a sound and efficient management of the EU Structural Funds savioli@feedback-bremen.de 4 History of shifts in approach to Evaluation II  Selection of evaluators, types of evaluators, capacity building  Direct grants to public agencies supporting structural funds implementation  Grants based on calls for proposals (NGOs, University institutes)  Public procurement (private research institutes, consultancies)  Setting up networks & partnerships  Informal network of ESF evaluators for exchange of experience, harmonisation of approaches, managed competition, active shaping monitoring via evaluation sub-committees of monitoring committees  EES – European Evaluation Society – Professionalisation, Research and exchange  German Evaluation Society (DeGEval) – Lobbying, standardisation and recommendations, professionalisation by training and topic related working groups

5. Twinning Project RO/06/IB/SPP/01 Support to Authority for Coordination of Structural Instruments (ACIS) to ensure a sound and efficient management of the EU Structural Funds savioli@feedback-bremen.de 5 Effects of EU evaluation request  Administration got used to it  Without Structural Funds this wouldn’t have happened  Number and scope of evaluation expanded (also outside SF)  Administration set up technical teams (capacity raised – partially by hiring former evaluators)  Administration took evaluations more under control (political risks)  Methodology applied evolved  Net effects (control /comparison group designs)  Impacts and sustainability – macro-economic modelling  Professionalisation in general  Standardisation  Post graduate courses  Business  BUT: Moderate influence on policy design  Evaluation usually ‘comes to late’ for political process

6. Twinning Project RO/06/IB/SPP/01 Support to Authority for Coordination of Structural Instruments (ACIS) to ensure a sound and efficient management of the EU Structural Funds savioli@feedback-bremen.de 6 Two highlights  German ESF Stammblatt – Evaluation shapes Monitoring  Common and overarching all ESF programmes approach for standardised data gathering on Projects (type of activity, target groups, benefits offered, region, financial data) Project providers (type, region, size, economic classification) Participants (target groups, gender, age, education, unemployment, participation in active measures, individual results, labour market status change)  Common classifications and indicator definitions  CIP EQUAL evaluation  Systematic multi-level evaluation: EU-wide /national /Projects (DP)  Intense exchange between national and DP-evaluations Guidance function of national evaluation Workshops for DP evaluators on SF-general, Innovation, Gender Mainstreaming, etc Handouts on topics of common concern and interest DP evaluators as source for national evaluation (regular questionnaires, interviews, etc.)  Tremendous increase in evaluation experience by number of involved people

7. Twinning Project RO/06/IB/SPP/01 Support to Authority for Coordination of Structural Instruments (ACIS) to ensure a sound and efficient management of the EU Structural Funds savioli@feedback-bremen.de 7 Positive influence of SF evaluation and promoting factors  Professionalisation on both sides  Administration  Evaluators  Good cooperation & coordination  Cooperation and professional exchange (even under potential competitors)  Active coordination of evaluation by MAs and monitoring sub- committees (  strong /well renowned topic owner)  Independence and fairness  Procurement rules can’t guarantee independence  Independence is a matter of professional attitude and its evidence is given in the process of examination and presentation of results  Evaluation is not the judge himself but provides valid information for rationale judgments

8. Twinning Project RO/06/IB/SPP/01 Support to Authority for Coordination of Structural Instruments (ACIS) to ensure a sound and efficient management of the EU Structural Funds savioli@feedback-bremen.de 8 Thank you for your attention! Now the floor is open for you !!!