European Social Fund Guidelines for a comprehensive Evaluation Plan Efie Meletiou Unit 03 Evaluation and Impact Assessment, DG EMPL Presentation to CY ESF Managing Authority Nicosia, 26 October 2010

European Social Fund A comprehensive Evaluation Plan should  Not be a list of planned evaluations!  Show how exactly you will use evaluation as a management tool!  Be based on The Intervention Logic!  …and on programme cycle  Be multi-annual, reviewed annually!  Identify information needs  …and link them to clear monitoring arrangements  Refer to management, timing, resources  Refer to dissemination and use of results

European Social Fund A comprehensive Evaluation Plan should be based on The Intervention Logic

European Social Fund A comprehensive Evaluation Plan should be based on the programme cycle Diagnosis Finalisation Implementation Application Preparation Ex-ante evaluation Evaluation on-going Ex-post evaluation

European Social Fund A comprehensive Evaluation Plan should identify information needs  Assess evolution of programme in relation to objectives  themes (strategic prior. / regulatory requirement)  judgment criteria  specific evaluation questions  Information collection (what/when/how)  links with monitoring  additional fieldwork  …and definitions/thresholds on “important divergence”  procedure for launching operational evaluation An Evaluation Plan should make an explicit commitment to all of the above

European Social Fund A comprehensive Evaluation Plan should identify information needs  Think of addressing Added Value  Volume effects: more/mirroring or boosting  Scope effects: extending coverage, different from volume  Role effects: novel actions/agenda setting  Process effects: Community way of doing things/varies from policy area to area

European Social Fund A comprehensive Evaluation Plan should identify methods of analysis  How to make use of existing indicators  How to make use of other tools  How to combine and analyse  Judgement criteria / baselines  Participative evaluation?  Meta-evaluations

European Social Fund A comprehensive Evaluation Plan should plan for resources  Internal arrangements: - Specific assignment of responsibilities?  Contractual arrangements: - Expert support to MA for whole period? - Framework contract for evaluations? - Contract for 2 evaluations per year?  Steering committee: - How inclusive?

European Social Fund A comprehensive Evaluation Plan should address the dissemination and use of results  Timing is crucial -policy cycle -budget review -stock taking need -opportunity for change  Who is in charge?  What are the procedures?  A learning organisational culture (Sweden)

European Social Fund Some final words Think of evaluation as a resource, not as a way of looking back and allocating blame; and not as an external imposition. Don’t carry out evaluations for their own sake. When planning them think of how they can improve citizens’ lives. Identify all interest groups and users. Ideally, the evaluation plan should be part of the programming document