Experiences in Finland on self- evaluation of Local Action Groups Seminar on Monitoring and Evaluation of the LEADER Approach Brussels, 25 September 2005 Päivi Pylkkänen, University of Helsinki, Ruralia Institute for Rural Research and Training, Mikkeli

Overview Definitions and characteristics Findings on the Finnish LAGs’ self- evaluations from the perspective of MTE Measures taken to improve the usefulness of the LAGs’ self-evaluations for the LEADER+ MTE Conclusions and recommendations

Definitions and characteristics - SE vs. PE LAG self-evaluationLEADER Programme evaluation Evaluative activity carried out at the level of a Local Action Group, not necessarily throughout by the LAG Principally guided by a LAG’s own needs and interests Recommended, yet a voluntary activity for a LAG Usually more geared towards learning and corrective actions than accountability (formative) Rather standardised activity carried out at the level of the LEADER Programme Guided by the EU regulations and Guidelines on M&E, together with national programme level interests (top- down needs and interests) Usually more geared towards accountability than learning (i,e summative); However, contains also learning objectives (policy level) The challenge from the Commission Guidelines for the Evaluation of LEADER+ (2002): Mid-term evaluations of the LEADER+ programme should, to the extent possible, draw on LAG self-evaluations

Findings on the Finnish LAGs´self- evaluations Self-evaluation recommended for all LAGs in the Finnish LEADER+ Programme ( ) Training on methods for (self-)evaluation provided by the MoA and the national network unit training sessions as from 2002, also trainings for individual LAGs by evaluation specialists Occurrence of planned and documented systematic self- evaluation grown from 25% of LAGs in 2003 to over 50 % by 2005, while all LAGs conduct some kind of evaluative self- reflection Varied focuses of the LAGs’ self-evaluation activities cover e.g.: LAG’s internal board working processes Client satisfaction and feedback on the LAGs’ services Stakeholders’ assessment on the LAGs’ impact

Findings on the Finnish LAGs´self- evaluations (cont.) Only about one fourth of the LAGs’ self-evaluations sufficiently documented from the point of view of programme level mid- term evaluation Following from the different key focuses of the self-evaluations and the programme evaluation, the former proved to be of limited usefulness for the MTE, particularly so in the 2003 MTE The above findings called for measures to improve the usefulness of the LAGs’ evaluative activities for the mid-term evaluation (for the 2005 MTE update and future evaluations)

Measures taken to improve the usefulness of the LAGs’ self-evaluations for the MTE Further evaluation capacity building through national training workshops, material, and tailored training sessions for individual LAGs Independently from diverse self-evaluations, reviewed guidelines for the LAGs’ annual reporting as from the year 2004 to serve as a foundation for the MTE update 2005: Aiming at more reflective, self-evaluative annual reporting by each LAG Merging mere standard recording and reporting with evaluative flavour Focus on selected common topics reflecting the overall MTE evaluation questions, e.g. -Realisation of the LEADER –method, particularly bottom-up, networking.. -Analysis of the LAG’s innovative results and processes -The long-term impacts of completed LAG projects (case studies!) …as perceived, argued and supported by relevant evidence by LAGs themselves

Conclusions and recommendations In the Finnish experience the LAGs’ self-evaluations and the programme evaluations can be mutually complementary: Some data best collected and analysed at the level of LAGs, some at the level of the programme LAG self-evaluations and reporting form a set of data for the programme level evaluations for further analysis, including their critical review Programme evaluations provide data for the LAGs self- reflection (For example in Finland, results from comprehensive questionnaires for funded projects and the stakeholders of the programme, broken down by each LAG, proved useful and interesting for LAGs as self-evaluation material)

Conclusions and recommendations (cont.) To be more useful for programme evaluations, LAGs’ self-evaluations should be: documented (both on methods and results) focused on (selected) common evaluation questions, particularly those related to the LEADER –method and value added of the LAG/the LEADER method in its area of operation systematic in terms of data used as evidence (the burden of evidence on reported result or impact claims rest with the LAGs) Mutual transparency and dialogue on evaluation plans crucial for optimal use of resources Beware: The top-down interests of the programme evaluations should not jeopardise the LAGs’ genuine self-evaluations geared towards LAGs own learning needs;resources needed for both interests!