“The organization of the on-going evaluation of rural development policy in Italy” 122 nd EAAE Seminar “Evidence-Based Agricultural and Rural Policy Making: Methodological and Empirical Challenges of Policy Evaluation” Ancona,18 febbraio 2011 Simona Cristiano, Alessandro Monteleone

Aim: Modelling the governance of the on-going evaluation in view of being influential for evidence-based policies. Methodology: Analysing the state of play of on-going-evaluation in Italy: the context, the framework, the governance models settled; Analysing the first findings Get to some reflections and insights on the way forward to a better use of the evaluations as based on a theoritical framework on influential evaluations Aim of the study and methodology

indicators  Timely establishing and quantifying baseline indicators and target levels monitoring with evaluation  Strengthening the linkage monitoring with evaluation in terms of data collection/provision capacity  Ensuring capacity building early on continuity  Ensuring continuity of evaluation activities, i.e. regular assessment of progress, annual reporting good practices  Supporting the establishment of good practices mid-term and ex-post  Preparing the ground for the mid-term and ex-post evaluations (2010, 2015) Objectives & approach of the EU ongoing evaluation Overall Objective: feeding the policy- makers; Quality of the implementation & and review of RDPs

Responsibilities – Responsibilities: MAs, M&E system and independent evaluators BurdensItaly: 21+1 – Burdens: setting up a M&E system at Programme level (Italy: 21+1) Deadlines – Deadlines: ex-ante, mid-term and ex-post. CMEF – A Common Monitoring and Evaluation Framework (CMEF) Measure-driven – Programme evaluation: Measure-driven overall approach Steps – Steps of the evaluation process Organization – Organization: stakeholders, steering group, evaluation need/services (ToR) questionsindicators –Tools: Common evaluation questions; Common indicators to be complemented by programme-specific ones (baseline-targets) Accountability 4 On-going Evaluation: EU framework

competitiveness I.Improving the competitiveness of rural areas (17 measures) environmentrural heritage II.Maintaining and preserving the environment and rural heritage (13 measures) rural economy quality of life III.Diversification of rural economy and proving the quality of life in rural areas (8 measures) IV.LEADER approach IV.LEADER approach (5 measures) The RDPs: three strategic objectives +a delivering axis 5 In Italy The context of the on-going evaluation in Italy + 21 Rural Development Programmes (regional level) + 1 National Rural Development Network Programme (national level)

evaluation needClair identification of evaluation need : beyond the requirerments; addressing the specific needs for knowledge; territorial approach (evaluation plan). LearningLearning process (mutual): implementation and effects of the policy; feeding the policy decision –maker (review and future programming period). Evaluation Culture & Ownership, through participative evaluation process and vocational activities The Italian approach 6 Ownership CapabilitiesUtilization-focused Ownership, Capabilities, Utilization-focused

… through … GovernanceGovernance: setting up proper structures (M&V unit and steering group); definition of the evaluation need; adequateness of financial resources. CommunicationCommunication: transaparency on results and participative dissemination. Participative processParticipative process: let emerging the evaluation need and governing the evaluation process. sharing practisesIdentify and sharing practises CoordinationCoordination: coordination with other territorial policies. Guiding role for the National Monitoring and Evaluation Network 7

Italy: the Evaluation budget (%)

Indipendent evaluators in Italy ESA-AGER (1) Agriconsulting (8)  Vary timing and Delays in selecting  21 RDPs/10 evaluators; 7 Partnerships  Evaluation “market”  Variety in budgets  Relevant Expertise:  Regional policy;  Technical Assistance on Rural policy Ecosfera (5+NRN) Agrotec – Disamis (1) ISRI (1) In house (1) ESA (1) IZI – Appollis (1) AGER – Starter (1) Agrotec–Rina Value (1)

10 Steering groups: setting up 10 Set up (6) Foreseen (10) Not esplicitely foreseen (5)

Governance Models in Italy 11

First findings  Evaluation Culture ⇒ Utilization-focused evaluations ⇒ Evidence-based policies  Increasing consciousness on the evaluation needs;Participative process: “accountability” vs. “learning process”  Steering group as pathawy of influence; Continuity: the role of the evaluation unit; Discussion on the results and Reviewing the programmes  Evaluation capabilities ⇒ Innovation ⇒ more Targeted Evaluations  Competitive evaluation market; Innovative approaches and methodologies; Increasing the skills and the exchange of practieses  Improve the avalability of data and their evaluation-utility  Awarness ⇒ Rural Activicitizenship ⇒ Need Assessment/Responsiveness  Awarness and communication ⇒ Rural Activicitizenship ⇒ Need Assessment/Responsiveness  Reaching the territory: stakeholders & rural population  Communication: innovative activities; Tailor-made  Sharing practises; second language; dialogue EvaluationNetworks  European and National Evaluation Networks as boosters

Reconducting the empirical evidence to the theoretical framework: evidence-based policy & evaluation influence (Patton, (1998, 2007, 2008); Preskill (1994, 1997, 1998); Kirkhart (2000); Mark & Henry (2003, 2004); Cousins (2005); Barca (2009); Verdung (2009): The way forward … Evidence-based policy and programmes Use of the evaluations and follow- up actions Utilization- focused evaluations

Thank you for attention Simona Cristiano: Alessandro Monteleone: For further information and updatings on the research, please take a look on the website: 14