Implementing the NATIONAL EVALUATION STRATEGY Niall McCann, Technical Assistance Project on Programming, Monitoring and Evaluation MINISTRY OF PUBLIC FINANCE MANAGING AUTHORITY FOR COMMUNITY SUPPORT FRAMEWORK Parliament Building, Bucharest, 2 November 2006

The Strategy’s fields of intervention – the activities, the indicators, and next steps Fields of intervention Fields of intervention 1. Training 2. Network and resource support 3. Evaluations! (including evaluations of EU funded programmes, and nationally-funded public interventions, initially through the “Evaluation Facility.” 4. Promoting the gathering of data relevant for evaluation 5. Embedding of evaluation practice in the design and management of public interventions through a normative act

1. Training Training will encompass two specific variants: Training will encompass two specific variants: 1. Training in how to do evaluations Target group – external evaluators in the academic, civil society and private sectors (consultancy companies, etc.) – funding source – Phare 2005 Target group – external evaluators in the academic, civil society and private sectors (consultancy companies, etc.) – funding source – Phare Training in how to manage evaluations Target group – civil servants and other public managers – funding source – OP ACD Target group – civil servants and other public managers – funding source – OP ACD

2. Network and resource support The MACSF ECU will be supported to: The MACSF ECU will be supported to: Build up its role as a source of evaluation expertise and a knowledge resource for other public bodies on all matters related to evaluation – funding source – OP ACD Build up its role as a source of evaluation expertise and a knowledge resource for other public bodies on all matters related to evaluation – funding source – OP ACD Liaise with an independent network of evaluation practitioners – funding source – Phare 2005 Liaise with an independent network of evaluation practitioners – funding source – Phare 2005 An independent network of evaluation practitioners will also be supported to nurture the development of a strong, independent evaluation community – funding source – Phare 2005 An independent network of evaluation practitioners will also be supported to nurture the development of a strong, independent evaluation community – funding source – Phare 2005

3. Evaluations! A number of distinct evaluation programmes will be supported: A number of distinct evaluation programmes will be supported: 1. Evaluation Facility 1 2. Evaluation Facility 2 3. Phare Interim Evaluation Scheme 4. Interim evaluations of the NSRF Operational Programmes 5. Interim evaluations of the Fisheries OP and the National Plan for Rural Development

3. Evaluations! (continued) 1. Evaluation Facility 1 – funding source – Phare 2005 – Public institutions, primarily those that may not have had any engagement with evaluation up to now, will be encouraged to have their public interventions evaluated by external evaluators. 2. Evaluation Facility 2 – funding source OP ACD – this will, in effect, be an expanded continuation of the pilot programme (above). Evaluations will cover all types of public interventions (projects, programmes, policies).

3. Evaluations! 3. Phare Interim Evaluation – funding source – Phare 2005 – each of the 10 Phare Monitoring Sectors will be evaluated on an annual basis. The successful consultants will also conduct ad hoc and thematic evaluations, as well as an annual country summary report. 4. Interim evaluations of the NSRF OPs – funding source – NSRF OP Technical Assistance Priorities, and OP Technical Assistance – the NSRF OPs will be subject to interim evaluations in both 2009 and Ad hoc and thematic evaluations will also be commissioned at either NSRF, OP, Priority Axis or Key Area of Intervention level, that will examine implementation or “horizontal” issues (equal opportunities, etc.) 5. Interim evaluations of the Fisheries Op and the NPRD

4. Promoting the gathering of data relevant for evaluation Funding source – OP ACD – aims to encourage development of monitoring and statistical systems that allow for easier access to relevant data for evaluation throughout the entire public administration. Funding source – OP ACD – aims to encourage development of monitoring and statistical systems that allow for easier access to relevant data for evaluation throughout the entire public administration.

5. Embedding evaluation practice in public interventions through a normative act A two-staged approach: A two-staged approach: 1. Researching and drafting of normative acts necessary to embed evaluation practice into the design and management of public interventions – funding source – Phare Assisting the public administration to implement changes necessary to comply with the legal requirements, following Parliament’s adoption of the normative acts.

Indicators to verify implementation of activities Indicators are identified for each Field of Intervention (FoI) Indicators are identified for each Field of Intervention (FoI) Indicators allow the NES Implementation Committee to determine whether the identified activities are being implemented, and should be updated annually. Indicators allow the NES Implementation Committee to determine whether the identified activities are being implemented, and should be updated annually. Some of the indicators are: Some of the indicators are: 60 students to graduate from evaluation training courses for external evaluators (Training FoI) 60 students to graduate from evaluation training courses for external evaluators (Training FoI) At least 2 persons from middle-level management of county and city councils trained in the commissioning and management of evaluation (Training FoI) At least 2 persons from middle-level management of county and city councils trained in the commissioning and management of evaluation (Training FoI) A central database (and website) of evaluations is managed by MACSF ECU, and updated on an ongoing basis (Resource and networking FoI) A central database (and website) of evaluations is managed by MACSF ECU, and updated on an ongoing basis (Resource and networking FoI) Evaluations of local service delivery commissioned by at least 5% of local public authorities each year (Evaluation Facility FoI) Evaluations of local service delivery commissioned by at least 5% of local public authorities each year (Evaluation Facility FoI)

Next steps… The National Evaluation Strategy Implementation Committee should convene and agree a mandate – February 2007 The National Evaluation Strategy Implementation Committee should convene and agree a mandate – February 2007 Within each Field of Intervention the first steps should be… Within each Field of Intervention the first steps should be… 1. Training The “Development of a Professional Evaluation Community” Phare project should be contracted – July 2007 The “Development of a Professional Evaluation Community” Phare project should be contracted – July 2007 The MACSF ECU, the OP ACD MA should agree a 2-year action plan on a framework for evaluation training courses within the public sector – May 2007 The MACSF ECU, the OP ACD MA should agree a 2-year action plan on a framework for evaluation training courses within the public sector – May Resource and network support The MACSF ECU should agree an annual Work Plan - August 2007 The MACSF ECU should agree an annual Work Plan - August 2007 The MACSF ECU, the “Development of a Professional Evaluation Community” consultants and members of the Romanian evaluation community interested to develop an independent network of evaluators should agree a plan of activities – September 2007 The MACSF ECU, the “Development of a Professional Evaluation Community” consultants and members of the Romanian evaluation community interested to develop an independent network of evaluators should agree a plan of activities – September Evaluations! 4. Promoting the gathering of data relevant for evaluation asdf asdf 5. Embedding evaluation practice in public interventions through a normative act asdf asdf

Next steps (continued)… 3. Evaluations! Phare 2005 consultants will agree the programme of evaluations with public administration bodies and the MACSF ECU – January 2007 Phare 2005 consultants will agree the programme of evaluations with public administration bodies and the MACSF ECU – January 2007 The Phare Interim Evaluation Scheme should be awarded – January 2007 The Phare Interim Evaluation Scheme should be awarded – January 2007 OP Managing Authority Evaluation should complete their OP MAEP – January 2007 OP Managing Authority Evaluation should complete their OP MAEP – January 2007 F4H.2 The MAC SF ECU should complete the NSRF MAEP – March 2007 F4H.2 The MAC SF ECU should complete the NSRF MAEP – March 2007 The Managing Authorities for the NPRD and the Fisheries OP should complete MA EP with the assistance of the MACSF ECU – June 2007 The Managing Authorities for the NPRD and the Fisheries OP should complete MA EP with the assistance of the MACSF ECU – June Promoting the gathering of data relevant for evaluation The MACSF shall oversee a programme of work to install proper, evaluation-friendly monitoring systems at all levels of the public administration – ongoing The MACSF shall oversee a programme of work to install proper, evaluation-friendly monitoring systems at all levels of the public administration – ongoing 5. Embedding evaluation practice in public interventions through a normative act Phare 2005 consultants will employ legal expertise to draft whatever normative acts are necessary in order to embed evaluation practice into the management of public interventions throughout the public administration – December 2007 Phare 2005 consultants will employ legal expertise to draft whatever normative acts are necessary in order to embed evaluation practice into the management of public interventions throughout the public administration – December 2007