W. Schiessl, AGRI E.II.4 Programme management and institutions involved in monitoring and evaluation

2 Purpose of presentation To show the importance of monitoring as part of the management of the IPARD programme by  identifying the legal basis and the provisions concerned  showing its main elements, including monitoring indicators  explaining the main actors involved and the links between them  discussing the setting-up of a monitoring system  summarizing the use of “monitoring” for the IPARD programs

3 Programme management and institutions involved in monitoring and evaluation Monitoring and evaluation is part of the logic of programming, which consists of -a description of a given situation -the identification of weaknesses and potentials, of risks and opportunities (SWOT) -the identification of objectives and targets -the strategy to meet them -the measures chosen to implement the strategy -the monitoring of the progress of implementation -the evaluation of the results achieved -the lessons learned to improve the programme or policies Monitoring = what is being done right now (immediate results) Evaluation = what was done, how was it done and what can be done better

4 Programme management and institutions involved in monitoring and evaluation Legal basis and provisions concerned Legal basis  Monitoring -Articles 36 and 38 of the Framework Agreement (FA) -Articles 60 – 62, 68 of the Sectoral Agreement (SA)  Evaluation -Articles 62 – 67 of the SA

5 Programme management and institutions involved in monitoring and evaluation Main elements of Monitoring Main elements of Monitoring 1 Annual reports 2 IPARD Monitoring Committee 3 IPA Monitoring Committee 4 Final report

6 Programme management and institutions involved in monitoring and evaluation Main elements of Monitoring Annual Report (Article 68 of SA)  Produced by the Candidate Country (CC) concerning civil year N -1  Shall cover the calendar year and shall include the cumulative financial and monitoring data for the whole period of implementation  Needs to be approved by the Monitoring Committee  Shall be submitted within six months of the end of each full calendar year of the implementation of the IPARD Programme – 30 June of year N

7 Programme management and institutions involved in monitoring and evaluation Main elements of Monitoring Annual Report - procedures Established by the Managing Authority in close cooperation with the IPARD Agency and submitted to the Commission before end of June Double check by Commission: completeness and quality  Completeness (admissibility): check within one month if report covers all elements of Article 68 (5). If not, payments to CC are interrupted (Article 41 of SA)  Quality: Commission has three months to comment The report, once accepted, shall be the basis for an review of the main outcome of the previous year and of comments and/or recommendations of the Commission for adjustments to the programme

8 Programme management and institutions involved in monitoring and evaluation Main elements of Monitoring Annual Report - content (Article 68 (5) of SA)  change in general conditions of relevance to the implementation of the IPARD programme  progress in implementation  steps taken by operating structures and MC to ensure the quality and effectiveness of implementation  steps taken to ensure co-ordination of all Community pre-accession assistance  financial tables showing Community, national and total expenditure by measure and, where appropriate, per sub-measure  summary of the on-going evaluation activities  description of activities carried out for implementation of the communication plan  Information should be based on the “common indicator tables for monitoring and evaluation of IPARD” established by the Commission

9 Programme management and institutions involved in monitoring and evaluation Main elements of Monitoring Annual Report - cumulative financial and monitoring data for the whole period of implementation (Articles 60, 62 and 68 of the SA)  “ Common indicator tables for monitoring and evaluation of IPARD programs 2007 – 2013  Developed by the Commission to provide a common basis for monitoring and evaluation and cross-cutting information to the public, based on exact and comparable data  They have to be seen in the context of each programme  Consist of a guidance document and common indicator tables in form of excel sheets  Common indicator tables consist of general tables (G), output and results indicator tables (O), baseline indicator tables (B) and impact indicator tables (I)  Shall be filled in by the Managing Authority in close cooperation with the IPARD Agency and shall be submitted to the Commission as annex to the annual report

10 Programme management and institutions involved in monitoring and evaluation Main elements of Monitoring Annual Report - cumulative financial and monitoring data for the whole period of implementation (Articles 60, 62 and 68 of the SA)  The guidance document includes the meanings of the financial terms and the common definitions to be used when completing the tables  The tables are structured around measure codes which are integrating the IPARD measures into the Rural Development Measure system of Member States. They should be used for the reporting and at a later stage also for the programmes  The tables shall still be discussed with the Candidate Countries

11 Programme management and institutions involved in monitoring and evaluation Main elements of Monitoring IPARD Monitoring Committee (Article 36 of FA, Article 61of SA)  Set up and chaired by the Candidate Country, it involves all relevant authorities and bodies, and appropriate economic, social and environmental partners (stakeholders)  The Commission, the Head of the IPARD Agency and the NAO shall participate in its work. It shall report to the IPA Monitoring Committee.  It shall examine the results of the IPARD program, consider, approve and propose to the Commission amendments to the program, consider and approve the annual and final reports, examine the on-going and interim evaluation, the communication plan  It is the main monitoring instrument and bears a specific responsibility for the successful implementation of the program.

12 Programme management and institutions involved in monitoring and evaluation Main elements of Monitoring IPA Monitoring Committee (Article 36 of FA, Article 61 (4) of SA)  Set up by the CC, the IPA Monitoring Committee shall ensure coherence and coordination in the implementation of the iPA components  It shall satisfy itself as to the overall effectiveness, quality and coherence of the implementation of all programmes and operations towards meeting the objectives set out in the multi-annual indicative planning documents (MIPDs) and the financing agreements (MFA)  It shall include among its members representatives of the Commission, the national IPA co-ordinator (NIPAC), the NAO and representatives of the operating structures  A representative of the Commission and the NIPAC shall co-chair its meetings

13 Programme management and institutions involved in monitoring and evaluation Main elements of Monitoring Final Report (Article 68 of SA)  To be drawn up by the Managing Authority, following consultation with the IPARD Agency  Shall contain the same information as the annual reports but shall cover the whole period of implementation and may include the last annual report  Shall be submitted at the latest six months after the final date of eligibility of expenditure under the IPARD Programme  Follows the same procedure as for annual reports

14 Programme management and institutions involved in monitoring and evaluation Main actors involved The main actors involved, their roles and interaction  Managing Authority – establishing a monitoring system which allows to gather, verify and process appropriate information and data for establishing the reports to the MC and thus informing on the progress made in implementation – close cooperation with IPARD Agency is necessary  IPARD Agency – establishing a system which allows to receive adequate information and data from the final beneficiaries, to verify and process these data and to inform the Managing Authority accordingly – close cooperation with the Managing Authority is necessary  IPARD Monitoring Committee – effectiveness and quality of program implementation in order to attain its objectives  IPA Monitoring Committee - overall effectiveness, quality and coherence of programme implementation towards meeting the objectives set out in the MIPD  NAO  NIPAC  Commission

15 Programme management and institutions involved in monitoring and evaluation Monitoring system Steps for establishing a monitoring system  Step 1 - Understanding the key success factors for setting-up a monitoring system  clear organisation in practice and in terms of procedures  information agreement between institutions involved  Step 2 - Deciding what to monitor and which indicators to use  proper mix of data (baseline, input, output, results)  sharing definitions of indicators  Step 3 - Deciding how to measure  integrate monitoring systems and electronic data exchange  Step 4 - Check the quality and timeliness of monitoring data  good record keeping  reliability and timeliness  Step 5 - Reviewing progress

16 Programme management and institutions involved in monitoring and evaluation Monitoring System Step 1: Understanding the key success factors for setting-up a monitoring system Drafting clear procedures and supporting documents (manuals, guidelines) for the organisation of the collection and transfer of data  What data are collected on what level  How data are transferred between the levels  Ensuring internal capacity for monitoring in all institutions involved in collecting data  Involving all relevant institutions  Information agreement - set of explicit and tacit agreements and understandings between and among participants on the monitoring process that leads them to generate, share and act upon information  Agreement amongst all actors participating in the implementation process is essential

17 Programme management and institutions involved in monitoring and evaluation Monitoring System Step 2 - Deciding what to monitor and which indicators to use 1) Indicators agreed in the IPARD programme and the “common indicators for monitoring and evaluation”:  Financial (Input): to monitor progress in commitment and payment  Output: relate to activity (measure), measured in physical (monetary) units  Results: to monitor direct and immediate effects on final beneficiaries ie. changes in farm management, business objectives  Baseline: to monitor socio-economic situation of the programme 2) Standardised data on beneficiaries (recommended)

18 Programme management and institutions involved in monitoring and evaluation Monitoring System Standardised data on beneficiaries (recommended)  Collected for all projects and final beneficiaries using standardised form  Type of data – as needed for monitoring and evaluation e.g.:  Gender, Age,  Main occupation  farm situation  etc.  Advantages of collecting data on beneficiaries: Preparation of monitoring system and stakeholders for requirements under IPARD Provision of standardised data on beneficiaries Possibility to analyse effects of the programme – survey of people leaving the measure Fulfilling information needs – use of data for reporting, public communications

19 Programme management and institutions involved in monitoring and evaluation Monitoring System Step 3 - Deciding how to monitor 1.How will the monitoring data be collected, how often and by whom? 2.How will the information be transferred between different levels of the IPARD program implementation system? 3.Who will ensure reporting requirements on the implementation of the program?  How to prevent from counting outputs twice (“double counting”)?  Do the proposed procedures comply with the data protection law (if any)?

20 Programme management and institutions involved in monitoring and evaluation Monitoring System (1) How will the monitoring data be collected, how often and by whom?  Ensure entry of data  Monitoring and evaluation requirements should be built into the calls for projects and proposal documents (application form)  Provision of data on indicators should be built into the contracts and beneficiaries shall be obliged to provide data systematically  Decide on monitoring timetable Entering data systematically, e.g. at the start of the project for information on beneficiaries Reporting intervals should be agreed and informed to all institutions participating in the monitoring system  Outline who does what: -Data collection -Data analysis -Use of information generated (e.g. decision on launching evaluation)

Programme management and institutions involved in monitoring and evaluation Monitoring System (2) How will the information be transferred between different levels of IPARD program implementation?  Key issues:  Setting up, maintaining and updating regularly the information system – manual/ electronic  Transfer and aggregation of information on monitoring indicators from projects/final beneficiaries to higher levels of the IPARD implementing system  Possible settings Excel sheets / Access database Internet based information exchange system

Programme management and institutions involved in monitoring and evaluation Monitoring System (3) Who will ensure the reporting requirements on the implementation of the IPARD program? Final beneficiaries – Provision of data IPARD Agency –Preparation of monitoring data/reports within its area of responsibility –Preparation of relevant sections of annual and final reports, within its area of responsibility Managing Authority –Coordination of programme monitoring –Preparation of annual and final reports IPARD Monitoring committee - examines achievement of the targets (for each measure and priority axis) –examines annual and final reports –reports to IPA MC on progress made in implementing the programmes (results achieved, financial indicators)

23 Programme management and institutions involved in monitoring and evaluation The use for IPARD Why should there be a Monitoring?  To fulfil regulatory requirements? rules on monitoring reporting on financial and physical progress of the programme implementation (indicators as agreed)  To improve programme management and implementation!  better managed, timely remedial actions => high quality of outcomes  avoid risks  independent judgement on progress

24 Programme management and institutions involved in monitoring and evaluation That’s why reliable and professional monitoring is important as part of the management of the IPARD programs! THANK YOU VERY MUCH FOR YOUR ATTENTION