Synergy of Government and Non-Government Bodies’ Involvement in Monitoring and Evaluation

Goal of M&E Process The ultimate goal of M&E process is to present to beneficiaries and stakeholders a fair and unbiased perception of project progress achieved. Progress measures with the extent which the original project goals have been achieved to.

Effectiveness of M&E process Effectiveness of M&E process measures with: Consistency of the fundamental principles that regulate project implementation and organizational arrangements such as: transparency; objectivity; centralization-vs- decentralization; delegation; accountability, coordination-vs-communication. Capacity to promptly identify, analyze and adjust whatever improprieties may occur. Capacity to provide an all-inclusive evaluation perspective, ie, covering all the phases of the project cycle.

Challenges to M&E Process The mainstay of issues to be considered is secure M&E effectiveness through involving outside, ie, non-government bodies. Why is that topical? 1.Including outside bodies stands for an integral and autonomous M&E structure which does not necessarily follow the administrative line structures. 2.Unlike decision making M&E needs intensive communication on the account of coordination, ie, on the account of rigid line administration’s restrictions to communicate and share knowledge. 3.Including outside bodies stands for 1) large participation of beneficiaries’ organizations; 2) building capacity at local level; 3) increasing effectiveness and efficiency of M&E. 4.Therefore, it would allow for overcoming the deficiencies of the centralized line administration without challenge it.

The presentation dwells on short falls and positive effects of country led evaluation systems based on a survey of monitoring and evaluation (M&E) procedures under two types of projects being implemented by Ministry of Labor and Social Policy: PHARE projects which are implemented by the EU Pre- Accession and International Projects Directorate with the Labor Ministry (for the current financial year the PHARE Program amounts to EUR 40 m round Social Investment and Employment Promotion Project, joint development project of Labor Ministry and World Bank of total capitalization EUR 66.7 m which is being implemented by the Bulgarian Social Investment Fund (semi-autonomous administrative body to the Labor Ministry)

M&E Patterns: PHARE Program with the Labor Ministry EU Pre-Accession and International Projects Directorate Financial Department - Financial Coordinator - Technical Coordinator - Financial Controller (Representative of the State Agency for Internal Financial Control) M&E Unit in charge of operational monitoring Regional Level (28 Regional Coordinators) Beneficiary Level

M&E Patterns: SIEP Project Steering Committee Based on Tri- Partite Principle Executive Director Departments TechnicalFinancial Advisory Services Legal Advisor M&E Coordinator Local Level Monitoring – By Competitively Selected Private Inspection Companies Local Level – Communities and Municipalities

Is it possible to decentralize M&E within the official structures? Both M&E patterns evidence a high level of centralization. The M&E arrangements follow the line administrative structures of the respective official institutions. No, because: 1.Both development projects and EU funded projects have been designed, planned and implemented at the Government’s discretion. They are Government led projects; they meet challenges the Government reckons relevant; the respective Government institutions have justified their relevance to the donors. 2.M&E is a multi-faceted process but financial/accounting audit plays the leading role within it. By rule financial audit is centralized because the official institutions which are in charge of project implementation use their own centralized bodies for this purpose. 3.Individual data collection and data processing through an integrated Management Information System requests centralization of communication

Evidences Mechanical division between financial monitoring and technical monitoring is available. Financial monitoring is formal; technical monitoring is substantial. Evaluating projects’ impact requests integrity of both formal and substantial approaches. This is an analytical work that requests specialized professional skills. The SIEP Project Key Performance Indicators evidence this requirement. (See next slide SIEP Project Key Performance). There is no specialized body to monitor and evaluate the effectiveness of each of the phases of the Project Cycle. The M&E specialist/unit comes too late when in the implementation and evaluation phases possible flaws of the previous phases may make themselves felt.

SIEP Project MIS

SIEP Project MIS – Key Performance Indicators

Evidences Regarding individual data collection and on-site control M&E effectiveness is additionally smothered by the centralized pyramidal project managing authorities’ administrations. M&E has not been fully institutionalized. M&E complements project implementation rather than to be an autonomous process which bears on the whole project cycle. M&E system has not been comprised in an integrated Management Information System (MIS) so far. The SIEP Project’s MIS made a substantial progress with this regard where its M&E Module had been linked to the other modules. The PHARE program is about to produce an integrated MIS too. Thus, on line information on the project’s current status is available. But generally, integration of individual projects’ MIS to line-ministries’ one is a challenge far from having been satisfactorily tackled so far. Centralization of information flows allows for expedient data processing and minimizing data distortion due to vested interests and lack of professional skills. The relevant principle to be followed with this regard is: 1) elaboration of as simple as possible measurable indicators (mandays created, long term unemployed hired, wage earned etc.) to be covered with individual data by local bodies; and 2) centralization of data collection and processing. Involving private companies specialized in monitoring and control is strongly recommended at the initial stage of data collection.

Alternatives Radical Alternative. It takes stock from the fact that M&E has not been fully institutionalized. Overcoming deficiencies of the now existing M&E systems that have been evidenced by the above projects stand for an autonomous M&E bodies where Government, Non-Government and private sector are represented on equal footing. In this way final beneficiaries would be much more effectively enabled to have a say on project implementation. But the line-ministries should keep their lead since the projects in question service primarily Government priorities and budget commitments. The M&E bodies would be involved in the whole Project Cycle along with the official institutions. The M&E bodies would develop flat administrative structure which is conducive to more effective communication and individual data collection. M&E should integrate financial and substantial (technical) audit/monitoring. Possible risk and weakness is the administration of integrated MIS and coordination of data processing and analysis, in particular.

Alternatives Realistic Alternative. It takes stock from the centralized way EU and international development projects, mostly WB funded, have been planned and implemented so far. This pattern of project financing persists. The only way to avoid the subsequent flaws is outsourcing. Outsourcing M&E services with independent private bodies would mitigate the negative effects of centralization. Primarily this would provide M&E with own quasi autonomous administration which would be free enough to set up formal relationships with beneficiaries’ monitoring bodies.