Monitoring and evaluation of Roma projects and policies 30 November 2010, Brussels György Molnár KIÚTPROGRAMKIÚTPROGRAM Monitoring and evaluation of the Roma pilot project – self-employment and microcredit

2/11 Main characteristics of Kiútprogram 1. Kiútprogram is a pilot project of Pan-European Coordination of Roma integration Methods – Roma inclusion: self-employment and microcredit, DG REGIO. Kiútprograms mission is empower people living in deepest poverty – mainly of Roma origin – to improve their situation with dignity, by providing them with financial services, information and social assistance. One of our central goals is to demonstrate: under or non-employment of people living in extreme poverty is not a welfare optimizing choice, but the absence of possibilities.

3/11 Main characteristics of Kiútprogram 2. To achieve our goals Kiútprogram provides unsecured loans for 1 year relatively small amounts (max Euros) to people living in poverty (upper income limit: 60% of the median) to create income-generating ability through self-employment Group-based lending no joint financial liability peer pressure instead of collateral sequential lending encouraging saving behavior, group saving

4/11 Success criteria 100 clients in the first, further 300 clients in the second year 75% repayment ratio of the first, and 80% of the second loan 70% of good performers come back for a second loan Income of households participating in the program increases School (incl. kindergarten) attendance of children of participating households improves Participation of the clients in local public affairs increases Subjective health and well-being improves Share of victims of usury among our clients decreases Increasing tax and social contribution payments Long-term goal: tax and social contribution surplus + saved social costs > direct costs of the program

KIÚTPROGRAM PLC RAIFFEISEN BANK EXPERTS, SHAREHOLDERS Government POLGAR Foundation Loans and savings Loan guarantee IT system Advocacy Feedback Networking Field work Accepting business plans Client and program administration Loan agency Monitoring & evaluation Quality control EU connections Main owner Civil connections Analysing social impact Financing welfare bridge and other related costs Regulation, laws World Bank/UNDP Impact evaluation Monitoring support Advocacy EU DG Region Roma integration pilot project Finance (1.43 mill. Euros) Control Institutional structure

6/11 Main steps of the lending process 1. 1.Settlement selection and analysis 2.Finding contact points, informing local leaders 3.Announcing the program 4.Forums, personal meetings with interested people 5.Baseline intake questionnaire with potential clients – prescreening social circumstances, indebtedness 6.First group meetings, discussing initial business ideas 7.Business questionnaire on formal or informal business activities in the recent past 8.Developing business plans 9.Debt management, if it is necessary

7/11 Main steps of the lending process Vocational training if it is necessary 11.Business score cards 12.Accepting business plans, finalizing the members of the groups 13.Group exam 14.Administration at the local municipality and the employment office 15.Disbursement of the loan 16.Weekly group meetings, consultations, payment 17.Control questionnaire at the end of the 1 year-long loan cycle in the case of clients contracted in the second year this would be later than the pilots final deadline.

8/11 The baseline questionnaires Main components Demographic profile of the household members Detailed income survey Credit and debts, including public utility debts Living standard assessment (housing, assets, etc.) Subjective wellbeing Ethnic identity, personal characteristics Former business experiences Double aim Baseline for impact evaluation by WB/UNDP Helps prescreening the potential clients and designing the activity of the fieldworkers (upper income limit, indebtedness, education, business plan, etc.) Harmonized with the UNDP questionnaire

9/11 Monitoring Operational database with the help of the WB –allows of scanning the activity of the fieldworkers –time consumption –costs –status of the potential clients –group activities MIFOS database made by the bank –financial activities (loan disbursement and payment) –to analyze business efficiency: outreach, portfolio, arrears, performance indicators Connection between the two databases, mutual openness M&E job financed by WB/UNDP –continuous feedback to Kiútprogram helping project implementation –coordinates the surveys, information for project evaluation

10/11 Conclusions Evaluation is not only a tool that allows to draw wider lessons, but it can directly improve the quality of project operations. To achieve this it is important to understand: we are sailing in the same boat. Necessary conditions to the successful cooperation: –the evaluators understand the project, the local circumstances and the motivation of the participants, –the project managers understand that without evaluation does not exist successful projects. Common work facilitates cooperation. Just coming and giving advices is not a good method. Conclusion for the granter. Project implementation and evaluation should be harmonized in time. In our case the extension both the pilot and the evaluation period would increase the efficiency of the invested sources: in the present setup the control questionnaire can be asked only from 100 clients that does not allows for real evaluation.

11/11 György Molnár Kiútprogram, Member of the Board