1 Monitoring and Evaluation in ECA Region Land Thematic Group Retreat November 19-20, 2007

2 ECA Overview  ECA Region – 29 countries ranging from recent EU entrants to IDA countries.  Several post-conflict countries..  Post-communist transition to market economies.  Land records universally out-of-date and neglected.  Privatization, de-collectivization, restitution issues to be resolved.

3 ECA Overview Project Type Closed Implementatio n Preparation Land reform, agriculture, farm restructuring 531 Registration, Cadastre 5121 Land Use Management 11

4 ECA Overview cont.  Approximately 30 projects in some 20 countries.  Supporting governments to restructure state and collective farms, privatize land, create modern cadastre and registration systems.  More recent projects also trying to support development of better land policy, state land management and land use planning.  The overall objective is to better develop land markets to support investment and more economic development.

5 Monitoring vs. Evaluation  Monitoring –Throughout project implementation. –To improve performance, aid in decision making, improve accountability, and enhance learning. –All current projects are regularly monitored.  Evaluation –What did the project achieve? And what difference did it make? –ICRs are completed at the end of each project. –But very few (if any?) impact evaluation carried out in the ECA region on land projects.

6 Monitoring in ECA Land Projects  Early projects (farm restructuring and registration/cadastre) heavily focused on inputs and outputs: –Number of titles issued –Crop yields  Monitoring in ECA land administration projects still focused on outputs (number of registrations), but more data being collected on land markets (price/m 2 ; mortgage market) and customer satisfaction. Moving towards outcomes and impacts.  In many countries difficulties with collection of baseline data.

7 Monitoring Indicators  Difficulties of comparability and consistency of indicators across projects and countries.  Often indicators are overly ambitious; too many indicators or the data is not available.  Indicators at appraisal may no longer be relevant at completion.  Important that monitoring data is used by the project – PIU, implementing agencies – for decision making and understanding.  Not just for reporting to the Bank team.

8 Evaluation  Completed projects in countries across the ECA region from Slovenia to Tajikistan.  ICRs for 7 projects – 4 registration and cadastre projects; 3 land reform/farm restructuring projects. ProjectRatingIEGSustainableIEG Kazakhstan Registration Pilot (2000) SMSLU Russia LARIS (2003) SSLL Armenia Title Registration (2004) HSHSHLHL Slovenia Registration (2005) SSLL Romania General Cadastre (2006) SHSLL Azerbaijan Farm Privatization (2003) HSHSHLHL Tajikistan Farm Privatization (2005) SSLL

9 Lessons Learned  Institutional: –Need political will to address issues of land policy and institutional reform. –This kind of reform takes time and does not always fit the Bank’s schedule – but if we stick with it, we can achieve results (LARIS, Romania). –Single agencies work best, but are not always feasible; do we have a back-up plan? –Supporting the institutions to move from output based, command and control to service focused and regulatory.

10 Lessons Learned  Capacity Building: –Necessary to building human and institutional capacity to carry out and sustain the reforms in the land administration sector. –Also must build capacity to manage the project and significant large contracts (IT). –Not just training, but professionalization of staff.

11 Lessons Learned  Automation and Information Technology –IT can play a role in helping to improve efficiency of the land administration agencies, but it is not a cure-all. –Not just a HW/SW issue – includes business process reengineering, understanding different users and their requirements. –Both the Bank and the client continue to underestimate the complexity and management requirements of these large IT implementations. –Have had better success with incremental, home grown solutions (Moldova, Kyrgyzstan), than with ‘big bang’ approach (Romania, Bulgaria).

12 Lessons Learned  Technical –Use of private sector surveyors has been positive – more effective than public sector. –Competition/ICB brings prices down –Use of new technologies, GPS, OPM, CORS, means surveys can be done cheap and accurate; $10/parcel or less! –Encouraging use of sporadic registration in urban areas.

13 Building Land Markets  But do we really know the impact?  Anecdotal evidence: –In 2001, the National Bank of Romania recorded 1,948 mortgage loans; in 2005 they recorded 82,675 mortgage loans –In Slovenia real estate taxes increased by 15% between 1999 and –In Armenia the number of real estate transactions more than doubled between 2000 and 2003.

14 Poverty Alleviation  ECA projects have not promised too much in this area – but it is clear that we do not know the poverty impacts our land projects have.  Is this an area for future evaluation?

15 Transparency  Transparency is important for land administration systems – part of good governance.  But difficult to measure... How do we monitor increased/improved transparency?  Is transparency also a question for the monitoring and evaluations systems? Is the monitoring data shared publicly? Should it be and how?

16 Assessing Performance  Projects seek to improve delivery of services to clients.  Regular customer surveys allow us to measure performance of the LA agencies and change in customer attitudes over time.  Regular customer surveys performed in Serbia, Macedonia, Croatia, Kyrgyz Republic.  Currently data is used project by project – can we use across projects to talk about results?

17 Assessing Performance -Example  Continuing successful implementation of the Land Real Estate Registration Project in Kyrgyz Republic.  Further support to the implementing agency – Gosregister – from BNPP to build capacity within Gosregister to measure performance in terms of stakeholder feedback.  Specifically to develop capacity and tools for staff to measure client satisfaction and derive evidence-based recommendations; and conduct an initial survey on quality of services delivered and collect baseline data in order to measure improvements over time.  Further information:

18 Future Plans  Croatia will do a follow-up study/evaluation of staff who participated in the training program over the course of the project ( ).  Consultant working on a desk review of monitoring and evaluation of land projects in ECA. Report should be ready by early  Bring together client survey results?