Part I: Impact Evaluation of Rural Roads Dominique van de Walle DECRG, October 2006

General belief that roads are good for development & living standards With limited redistributive instruments, we look to sectors such as rural roads to achieve distributional objectives General belief that roads are good for development & living standards But, little is known about impacts

What Is Different About Roads? Impacts are indirect: roads are an intermediate service; impacts depend on interactions w/other investments, household & community factors Important to control for heterogeneity of other factors But, impacts can be economy wide: potential controls may have been determined by road investment Roads are not randomly placed

Therefore: Traditional methods of conducting evaluation may be contaminated by the project, directly or indirectly Potential for randomization is limited Need: baseline and panel appropriate controls for exogenous time varying factors i.e. double difference + controls for observed time varying factors

Getting at distributional impacts: On average benefits may be positive But, there may be both gainers & losers Short & longer term distributional impacts may be different So need: ability to differentiate between welfare groups sufficiently long post-project follow-up

Learning from the Ex-Ante Evaluation Think about evaluation from day 1 of project ‘Appraisal’ stage can be thought of as ‘ex-ante evaluation’ How are intervention areas picked? How are road links chosen? Helps to understand biases in ex-post evaluation

Part II: Application to Rural transport project (RTPI) in Vietnam Rural roads rehabilitation project in 18 provinces of Vietnam, 1997-2001 To link communities with markets and reduce poverty Rehabilitation of rural roads; no new roads are to be built. Selection criteria: costs less than $15,000 per km; population served is at least 300 per km.

Evaluation Questions What are the impacts of rural roads on living standards (broadly defined) and their distribution? What factors influence those outcomes? How much do benefits depend on other investments (e.g. human capital)? In what ways do first-round impacts differ from longer term impacts?

Survey of Rural Road Impacts in Vietnam (SIRRV) Pre-project baseline in 1997 Post-project follow-up rounds every two years: 1999, 2001 and 2003. 100 “treatment,” 100 “comparison” communes in 6 provinces; randomly chosen 15 households per commune: stratified sampling Panel of 200 communes & 3000 households. District and project data bases No welfare indicator; possibility of linking up with VLSS98

Risks timing: project delays commune splits weather: typhoons, flooding etc waning interest fungibility: project aid displacement

Evaluation methodology Diff-in-diff with propensity score matching Propensity-score matching Using baseline characteristics likely to affect selection into the project, and outcomes This deals with observable heterogeneity in initial conditions that can influence subsequent changes over time Diff-in-diff Difference in outcomes over time between matched project and non-project communes Purges additive time-invariant unobservables

Evaluation methodology Logit model of program participation on pooled project and non-project samples. Create predicted values from the logit regression – the propensity scores Check for common support Nonparametric kernel matching is used for one estimator An alternative estimator uses the propensity scores to weight the DD

Impacts on rehabilitated road kms? Impacts on rehabilitated road km were less than intended However, more roads were built in project areas. Thus, we find fungibility within the sector, but evidence of a flypaper effect of aid on roads Spending on rehab + building accords reasonably closely to total amount allocated by the project

Other findings so far: Impacts on market development, availability of goods, household involvement with markets But, very heterogeneous impacts across communes, depending on initial conditions.