1. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE What are the Effects of Land Use Restrictions on Local Communities? Evidence from an Impact Evaluation of Costa Rica’s Protected Areas Kwaw S. Andam International Food Policy Research Institute Perspectives on Impact Evaluation: Approaches to Assessing Development Effectiveness 31 March - 2 April 2009, Cairo, Egypt

2. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Acknowledgements  Co-authors: Paul Ferraro (Georgia State Univ.) and Margaret Holland (Univ. of Wisconsin-Madison)  Evaluation Office, Global Environment Facility (GEF)

3. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Research Question  How different would socioeconomic outcomes have been in the absence of protected areas?

4. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Policy Context  Protected Areas: Most widely used conservation tool Role in climate change policy? More planned for future

5. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Motivation  Strong debate: how do protected areas affect local people?  Most studies focus on environmental impacts only

6. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE The Evaluation Challenge  Protected Areas placed selectively  Selection bias  Lack of common support Millennium Ecosystem Assessment (2005): “Many protected areas were specifically chosen because they were not suitable for human use.” Empirical evidence: e.g. Costa Rica (Powell 2000; Sanchez-Azofeifa) ; Nepal (Hunter & Yonzon 1993) ; Australia (Pressey 1995) ; United States (Scott et al. 2001) ;

7. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Therefore, an evaluation must… 1)Objectively measure indicators of human welfare 2)Measure indicators before and after establishment of protected area 3)Measure indicators in both “treated” and “control” areas 4)Measure baseline characteristics that affect both location of protected areas and how indicators change over time

8. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Study site – Costa Rica

9. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Costa Rica’s Protected Areas

10. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Approach Estimate impact of protected areas established before 1980 on changes in census tract-level socioeconomic indicators between 1973 and 2000

11. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Key question How different would socioeconomic outcomes have been in the absence of protected areas?

12. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Data  Quantitative indicators of change in socioeconomic outcomes (infrastructure, assets, poverty indices)  Control variables including land use productivity, forest cover (1960), accessibility (distance to markets, ‘road-less volume’), baseline (1973) indicator  Measures near time of and after establishment of protected areas

13. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Matching Methods  Select control communities similar to communities near protected areas (treated) in terms of pre-protection characteristics  Key assumption: without protection (and conditional on control variables), control and treated communities would, on average, have similar socioeconomic outcomes in 2000  Treatment: at least 20% protected before 1980 >>> any remaining differences in outcomes are due to protection

14. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE VariableSampleMean Value Protected Segments Mean Value Control Segments* Diff Mean Value Avg. Raw eQQ ‡ Forest Area in 1960 (km 2 )Unmatched Matched 15.816 0.833 8.618 14.983 7.198 14.897 6.543 High Productivity Land ◘ (km 2 ) Unmatched Matched 0.885 0.540 1.012 0.345 -0.127 0.299 0.191 Medium Productivity Land (km 2 ) Unmatched Matched 1.922 0.402 1.671 1.520 0.251 1.467 0.263 Medium-Low Productivity Land (km 2 ) Unmatched Matched 4.375 0.650 3.584 3.725 0.791 3.594 0.719 Roadless Volume (km 3 )Unmatched Matched 319.040 21.322 172.510 297.718 146.530 291.960 133.220 Distance to City (km)Unmatched Matched 53.836 35.345 53.219 18.491 0.617 18.435 4.476 Poverty Index in 1973Unmatched Matched 11524 8574.200 12016 2949.800 -492.000 2950.100 964.230 Percent of houses in bad condition in 1973 Unmatched Matched 18.343 15.017 19.573 3.326 -1.230 3.504 1.313 Percent of houses in slums in 1973 Unmatched Matched 1.454 1.942 1.486 -0.488 -0.032 0.603 0.150 Percent of houses without telephones in 1973 Unmatched Matched 99.304 93.607 99.468 5.697 -0.164 5.810 0.144 Percent of houses without electricity in 1973 Unmatched Matched 4.365 3.015 3.832 1.350 0.533 1.256 0.544 Percent of houses without access to water supply in 1973 Unmatched Matched 39.462 16.321 40.458 23.141 -0.996 23.064 1.248 ◘ Low productivity land is the omitted category. * Weighted means for matched controls. ‡ Mean difference in the empirical quantile-quantile plot of treatment and control groups on the scale in which the covariate is measured.

15. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Results 1234 Outcome% Houses in bad condition Poverty Index % Houses in slums % Houses without telephones Matching Estimators (Effect of protection on change in outcome 1973-2000) Covariate matching – Mahalanobis -9.3*** (2.5) -987.6*** (248.9) -3.4*** (0.8) 1.2 (1.5) Covariate matching – Mahalanobis with calipers -3.0** (1.2) -500.2*** (134.6) -1.3*** (0.5) 1.3 (1.3) Replicating Conventional Methods (Effect of protection on outcome in 2000) Difference in means N treated (N available controls) 399 (16002) 399 (16002) 399 (16002) 399 (16002) *** significant at.01; ** p<0.05; * p<0.10

16. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Additional Results 123 OutcomeUnsatisfied basic needs (3 or 4) % Houses without electricity % Houses without water supply Matching Estimators (Effect of protection on outcome in 2000) Covariate matching – Mahalanobis -14.1*** (3.0) -20.7*** (4.6) -10.7** (4.4) Covariate matching – Mahalanobis with calipers -8.5*** (1.3) -9.4*** (1.5) 1.7 (2.0) Replicating Conventional Methods (Effect of protection on outcome in 2000) Difference in means8.7***16.8***31.1*** N treated (N available controls) 399 (16002) 399 (16002) 399 (16002) *** significant at.01; ** p<0.05; * p<0.10

17. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Caveats  Average effects only (adverse effects on subgroups may still be present)  Costa Rica may be very different from other nations  Indicators may not capture other aspects of poverty

18. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Ongoing work (1)  Census segments change over time from 1973 to 2000  Need to account for changes in unit of analysis over time

19. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE 1973: 4,694 segments

20. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE 2000: 17,264 segments

21. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Ongoing work (1)  Use simple areal weighting to reconcile 1973 segments with disaggregated segments in 2000, and analyze at level of 1973 segments  Findings confirm analysis with disaggregated segments

22. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Ongoing work (2)  Spillover effects: do effects of protection spillover onto nearby ‘unprotected’ communities?  Findings: None to small positive spillover effects

23. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Ongoing work (3)  Test sensitivity of results to hidden bias (unobservable characteristics that affect likelihood of protection and socioeconomic outcomes)  Test using Rosenbaum bounds tests  So far, mixed results

24. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Summary  No evidence that protected areas had negative effects  Protected areas may have had positive effects  But conventional methods (erroneously) imply the opposite

25. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Discussion  Why differences in estimates? Simply comparing post-protection outcomes does not account for pre-existing differences correlated with poverty indicators  How does protection have positive effects? Ecotourism Protected area infrastructure Initiatives to ease deforestation pressures Ecosystem services?

26. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Thank you! Kwaw S. Andam International Food Policy Research Institute Addis Ababa, Ethiopia Email: k.andam@cgiar.orgk.andam@cgiar.org Phone: ++251.617.2507 www.ifpri.org

27. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Appendix  More comparisons with conventional estimates

28. INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Replicating conventional methods 1234 Outcome% Houses in bad condition Poverty Index% Houses in slums % Houses without telephones Matching Estimators (Effect of protection on change in outcome 1973-2000) Covariate matching – Mahalanobis -9.3*** (2.5) -987.6*** (248.9) -3.4*** (0.8) 1.2 (1.5) Covariate matching – Mahalanobis with calipers -3.0** (1.2) -500.2*** (134.6) -1.3*** (0.5) 1.3 (1.3) Replicating Conventional Methods (Effect of protection on change in outcome 1973-2000) Difference in means 2.8*** 129.1***1.2**23.4*** OLS 2.4*** (0.8) 709.6*** (114.8) 0.6* (0.3) 11.3*** (1.5) N treated (N available controls) 399 (16002) 399 (16002) 399 (16002) 399 (16002) *** significant at.01; ** p<0.05; * p<0.10