Sudan Community Development Fund: Preliminary Slice I Impact Evaluation Results and Needs for Future Evaluations Abdulgadir Turkawi, Krishna Pidatala, Tei Fujiwara, Ryan Sheely

 Community Development Fund (CDF) National program that uses CDD approach to rapidly provide basic social infrastructure and services to war-affected and underdeveloped areas of North Sudan – i.e. In 4 states out of 15 states  Two Phases  Slice-1 ( ) - $25 million  Slice-2 ( ) - $50 million CDF – Project Background

Project Background (2)  Slice I ( ) - US$ 25 million  4 states out of 15 states; 10 neediest localities within these 4 states; 20 communities per locality = 200 total communities  Slice II ( ) - US$ 50 million  4 states out of 15 states; 6 more communities added to the original 10 Slice-I localities; 19 new localities added in the 4 states in Slice-II to increase coverage within the states.  Total Slice-I communities = 260  Total Slice-II communities = 380

Implementation Overview  Baseline 1 – May 2007 (3 treatment communities per locality; 2 control communities per locality; 27 random Households in each community )  Follow-up Survey/Baseline Survey 2 – June 2008  Follow-up survey for Slice1 - panel survey for same households as in baseline 1 for Slice1 ;  Baseline survey for Slice2 - 4 treatment communities per each new locality; 2 control communities per each new locality; 24 random Households in each community  Going Forward – Final survey expected in 2011

Evaluation Strategy and Survey Design  Targeting – based upon poverty & population assessments; 20 lowest ranked communities in each locality were selected to receive the CDF program. 21 st and 22 nd lowest communities in each locality picked as the control group  Treatment & Control groups – selected communities with similar characteristics  Cluster Random Sampling – Households in Treatment and Control groups were selected randomly

Selection of Treatment Communities  Total of 20 Communities per Locality  Needed to Ensure that there was at least one treatment Community per Administrative Unit  Within each Administrative Unit, communities were ranked based on poverty, population, availability and condition of Basic Services and Population  Number of Communities Chosen for Treatment Per Administrative Unit - based on above ranking (Poverty, population, availability & condition of basic infrastructure & services)

Selection of Control Group  To construct a control group, all communities within each locality were ranked based on the number and condition of basic services and population  The 21 st and 22 nd communities on the list were selected as a control group  These were the communities that were not selected that were most similar to the selected communities

Survey Methodology  Community and Household Questionnaires  50 Communities Chosen from all 10 Localities – 5 from each locality  3 Treatment Communities Randomly Chosen From Each Locality  2 Control Communities From Each Locality  Selection of Households  27 Households Randomly Selected from Each of the 5 Communities  Sampling Frame-Household Lists where available, “Spin the Pen” method where not available

Evaluation Questions  The Slice 1 Baseline and Follow-Up Surveys were designed to assess the overall effectiveness of the project at meeting its objectives: Measurements :  Access to Education, Health, and Water?  Good Governance?  Participation and Social Capital?

Results of Slice I Impact Evaluation  Results Estimated Using Difference-in- Difference Approach  Education  Gains in enrollment, reduction in dropouts ▪ Decline in female dropouts  Increases in number of classrooms, toilets, benches, and teachers dormitories  Treatment communities 34% more satisfied with education after intervention, compared to control group

Results of Slice I Impact Evaluation  Health  Fewer statistically significant increases in health center functionality  Increase in frequency of health center visits  Satisfaction with health facilities significantly increased  Water  Fewer statistically significant increases in water quality  Increased consumption of water  Increased Number of Pump sets  Increased Satisfaction with Access to Water

Results of Slice I Impact Evaluation  Governance  Increase in Reported Rates of Leader Compliance with Community Needs and Leader Responsiveness  Decrease in Ease of Changing Leader  Participation and Social Capital  Increase in Community’s Ability to Solve Development Problems  No significant increase in Participation in Community Activities or Meetings

Evaluation Challenges & Lessons Learned  Sample attrition – possibility that some households could have moved by 2011  Data Management – difficulty in matching of some households from baseline & follow-up surveys  Gender sensitivity & participation – 1 st baseline survey did not have any female respondents. Addressed this shortcoming in 2 nd baseline survey for Slice-II.  Survey questionnaires too long – Need to condense follow-up questionnaires. Interview takes 1 hour 15 minutes.

Looking forward – Impact Evaluation  Final survey expected to be undertaken  Impact Evaluation – Need for continued support from DIME for facilitation & technical expertise to the project  Phase-II – Dependent upon the Referendum in January 2011 and the political landscape there after. We expect a more rigorous IE design for the next phase/project.  Evaluation design - possible sub treatment interventions  Survey design and management  Sampling

Questions & Feedback Needed: Slice II Follow-Up Survey  Budget constraints – For the final survey, should we reduce – (a) the number of households per community,or (b) the number of communities per locality ?

Preparing for Future Impact Evaluations  Build Local capacity – involve local counterparts in IE technical design & analysis (as far as possible)  Gender Sensitivity/Participation – IE expert on team to be a woman (based upon past experience)  Focus on Project/Program – Develop project/program questions to be answered by IE  National Statistics Bureau – Look at possibility to involve them in some way to build their capacity (most projects ignore them)  DIME Support– need continued support from them. DIME to provide technical expertise, oversee IE work, ensure quality of work, etc.

Questions For Future Evaluations  What is the effect of installation of solar electricity on health, education, and security outcomes?  What is effect of social accountability mechanisms on Infrastructure Functionality?  Due to implementation in progress, may not be able to evaluate until Phase II  Will look for opportunities for evaluations in Slice II

THANK YOU