1. State of the Field: The Need to Understand and Incorporate Variation in Impact in Seeking to Influence Outcomes for Women and Children Kate Schwartz & Yeshim Iqbal

2. Outline Impact Evaluations (scale/cost) What we Know (and what we don’t) Women’s Economic Empowerment Child Development How to Know More Overview of Additional Insights Year 2 Goals

3. Impact Evaluations  2,640 (1,721 RCTs) published impact evaluations conducted over the past 3 decades in 120+ low- and middle-income (LAMI) countries  863 (648) in Sub-Saharan Africa  655 (352) in Latin America and the Caribbean  534 (340) in South Asia & 444 (276) in East Asia and the Pacific  112 (91) in the Middle East and North Africa

4. Impact Evaluations EXPONENTIAL growth in impact evaluations in recent years….  52% (N=1372) of published impact evaluations (and RCTs, N=895) conducted in LAMI countries have been conducted since 2010  Especially true in Middle East and North Africa (64% since 2010)  410 (271 RCTs) in 2012 alone compared with 45 (28 RCTs) in 2002

5. Impact Evaluations EXPONENTIAL growth in impact evaluations in recent years….  The World Bank averaged 16 per year from 1999-2004 and 62 per year from 2005- 2010.

6. Impact Evaluations EXPONENTIAL growth in impact evaluations in recent years….

7. Impact Evaluations Cost of the average evaluation is around $500,000 USD, exclusive of all intervention costs Yet, this is only 1.4% of the total cost of the evaluated intervention. (IEG, 2012, Thomas, 2014)

8. So what do we know from all of these? “What I want to talk shit on is the paradigm of the Big Idea—that once we identify the correct one, we can simply unfurl it on the entire developing world like a picnic blanket...There are villages where deworming will be the most meaningful education project possible. There are others where free textbooks will. In other places, it will be new school buildings, more teachers, lower fees, better transport, tutors, uniforms… The point is, we don’t know what works, where, or why…” Michael Hobbes, Human Rights Consultant in Berlin

9. So what do we know from all of these?  79% of completed World Bank IEs & 35% of completed IFC IEs have estimated distribution of program impacts  27% evaluated the specific contribution of specific elements of treatment  21% and 46% looked at cost- benefit, rate of return, or cost effectiveness across programs  11% and 20% looked at medium/long-term impacts

10. So what do we know from all of these?

12. DE-WORMING INTERVENTION

13. So what do we know from all of these? DE-WORMING INTERVENTION Kenya, RCT, ~30,000 students (Baird et al., 2014; Miguel & Kremer, 2004)  Absence rates decline 25%  Health increases  Educational attainment and salaries increase (long-term outcomes) Cited 1,266 times Featured in/by the Atlantic, Huffington Post, NY Times, World Economic Forum Newsletter, the Clinton Global Initiative, and more

14. So what do we know from all of these? Kremer founded Deworm the World in 2007 Large scale replication (largely un-evaluated) YET, Taylor-Robinson et al. (2000, 2009, 2012) find very low to low evidence that deworming affects weight, height, cognitive tests, or school attendance in multiple systematic reviews.  2000: 30 RCTs in 17 countries  2009: 34 trials (10 new, 6 prior ones excluded due to updated stricter inclusion criteria)  2012: 41 trials in 23 countries

15. So what do we know from all of these? TEXTBOOKS

16. So what do we know from all of these? MICROFINANCE  Hyderabad, India: No effect on women’s economic empowerment  Andhra Pradesh, India: Positive effect  Bangladesh: Positive and negative effects  Qualitative reviews: Overwhelmingly positive “It remains unclear under what circumstances, and for whom, microfinance has been and could be of real, rather than imagined, benefit to poor people. Unsurprisingly we focus our policy recommendations on the need for more and better research… on whom, where, and when (e.g. under what circumstances), and the mechanisms which account for these effects…While there is currently enthusiasm for RCTs as the gold standard for assessing interventions, there are many who doubt the universal appropriateness of these designs. Indeed there may be something to be said for the idea that this current enthusiasm is built on similar foundations of sand to those on which we suggest the microfinance phenomenon has been based.”

17. How to Know More For Whom, Under What Conditions, How? Mixed Methods Approaches Including Community more in Process Standardized Measures across Evaluations Quasi-Experimental Designs Strategic, Coordinated Impact Evaluation Designs Translating as opposed to transferring findings

18. Additional Insights: 21 st Century Strategies to Promote Economic Empower ment and Child Developm ent

19. Systematic review of available data sets and available data YEAR TWO

20. Additional Insights: 21 st Century Strategies to Promote Economic Empower ment and Child Developm ent All of you!

21. Thank you!

22. Impact Evaluations  300 (146 RCTs) related to women’s economic empowerment*  76 (51) in Sub-Saharan Africa  100 (53) in Latin America and the Caribbean  85 (24) in South Asia & 29 (12) in East Asia the Pacific  5 (3) in the Middle East and North Africa * Tagged sub-group Gender AND at least one of the following sectors: Agricultural Reform, Agricultural Credit, Rural Livelihoods, Economic Policy, Girls’ Education, Vocational/Technical Education & Trade. Scholarships, Finance, Conditional Cash Transfers, Community Action Program, Community Driven Development, Social Funds, Private Sector Development, Social Protection

23. Impact Evaluations  794 (146 RCTs) related to child development*  188 (144) in Sub-Saharan Africa  290 (174) in Latin America and the Caribbean  134 (86) in South Asia & 140 (95) in East Asia the Pacific  27 (24) in the Middle East and North Africa * Tagged as at least one of the following sectors: Educational Inputs; Girls’ Education; Pre-Primary and Primary Education; Public/Private Sector Education; Secondary Education; Scholarships; Child Nutrition; Conditional Cash Transfers; Early Child Development Programs