EPAG Study Team (World Bank / MoGD/ Subah-Belleh Associates) Making Cents / Youth Economic Opportunities Conference 12 September 2012 EPAG Impact Evaluation: Preliminary Midline Results

Countries: Liberia, Nepal, South Sudan, Afghanistan, Rwanda, Jordan, Laos, Haiti The Adolescent Girls Initiative Liberia Rwanda South Sudan Afghanistan Nepal Haiti Jordan Laos

Objective: Vocational and entrepreneurship training for young women Design Features: – Private sector/NGO training providers competitively selected to provide market-relevant skills – Vocational Skills for wage or self employment – Life skills training to address girls’ vulnerabilities – Financial literacy and business development – Stipend (Liberia, Rwanda, Afghanistan) – Job placement through performance-based contracts The Adolescent Girls Initiative

Cross-country learning – Core evaluation team works across countries – Reports shared across countries – International workshops at different stages of project to share lessons learned Expansion: Potential to influence AGI programs in new countries Sustainability: By comparing outcomes, we can learn what methods work best.  These lessons can inform design of future programs and the scale-up of these pilot projects Global Perspective of the AGI

Implemented by Ministry of Gender and Development Design Features: – 4 service providers competitively selected (plus 4 more sub-contracted) – Training delivered in 2 rounds in 9 communities – In Round 1: Job Skills (35%) and Business Development Services (65%) – Wide variety of job skills areas: painting, hotel/ restaurant work, driving, etc. – Coverage: 2500 girls in Greater Monrovia and Kakata (1191 in Round 1; 1300 in Round 2) – Six month follow-up period for job placement AGI in Liberia (EPAG)

Like other evaluations, IE focuses on outcomes – What is the effect of a specific program on specific outcomes? But unlike other evaluations: 1.IE starts before the project begins 2.IE compares the beneficiary group with a similar group of individuals who do NOT receive the project This method allows us to attribute causality: – How much better off are beneficiaries because of the program? – How can we know that the outcomes we see are due to the program, rather than other factors? Impact Evaluation

Objective: To measure the impact of the program on the well-being of participants and their families Methodology: Randomized selection to treatment and comparison groups Data collection using a series of household and individual surveys: – Baseline: 2010 – Midline: 2011 – Endline: 2012 (in the field now) Research Design

What we will learn: Does the program improve the economic well-being of young women who participate? What is the impact on employment, earnings, investment, savings, borrowing, and lending? What is the impact of the program on a wide range of socioeconomic behaviors and outcomes, such as reproductive health, time management, experience of gender-based and other violence, and attitudes toward risk? Does the program promote the empowerment of participants, as measured by proxies such as decision-making, aspirations for the future, and control over household resources? How do the program impacts vary according to the demographic and personal characteristics of the participants? Research Questions

Research Design

Sample size 2106 Originally recruited 116 Never started training 1273 Assigned to round Control group 39 Re-assigned to round Assigned to round Started training 1191 Entered round 1 25 Very pregnant (assigned to round 2) 34 Started training 1131 Completed round 1

Response Rates Table 1. Response rates for baseline and midline surveys Midline Not interviewedInterviewedTotal Baseline Not interviewed Interviewed Total

Benchmark the socioeconomic status and demographic characteristics of the young women who would be participating in the EPAG program to provide a basis for comparison in the future Validate whether the treatment and control groups are statistically viable comparison groups for the impact evaluation Should not be used to make generalizations about adolescent girls and young women in Liberia overall – EPAG girls are more educated than the “average” girl – EPAG girls live in more urban areas than the “average” girl Baseline Survey

Results: Employment Notes: The treatment group in the graph includes only those who actually started the training. The overall midline employment rate for the treatment group is 67%. This includes the Job Skills (52%) and BDS (76%) tracks.

Results: Earnings Table 2. Impact on Weekly Earnings (LD) Indicator Treatment Effect T- statistic Number of Observations (1)Entire sample ***3608 (2)Those who took part in program1, ***3467 (3) Those working (4) Those who took part in program and are working *** significant at 1% Note: An increase of 1,109 LD per week is roughly equal to 58 USD per month. The average earnings at baseline was 43 USD per month.

Results: Savings Table 3. Impact on Savings Treatment EffectT- Statistic Number of Observations (1)Any Savings? (Yes/No) *** 3591 (2)Amount saved at home (USD) (3)Amount saved at bank (USD) *** 3590 (4)Amount saved at credit group( USD) * 3608 (5)Amount saved at susu( USD) ** 3607 (6)Amount saved at Nigerian susu (USD) *** 3608 (7)Amount saved total (USD) *** 3563 * significant at 10%; ** significant at 5%; *** significant at 1% Note: EPAG participants were given small stipends and a $25 bonus for completing the course.

Results: Self-Confidence Table 4. CHANGES IN ATTITUDES OVER THE PAST YEAR (only asked at midline) Treatment effectT- statistic Number of observations (1) I feel more able to work well with people now than a year ago ***1666 (2) I feel more comfortable with who I am now than a year ago ***1670 (3) I feel more in control of my life now than a year ago ***1670 (4) I feel more able to call upon my friends for support than I was a year ago ***1666 (5) I am more able to help my friends now than I was a year ago ***1668 (6) I am more comfortable in situations now with people I do not know than I was a year ago ***1665 (7) I am more outgoing now than I was a year ago ***1650 Note: A higher number indicates more likely to agree or strongly agree with the statement. * significant at 10%; ** significant at 5%; *** significant at 1%

Summary What have we learned? Large increases in employment (55% increase) and earnings (115% increase) Positive impacts on savings and self- confidence Stronger effects for Business Skills trainees than for Job Skills trainees – But the business skills income is total enterprise revenue, not earnings or profits.

Summary Next Steps: Examine impact on households Investigate heterogeneous impacts: by community, type of training, age, etc. (After completion of endline survey) Look at longer-term outcomes: do these positive effects persist, grow, or weaken over time? Understand profits and incomes of businesses.

Thank you