Evaluation of Education Maintenance Allowance Pilots Sue Middleton - CRSP Carl Emmerson - IFS

The Policy Problem (1) Very large increases in participation in the later 80s and early 90s had levelled off by the mid-90s

The Policy Problem: Post-16 Participation and Socio-Economic Group Source: Youth Cohort Study, Statistical Bulletin 02/2000

The Policy Problem: Post-16 Participation and Gender Source: DfES Statistics

Reasons for Non-participation Little evidence about the reasons Money is one factor among many...not surprisingly, money looms large in the accounts given by disaffected young people of their lives. They report one of the key barriers to further and higher education to be (lack of) money.(Newburn, 1999)

The Policy Response Multiple policy responses Education Maintenance Allowance - to encourage participation, retention and achievement in post-16 education - focusing on young people from low income families

Education Maintenance Allowance Household income < £13k - weekly allowance up to £40 per week Household income £13k - £30k - weekly allowance tapers to minimum £5 per week Retention and achievement bonuses - available to ALL awarded EMA Receipt subject to compliance with Learning Agreement 4 Variants being piloted

EMA Variants Maximum Weekly Amount Retention and Achievement Bonuses Paid to: Variant 1£30£50Young person Variant 2£40£50Young person Variant 3£30£50Parent Variant 4£30£80 (retention) £140 (achievement) Young Person

EMA Evaluations EMA Main – 10 LEAs Leeds and London – 5 LEAs EMA Transport – 5 LEAs EMA Extensions – 4 LEAs

Evaluation in the LEAs ROUND TABLE DISCUSSIONS EMA Implementation groups DATA COLLECTION Labour market Education profile Take up of EMA Socio-demographics INTERVIEWS LEA administrators Careers Service representative TEC representative Employer organisations AREA VISITS

Qualitative Interviews with Young People/Parents Year 1 – Participants/Non-participants (young people and parents) Year 2 – Longitudinal interviews with young people and early leavers

WAVE 1WAVE 2WAVE 3WAVE 4 Face to face October 1999 Telephone October 2000 Telephone October 2001 Telephone October 2002 COHORT 1 COHORT 2 Quantitative Design Face to face October 2000 Telephone October 2001 Telephone October 2002 Telephone October 2003

The data Questionnaires have detailed information on: - all components of family income - household composition - GCSE results - mothers and fathers education, occupation and work history - early childhood circumstances - current activities of young people

Matching approach Involves taking all EMA eligible young people in the pilot areas and matching them with a weighted sum of young people who look like them in control areas Difference in full-time education outcomes in pilot and control areas in this matched sample is the estimate of EMA effect Crucial assumption is that everything is observed that determines education participation

How is this done? Dont match on all Xs, but can instead match on the propensity score (Rosenbaum and Rubin, 1983) Propensity score is simply the predicted probability of being in a pilot area given all the observables in the data Use kernel-based matching (Heckman, Ichimura & Todd, 1998) The matching is undertaken for each sub-group of interest

Family background - household composition, housing status, ethnicity, early childhood characteristics, older siblings education and parents age, education, work status and occupation Family income - current family income, whether on means-tested benefits Ability (GCSE results) School variables Indicators of ward level deprivation Variables are matched on:

Analytic Strategy for EMA Propensity Score Matching: Measures the Impact of EMA BUT Requires Large Sample Sizes Weighting Issues Limited Disaggregation Descriptive Analysis: Provides Contextual Detail Allows Disaggregation Overcomes Weighting Issues BUT Cannot Measure Impact

The Impact of EMA on Participation EMA has increased participation by 5.9 percentage points EMA had a larger effect on young men than young women Base: Eligible young people in Cohort 1 & 2

The Impact of EMA on Participation Draw is from both those who would have been in work and those who would have been NEET Base: Eligible young people in Cohort 1 & 2

EMA and Retention at Year 13 Slightly larger impact on participation in year 13 Suggests that retention has not fallen Base: Eligible young people in Cohort 1 & 2

Participation and Retention by Variant

Conclusions (1) EMA effect around 6 percentage points Plays an important role in reducing gender differences in post-16 participation Important to control for local area effects - matching on ward level data important

Conclusions (2) More effective paying EMA to young person rather than parent Bigger retention bonuses have significantly larger effect on retention than other variants Increase in participation drawn from both work and other groups

What We Learn, And When: From PSM SurveysInformation onBest Impact Measure Wave 1Participation2002 Report 2 Wave 2Participation and Retention in Year Report 3 Wave 3Retention in Year 14 Achievement Higher Education Entry 2004 Report 4

What We Learn And When: From Descriptives (1) SurveysInformation on Wave 1Year 11 decision making Awareness, Applications and Receipt Year 12 Courses Wave 2Retention bonuses Year 12 achievement Destinations

What We Learn And When: From Descriptives (2) SurveysInformation on Wave 3 (& 4)Achievement Bonuses Year 13 Achievement Higher Education: - Courses and Institutions - Financial Support Destinations over time

What We Could Learn SurveysInformation on All surveysDestinations over time including labour market and higher education entry (for sub-groups) Part-time work, hours and earnings Expenditure patterns and responsibility