School of Education Investigating influences on post-16 progression in science in England using quantitative national data Matt Homer UK Science Education Research Conference – Researchers sharing with researchers 2-4 July 2012 National Science Learning Centre, University of York

School of Education Some background issues… In the UK we have a lot of national assessment data. How can and should it be used? –Researchers, policy makers, schools, others? What are the limitations of findings from such data? The context: what does post-16 science participation look like – what can national data tell us about key influences on this? –How might curriculum reform change patterns of participation? 2

School of Education Overview of talk Policy and project (EISER) background National data sources – the NPD Known influences on post-16 science participation Appropriate statistical methods Results: Descriptive analysis – gender/SES/14-16 pathway Results: Modelling participation Conclusion/Discussion – implications for policy/further research? 3 3

School of Education Policy reform A new science curriculum for year olds introduced in England in 2006 Flexibility: a greater variety of ‘routes’ through (KS4) science. A focus on teaching about the nature of science and socio- scientific issues – How science works Enhanced presence of vocational science courses (‘applied sciences’). Also, ‘Entitlement’ to Triple award (separate sciences at KS4) from 2008 – big increases in students 4

School of Education of-science-education-reform-eiser Our project: Enactment and Impact of Science Education Reform (EISER) Mixed methods, : jointly funded by the Gatsby Charitable Foundation and the Economic and Social Research Council This study examines school responses to this major curriculum reform. A particular focus is teacher enactment of the science curriculum in the classroom. The study is also investigating the initial impact of these reforms on student achievement, attitudes towards science education and participation in post-compulsory science courses. Document analysis, interviews with teachers and students, National data analysis Jim Ryder, Indira Banner, Matt Homer, Jim Donnelly… 5

School of Education ImmediateLonger Term Increase student interest in their science education Improve student attainment as measured through external examinations Support students in engaging effectively with science-related issues as citizens Increase post-compulsory participation in science education Ensure adequate supply of scientists/engineers Increase the employability of students Improve social mobility and inclusion Multiple aims of reform Ryder and Banner (2011) 6

School of Education Data source: the National pupil database (NPD) What is in it? All pupils in state-funded schools in England. Pupil level assessment data (primary to post-16) Pupil level personal data (e.g. gender, ethnicity, SES...). School level data (phase, selection policy, governance,…) Files linked by a unique pupil ID number. Large files: e.g. 600, 000 per cohort so KS4 results file – 6 million records for a particular year – 1 per qualification (GCSE…), i.e. multiple records per pupil. A powerful resource – the national picture and how it is changing over time. 7

School of Education Use of NPD datasets in EISER What are the patterns of participation and attainment across KS4 and KS5 science courses? How is this changing over time? What are the influences on participation and attainment? Five successive KS4 cohorts Two pre-2006 reform: 04-06, Three post-reform: 06-08, 07-09,08-10 Initial focus on KS4 – longitudinal change Have recently started focussing on post-16… 8

School of Education Our post-16 analysis is limited for two main reasons: NPD AS data is problematic (~20% missing due to cashing/not cashing in issues) – fixed(?) in newest data As of Jan 2011, we had only one post-reform full A-level cohort of data – hence it is probably too early to see longitudinal ‘impacts’ of the reforms on post-16 participation and attainment. Have investigated first post-reform cohort...(KS4: 06-08, A-Level 2010) Work in progress… Problems with post-16 data 9

School of Education Post-16 science – influences on participation These are some known factors in uptake: attainment at pathway – Triple award vs. Dual award vs ‘Other’ Gender – e.g. biology vs. physics Socio-economic status – lower groups less likely... (Plus teachers, student identity, school policy…) But… What are the relative ‘sizes’ of these effects? How do they vary across the sciences (biology, chemistry, physics) and against other subjects? To what extent are supposed ‘science’ problems with participation specific to science? 10

School of Education Methods: progression to eight courses compared Name of qualificationType of qualificationMore details of qualification types Science courses A-levels (formally Advanced Level General Certificate of Education) are usually studied over a two-year period, and entrance to university is normally predicated on sufficiently high achievement in these qualifications. Applied A-levels are A-levels with a vocational emphasis. BTEC National Awards are equivalent to A-levels in terms of academic value, but have a vocational focus. BiologyA-level ChemistryA-level PhysicsA-level Single Award Applied Science Applied A-level – vocationally orientated BTEC National Award Applied Science A vocationally orientated qualification Comparator courses MathematicsA-level PsychologyA-level HistoryA-level 11

School of Education Methods – two approaches 1. Descriptive statistics: comparing participation rates by Gender, SES and pathway separately Also, controlling for prior attainment at 16 in science 2.Multi-level modelling: predicting participation for each course based on gender, SES and pathway plus: attainment in science and maths at 16, socio-economic status (2 measures - free school meal, FSM; and Income Depravation affecting children index, IDACI) whether school also teaches to 18 This modelling gives measures of the INDEPENDENT effect of each variable, and also gives an indication of how important schools are in determining participation. 12

School of Education Descriptive results: overall participation post Number and percentage of full cohort at 16 taking each qualification Main messages Participation rates are generally quite low (a few per cent) The vocational routes are small (A-levels etc in 2010)

School of Education Red: all students doing the course Blue: all students doing the course who achieved a (mean) grade ‘B’ in science at 16 Main messages Wide variation in participation rates across both sciences and non-sciences – e.g. biology/physics, maths/psychology The role of KS4 attainment – it certainly does not account for the differences –within courses red bars similar to blue in length Descriptive results: participation by gender Percentage of female students within courses 14

School of Education Main messages Red: wide variation in participation rates across both sciences and non-sciences – but generally low – vocational courses the exception Blue: Controlling for prior attainment partly ameliorates the under-representation. But not entirely. So FSM students are generally less likely to participate (e.g. physics) but this is not a ‘science’ problem per se (e.g. history). Descriptive results: participation by FSM Percentage of FSM-eligible students within courses 15

School of Education Main messages Red: As we would expect, TA students are heavily represented in ‘traditional’, but not in vocational, sciences (esp. physics) Blue: Controlling for prior attainment makes little difference. Descriptive results: participation by pathway Percentage of TA students within courses 16

School of Education Multi-level modelling results Coefficients are odds ratio of participating compared to not An odds ratio of 1 means no effect on participation, >1 implies more likely to participate… All significant at 5% level except for shaded cells Post-16 course Odds ratios estimates for participation Percentage of additional variation at school level Mean prior attainment in science at 16 Prior attainment in maths at 16 Gender: Male Measures of socio-economic status science pathway school teaches up to age 18 FSMIDACITAOther Biology Chemistry Physics Single Award Applied Science BTEC National Award Applied Science Mathematics Psychology History Odds ratio estimates are significantly different from 1 at 5% level except for those shaded. 17

School of Education Implications for policy? Odds ratio estimates are significantly different from 1 at 5% level except for those shaded. Gender: Girls are ‘missing’ from physics but so too are boys (a little) from biology – does it matter? TA tends to encourage participation in science but is this weakening over time? More to do longitudinally Students from lower SES backgrounds are not discriminated against in terms of progression to the sciences (once prior attainment is accounted for). –Other subjects are worse e.g. psychology, history Vocational sciences are of a different character – and schools play a much bigger role in promoting progression More to do…what influences attainment? 18

School of Education Finally… The NPD is undoubtedly very useful in terms of assessing the macro picture following reform, and tracking longitudinally. Also need a mixed methods approach to find out what (and the why) is going on in schools, attitudes science etc. More changes working their way through the system –EBacc, National curriculum review(?), changes to accountability measures (league tables, what counts and how much), O-levels (!) 19

School of Education References Banner, I; Donnelly, J; Homer, M; Ryder, J (2010) The impact of recent reforms in the key stage 4 science curriculum In: School Science Review 92 (339) pp. 101 – 109 Homer, M., Ryder, Jim & Donnelly, J., Sources of differential participation rates in school science: the impact of curriculum reform. British Educational Research Journal, pp Homer, M., Ryder, Jim & Donnelly, J., The use of national data sets to baseline science education reform: exploring value-added approaches. International Journal of Research & Method in Education, 34, pp Ryder, J; Banner, I. (2011) Multiple aims in the development of a major reform of the national curriculum for science in England In: International Journal of Science Education 33 (5) pp. 709 – 725 Thank you – questions? Matt Homer: 20