Dr Wai Yi Feng University of Cambridge, Faculty of Education Royal Society Ogden Education Research Fellow Mark Williams Engineering Development Trust (EDT) Education Director & Industrial Cadets Quality Assurance Manager Measuring Impact of Employer Engagement Programmes in STEM London Conference on Employer Engagement in Education and Training July 2016

STEM Enrichment “Activities not prescribed by the school curriculum, which aim to enhance students’ experience of STEM subjects” (e.g. Correll, 1978; Clendening & Davies, 1980) Programmes, interventions, activities in and out of school: Lessons incorporating “rich” tasks designed to support exploration and problem solving School clubs Competitions Masterclasses Summer schools / residential programmes Outreach / WP activities (inc. working with scientists) Extended investigations / projects Mentoring programmes Research / work placements Designed & run by teachers in lessons / sponsored & delivered by external agencies and institutions

EDT has engaged > 250,000 students Track record - EDT a charity established in 1984 Scale/reach - EDT is the leading UK-wide provider of work- related learning programmes for 11–21 year-olds and annually involves over 30,000 students Impact - EDT inspires and motivates young people towards science, technology, engineering and maths (STEM) careers through real life industry experiences Continuum of schemes - from half a day through to a full twelve months: > 500 companies > 1400 schools > 45 universities

Placements Tasters Projects EDT TASTERS raise awareness of careers, inform choice and stimulate young people to consider a future in science, technology, engineering and maths EDT PROJECTS develop skills, shift perceptions, inform career choice and raise awareness of science, technology, engineering and maths in industry EDT PLACEMENTS develop skills, inform career choice, raise awareness and allow young people to gain workplace experience in science, technology, engineering and maths

EDT has a range of programmes to facilitate engagement Tasters Projects Placements First Edition STEM Family Challenge Go4SET Routes into STEM Inspire (girls) Engineering Education Scheme Headstart Year in Industry Industrial Cadets is an industry-led accreditation providing a skills based school engagement structure led by industry and created for industry

Types of Impact Support for learning in school Enhanced understanding of STEM topics/disciplines, linked to more positive views/attitudes Development of skills and learning processes Personal and social development Insights into STEM-related study/careers, leading to increased likelihood of participation Typology identified in WYF’s ESRC- & RS-funded research: Stage 1: literature review – document analysis – observation – interviews; Stage 2: development of common profile instrument – measurement of impact

Support for Learning in School “Some of the things I have come across [in Outreach] I later met in lessons. So I understand it better when Miss introduced it in class. And because it was different – how it was taught to us [in Outreach] – I see a different side of it.” (Student on NRICH Outreach) Helped me better understand how things I learn in lessons can be applied in the real world Gave me a better appreciation of how the things I learn in school will be of use to me in future Taught me new skills or techniques which I can apply in my school work Answered questions I had which my teachers were not able to / did not have time to explain Motivated me to persevere more with school work

Enhanced Understanding of STEM – More Positive Views/Attitudes “I have not come across this whole area of science – didn’t know it makes such an important difference to us – to people. […] I was interested in science before, but this has definitely made me more interested, I’d say.” (Students on Nuffield Research Placement) Gave me insights into aspects of [STEM] I was not aware of Gave me experience into [STEM] I would not otherwise have come across Gave me insights into the links between different [STEM] areas Helped me better understand how [STEM] is done Made me like / enjoy [STEM] more Gave me better appreciation of the contribution of [STEM] to society

Development of Skills & Learning Processes “I have better problem-solving skills and team-working skills now. When I see a problem I’ve not seen before, I have a go at it and […] try to break [it down] – step by step. And I know how to manage my time and work better as a team with other people – so each team member contribute their strength.” (Student on CREST Award) Improved my communication skills Taught me how to learn / work more effectively Improved my practical skills in [STEM] Improved my project planning & management skills Improved my report-writing skills Improved my skills in presenting my work to others

Personal & Social Development “I’ve made a lot of friends across year [groups]. I was quite shy when I joined, but I’ve grown as a person. I am more confident now – even giving talks to people, which I wouldn’t have thought I would do. […] The places you see are amazing. You see the world – the poverty compared with what we’ve got here.” (Student on Royal Society Partnership Grant - Conservation Project) Made me a happier / more fulfilled person Brought me together with like-minded people my age Introduced me to inspirational mentors, supervisors or role models Made me more mature as a person Made me more motivated to do well Made me feel a part of the [STEM] community

Insights into STEM Study / Careers – Increased Likelihood of Participation “The mentors told us a lot about their work and how what we do relates to it and builds up – like to the context of it. I didn’t think what we do in school could relate to [work that’s done in industry]. And then, there’s so much more – like a whole new level – the stuff they have and use in the company. It made me want to have a go – maybe study it further, get to know a bit more.” (Student on Go4SET) Gave me better understanding of expectations in work / employment Helped me to think about what I might do after I leave school Gave me insights into what it is like to study at university Gave me experience of working in a university / company Introduced me to people who gave me good careers / study advice Made me more likely to choose to study [STEM] related subjects in future

Questionnaire (Profiling Tool)

Scoring (EES Student Data, n=155)

Example Profiles

EES: Students’ Experience vs. EDT Expectations Types of Impact Met expectations (90%–100%): Support for Learning in School (91%) Better Understanding of STEM (93%) Personal and Social Development (100%) Working towards expectations (75%–90%): Development of Skills (85%) Insights into STEM Related Study / Careers (81%) Similar profiles EDT staff have good, shared understanding of EES performance, strengths, weaknesses EES performs well relative to EDT expectations Indicators Exceeded expectation (>100%): Made me like or enjoy STEM more (116%) Increased my self-confidence or resilience (102%) Met expectation (90%–100%): Deepened my understanding of STEM (96%) Gave me experience in STEM I would not otherwise have come across (91%) Working towards expectation (75%–90%): Taught me new skills or techniques which I can apply in my school work (87%) Falling short of expectation (<75%): Improved my organisational skills (73%)

Students’ Experience: EES vs. Go4SET Go4SET is intended to be a junior version of EES, targeted at a younger age group Similar profiles provide first evidence that vindicates design decisions Examination of differences enables better understanding of both schemes

Implications for EDT Greater and deeper understanding of the Types of Impact Impact assessment for existing programmes – programme modification EDT Quality Assurance Influence and guide new programme development Influence and guide building of EDT programme portfolio

Other Questions Differences: different models of operation gender, background or prior experience of students Tracking: changes in outcome reported by a cohort over time programme performance against programme changes over time Benchmarking / Target setting “Value for money” / “Best buy” ? Create programmes that match needs using “basic building blocks” Curate portfolio of complementary programmes within / across institutions, catering for different age groups and levels of investment (e.g. budget, time)

Thank you For further information on EDT: Website For further information on Industrial Cadets: Website To contact presenters: Dr Wai Yi Feng - University of Cambridge Mark Williams - EDT