WP Research and Evaluation Manager Using an outcomes based logic model to evaluate Access, Success and Progression initiatives Katherine Sela WP Research and Evaluation Manager

Introduction Benefits of using logic models in evaluation Developing outcomes Evaluating programmes Challenges faced and solutions #universityofsurrey

“A logic model is a graphic display or ‘map’ of the relationship between a programme’s resources, activities, and intended results, which also identifies the programme’s underlying theory and assumptions.” (Kaplan and Garrett, 2005) #universityofsurrey

Continual improvement Helps with planning Helps visualise the strategy Why use logic models Continual improvement Helps with planning Helps visualise the strategy Unpacks processes that lead to outcomes Integrated with formative evaluation Lends itself to qualitative methods #universityofsurrey

Outcomes based logic model Situation Situation Inputs & Activities Inputs & Activities Short term Short term Medium term Medium term Medium term Long term Behavioural Outcomes Impact Remit to ensure that under-represented students are supported to access, succeed in and progress beyond higher education Students from KS2 to 5 (including mature learners) Knowledge Outcomes Knowledge Outcomes Attitudinal Outcomes Increased aspiration and attainment of students from underrepresented groups Increased progression to HE by students from underrepresented groups Increased success and progression of WP students within and beyond higher education Parents and carers Increase engagement with outreach activities and educational opportunities Teachers and support staff Increase engagement with outreach activities and Higher Education WP UG students at the UoS Increased retention, success and progression indicators

Setting outcomes An outcome is a clear statement of the change in behaviour, attitude, knowledge or skills you are trying to achieve Behavioural Attitude Knowledge (long term) (medium term) (short term) Outcomes need to be: Realistic Meaningful Agreed on Evaluable #universityofsurrey

Outcomes – Pre-entry and Mature students Knowledge Outcomes Attitudinal Outcomes Knowledge of the different pathways to HE Understanding of the financial implications of going to HE Knowledge of academic and pastoral support available within HE Increased confidence in their ability to make informed and independent decisions Increased confidence to make the transition to HE Increased commitment to learning

Students, Year 10 Programme 11-week programme on Saturday mornings (Tasters; study skills; university exchanges) 3 sessions include parents and teachers Aims to increase GCSE attainment 80% agreed or strongly agreed that they better understood the different pathways to HE Students, Year 10 Programme ‘It has boosted his confidence and given him a great insight to what university can offer’ Parent, Year 10 Programme #universityofsurrey

Success and Progression initiatives Academic Skills Development (SPLASH) Sprint Development programme for female students Additional Learning Support Early Induction Programme for Autistic students Academic Skills Development (SPLASH) Maths and Statistics Hub Centre for Wellbeing Vet School Peer Support Scheme Global Engagement Summer School Travel Bursary Careers & Employability Employability Award Additional Learning Support Mental health workshops Faculty For Arts and Social Sciences Foundation Year

Outcomes – undergraduate students Knowledge Outcomes Attitudinal Outcomes Knowledge of personal and professional development skills and how to apply these (eg communication, team work) Knowledge of key academic skills and how to apply these (eg research, critical thinking) Knowledge of key support services and how to access them Increased confidence in applying professional development skills Confidence to engage in extra- curricular activities Increased confidence to seek support from support services Increased sense of belonging

Sprint evaluation ‘I feel more confident to apply for opportunities which come my way’ ‘I have grown in confidence during the course of the programme’ 81% of students felt confident in getting involved with extra-curricular activities 95% of students said they felt confident to network with others #universityofsurrey

Challenges and possible solutions Some solutions Access and Participation Advisory Group Consent to contact at Registration programmes targeted directly at students in most need WP success and progression dashboard Evaluation and Data Officer (Success and Progression) Challenges Institutional ‘buy in’ Identifying WP Students within the university Limited uptake and success of programmes Compliance with data protection legislation Staff resources

Email: k.sela@surrey.ac.uk Any Questions? Email: k.sela@surrey.ac.uk