Dr Stephen Haddelsey Learning Gain Project Manager Dr Linda Speight Learning Gain Data Analyst

The Lincoln-Huddersfield Learning Gain Project Three-year, longitudinal, mixed methods study Focus on the development of ‘employment competencies’ Commenced November 2015 Led by the University of Lincoln working in partnership with the University of Huddersfield

Methodology Student Self-Assessment (SSA): in-house development (duration: 10 minutes) Situational Judgement Test (SJT): A&DC psychometric test (duration: 30 minutes) Additional sources: Progression data Attendance Student engagement data: clubs, volunteering, job experience, democracy Student interviews and focus groups

Status 272 Level 1 students (2015 intake) at Lincoln completed one, or both, exercises and gave consent (47% of the target group) Of these, 198 students have completed a second administration (73% of the original cohort) 397 Level 1 students (2016) have also completed one or both exercises (60% of the target group) A second administration is planned for September 2017

Communicating & Influencing What was the…? Percentage of 1st year students who scored ‘above average’ on the SJT Weakest skill set identified by 1st year students in the SJT Key thing that 3rd year students felt had improved at university? Percentage of 1st year students’ comments which identified ‘academic activities’ as helping improve skills Percentage of 1st year students involved in sports or societies Percentage of 2nd year students involved in volunteering 8% Communicating & Influencing Confidence 38% 34% * 11% * * percentages are based on % of learning gain cohort and may not represent the University of Lincoln as a whole

The Issue of Student Engagement Poor engagement has necessitated extended administrations and introduction of incentives Students are taking ~20 minutes to complete the SJT (33% less time than the providers suggest) 39% of students took <20 minutes to complete Integrity of the SJT results are, therefore, questionable These findings reflect local anecdotal concern that students do not properly engage with employment-related activities until their 3rd year of undergraduate study

Next Steps Continue annual administration of Student Self-Assessment Survey Develop Student Profiles (progression data, test results, student engagement data etc) Collect and incorporate additional data, including: Pre-enrolment survey (Get Set) Qualitative interview data Contribute to body of knowledge on learning gain

Get Set Survey, 2016 Designed to support TEF by establishing Level 1 students’ self-perception against a range of skills & competencies Intended to inform personal tutoring conversations and to identify development needs at a corporate / school / subject level 1865 Level 1 students completed the survey in August / September 2016 (50% of intake) But…. Reporting was limited to exported graphs from Survey Monkey, manually collated and emailed to relevant staff Difficult to obtain an overview of the data, compare between colleges, or for personal tutors to access individual reports unless initiated by the student Results not used?

Get Set Survey, 2017 DVC decision to roll-out survey to all students (still on a voluntary basis): Continues to support TEF Provides additional data for inclusion in Learning Gain Project reporting Survey Monkey replaced by Qualtrics: More professional in appearance Works across computer and mobile platforms Improved reporting functionality Easy to export data Question set: Now incorporates questions around student expectations, adding value and utilisation potential Response options changed from a four point to a five point scale. Careers questions changed to directly match those used by the Career Service Group Status (at 11 September 2017): 1926 L1 completions 90 L2-4 completions Administration scheduled to close 30 October 2017   .

Get Set 2017 Question Set (1st Yr)

Get Set Reporting What uses do you see for the data? We see two potential benefits of the Get Set data: Improving tutorial support Improving teaching and learning within schools   What uses do you see for the data? Who should receive reports? What would you like to see? When would you want to receive it? Data available ASAP Data at school level Tabular data Trackable data  

Contact details: Dr. Stephen Haddelsey Telephone: 01522 88 6034 Email: shaddelsey@lincoln.ac.uk Dr. Linda Speight Telephone: 01522 83 7077 Email: lspeight@lincoln.ac.uk