1. Dr Stephen Haddelsey Learning Gain Project Manager
Dr Linda Speight
Learning Gain Data Analyst

2. The Lincoln-Huddersfield Learning Gain Project
Three-year, longitudinal, mixed methods study
Focus on the development of ‘employment competencies’
Commenced November 2015
Led by the University of Lincoln working in partnership with the University of Huddersfield

3. Methodology
Student Self-Assessment (SSA): in-house development (duration: 10 minutes)
Situational Judgement Test (SJT): A&DC psychometric test (duration: 30 minutes)
Additional sources:
Progression data
Attendance
Student engagement data: clubs, volunteering, job experience, democracy
Student interviews and focus groups

4. Status
272 Level 1 students (2015 intake) at Lincoln completed one, or both, exercises and gave consent (47% of the target group)
Of these, 198 students have completed a second administration (73% of the original cohort)
397 Level 1 students (2016) have also completed one or both exercises (60% of the target group)
A second administration is planned for September 2017

5. Communicating & Influencing
What was the…?
Percentage of 1st year students who scored ‘above average’ on the SJT
Weakest skill set identified by 1st year students in the SJT
Key thing that 3rd year students felt had improved at university?
Percentage of 1st year students’ comments which identified ‘academic activities’ as helping improve skills
Percentage of 1st year students involved in sports or societies
Percentage of 2nd year students involved in volunteering 8%
Communicating & Influencing
Confidence
38%
34% *
11% *
* percentages are based on % of learning gain cohort and may not represent the University of Lincoln as a whole

6. The Issue of Student Engagement
Poor engagement has necessitated extended administrations and introduction of incentives
Students are taking ~20 minutes to complete the SJT (33% less time than the providers suggest)
39% of students took <20 minutes to complete
Integrity of the SJT results are, therefore, questionable
These findings reflect local anecdotal concern that students do not properly engage with employment-related activities until their 3rd year of undergraduate study

7. Next Steps
Continue annual administration of Student Self-Assessment Survey
Develop Student Profiles (progression data, test results, student engagement data etc)
Collect and incorporate additional data, including:
Pre-enrolment survey (Get Set)
Qualitative interview data
Contribute to body of knowledge on learning gain

8. Get Set Survey, 2016
Designed to support TEF by establishing Level 1 students’ self-perception against a range of skills & competencies
Intended to inform personal tutoring conversations and to identify development needs at a corporate / school / subject level
1865 Level 1 students completed the survey in August / September 2016 (50% of intake)
But….
Reporting was limited to exported graphs from Survey Monkey, manually collated and ed to relevant staff
Difficult to obtain an overview of the data, compare between colleges, or for personal tutors to access individual reports unless initiated by the student
Results not used?

9. Get Set Survey, 2017
DVC decision to roll-out survey to all students (still on a voluntary basis):
Continues to support TEF
Provides additional data for inclusion in Learning Gain Project reporting
Survey Monkey replaced by Qualtrics:
More professional in appearance
Works across computer and mobile platforms
Improved reporting functionality
Easy to export data
Question set:
Now incorporates questions around student expectations, adding value and utilisation potential
Response options changed from a four point to a five point scale.
Careers questions changed to directly match those used by the Career Service Group
Status (at 11 September 2017):
1926 L1 completions
90 L2-4 completions
Administration scheduled to close 30 October 2017 .

10. Get Set 2017 Question Set (1st Yr)

11. Get Set Reporting What uses do you see for the data?
We see two potential benefits of the Get Set data:
Improving tutorial support
Improving teaching and learning within schools
What uses do you see for the data?
Who should receive reports?
What would you like to see?
When would you want to receive it?
Data available ASAP
Data at school level
Tabular data
Trackable data

12. Contact details:
Dr. Stephen Haddelsey Telephone:
Dr. Linda Speight
Telephone: