Retention Convention Leeds 3-4 March 2010 Identifying students ‘at risk’ of early withdrawal: issues and dilemmas Dr Elena Bedisti, Dr Kirsten Hall, Dr Maura O’Regan, Dr Sue Robbins

National Audit Office Report (2007): Staying the Course. The retention of students in Higher Education:  Strong performance of HEIs in retaining their students  Wide range of advice on good practice in supporting and retaining students.  Little evaluation of the impact and transferability of this practice Following the NAO report, HEFCE and the Paul Hamlyn Foundation made available a total of £1 million from to support projects that identify, evaluate and disseminate institutional analysis and good practice relating to student retention. The ‘What Works? Student Success and Retention Programme’ aims to analyse and share good practice about the most effective strategies in order to ensure high continuation and completion rates within HE. Seven universities were selected and have been awarded grants to run the projects. All but one are collaborative projects. In total, 21 institutions are directly involved in the grants programme.

‘What works? Student Retention and Success Programme’ – 2 key areas of work  providing academic support to students  promoting academic and social integration into the institution to promote a sense of belonging Reading/Oxford Brookes Collaborative project: Investigating the impacts on student retention of different approaches to Supporting students through study advice and personal development

Identifying ‘at risk’ students HEFCE Criteria Robbins method for science students in the School of Life Sciences Both methods work within their own context – significant correlations between ‘at risk’ students and withdrawal rates Implications: 1. A more interventionist Approach to providing support (PASS) Implications : 2. A more active engagement of Personal Tutors and other staff in signposting support available Students either self refer or are referred by their Personal Tutors Measuring and monitoring engagement Identifying ‘at risk’ students in order to provide support or signpost support available to them Study Skills Support / Study Advice The role of the Personal Tutor What works?

‘At risk’ HEFCE Risk Factors:  Qualification on entry  Tariff Points  School Type  POLAR 2 – Postcodes which are identified as low HE participation quintiles (quintiles 1 and 2)  Socio-economic classification code (risk factors between 4-8) SEC CodeSocio-economic classification Not known 1Higher management & professional occupations 2Lower management & professional occupations 3Intermediate occupations 4Small employers & own account holders 5Lower supervisory & technical occupations 6Semi-routine occupations 7Routine occupations 8Never work & long-term unemployed 9Not classified

Reading ‘at risk’ YearWithdrawal rate of total cohort Withdrawal rate of ‘at risk’ students Withdrawal rate of not ‘at risk’ students 2006/077%27%6.2% 2007/086%45%5.3% *2008/09 cohort: N = 2,510 Total ‘at risk’ = 214 ‘At risk’ students withdrawn = withdrawals 6 withdrawn as a result of academic failure 1 student was suspended * Expecting complete withdrawal data/retention rates from Planning *2009/10 cohort: N = 2,944 Total ‘at risk’ = 190 Total withdrawals from cohort: 33 ‘At risk’ students withdrawn = 9 * Preliminary data (March 1 st 2010)

Reading ‘at risk’ Health Reasons2.5%4 Family illness/bereavement1.9%3 Personal Reasons (2)10.1 %16 Course not what I expected (1)7.6%12 Struggling academically (1)8.2%13 Homesickness (1)6.3%10 Feeling isolated7.0%11 Course resources not what I expected (1) 2.5%4 Financial reasons (2)8.2%13 Survey (2) – all Part 1 Undergraduate Students (158 responses): ‘I have thought about withdrawing from University’ 27 strongly agreed/agreed with the statement 2 were part of our ‘at risk’ students (‘I thought I might as well make it to the end of the academic year’, ‘stayed because of the benefits of a full degree’)

Survey (1_short) – all Part 1 Undergraduate Students (231responses): ‘I have thought about withdrawing from University’ No respondent belonged in the ‘at risk’ category

Short survey Survey (1_short) – all Part 1 Undergraduate Students (231responses): What the found helpful in their decision to stay:

Oxford Brookes ‘at risk’ (School of Life Sciences) – The Robbins Method  The HEFCE method identifies students as being of High, Medium or Low risk dependent on their entry qualifications.  The Robbins method asks students to fill in a questionnaire that asks them a number of questions, including questions about their previous science qualifications. This enables the PASS team to ‘score’ for relevance the subject areas and grades that a student has obtained before entering the School of Life Sciences. Hence students are designated as:  At risk  Not at risk  Disengaged (if they do not respond to requests to fill in the questionnaire)

Oxford Brookes ‘at risk’ (School of Life Sciences) – The Robbins Method SubjectsSkills Science 1Biology, Chemistry Human Biology Provides subject specific knowledge and scientific methods Science 2 Physics, Maths, Statistics, Environmental Science Provides limited subject specific knowledge and limited scientific methods Science-related Geography, Sports Studies, PE, Home Economics, Psychology, ICT, Computing Provides limited relevant content and knowledge Non ScienceAll other subjectsShows ability to study Once the above profiling is established scores are assigned: A2 grade A-BC-DE Science Science 2 Science-related531 Non Science421

Oxford Brookes ‘at risk’ (School of Life Sciences) – The Robbins Method A total A2 score of combined cohorts to For each A2 Robbins scoring band, percent of students in that band who passed/failed modules in year 1 Nearly half of students without A level entry qualifications fail at least one module in their first year. These students include those entering from ACCESS programmes, BTECs as well as baccalaureates. It appears that there is a problem if students do not have an academic background and lack academic skills

Oxford Brookes ‘at risk’ (School of Life Sciences) – The Robbins Method vs HEFCE  Robbins method identifies 230 students as being ‘At risk’ whereas HEFCE identifies 88 students  Of those 88 students only 21 (24%) are identified as ‘At risk’ by the Robbins method  93 (36%) of students identified as ‘At risk’ by the Robbins method are identified as ‘low risk’ by the HEFCE method  25 (28.5%) students are identified as ‘high risk’ by HEFCE whereas using the Robbins method they are categorised as ‘disengaged’

Oxford Brookes ‘at risk’ (School of Life Sciences) – The Robbins Method vs HEFCE ‘at risk’ criteria Academic outcomes for Robbins method of identifying ‘at risk’ students Of the 210 students who failed one or two modules in year 1, 68% were identified by the Robbins method as being ‘at risk’ or ‘disengaged’

Oxford Brookes ‘at risk’ (School of Life Sciences) – The Robbins Method vs HEFCE ‘at risk’ criteria Academic outcomes for HEFCE risk criteria of identifying ‘at risk’ students Of the 210 students who failed one or two modules in year 1, 59.05% were identified by the HEFCE ‘at risk’ criteria as being of ‘High’ or ‘Medium’ risk of failure

Oxford Brookes ‘at risk’ (School of Life Sciences) – The Robbins Method vs HEFCE ‘at risk’ criteria Conclusion:  The Robbins method identified 451 students as ‘at risk of failure’ or as ‘disengaged’. Of the 210 students that failed one or more year 1 modules, 68.1 were in this group of 451 students  The HEFCE categories identified 348 students as being at ‘high’ or ‘medium’ risk of failure. Of the 210 students that failed one or more Year 1 modules, 59.05% were in this group of 348 students The Robbins method successfully identified nearly 10% more of the 210 students who went on to fail one or more year 1 modules

Self-reported academic confidence Students were asked to rate their confidence (1-4) in a range of skills: speaking & listening, reading and researching, writing, time management, numeracy and IT. Preliminary results (4 consecutive years were aggregated N=681):  The whole cohort is more confident about reading and researching than speaking and listening or writing. Time management confidence levels were somewhere between. Students’ confidence levels at numeracy approach those of reading and researching.  No correlation (positive or negative) was found in any skill area between self reported confidence at academic skills and academic results. This would indicate that students’ self assessment is not particularly accurate. However, some interesting features still emerged from the data.  One quarter of the students who failed one or more module had extremely high confidence in their reading and researching, and their numeracy skills. While Life Sciences modules depend to differing extents on students’ numeracy skills, all modules demand reading and researching skills. This would suggest that a high proportion of failing students have misplaced confidence in their reading and researching skills.

Self-reported academic confidence Students were asked to rate their confidence (1-4) in a range of skills: speaking & listening, reading and researching, writing, time management, numeracy and IT. These data were also broken down into age groups: 18-20, 21-29, 30+. This showed up certain features of the 30+ age group, which did not appear in other age groups:  Those who were confident at speaking and listening were more likely to be those who failed. Those less confident in this area were much less likely to fail.  This age group tend to be more confident than others at reading and researching, and at time management. As would be expected, they are less confident at IT skills.  The extremely confident at writing, at reading and researching, and at numeracy are more likely to fail. Their misplaced confidence may be through not knowing the requirements, having been out of education for the longest time.  One third of failing 30+ students have medium to high ‘life concerns’ that are affecting their studies. In fact, all of the most highly concerned students failed one or more module.

conclusions:  Identifying ‘at risk’ students using the HEFCE method seems to be ‘working’ to a certain extent  In the case of the School of Life Sciences, a more discipline specific method identifying students ‘at risk’ appears to be more successful (Robbins method)  Preliminary results show that there does not appear to be a correlation between self reported academic competence at academic skills and academic results Some thoughts…  Identifying students ‘at risk’ – why?  Identifying students ‘at risk’ to target support or provide support for all  Identifying students ‘at risk’ – one method fits all?  Ethical implications about identifying students ‘at risk’