1. Improving retention: Using a voluntary diagnostic quiz
eSTEem April 2016

2. The issue

3. The issue
When science, technology, engineering and mathematics students are considered together, they are less likely to continue to a second year of study than students following other subjects.

4. The issue

5. and the chart that started it all !!
Our issue
and the chart that started it all !!
In October 2014 we had
2 972 students register on a particular maths module (MST124) -by June 2015
2 473 were still with us two weeks after module start
passed

6. A solution
Calvert, C. (2014) Developing a model and applications for probabilities of student success: a case study of predictive analytics. Open Learning: the Journal of Open, distance and e learning, 29:2,
There was a high quality pre existing diagnostic quiz “Are you ready for MST124”
Previous work had established a logistic regression model which provides early indicative predictions of student success. The model generates a probability, based on performance of similar students in the previous year, that is linked to whether students will pass key points in the module
The factors used in the logistic regression were chosen to be used across all the OU undergraduate modules
However the process can be readily modified and run for individual modules to include information specific to a particular module. In this study the MST124 diagnostic quiz results were included as a potential factor for students on MST124 – and they clearly dominated all other factors

7. Getting more students to take the quiz
A solution
Getting more students to take the quiz
We started in May with students who had registered on MST124
We spoke directly with students who had taken the quiz and got a low score and offered advice on additional resources and / or alternative modules
We ed or spoke to students who had not taken the quiz and again offered advice
We changed the module web page so the WELCOME message pressed students to take the quiz and contact their tutor or the support teams
We have tutors the quiz scores of their students so they could see who was struggling before the module began
Tutors urged their students to take the quiz if they had not already done so

8. The Outcome Numbers to date – April 2016
Student numbers taking the diagnostic quiz increased from around 200 in October to 700 in October 2015
At the highest number of registrations we had 232 more students in 2015 but in Mid October we had 400 more students. So we had retained over 200 students more than expected.
Watch out for evaluation TOO early- we had 200 more students than we’d have expected and now it is around 50

9. A changed pattern of withdrawal
The Outcome
A changed pattern of withdrawal
More students withdrew before module start – a success for various reasons
The very steep de registration in the period when module materials are received / module starts was reduced

10. The Outcome The 2015 students
Comparisons with are valid until the end of November
Other interventions occurred from Dec 2015 onwards that did not occur in 2014

11. Key Issues Having a good diagnostic Having an alternative
Having everyone signed up
Sustainability
Cognitive dissonance

12. Carol.Calvert@open.ac.uk Rachel.Hilliam@open.ac.uk
Questions