Using Predictive Analytics to Enhance Student Performance and Reduce Student Attrition Ronald Sarner SUNY Polytechnic Institute January 2018

The Problem “Joe Smith” - pseudonym, real student CS major, HS Avg 82.6, SAT Verbal 700, SAT Math 630, Tier II First semester course selections – traditional course selection methods CS 108 – Computing Fundamentals (4) MAT 115 – Finite Mathematics (4) ENG 101 – Freshman Comp (4) HIS 102 – American History 1865- (4) FYS 100 – Freshman Experience (1)

Poor Joe Joe has been set up to fail and possibly drop out – three “at risk” courses Risk analysis – probability of getting less than a “C” for students with the same academic profile as Joe. CS 108 - 47% MAT 115 – 12% ENG 101 – 38% HIS 102 - 44%

Smiling Joe Course selection using predictive analytics – no “at-risk” courses CS 100 – Computing Seminar (4) – 2% MAT 115 – Finite Math (4) – 12% HIS 101 – American Hist I (4) – 22% BIO 105 – Ecology (4) – 26% FYS 100 – Freshman Experience (1)

Who Attrits? Largest group of attrits are students in academic difficulty (GPA < 2.00)

Proposition Attrition can be reduced by placing students into classes where there is empirical evidence that they are more likely to succeed. It is particularly important that students are not taking multiple “at-risk” courses in the same semester – particularly in the first semester of the freshman year.

Data Collection (cont'd) Data Set II – Demographic Data One record per student Student ID First semester enrolled Major Gender HS Avg SAT scores (Verbal/Quant) Athlete? EOP? Resident/Commuter Admissions Tier Financial aid? Scholarship amount Entered as freshman or transfer

Data Preparation Data sets joined so that grade records included demographic data.

Course Groupings Courses taken by students in the first semester of their freshman year – minimum cumulative enrollment of 100 cases and offered Fall 2017 BIO 215 CET 101 CHE 110 CHE 130 CS 100 CS 108 ENG 101 ESC 110 HIS 101 HIS 102 IDS 103 MAT 110 MAT 115 MAT 120 MAT 151 MAT 152 MTC 101 NCS 181 PHY 101 PHY 201 PSY 100 SOC 100

Course Groupings Courses taken by students who entered as freshmen, minimum cumulative enrollment of 100 over five years regardless of what semester taken, not in prior group, and offered Fall 2017. ACC 201 AST 222 BIO 105 BUS 101 BUS 105 CET 102 COM 112 CTC 162 ECO 110 IDS 102 MAT 112 MAT 121 MAT 122 MAT 230 MAT 253 PHY 102 PHY 202 SPA 100 SPA 200

Model Development SPSS Decision Trees used Grade recoded to S/U (S >= 'C' course grade, 'U' otherwise). Recoded grade is the dependent variable.

Model Development Model developed for all Group I courses based on first-semester freshman enrollment Model developed for Group II courses Based on students who entered as freshmen regardless of when course was taken.

Sample Model – ENG 101

Freshmen CS Majors Pre-registered avoiding at-risk courses (risk>30%)

Individual Risk Analysis Given to all incoming CS freshmen at orientation for guidance in making schedule changes

Short-term Future Assess whether freshmen so advised have a reduced rate of poor academic performance.

Recommendations Expand to all incoming freshmen – Fall 2018 Expand to all students – requires generating other models based on both freshmen and transfer students. Use decision rules against actual current course enrollment to identify at-risk students in the first week of the semester; engage Learning Center early.

Recommendations Develop stand-alone apps (web-based, Android, IOS) for students to ascertain risk levels. Integrate models into Banner so that students are provided with risk data at the time of registration. Investigate further to ascertain if models can be developed that are as effective (or better) than placement tests in math and writing.