HLC Academy on Student Persistence and Completion – A Presentation on Statistical Analyses of Illinois Tech Data May 24, 2016 Illinois Institute of Technology Siva Balasubramanian (Team Leader) Ray Trygstad (Member) Tyyab Arshad (Member) Charles Uth (Member) Tianyu Zhao (Student Assistant)

Initial Analysis of Illinois Tech Data ( ) These data include students who were registered as degree-seeking from fall 2008 through fall unique student records in our dataset

Overall Persistence

Persistence Results for students with Cumulative GPA>=3.5 This group of 149 Students that do not persist (7.7% of all those in Cum GPA>=3.5 group) includes those in high academic standing who left the university. We call them the invisible group, because no visible reasons exist to suspect that these students will not persist

Persistence Result for students with Cumulative GPA<3.5 The percentage of students who do not persist goes up from 7.7% in the GPA>=3.5 group to 20.2% in the GPA<3.5 group. In other words, the persistence rate is almost 3 times worse in the GPA =3.5 group.

Summary of Initial Analysis and Results We ran three analyses. The first analysis included all 7205 records. The second one focused on the students with a 3.5 or higher cumulative GPA. The third analysis considered students with a 3.75 or higher cumulative GPA. In all our analyses, the dependent variable is student persistence. Using logistic regression, we consider several factors that may affect student persistence. In each of these analyses, tests indicate that our statistical model fit is good.

First Analysis - Whole Sample In our whole sample analysis, we found that after controlling other factors: Compared to students with math grade (MATH 148) lower than C, those with a C or higher math grade (MATH 148) have 51.3% more probability to persist at Illinois Tech (Odds Ratio=1.513; p=0.016). For a unit increase in cumulative GPA, there is a 404.4% higher probability to persist (Odds Ratio=5.044; p<0.001).

Results for GPA cutoffs at 3.5 and 3.75 In GPA>=3.5 group, when compared to students with a math grade (MATH 148) lower than C, those with a C or higher math grade (MATH 148) have a 288.6% higher probability to persist at Illinois Tech (Odds Ratio=3.886; p=0.074). In GPA>=3.75 group, math grade appears to have an important role, but this result is not statistically significant at 0.1 level (Odds Ratio=8.424; p=0.118).

Analyses with More Recent Data In May 2016, we ran several additional analyses with more recent data that provide useful insights

Analysis of SI (Supplementary Instruction) treatment and MATH performance outcomes As we have shown in our earlier analysis, students’ performance on MATH 148 plays an important role in retention In the results presented earlier, when compared to students with a MATH 148 grade lower than C, those with a C or higher Math grade have a 51.3% higher probability of remaining at the university (OR=1.513; p=0.016). Follow the suggestions from HLC reviews, we now focus on the following question: is there any difference between students exposed to MATH 148 course with SI students who have already passed MATH 148 (the SI treatment), and those who took MATH 148 course without SI treatment.

Data for SI We collected data between Fall 2013 and Spring records for MATH 148 were included in our data Excel Pivot Table was applied in the SI part of the analysis We matched the SI information with MATH 148 course results by using key factors, such as course name, course academic period and SI student academic period.

Results of SI Comparison With SI Treatment MATH 148 course score with C and Higher percentage: 76.88% MATH 148 course score with B and Higher percentage: 48.55% Without SI Treatment MATH 148 course score with C and Higher percentage: 63.04% MATH 148 course score with B and Higher percentage: 36.96% Conclusion: SI treatment significantly improves performance in MATH 148 course.

Data for GPA and MATH 148 grade – new analysis Updated data From Fall 2012 through Fall 2016 For each student, we tracked the latest academic period record to identify his/her persistence unique student records in our dataset. We merged data from different sources by student UID as the key variable. Following the results presented earlier, we re-focus attention on how GPA (Major and Cumulative) and MATH 148 grade affects students’ persistence. SPSS Logistic regression was used in this part of our analysis.

Measure of Persistence Active records indicate persistence. Records that are inactive due to graduation are treated as persistent. 1 record represented an admitted student with no deposit – therefore, this was excluded from our analysis. All other records were recoded into Inactive or Withdrawn, which means that those students are not persistent. Frequency of Records

Persistence Results Overall For a few students, we were unable to compute the cumulative GPA. Therefore, this simply represents an overall view of persistence of students.

Persistence Results for those with Cumulative GPA>=3.5 This group of 12 Inactive Students (0.6% of all Cum GPA>=3.5 group) represent good students who left the university. Based on previous analyses, we call them the invisible group Unfortunately however, there are simply too many reasons that motivate this invisible group to leave the university.

Persistence Result Cumulative GPA<3.5 The percentage of students who do not persist goes up from 0.6% in the GPA>=3.5 group to 1.8% in the GPA<3.5 group. In other words, the persistence rate is 3 times worse in the GPA =3.5 group.

Continued.. The second set of recent analyses indicate that results highlighting the role of Cumulative GPA in shaping student persistence outcomes is consistent with our initial analyses. Both sets of analyses show that Cumulative GPA is the most important factor that influences student persistence.

GPA and Persistence We focus on GPA and Credits that students earned at the university Major GPA does not show any significant effect on student persistence. However, a unit increase in Cumulative GPA results in a 828.1% higher probability of persistence (OR=9.281; p<0.001). In other words, eight times higher. The direction of this result is consistent with our initial analyses.

Continued

MATH Score and Persistence The Logistic regression results for MATH grade is consistent with results from our initial analyses. When we focus on the relationship between the MATH grade (MATH 148) and student persistence, a unit increase in MATH Grade (MATH 148) results in a 55% higher probability of student persistence (Odds Ratio=1.550; p<0.001).

Continued

Conclusions SI (Supplementary Instruction) significantly improves performance in MATH 148 course GPA and MATH 148 performance reported in the updated dataset appear to influence student persistence in a manner consistent with our initial analyses.