Developing Honors College Admissions Rubric to Ensure Student Success Lizmarie Maldonado, MSPH, Senior Statistician College of Charleston #RIConf2019

Quick Facts: College of Charleston Location: Charleston, SC Public Liberal Arts and Sciences university 61 majors and 78 minors; 24 master’s degree programs Fall 2018 Enrollments: 9,880 Undergraduate; 903 Graduate Founded in 1770 (oldest educational institution south of Virginia and 13th oldest in the US) Honors College established 1967 with close to 700 students today

Background Objective: Determine factors most likely to predict success in the Honors College at the application stage Measures of success: One-year College of Charleston retention One-year Honors College retention One-year cumulative College of Charleston GPA One-year cumulative Honors College GPA

Honors College Admissions Standardized test scores (SAT/ACT)*: 10-point scale based on highest test score (ACT scores converted to SAT equivalent) Essay: 12-point scale following rubric Transcript*: 10-point scale Resume*: 12-point scale Total Possible Score: 44 *New rubrics developed for point-assignment; resumes scored separately on style and content

Study Timeline Fall 2017: Developed New Admissions Rubric + Tested on Fall 2016 Cohort Fall 2018: New Rubric Implemented for Entering Freshmen Fall 2019: Assessment of Full Freshman Cohort

Honors College Academic Profile Fall 2016 Entering Freshmen Class: 194 95 Residents 99 Non-residents Average SAT score: 1287 Average High School GPA: 4.46 One-year retention: 89.2% One-year Honors College retention: 77.8% Average cumulative CofC GPA: 3.56 Average cumulative Honors College GPA: 3.59

Sample of Fall 2016 Cohort Fall 2016 Honors College entering freshmen cohort A simple random sample of 100 students selected (50 residents/50 nonresidents) Student applications rescored based on new rubric Average total application score: 31 Outcomes One-year CofC & Honors College retention (retained in Fall 2017) One-year cumulative CofC & Honors College GPA (2016-2017 Academic Year)

Analyses in Veera Predict One-year CofC & Honors College retention: Logistic regression Models: Total Application Score Test Score + Resume + Transcript + Essay One-year cumulative CofC GPA & Honors College GPA: Linear regression

Results One-year CofC & Honors College retention: No correlation found One-year cumulative CofC GPA & Honors College GPA: Correlation to Total Application Score Correlation to Test Score, Transcript, and Resume Scores Fall 2016 Honors College Cohort Cumulative CofC GPA by Application Score

Summary Provided assurance that historical admissions process achieved the goal of selecting successful students Provided baseline for how new admissions process might perform Provided insight into which Honors College application components were most predictive of success

Decision Tree Analyses in Veera Predict

Chi-squared Automatic Interaction Detector (CHAID) Analysis Tree classification method Continuous predictors are split into categories with approximately equal number of observations Examines all possible cross tabulations between each categorical predictor and the outcome Identifies optimal splits

Building a Decision Tree Model in Veera Predict

Decision Tree Models Different iterations: Residency Gender Total score/individual app components

Initial Results Sample Total App Score < 24 Sample of Fall 2016 cohort One-year CofC cumulative GPA and total application score Sample Avg. GPA = 3.61 Total App Score < 24 Avg. GPA = 2.97 24 ≤ Total App Score < 32 Avg. GPA = 3.50 Score ≥ 32 Avg. GPA = 3.81

Additional Models Random sample of 60 students from Fall 2017 cohort 30 resident 30 nonresident Combined Fall 2016 and Fall 2017 = 160 students

Model Results Sample Total App Score < 28 Combined sample of Fall 2016 and Fall 2017 cohorts One-year CofC cumulative GPA and total application score Sample Avg. GPA = 3.60 Total App Score < 28 Avg. GPA = 3.32 28 ≤ Total App Score < 32 Score ≥ 32 Avg. GPA = 3.77

Other Findings One-year CofC cumulative GPA: Test score most significant split node for Fall 2016 (suggestive of a minimum 1300 SAT) Honors essay most significant split node for in combined cohorts An examination of two-year CofC cumulative GPA produced similar results One-Year Honors cumulative GPA: Honors essay most significant split node followed by test score for Fall 2016

Implementation First admissions cycle: Fall 2018 Applicants with a total score of at least 28 were admitted Resulted in an Honors College entering freshmen class of 264 Second admissions cycle: Fall 2019 Projected enrollment: 277

Next Steps Assessment of full Fall 2018 cohort Impact on retention and cumulative GPA

Thank you! Questions/Comments? Lizmarie Maldonado, MSPH, Senior Statistician College of Charleston maldonadol@cofc.edu #RIConf2019