Multi-Institutional Data Predicting Transfer Student Success Denise Nadasen Anna Van Wie Institutional Research University of Maryland University College

Outcomes for this Session You will learn about the: –Goals for this grant and the research project –Process for integrating a multi-institutional data base –Research questions, methods, and findings –Lessons learned and next steps 2

Goals of the grant Collaborate with the community colleges Define research questions and variables Build a dataset for transfer students Explore predictor/outcome variables Predict student success Report the results at national conferences Use the results to inform policy and practice to better serve transfer students 3

Collaborative Partners UMUC is an online institution that enrolls over 90,000 diverse students each year worldwide Prince George’s Community College is located within two miles of UMUC’s Academic Center and enrolls over 37,000 diverse students. Montgomery College is located within 10 miles of UMUC’s largest regional center, and enrolls over 35,000 diverse students. 4

The Team PI– President, Provost Sponsor – Institutional Research Partners –Montgomery College and Prince George’s Community College –Undergraduate retention and data mining specialist –External evaluators Researchers: –Cheoleon Lee, Jing Gao, Futoshi Yumoto, Husein Abduhl-Hamid Data Mining Specialists –Stephen Penn, The Two Crows 5

The Student Population Students enrolled at UMUC between 2005 and 2011 PG and MC transfer students –Direct compare (32,000) –National Student Clearinghouse (12,000) –UMUC records (8,000) 6

Merging Multi-Institutional Data Protect this data! Balance institutional-specific protocols with research-based definitions Address data anomalies Distinguish student level vs. course level Define LMS data –Limits on data extract 7

KDM Integrates student data –Community College and UMUC SIS –Demographic –Courses –Performance –Classroom behavior (LMS) 300 source and derived variables Gather from disparate sources One time snapshot 8

WT Online Classroom PeopleSoft Live SIS with UMUC students PGCC students and class data MC-PGCC-UMUC Transfers UMUC students who transferred from MC or PGCC and were matched in the BASE file Base Extract UMUC undergrad students enrolled between Spring 2005 and Spring 2011 Data Warehouse UMUC students from PeopleSoft Daily Update MC students and class data WT extract Classroom activity Prior Work derived data for transfer students

Question What barriers would your institution face in merging multi-institutional data? 10

Research Goals Define outcome variables Define predictor variables Model the student lifecycle Determine the success and failure factors Develop and implement interventions Impact outcomes 11

Outcome Variables Successful course completion (percent) First term GPA (dichotomized) Reenrollment in next term (Y/N) Retention (12 month window – Y/N) Student Classification (Slackers, Splitters, Strivers, and Stars) 12

Transfer Student Progressions 13 cc First Semester Semester 2 Last Semester Four-Year Institution Demog and Other Academic Work Transfer Graduate School

Research Studies 14

Which variables contribute to the prediction of online course success? The data: 4,558 new, undergraduate, first bachelor- degree seeking enrollments in 15 UMUC online gateway courses in Spring Transfer data on students from partner institutions, Montgomery College (MC) and Prince George's Community College (PGCC).

Methodology Exploratory factor analysis (EFA) was used to identify key covariates. Logistic regression was used to predict course success.

Findings Total number of transferred credits is the best predictor of course success –pseudo R 2 value around.12 GPA from transferred credits is the second best predictor of course success –pseudo R 2 around.11 Semester course load contributes less to course success than other covariates.

Findings Four of five predictors derived from online student behavior show a strong contribution to successful course completion.

Final Predictive Model Significant Variables Total number of transfer credits Summary of students’ week 0 behavior prior to the first day GPA from transferred credits Semester course load Amount of time since students attended the last institution Significant Online behaviors Read a conference note Entering a class Created a conference note Created a response note

Which variables predict retention in an online environment? The same data set for the prediction of course success Add in retention status from Summer 2011, Fall 2011, and Spring 2012.

Methodology Logistic regression Preliminary analysis focused on the evaluation of covariates (as identified in the previous analysis) predictors based on the students’ coursework behavior, and course success.

Findings The covariates and student behavior variables made less of a contribution to this model than the prediction of course success. These results indicate that course success may be a good predictor of retention.

What is the relationship between prior academic coursework and UMUC first semester gateway course on re- enrollment? The population: Students new to UMUC in Fall 2008 to Fall 2010 Took ACCT220, BMGT110, CMIS102, GVPT170, or PSYC100 in their first semester.

Methodology Association algorithm Apriori to determine relationships between courses in previous academic work and re-enrollment rates. The algorithm indicates when a certain condition is found another condition can be expected.

Findings Significant Course Relationships Community College Course Disciplines First UMUC Course Math or BusinessACCT 220 Math or BusinessCMIS 102 Business or ScienceBMGT 110 Science or communicationGVPT 170 Math or communicationPSYC 100 We cannot assume causality

Current Studies 26

Examine CC Courses Explore relationship between CC courses and first term GPA Identify courses of interest Developmental Ed sequencing Successful completion of CC course Mixing course level and student level data 27

Predicting First Term GPA What CC variables predict first term GPA of 2.0 or higher? Course Efficiency CC courses –English, Math, Speech, Computer, Honors, On- line Course, Remedial Demographics –Age, Gender, Race, Marital Status, Cohort, Community College Origin, Terms skipped 28

The Population 9,063 students from MC and PGCC Mostly Single, African-American, and female Most do not skip terms Most get A’s and B’s at CC Most have >2.0 at UMUC Only PGCC offered online courses 29

Predictors of Success Race Gender Math Computer On-line Age Speech Course Efficiency English UMUC Marital Status C.C. Courses Honors Remedial Logistic Regression 30

Question What CC variables do you think are good predictors? 31

Findings Predictive variables: –Age, marital status, and under-represented minorities have predictive power –Math and Honors courses have positive effects –Remedial and Online have a negative effect –Course efficiency has a positive effect 32

Predicting Student Clusters Dataset includes all PGCC and MC students who transferred Student level derived variables Cluster students based on retention and first term GPA at UMUC Predict clusters from prior CC work and demographic variables 33

34 StriversStarsSlackersSplitters Success Quadrants Retention Yes Retention No GPA > 2.0 GPA < 2.0

Stay Tuned …. Data mining continues –So far, Stars appears to have distinguishing features Focus on top 50 CC courses and combinations of courses as predictors Focus on performance in gateway courses at UMUC as outcomes 35

Summary of Findings Positive effects –Transfer credit, prior GPA, math, honors, course efficiency, online activity, age, marital status –Course success can predict retention Negative effects –remedial, online, minority status 36

Interventions Identify areas of risk Collaborate with CC Develop intervention strategies –Advising –Messaging –Learning community –Course development 37

3 Projects Synergizing Kresge PAR Civitas 38

Next Steps Examine course success at the CC Implement/evaluate interventions Update KDM with more data Develop, understand, and explain predictive models to identify at risk students at the CC 39

Lessons Learned Long term plan for data up front Get a project manager Manage expectations Communicate progress far and wide 40

Questions Anna Denise 41