Examining Retention of Sophomores from a Consumer Satisfaction Perspective Authors: M. Rita Caso, Xiaohong Li Rebecca Bowyer & John J. Scariano O FFICE OF I NSTITUTIONAL R ESEARCH & A SSESSMENT Sam Houston State University A Member of The Texas State University System O FFICE OF I NSTITUTIONAL R ESEARCH & A SSESSMENT Sam Houston State University A Member of The Texas State University System

Absent Author 2 Rebecca Bowyer

Evolution of Research Objective Why Sophomores? Sophomores often represent the university’s second largest attrition group. Much research devoted to predicting freshman retention but less to sophomores 3

Evolution of Research Objective (cont.) Much research has predicted retention from key variables student attrition theories Authors: Tinto, Bean, Dey, Terenzini & Pascarella Variables: Major Certainty, GPA, Institutional Fit, Social Integration, Finances/Support 4

Evolution of Research Objective (cont.) Variables that are historically associated with persistence/ attrition, which were available : — Entry At-Risk Individual Attributes, Tinto 1975 —1 st Sophomore Semester GPA Academic Integration, Tinto 1975 —Campus Residence (FTF) Academic Integration, Tinto 1975 —Minority Individual Attributes, Tinto 1975 —Financial Aid Family Background and Individual Attributes, Tinto

Evolution of Research Objective (cont.) At our university, research on retention of Sophomores to Junior year using traditional attrition-theory variables had not produced useful prediction or explanation 6

Evolution of Research Objective (cont.) Other Possible predictors of persistence or attrition: Langbein & Snider (1999) and Ronco & Cahill (2004) – Classes taught by faculty who are not tenured – Favorable rating of classes in the university’s class evaluation survey system as related to tenure/tenure-track level of instruction Suddarth (1975) – Classes that are required as part of a General Education/ Core Curriculum program ** In our current study we add Required Remedial Courses to General Education Courses 7

Evolution of Research Objective (cont.) Could student satisfaction with class experiences significantly influence the decision to persist? Student as Consumer  Consumer Satisfaction –Brand Loyalty Student-Consumer Satisfaction  Decision to Stay –Abandonment of Brand Student-Consumer Dissatisfaction  Decision to Leave 8

Variables of Interest Hypothesized predictors of Sophomore-to-Junior persistence representing the customer satisfaction perspective : High sophomore year exposure to classes positively rated overall* High sophomore year exposure to classes positively rated for instruction* High sophomore year exposure to classes taught by tenured or tenure- track faculty in sophomore year Low sophomore year exposure to required General Education (Core Curriculum) or Remedial courses (* hypothesized to be a direct indicator of customer satisfaction) 9

Variables of Interest from IDEA Operational Definitions Percentage of a student’s sophomore year class schedule spent in classes positively rated* overall - IDEA class evaluation Question 42 – “Overall, I rate this course as excellent.” – Scale (1 = Definitely False, 5 = Definitely True) Percentage of a student’s sophomore year class schedule spent in classes positively rated* for instruction - IDEA class evaluation Question 41 – “ Overall, I rate this instructor as an excellent teacher.” – Scale (1 = Definitely False, 5 = Definitely True) *Positively Rated classes have mean class ratings >=4 10

IDEA Class / Instructor Evaluation System IDEA Class/Instructor Evaluation – the standardized, national, classroom evaluation system used at SHSU by students near the end of each semester to rate their perceptions about classes and instructors. 11

Variables of Interest: Operational Definitions in the Study Data Percentage of student’s sophomore year class schedule spent in classes that were taught by Tenured/Tenure-Track faculty Percentage of student’s sophomore year class schedule spent in classes that were required General Education (Core Curriculum) or Remedial courses 12

Persistence-Related Covariates: Operational Definitions Entry At-Risk - Whether a student was originally admitted with one or more College Readiness Deficiencies GPA in 1 st Sophomore semester - End of semester GPA in first semester of Sophomore year Campus Residence (Off/On) - Residence off vs. on campus in 1 st semester of entry year. Financial Aid - Whether or not the student received financial aid during Sophomore year Minority – Hispanic, African-American, Native-American Vs. White, Asian-American, International Entry Type - Entry as First Time Freshmen (FTF) vs. First Time Transfer (FTT) 13

The first Fall semester by which a student has completed between 30 and 59 semester credit hours defines her/his Sophomore Cohort. E.g., – If a student is first classified as a sophomore in Spring 2004 and is still classified as a sophomore in Fall 2004, that student is a member of the Fall 2004 Sophomore Cohort. – If a student is first classified as a sophomore in Fall 2004, but is still classified as a sophomore in Fall 2005, that student is still a member of the Fall 2004 Sophomore Cohort 14 Sophomore Population Operational Definition

One-Year retention is defined as continued enrollment in the fall term following the first fall term in which the student meets the criteria for sophomore classification –The student is considered retained even if he/she has not earned enough credits to progress to junior classification. 15 Dependent Variable Operational Definition

SPSS Procedure Backward, Stepwise,Binary Logistic Regression 1.Under Tool Bar Analyze, select Regression - Binary Logistic 2.Select variable ‘Retention1Yr’ as the dependent variable. 3.Select the variables of interest as Covariates 4.Method - Select Backward Stepwise Likelihood Ratio 5.Click Option in the logistic Regression Dialog box 1.Select Hosmer-Lemeshow goodness-of-fit to test the model fit 2.Enter.2 as the classification cutoff point 3.Keep the default Probability for stepwise setting - Entry Probability.05 and Removal Probability.10. Example of The syntax: LOGISTIC REGRESSION VARIABLES Retention1Yr /METHOD = BSTEP(LR) PercentQ42Coded PercentQ41coded PercentRemiCore PercentTenureCourses Enter_Risk Enter_TYP EUGACoded FAID Minority /CONTRAST (Enter_Risk)=Indicator /CONTRAST (Enter_TYP)=Indicator /CONTRAST (FAID)=Indicator /CONTRAST (Minority)=Indicator /CONTRAST (PecentQ42Coded)=Indicator /CONTRAST (PercentQ41coded)=Indicator /CONTRAST (EUGACoded)=Indicator /PRINT = GOODFIT /CRITERIA = PIN(.05) POUT(.10) ITERATE(20) CUT(.2). 16!

Binary (or binomial) logistic regression is a non-linear form of regression which is used when the dependent is a dichotomy (binary) and the independents are of any type (continuous, categorical, binary). Backwards stepwise logistic regression methods allow one to enter all variables at once. Backwards stepwise logistic regression methods determine automatically which variables to drop from the model. Hosmer-Lemeshow goodness-of-fit test is a recommended test for the overall fit of a binary logistic regression model – A finding of non-significance allows the researcher to conclude that the model adequately fits the data. Well-fitting models show non-significance, indicating that model prediction is not significantly different from observed values. 17! Methodological Definitions

Coefficient (B ) - The coefficient (B)’s estimations tell the amount of increase (or decrease, if sign is negative) in the predicted log odds of retention by an increase or decrease in the variable’s values, holding all others constant. The proportional change in the odds of being retained, for a one unit change in the independent variable Odds Ratio = Exp(B) - General speaking, the odds ratio is a measure of the strength of association between a predictor and the response of interest. Can be used to compare whether the probability of a certain event is the same for two groups. The value of odds ratio is from 0 to infinity. If the odds ratio is one, there is no association, which implies that the event is equally likely in both groups, and an odds ratio greater than one implies that the event is more likely in the first group. An odds ratio less than one implies that the event is less likely in the first group. 18! Methodological Definitions

PopulationsPopulations “Combined” - Sophomore Cohort population (n=10983) includes any enrolled student who has completed between 30 and 59 semester credit hours at the start of Fall 2005, Fall 2006, Fall 2007, or Fall 2008 regardless of whether he/she originally entered the university as a First Time Freshman (n=5568) or a First Time Transfer (n=5415). “FTF” - F Sophomore Cohort members who entered as First Time Freshman (FTF) (n=5568) “FTT” - F Sophomore Cohort members who entered as First Time Transfer Cohort Population (FTT) (n=5415) 19

Continuous Variables Recoded as Categorical 20 PopulationEUGAcoded%Q42coded%Q41codedCodeFrequency% CodeFrequency%Code Frequency% Combined 1 >= >=50% >=66% >=3.0 & < >=40% & <50% >=50% & <66% =>2.5 &< >=.30% & <40% >=38% & <50% =>2 & < >=20% & <30% <38% =>1.5 & < <20% < FTF 1 >= >=50% >=66% >=3.0 & < >=40% & <50% >=50% & <66% =>2.5 &< >=.30% & <40% >=38% & <50% =>2 & < >=20% & <30% <38% =>1.5 & < <20% < FTT 1 >= >=50% >=66% >=3.0 & < >=40% & <50% >=50% & <66% =>2.5 &< >=.30% & <40% >=38% & <50% =>2 & < >=20% & <30% <38% =>1.5 & < <20% <

Hosmer and Lemeshow Test 21 Combined Population Analysis Model Fit Backward Stepwise StepChi-squaredfSig

22 Combined Population Analysis Variables in the Equation at Final Step Combined Population Analysis Variables in the Equation at Final Step Variable Name Values BdfSig. (2-tailed)Exp(B) %Q42Coded REF -%Q42Coded(5) - <20% %Q42Coded(1) >=50% %Q42Coded(2) >=40% & <50% %Q42Coded(3) >=30% & <40% %Q42Coded(4) >=20% & <30% %Q41coded REF - %Q41Coded(4) -<38% %Q41coded(1) >=66% %Q41coded(2) >=50% & <66% %Q41coded(3) >=38% & <50% %RemiCoreContinuous EUGACoded REF - EUGACoded(6) - < EUGACoded(1) >= EUGACoded(2) >=3.0 & < EUGACoded(3) =>2.5 &< EUGACoded(4) =>2 & < EUGACoded(5) =>1.5 & < Constant

End of 1 st Sophomore Semester GPA, EUGAcoded, is the most important factor is the persistence-related covariate. EUGAcoded is statistically significant overall, and for each of its dummy- coded values. – Sophomores with End-of-1 st -Semester GPAs between were the group most likely to be retained into next year. – Interesting that students in the highest GPA group ( )-EUGACoded(1)- with an odds ratio of 8.24, had slightly lower odds of being retained than students in the lower GPA group ( EUGACoded(3)- with odds ratio of Combined Population Interpretation of Coefficients, Odds Ratios

The effect of Percent of required General Education or Remedial Classes in student’s schedule, %RemiCore, is smaller than that of 1 st semester sophomore GPA The relationship to retention is negative For each one point increase in the % of required General Education or Remedial courses in a student’s sophomore schedule, the log odds of the student being retained decreased by Combined Population Interpretation of Coefficients, Odds Ratios

Both of the variables representing our customer satisfaction hypothesis (%Q41Coded and %Q42Coded) are statistically significant overall However, some of their dummy-coded value levels are not statistically significant. There is a NEGATIVE relationship between retention and the groups with the highest percent of positively rated classes, e.g.,: – %Q41Coded(1) = >66% has Coefficient=-.12 and Odds Ratio = 0.89 – %Q42Coded(1) = >50% has Coefficient=-.28 and Odds Ratio = 0.76 Students with 50% or more classes with favorable overall ratings have odds of NOT being retained that are 1.24 times higher than for those with the lowest % of favorably rated classes ( %Q42Coded(5)) All other dummy-coded levels have positive relationships with retention. 25 Combined Population Interpretation of Coefficients, Odds Ratios

26 Are FTF and FTT Populations Different? Cross Tabs – Retention * Entering Type FTFFTTCombined Freq Count Freq % Freq Count Freq % Freq Count Freq % Retention 1 YR 0 – Not Retained % % % 1 – Retained % % % Total % % % Chi-Square Test Valuedf Asymp. Sig. (2-sided) Exact Sig. (2-sided) Exact Sig. (1-sided) Pearson Chi- Square Fisher’s Exact Test.000 N of Valid Cases 10983

Hosmer and Lemeshow Test 27 FTF Population Model Fit StepChi-squaredfSig

28 FTF Population Variables in the Equation at Final Step FTF Population Variables in the Equation at Final Step Variable Name ValuesBdfSig. (2-tailed)Exp(B) %Q42Coded REF- Q42Coded(5) - <20% %Q42Coded(1)>=50% %Q42Coded(2) >=40% & <50% %Q42Coded(3) >=30% & <40% %Q42Coded(4) >=20% & <30% %RemiCore Continuous Enter_Risk(1)Binary EUGACodedREF-EUGACoded(6) - < EUGACoded(1)>= EUGACoded(2) >=3.0 & < EUGACoded(3) =>2.5 &< EUGACoded(4) =>2 & < EUGACoded(5) =>1.5 & < ONOFF(1)Binary Constant

FIVE variables influence retention of sophomore students who entered the university as FTFs: 1. EUGAcoded 2. %RemiCore 3. Q42coded 4. ONOFF 5. Enter_Risk 1.First Semester Sophomore GPA is the most important variable in predicting retention. The overall EUGAcoded variable is statistically significant as are each of its dummy-coded values. a.Students with end of semester sophomore GPAs between have times higher odds of being retained than students in the EUGACoded(6) group who have GPAs <1.5 2.The Percent of GenEd/Remedial Courses effect is smaller, and NEGATIVE in relation to retention. a.For each one point increase in a student’s %RemiCore, the odds of NOT being retained increase by ! FTF Population Interpretation of Coefficients, Odds Ratios FTF Population Interpretation of Coefficients, Odds Ratios

1. Percent of Favorably Rated Courses Overall, is statistically significant, although two of its dummy-coded values are not. a.There is a NEGATIVE relationship between retention and membership in the %Q42Coded(1) group, despite the fact that this group has the highest % of classes that were rated favorably overall(>= 50%) b.In contrast, there is a POSITIVE relationship between retention and membership in the %Q42Coded(3) group whose members have much lower exposure to favorably rated classes ( 30% -39%) 2. The ONOFF odds ratio of 0.724, indicates that sophomores who entered the university as FTFs and did NOT live on campus in their entering semester were 1.27 times more likely to NOT be retained than those who lived on campus 3. The Enter_Risk effect is the smallest, of all the variables. a. There is a negative relationship between having entered the university at risk and being retained. b. With an odds ratio of 0.773, students who entered at risk have 0.22 greater odds of NOT being retained than students who do not enter at risk. 30! FTF Population Interpretation of Coefficients, Odds Ratios

31 FTT Population FTT Population Variables in the Equation at Final Step FTT Population FTT Population Variables in the Equation at Final Step Values BdfSig. (2-tailed)Exp(B) %Q42Coded REF-%Q42Coded(5) - <20% %Q42Coded(1)>=50% %Q42Coded(2) >=40% & <50% %Q42Coded(3) >=30% & <40% %Q42Coded(4) >=20% & <30% %Q41coded REF-%Q41Coded(4) -<38% %Q41coded(1) >=66% %Q41coded(2) >=50% & <66% %Q41coded(3) >=38% & <50% %RemiCore Continuous EUGACoded REF-EUGACoded(6) - < EUGACoded(1)>= EUGACoded(2) >=3.0 & < EUGACoded(3) =>2.5 &< EUGACoded(4) =>2 & < EUGACoded(5) =>1.5 & < Constant

Same 4 variables influence retention of sophomore students who entered the university as FTTs, as for Combined Population: 1. EUGAcoded; 2. %RemiCore ; 3. %Q41coded ; and 4. %Q42coded 1.The most important factor is EUGAcoded, which is statistically significant overall and for each dummy-coded value. a.Students in the EUGAcoded(2) group have GPAs between and the highest coefficient and odds ratio of all EUGAcoded value groups. With an odds ratio = 7.91, the odds of students in this group being retained is 7.91 times greater than for students in the EUGACoded(6) reference group who have GPAs < The %RemiCore effect is smaller, and NEGATIVE in relation to retention. For each one point increase in a student’s %RemiCore, the odds of NOT being retained increase by ! FTT Population Interpretation of Coefficients, Odds Ratios FTT Population Interpretation of Coefficients, Odds Ratios

Both %Q41Coded and %Q42Coded are statistically significant overall, however, some of their dummy-coded value levels are not statistically significant. 3.%Q41Coded (3), the group with 38-49% exposure to courses positively rated for instruction is 1.38 times more likely to be retained than the group with <38% exposure However, %Q41Coded (2), the group with >50% exposure to courses positively rated for instruction has slightly lower odds of retention (1.27) 4.Curiously, for %Q42Coded the only value level with a significant relationship to retention is %Q42Coded (1), which is a negative one. Students who have a >50% exposure to courses positively rated overall have 1.29 times greater odds of NOT being retained than students in the reference group, who had only <20% exposure to courses positively rated overall 33! FTT Population Interpretation of Coefficients, Odds Ratios FTT Population Interpretation of Coefficients, Odds Ratios

34 Variables Name CombinedFTFFTT BSig.Exp(B)BSig.Exp(B)BSig.Exp(B) PecentQ42Coded 0.00 PecentQ42Coded(1) PecentQ42Coded(2) PecentQ42Coded(3) PecentQ42Coded(4) PercentRemiCore EUGACoded 0.00 EUGACoded(1) EUGACoded(2) EUGACoded(3) EUGACoded(4) EUGACoded(5) PercentQ41coded 0.00 PercentQ41coded(1) PercentQ41coded(2) PercentQ41coded(3) Enter_Risk(1) ONOFF(1) Comparing Combined, FTF and FTT Population Variables in the Equation at Final Step Comparing Combined, FTF and FTT Population Variables in the Equation at Final Step

1.For all 3 populations,GPA at the end of 1st sophomore semester is extremely important for retention into next year, and this variable is relatively more important in relation to other variables in the model for sophomores who entered as FTF students 2.The % of General Education or Remedial Courses is the next most important variable common to all three populations. This variable is a consistent NEGATIVE predictor to retention. 3.Less clear in its predictive importance for all three populations is the % of classes favorably rated overall. However, students in the group with >50% exposure to favorably rated classes are consistently less likely to be retained than students with much lower (<20%) exposure to favorably rated classes.. 35 Comparing Combined, FTF and FTT Populations Interpretation Comparing Combined, FTF and FTT Populations Interpretation

DiscussionDiscussion EUGACoded - Common Sense and 30 years of literature on Retention by Tinto, Dey, Bean, Astin, Pascarella, Terenzini support & explains the association of GPA with retention. %RemiCore - The experimental research findings of Betty Suddarth, at Purdue University (1975), indicated a direct relationship between retention and removing General Education course requirements, as well as a direct relationship with students’ expressed satisfaction with the university experience and removing General Education course requirements Enter_Risk & ONOFF - The specific indicators of Risk and Off/On Campus Residence had been associated with retention of freshmen and retention of Sophomores in previous studies conducted at our university ( i.e., Li,X & Caso,R, Spring 2009 & Fall 2009 and Yi,J, Caso,R, Li,X, Kokatla,L, Li,Q, Spring 2009) 36

4.Less clear and also less consistent in its predictive importance is the % of classes positively rated for instructor, which is significant for the retention of Combined and FTT populations: a.There seems to be a significant relationship between retention and being a member of either of the two groups who have 38-49% and 50-65% exposure to classes that were favorably rated for instructor. This is not so for members of groups with higher or lower percentages of exposure. 5.For Sophomores who entered as FTFs their initial Entry-at-Risk and On- Campus Residence in first semester at the university seem to continue to play a role in persistence from sophomore into junior year. Neither of these variables impacted retention of students who entered as FTTs and neither contributed to retention in the combined population. 37 Comparing Combined, FTF and FTT Populations Interpretation Comparing Combined, FTF and FTT Populations Interpretation

5.Variables rejected in models for all populations: % Exposure to Tenured/Tenure-track Faculty Financial aid Minority vs. Non-Minority status Commitment to major had been ruled out in preliminary examinations and was never entered into any of the models. 38 Comparing Combined, FTF and FTT Populations Interpretation Comparing Combined, FTF and FTT Populations Interpretation

DiscussionDiscussion %Q42Coded & %Q41Coded – Represent the Customer Satisfaction hypothesis in this study, and while both of these variables are statistically significant contributors to the prediction of sophomore retention, the observed nature of their relationship to retention is not logically explained. – We conclude that these particular measures may not have been the most appropriate indicators of the type of customer satisfaction / dissatisfaction that drives brand loyalty or brand abandonment for students at this university – A survey study of the sophomores conducted at our university in winter-spring of 2010 ( Rogers,K, May 4,2010) suggests that non-retention at this university is not driven by educational dissatisfaction. 39

Questions & Comments? THANK YOU! 40

ReferencesReferences Langbein, Laura I. &Kevin Snider. “The Impact of Teaching on Retention: Some Quantitative Evidence”. Social Science Quarterly 80.3, Sept Li,Xiaohong ; Li,Qiyu &Caso, Rita. “Looking Backwards for Profiles of Success”. Paper presented at the 2009 TAIR Conference, Lubbock, TX, March Li,Xiaohong ; Li, Jia; Li, Qiyu; Kokatla, Lakshmi & Caso, Rita. “Exploration, Examination, Excavation and Explanation”. Paper presented at the 2009 TAIR Conference, Lubbock, TX, March Rogers, Keri. “Success Advisory Board 2010 Sophomore Survey Study”. Study results presented at the May 4, 2010 meeting of Sam Houston State University Success Advisory Board, Huntsville, TX. Ronco, Sharron,& Cahill, John. "Does it Matter Who’s in the Classroom? Effect of Instructor Type on Student Retention, Achievement & Satisfaction". Paper presented at the 44th Annual Forum of the Association for Institutional Research, Boston, Massachusetts, May ) Suddarth, Betty, ”An Investigation of General Education Requirements in College Curricula”. Research in Higher Education, Vol. 3, No. 3 (1975), pp Springer. Stable URL: Research in Higher Education Tinto, Vincent. “Dropout from Higher Education: A Theoretical Synthesis of Recent Research.” Review of Educational Research, Vol. 45, No. 1 (Winter, 1975), pp American Educational Research Association. Stable URL: 41

Contact Information M. Rita Caso – – Phone: Xiaohong Li – – Phone: Rebecca Bowyer – – Phone: