In Search of a Grail: Identifying Best Practices for Attracting and Retaining Students Don Hossler Mary Ziskin ( ) Jacob P.K. Gross Indiana University – Project on Academic Success Lauck Parke Vice President for Undergraduate Studies University of Vermont

2 Overview What student retention efforts seem to work (or not)?What student retention efforts seem to work (or not)? But why?But why? Where does this leave us?Where does this leave us? What are we learning from pilot studies on student retention?What are we learning from pilot studies on student retention?

3 A Sampling of Retention Initiatives Supplemental instructionSupplemental instruction Major/Career counseling programsMajor/Career counseling programs Learning communitiesLearning communities First-year experience seminarsFirst-year experience seminars

4 Supplemental Instruction Participants more than twice as likely to persist year-to-year as nonparticipantsParticipants more than twice as likely to persist year-to-year as nonparticipants Undergraduates were trained in cooperative learning strategies & stuck to this approachUndergraduates were trained in cooperative learning strategies & stuck to this approach They had a training manualThey had a training manual In other words, the institution devoted time & resources to training – it continues to do soIn other words, the institution devoted time & resources to training – it continues to do so

5 Major/Career Counseling Program for undeclared students at a church- affiliated residential institutionProgram for undeclared students at a church- affiliated residential institution Participants 6 times more likely to persist to degreeParticipants 6 times more likely to persist to degree Strong institutional focus – campus has a director for this programStrong institutional focus – campus has a director for this program Could success be linked to faith-based approach?Could success be linked to faith-based approach? Program is now required & we are continuing study of the effectsProgram is now required & we are continuing study of the effects Program for undecided students at a commuter campus run by career centerProgram for undecided students at a commuter campus run by career center Participants were over 8 times more likely to persist year-to-year as non participants Participation was strongly encouraged by campus Program connects students to staff advisors, faculty mentors, and potential employers

6 Learning Communities on a Commuter Campus Had positive impact at a nonselective institution on first-to-second-semester persistenceHad positive impact at a nonselective institution on first-to-second-semester persistence No effects on year-to-year persistenceNo effects on year-to-year persistence We know little of how the program was implemented and sustained over timeWe know little of how the program was implemented and sustained over time Has this been a success?Has this been a success?

7 First-Year Seminars at a Community College Program implemented at three separate campuses within one regionProgram implemented at three separate campuses within one region IPAS undertook program evaluation studies using both qualitative and quantitative methodsIPAS undertook program evaluation studies using both qualitative and quantitative methods No evidence of program impact from these studiesNo evidence of program impact from these studies Little training of or coordination among faculty instructorsLittle training of or coordination among faculty instructors No additional institutional resources devoted to programNo additional institutional resources devoted to program Why would we expect this to work?Why would we expect this to work?

8 Some Caveats Student self-selection into programs likely contributes to positive findingsStudent self-selection into programs likely contributes to positive findings The time commitments of participating campuses & their lack of experience with doing evaluation research resulted in their not collecting key data elements needed for rigorous assessmentThe time commitments of participating campuses & their lack of experience with doing evaluation research resulted in their not collecting key data elements needed for rigorous assessment

9 Where Does This Leave Us? We remain interested in better understanding how campuses can intervene to positively influence persistence.We remain interested in better understanding how campuses can intervene to positively influence persistence. Because the commitments of campuses to the way they deliver programmatic initiatives is so variable, we are also interested in how campuses organize themselves to address issues of student persistence. We think this might be an important factor.Because the commitments of campuses to the way they deliver programmatic initiatives is so variable, we are also interested in how campuses organize themselves to address issues of student persistence. We think this might be an important factor.

10 And…We Are Trying To Do This! To understand how campuses can adopt policies that enhance persistenceTo understand how campuses can adopt policies that enhance persistence We next report on two pilot study efforts we have underway, both funded by the College Board.We next report on two pilot study efforts we have underway, both funded by the College Board. One is a student survey that helps campuses identify the policy levers they can use to improve persistence.One is a student survey that helps campuses identify the policy levers they can use to improve persistence. The other survey looks at how institutions organize themselves to enhance persistence & how this might differentially affect persistence rates.The other survey looks at how institutions organize themselves to enhance persistence & how this might differentially affect persistence rates.

11 College Board Pilot Study on Student Retention A survey of first-time, full-time, first-year students at 8 four-year institutionsA survey of first-time, full-time, first-year students at 8 four-year institutions Students surveyed at the end of their first year (spring 2006)Students surveyed at the end of their first year (spring 2006) Web-based instrument, orWeb-based instrument, or In-class paper-and-pencil administrationIn-class paper-and-pencil administration Response rates varied widely from under 10% to over 35%Response rates varied widely from under 10% to over 35% Follow-up data collected from institutions to show enrollment in fall 2006Follow-up data collected from institutions to show enrollment in fall 2006 Allows us to look at persistenceAllows us to look at persistence

12 A few examples of survey questions How certain are you that you have received useful academic advising at this college?How certain are you that you have received useful academic advising at this college? How often have you participated in classroom discussions… that included contributions from students with diverse backgrounds?How often have you participated in classroom discussions… that included contributions from students with diverse backgrounds? How certain are you that you've taken advantage of all federal and state financial aid programs you are eligible for?How certain are you that you've taken advantage of all federal and state financial aid programs you are eligible for? How often have you been unable to register for a course that you needed…?How often have you been unable to register for a course that you needed…? How many times per semester have you received prompt feedback…from Instructors?How many times per semester have you received prompt feedback…from Instructors? How often have you felt out of the loop with regard to campus policies and procedures?How often have you felt out of the loop with regard to campus policies and procedures?

13 Participating Campuses Campuses includedCampuses included 3 commuter campuses3 commuter campuses 2 small private liberal arts colleges2 small private liberal arts colleges 2 residential public universities2 residential public universities 1 public HBCU1 public HBCU Institutions in four statesInstitutions in four states

14 Institution-Specific Analyses Descriptive informationDescriptive information Participation in student programsParticipation in student programs Classroom experiencesClassroom experiences Time diary itemsTime diary items SatisfactionSatisfaction Inferential analysesInferential analyses Explore factors associated with intent to persistExplore factors associated with intent to persist Merge data with SAT Questionnaire program and fall 2006 enrollment data to explore covariates of persistenceMerge data with SAT Questionnaire program and fall 2006 enrollment data to explore covariates of persistence

15 Preliminary Results from Residential Campuses A high proportion of students (87-92%) intend to persistA high proportion of students (87-92%) intend to persist Factors capturing aspects of academic engagement emerge on one campusFactors capturing aspects of academic engagement emerge on one campus Campus 2: High Academic Engagement (α=.629 )Campus 2: High Academic Engagement (α=.629 ) Campus 2: Use of Public Space for Learning (α=.607)Campus 2: Use of Public Space for Learning (α=.607) Logistic regressions showed that a traditional persistence model enhanced the prediction of which students did not intend to persistLogistic regressions showed that a traditional persistence model enhanced the prediction of which students did not intend to persist Variables that contribute significantly to intent to persistVariables that contribute significantly to intent to persist Campus 1: development of friendship networks, class attendance, and positive perceptions about placement practicesCampus 1: development of friendship networks, class attendance, and positive perceptions about placement practices Campus 2: high combined SAT scoreCampus 2: high combined SAT score Variables that detract significantly from respondents’ intent to persistVariables that detract significantly from respondents’ intent to persist Campus 2: distance of residence from campus, time spent preparing for classCampus 2: distance of residence from campus, time spent preparing for class

16 Results from Commuter Campuses A high proportion ( %) intend to persistA high proportion ( %) intend to persist Logistic regressions showed that a scaled-down traditional persistence model enhanced the prediction of which students did not intend to persistLogistic regressions showed that a scaled-down traditional persistence model enhanced the prediction of which students did not intend to persist Campus 1: 71.4% correctly predictedCampus 1: 71.4% correctly predicted Campus 2: 70.0% correctly predictedCampus 2: 70.0% correctly predicted However…However… Overall variance explained by the models was relatively low ( %), though comparable to other research on persistenceOverall variance explained by the models was relatively low ( %), though comparable to other research on persistence Academic engagement variables included in the models did not show a significant effectAcademic engagement variables included in the models did not show a significant effect Variables that contribute significantly to intent to persistVariables that contribute significantly to intent to persist Campus 1: development of friendship networksCampus 1: development of friendship networks Campus 2: certainty of being able to pay for collegeCampus 2: certainty of being able to pay for college

17 Factor Analysis FactorVariablesAlphas Perception of openness Ratings of social & academic climate on campus.70 to.89 Interaction w/ students Support, advice, and academic assistance from peers.68 to.87 Interaction w/ faculty Support, advice, assistance, and feedback from instructors.73 to.85 Satisfaction Satisfaction ratings on several aspects of college experience: quality of teaching, support for students, financial aid, overall educational experience.85 to.90 Certainty of college choice Ratings of institutional quality, advising practices, and own choice of college.70 to.80

18 Cross-Case Findings Differences across campuses are evidentDifferences across campuses are evident Robust factors emergeRobust factors emerge Early analyses show how policies, practices and environment play into intent to persist, institution-by-institutionEarly analyses show how policies, practices and environment play into intent to persist, institution-by-institution

19 Lessons Learned: Pilot Year-1 Small residential campuses have the highest response rates, commuter campuses the lowest.Small residential campuses have the highest response rates, commuter campuses the lowest. Paper-and-pencil administration on commuter campusesPaper-and-pencil administration on commuter campuses Increased costIncreased cost Much higher response rateMuch higher response rate More complicated to administer, but may also be a good indicator of…how serious campuses are about improving student persistenceMore complicated to administer, but may also be a good indicator of…how serious campuses are about improving student persistence Timing: We hope earlier administrations will improve response rates for residential campuses that use a Web-based survey.Timing: We hope earlier administrations will improve response rates for residential campuses that use a Web-based survey. Studying and improving student persistence is difficult. It takes institutional commitment.Studying and improving student persistence is difficult. It takes institutional commitment.

20 Next Steps We will merge fall enrollment records to see who actually returned and then re-run our current set of analyses.We will merge fall enrollment records to see who actually returned and then re-run our current set of analyses. Then we will also merge financial aid data, and SAT questionnaire data to see how these data affect our results.Then we will also merge financial aid data, and SAT questionnaire data to see how these data affect our results. We will readminister the survey in the winter of 2007.We will readminister the survey in the winter of 2007.

21 Institutional Survey Sneak Preview Survey of four-year institutions in California, Georgia, Indiana, New York, and TexasSurvey of four-year institutions in California, Georgia, Indiana, New York, and Texas Web-based administration, summer 2006Web-based administration, summer % response rate32.8% response rate Preliminary findingsPreliminary findings 57.1 % of responding institutions have a retention coordinator57.1 % of responding institutions have a retention coordinator 97.3% of institutions analyze retention data annually97.3% of institutions analyze retention data annually Annual analyses, broken out by race/ethnicity, 87.8%Annual analyses, broken out by race/ethnicity, 87.8% Annual analyses, broken out by major, 70.8%Annual analyses, broken out by major, 70.8%

22 Institutional Survey Preliminary Findings (Continued) 43.5% report having semester-long orientation programs43.5% report having semester-long orientation programs 60.5% reported average class size for first-year students at 30 or lower60.5% reported average class size for first-year students at 30 or lower 82.9% require students to meet with advisors each term82.9% require students to meet with advisors each term 44.2% of retention coordinators rated the availability of academic support at their institutions as higher than at similar institutions44.2% of retention coordinators rated the availability of academic support at their institutions as higher than at similar institutions

23 Final Thoughts Both surveys are works in progress, but show potentialBoth surveys are works in progress, but show potential We will refine both instruments and re-test them early second semesterWe will refine both instruments and re-test them early second semester We hope to shed light on how institutions organize themselves and what they can do to enhance student persistenceWe hope to shed light on how institutions organize themselves and what they can do to enhance student persistence

24 Contact Us Indiana University Project on Academic Success

Appended Slides

26 Data & Case Descriptives A Public Residential UniversityA Public Residential University Public, residential, research-extensive universityPublic, residential, research-extensive university Approximately 9,000 undergraduatesApproximately 9,000 undergraduates 92% White; next highest group: APA (2.5%)92% White; next highest group: APA (2.5%) 49% expressed confidence that their families would be able to pay for college49% expressed confidence that their families would be able to pay for college 87% reported an intent to persist87% reported an intent to persist Data for this institutionData for this institution Response rate 22%Response rate 22% Men underrepresented among respondents (46% of population; 27% of respondents)Men underrepresented among respondents (46% of population; 27% of respondents)

27 A Public Residential University

28 Findings in 4 cases Campus A (Nagelkerke=.245)Campus A (Nagelkerke=.245) (+) Friendships/Social network (p<.01)(+) Friendships/Social network (p<.01) (+) Positive perceptions of English placement (p<.01)(+) Positive perceptions of English placement (p<.01) (-) Missing class (p<.05)(-) Missing class (p<.05) Campus B (Nagelkerke=.342)Campus B (Nagelkerke=.342) (+) SAT (p<.05)(+) SAT (p<.05) (-) Distance (p<.05)(-) Distance (p<.05) Campus C (Nagelkerke=.320)Campus C (Nagelkerke=.320) (+) Certainty of major (p<.05)(+) Certainty of major (p<.05) (-) Discussions with peers (p<.05 )(-) Discussions with peers (p<.05 ) Campus D (Nagelkerke=.209)Campus D (Nagelkerke=.209) (+) Friendships/Social network (p<.05)(+) Friendships/Social network (p<.05) (+) Feedback from instructors (p<.05)(+) Feedback from instructors (p<.05)