1. Monroe Community College Summary of Monroe Community College’s Online Student Retention Research Project Dr. Jeffrey P. Bartkovich Marie J. Fetzner Spring 2004

2. Monroe Community College 2 Purpose of Presentation Provide an archived overview of two MCC online student retention/withdrawal studies  The problem, the research, the findings and the applications Demonstrate retention strategies and policy Discuss a research agenda to inform practice  Theory development, defining retention, managing services and student-to-student advice

3. Monroe Community College 3 MCC Overview MCC is part of SUNY  Founded in 1961; 13,326 FTE in 2003 Joined SUNY Learning Network (SLN) in 1997  SLN is totally asynchronous Spring 2004 MCC SLN Summary Data  Headcount = 3,434;  Online FTE = 328.43  Courses = 86; Sections = 121

4. Monroe Community College 4 MCC Online Courses and Sections Fall 1997 to Spring 2004

5. Monroe Community College 5 MCC Online Program Support MCC is western NY SLN faculty training site MCC online faculty supported via The Monroe Model  SLN and local campus participation  Cross divisional membership  Comprehensive on-site support Students supported via SUNY HelpDesk and MCC online and on-site resources

6. Monroe Community College 6 MCC Educational Technology Services (ETS) Organization CIO Communications & Network Services Telecommunication & PC Support Computing ServicesLibrary Services Instructional Technologies Distance Learning Electronic Learning Centers

7. Monroe Community College 7 MCC Online Student Retention Study #1 Characteristics and Attitudes of Non-Retained Online Students Office of Educational Technology Services Monroe Community College Dr. Jeff Bartkovich and Marie Fetzner

8. Monroe Community College 8 The Research The Problem/Research Question Method  Study limitations Descriptive Statistics Correlations  Significant findings Conclusions and Applications to Practice

9. Monroe Community College 9 The Problem Why are certain MCC online students (those with grades of F/W) not succeeding in their online courses?

10. Monroe Community College 10 Online Retention Study Method Archival Data Analysis  Grades and demographics for online courses retrieved from student records database Student Survey  Survey created and administered to sample of students who received an “F” or “W” in an online course for the Fall semesters in 2000, 2001 and 2002

11. Monroe Community College 11 The Survey Basic demographic data Questions derived from  Online Faculty  MCC Student Retention Survey  E-resources

12. Monroe Community College 12 Survey Instrument Total of 45 questions in three areas  Expectations at time of registration  Satisfaction at time of withdrawal  Reasons for withdrawal  Likelihood of enrolling in another online course

13. Monroe Community College 13 Research Limitations Specific  Sample of convenience  Problems with generalizability (n=201)  Difficulty in contacting students who do not successfully complete the course General  Orientation procedures and other online student services evolve over time  Course design and faculty experience improve constantly

14. Monroe Community College 14 Basic Student Demographics OL and Site-based, By Percent Spring2002Fall2002 OLSite-basedOLSite-based Female68557155 Male32452945 Minority20251924 Non-Minority80758176 Less than 2018271530 20-2433 3532 25-2916111511 30 and over33293427

15. Monroe Community College 15 Archival Data Analysis MCC Online vs. On-site Grade Distribution Grade Rates Success Rates in Matched (ftf vs. same section online) Courses

16. Monroe Community College 16 Overall Grade Distribution OL vs. Site-based, By Percent Spring2002Fall2002 GradeOLSite-basedOLSite-based A34.532.834.028.7 B20.526.119.726.2 C10.015.310.615.2 D3.95.03.75.1 F12.98.214.17.4 W16.311.414.510.7

17. Monroe Community College 17 Grade Rates OL vs. Site-based, Fall Semester 1999200020012002 OLN9701,4171,9162473 C and Higher69.7%65.8%65.3%63.6% F/W24.8%26.4%29.7%30.2% MCCN43,77945,23548,28852,428 C and Higher70.3%69.7%70.7% F/W20.0%21.3%20.6%20.5%

18. Monroe Community College 18 Success Rates in Matched Online and Site-based Courses On-campus, n = 16,291 Online, n = 1,719 % C or BetterPercentage Point Difference Online Site-based Total64.269.6-5.4 Full-time Part-time 56.4 72.3 70.3 66.7 -13.9 5.6 1 st time FT 1 st time & at risk 1 st time & not at risk 39.7 47.2 66.3 72.2 66.6 74.0 -32.5 -19.4 -7.7 Under 25 years of age 25 years of age or older 54.5 75.2 67.8 75.7 -13.3 0.5

19. Monroe Community College 19 Success Rates in Matched Online and Site-based Courses On-campus, n=16,291 Online, n = 1,719 % C or BetterPercentage Point Difference OnlineSite-based Part-time72.366.75.6 1 st time, PT & not at risk73.566.47.1 Taking Eng, History, Math, Speech, Comm, Music, HVAC 51.665.3-13.6 Not Taking Eng, History, Math, Speech, Comm, Music, HVAC 74.173.70.4

20. Monroe Community College 20 Retention Sample MCC vs. Retention Sample Gender Age Ethnicity

21. Monroe Community College 21 MCC, MCC SLN and Sample Profile Gender and Age 2002-03MCC ProfileMCC SLN Profile Sample Profile Female Male 54.7% 45.3% 70.8% 29.2% 62.2% 34.8% Under 20 20-24 25-29 30-34 35-44 45-59 60 and over 31.8% 32.2% 11.0% 7.6% 11.2% 6.0% 0.2% 14.3% 36.9% 14.0% 11.6% 16.7% 6.5% 0.0% 14.3% 42.9% 15.7% 8.6% 11.4% 7.1% 0.0%

22. Monroe Community College 22 MCC, MCC SLN and Sample Profile Ethnicity 2002-03MCC ProfileMCC SLN Profile Sample Profile Black Hispanic Amer. Ind. NRA Asian Unknown White 16.2% 5.1% 0.6% 0.5% 3.2% 0.0% 74.4% 11.0% 4.4% 0.4% 0.3% 2.1% 0.0% 81.7% 12.9% 5.5% 0.0% 3.5% 4.5% 73.6%

23. Monroe Community College 23 First-time vs. Experienced Online Student Sample by Gender First-time online students (n = 132; 68%) Experienced online students (n = 62; 32%) ExperiencedFirst-time 82 19 50 43 n = 194 7 Unknown

24. Monroe Community College 24 First-time vs. Experienced Online Student Sample by Ethnicity

25. Monroe Community College 25 Expectations* at Time of Registration % First-Time Students % Experienced Students Can Begin Anytime 41.418.8 Online Easier than On-Campus 40.422.3 Less Homework than On-Campus 40.020.0 Faculty Interaction Required 35.615.0 Need Basic Computer Skills 28.314.1 Participate Independently 18.935.0 *Definitely what I expected + Sort of what I expected

26. Monroe Community College 26 Satisfaction* at Time of Withdrawal % First-Time Students % Experienced Students Own Performance in Course 30.014.0 Technical Help with the Course 28.613.5 Registration/Orientation 26.49.8 SLN in General 23.413.9 Directions Provided by Faculty 22.814.0 Directions to Get Started 10.915.1 *Somewhat satisfied + Extremely satisfied

27. Monroe Community College 27 Explanation for Non-Success in Online Course Top Ten Factors % First-Time Students % Experienced Students Lack of Motivation 46.324.4 Course Taking too Much time 43.018.8 Instructor’s Teaching Style 43.018.4 Too Many Technical Difficulties 41.919.8 Got Behind/Couldn’t Catch Up 41.722.1

28. Monroe Community College 28 Explanation for Non-Success in Online Course Top Ten Factors (Con’t) % First-Time Students % Experienced Students Too Much Reading 41.420.9 Signed Up for too Many Courses/Had to Drop 41.120.4 Course too Unstructured 41.021.1 Not Interested in Subject 40.820.3 Couldn’t Handle Study Plus Other Responsibilities 38.921.6

29. Monroe Community College 29 Number One Reason for Non-Success—Ranked Order 1.Got behind and couldn’t catch up 2.Too much reading 3.Course taking too much time 4.Course too unstructured 5.Too many technical difficulties

30. Monroe Community College 30 Number One Reason for Non-Success—Ranked order 6. Not interested in subject matter 7. Course too difficult 8. Signed up for too many courses 9. Couldn’t handle school with other duties 10. Didn’t know where to go for help

31. Monroe Community College 31 Likelihood to Take Another Online Course Survey Not a chance/not likely It’s possible Somewhat likely/very likely Fall 200031.8%11.6%56.5% Fall 200158.1%12.9%29.0% Fall 200255.9%11.8%32.3% 3 Year Total48.5%12.2%39.3%

32. Monroe Community College 32 Likelihood to Take Another Online Course by Age Age Range Not a chance/not likely It’s possible Somewhat likely/very likely <2043.59%15.9%40.6% 20-2434.4%15.6%50.0% 25-2946.4%14.3%39.3% 30-3464.1%7.7%28.2% 35-4463.6%0%36.4% 45-5950.0%0%50.0% 60 and overN/A

33. Monroe Community College 33 Likelihood to Take Another Online Course by Gender Not a chance/not likely It’s possible Somewhat likely/very likely Female52.4%8.1%39.5% Male40.9%19.7%39.4%

34. Monroe Community College 34 Likelihood to Take Another Online Course by Ethnicity Ethnicity Not a chance/not likely It’s possible Somewhat likely/very likely Black48.0%16.0%48.0% Hispanic45.5%18.2%36.3% Asian49.2% 1.6%49.2 White41.3%11.7%40.0%

35. Monroe Community College 35 General Findings Archival Analysis When compared to site-based courses at MCC Online students are more likely to earn a grade of C or better if they are >25 years of age and are part-time students First-time, full-time online students are least likely earn a grade of C or better

36. Monroe Community College 36 General Findings Survey Analysis Levels of Satisfaction First-time students more satisfied at time of withdrawal in general, and with their own performance Experienced students less satisfied with  Registration procedures  SLN in general  Directions from faculty

37. Monroe Community College 37 General Findings Survey Analysis Expectations First time students more likely to expect  Fluid beginning and end dates  Less homework  Online easier than site-based courses Experienced students more likely to expect  Limited faculty interaction  Independent participation in course

38. Monroe Community College 38 General Findings Survey Analysis Expectations Online format expectations not accurate Level of online interaction with faculty and other students not accurate Technical expectations (needed PC and typing skills) not accurate Percentage of W/F students’ likelihood to take another online course is decreasing Approximately 1/3 are likely or somewhat likely

39. Monroe Community College 39 General Findings Survey Analysis Main reasons for withdrawal Got behind and couldn’t catch up Course had too much reading, took too much time, was too unstructured and too difficult Had too many technical difficulties and didn’t know how to get help Not interested in subject matter and dropped course to cut course load Couldn’t handle school, family, work, etc.

40. Monroe Community College 40 Likelihood to take another online course SLN in general (.604) Interaction with other students (.518) Directions provided by faculty (.491) Directions to get started (.483) Interaction with the faculty (.428) Technical help with course (.408) Significant Positive Correlations (GPA and credit hours not significant with likelihood to take another course) Significant at 0.01 level)

41. Monroe Community College 41 Significant Positive Correlations Lacked PC skills and  Lacked typing skills (.970)  Too many tech difficulties (.741) Lack of access to PC and  Lacked PC skills (.766)  Financial problems (.731)  Too many tech difficulties (.688) Course too difficult/lacked PC skills (.629) (Significant at 0.01 level)

42. Monroe Community College 42 Significant Negative Correlations Likelihood to take another online course Didn’t like the online format (-.569) The course was too unstructured (-.485) Felt too alone, not part of the class (-.435) Didn’t know where to get help (-.324) (Significant at 0.01 level)

43. Monroe Community College 43 Factor Analysis Summary Factor analysis conducted  Data converged into four key groupings in seven iterations  Further discussion on this analysis is described later in this presentation

44. Monroe Community College 44 Advice from the Literature The course design/level of interaction The experience of the faculty The level of technical support A student pre-course orientation Manage student expectations upfront Professional development and training for faculty Standardized course management system

45. Monroe Community College 45 Summary of Key Findings Retention of online (W/F) MCC students is impacted by these characteristics:  First-time (vs. Experienced) online students  Amount of previous higher education credits for full-time online students  Student age (<25)  Lack of PC skills and lack of access to a PC

46. Monroe Community College 46 Research Informing Practice Applications Online student orientation CD project  Development and distribution of the CD  Phone calls made to first-time online students On-site student orientations implemented*  Development of Ten Myths Videostream Expansion of PC access for MCC students  Wireless laptops for check-out in Libraries  Learning Center support for online students *An analysis was conducted on online student performance outcomes for those students who attended on-site MCC orientations

47. Monroe Community College 47 Research Informing Practice Additional materials development  SLN Student Awareness Sheet Retention strategies included in online faculty course development trainings  Emphasis placed on pre-course activities and first week of course interactions Promote self-paced free technology training to MCC students Preliminary discussion of MCC policies  Forwarded to Academic Leadership Council

48. Monroe Community College 48 Next Steps Conduct further analysis to investigate the relationship between online student reasons for non-success, and satisfaction variables for successful online students Investigate students’ technology readiness Continue evaluation of services and activities that were implemented as a result of the retention study

49. Monroe Community College 49 MCC Online Student Retention Study #2 An Analysis of SLN Students’ Performance and Differentiation Office of Institutional Research Monroe Community College Angel Andreu

50. Monroe Community College 50 The Study Research The Problem/Research Question Observations The Data Analysis Conclusions and Policy Implications

51. Monroe Community College 51 The Problem It is observed that the failure and withdrawal (F/W) grades for online students are increasing in number and percentage.

52. Monroe Community College 52 The Observation Percentage of F/W Grades, Fall Semester 1998 1999 2000 2001 2002 Online*19 25 26 30 30 College17 20 21 21 21 *N in 1998 was 409; 2002 was 2,473

53. Monroe Community College 53 The Research Previous research identified characteristics of students at risk of F/W grades in online courses. These were: First-time online students Full-time students with less than 30 earned credits Students under the age of 25 Minority students

54. Monroe Community College 54 The Research The MCC “Andreu” study affirmed these characteristics and looked specifically at: Time of registration Basic academic skills

55. Monroe Community College 55 The Data: Time of Registration Only 7% of the “C or better” students registered during the first week of classes, compared to 16% of the “F/W” students Students who registered during the first week of classes had a 50/50 chance of earning a grade of C or better The odds of getting C or better decreased by 13% for each week closer to the first week of classes

56. Monroe Community College 56 The Data: Age 65% of the F/W grades were among students under the age of 25 Between the ages of 21 and 22, a student had a 50/50 chance of earning a C or better

57. Monroe Community College 57 The Data: Age

58. Monroe Community College 58 The Data: Age and Time of Registration Age and time of registration are each significant contributors to grade performance Their interaction is not significant The best chance of getting a C or better  Register 5 weeks before the start of the semester  Be age 25 or above

59. Monroe Community College 59 The Data: Experience (i.e. number of accumulated credit hours) The rate of F/W grades between first and second year students is significant Experience is not significant when combined with age or time of registration

60. Monroe Community College 60 The Data: Part/Full Time Status Part/Full time status is not significant The interaction of status and age was significant

61. Monroe Community College 61 The Data: Part/Full Time Status

62. Monroe Community College 62 The Data: Basic Academic Skills MCC’s ACCUPLACER tool  The variables used included reading and sentence scores  The metrics are placement scores at the college English level (78 for reading, 83 for sentences)

63. Monroe Community College 63 The Data: Basic Academic Skills For students to have a 50/50 chance of earning a C or better in an online course, their basic academic skills should be at the college English level

64. Monroe Community College 64 What Does the Analysis Show? Factors important to success in online courses: Time of registration The age of the student The academic preparedness of the student

65. Monroe Community College 65 What Does the Analysis Show? Factors in the decision model College is enrollment driven The online program is supporting enrollment growth The demographics of enrollment growth for the College are the same demographics of non-success in online courses

66. Monroe Community College 66 What Does the Analysis Show? Policy Factors Should policy account for motivation At what “odds of failure” do we restrict enrollment Should online growth be allowed to continue to support college enrollment growth What is the balance between student success and online enrollment growth

67. Monroe Community College 67 Policy Development 1 In all cases, a student with an ACCUPLACER score below 78 (reading) and/or 83 (sentence) is prohibited from taking an online course at MCC. This prohibition is lifted at such time that the student successfully completes the Transitional Studies coursework appropriate to address the deficiency. All online sections will be closed to registration one week before the start of class. Qualified students (see definition on next slide) who wish to enroll after this date will be allowed to “green slip” in until the College’s ‘Add’ deadline.

68. Monroe Community College 68 Policy Development 1 Qualified students include mature students with a record of success in college-level coursework. Student must also have satisfactory ACCUPLACER scores to have successfully completed the Transitional Studies coursework appropriate to address any deficiency. Students who have successfully completed an online course previously are also considered qualified. Note that data indicate that students 22 years old or older have a significantly greater chance of success in online coursework.

69. Monroe Community College 69 Policy Development 2 Considerations Forget policy, make it a course prerequisite Must be programmable Cannot rely upon faculty involvement for exceptions due to availability Does resolution of the online problem become a standard

70. Monroe Community College 70 Policy Development 3 Draft X Students in remedial English courses are prohibited from enrolling in online courses All registrations for online courses are closed on the first day of classes

71. Monroe Community College 71 Policy Development 4 Current Policy Registration for online courses closed week before the first day of classes Late entry by permission

72. Monroe Community College 72 Research to Inform Practice: An Online Research Agenda MCC’s Online Research Agenda: Level 1 – Descriptive Basic demographics Basic production Basic enrollment Basic finance

73. Monroe Community College 73 An Online Research Agenda MCC’s Online Research Agenda: Level 2 – Evaluative Basic performance Student and faculty satisfaction Integration with campus IR Modify existing surveys “to fit”

74. Monroe Community College 74 An Online Research Agenda MCC’s Online Research Agenda: Level 3 – Comparative Statewide benchmarks National benchmarks Complete surveys-–be a case study MCC’s Online Research Agenda: Level 4 – Theoretical External – in the literature Internal – on the campus

75. Monroe Community College 75 Research to Inform Practice: SUNY Research Reasons for Taking an OL Course* Reasons for Taking Course Online Conflict with Personal Schedule 49% Family Responsibilities 23% Distance or Transportation 10% Other 10% Course Not Offered on Campus 5% Interest in Technology/Internet 3% *Retained Students

76. Monroe Community College 76 SUNY Research Reasons for Taking an OL Course MCC vs. all other SUNY Students* Primary ReasonMCC* SUNY* Personal Schedule Conflict 49% 31% Family Responsibilities 23% 16% Distance or Transportation 10% 20% Other 10% 12% Not Offered On Campus 5% 15% Interactive Technology 3% 7% *Retained Students

77. Monroe Community College 77 Research to Inform Policy: Theory Development Four major barriers to persistence in distance learning (Garland, 1993) Situational – students’ general environment (social, economic, family) Institutional – college’s programs, policies, and procedures Dispositional – student’s personal background Epistemological – academic environment (course, pedagogy, prerequisites)

78. Monroe Community College 78 Theory Development 2 1.Classified the 22 reasons for non- persistence according to Garland’s four barriers 2.Conducted Principle Components Factor Analysis 3.Analysis and Definition of the Factors

79. Monroe Community College 79 Research to Inform Practice: On-campus Orientation Question Is there value to an on-campus orientation course for online performance?

80. Monroe Community College 80 On-campus Orientation On-Campus Orientation Results Most students felt they had the necessary skills to be successful Skill preparation is not related to performance Students who attended the on-campus program were more likely to earn a C or better C/BetterF/W On-campus Orientation 78%18% Online Orientation only 64%30%

81. Monroe Community College 81 On-campus Orientation To Continue the Course Practice Demonstration Course Structure not Skills Market the grade differential Keep faculty involved in the program to provide “authentic experience”

82. Monroe Community College 82 Research to Inform Practice: Student Advice Question to online students who received F/W grade on their online course: “What advice would you give a fellow student considering registering for an online course?”

83. Monroe Community College 83 Student Advice Results – Top Five Topics Be ready for independent learning. Must be self-motivated. Once you get started, stay on track. Online courses require lots of reading, homework and research. Get ready for the demands. Must develop a schedule to go online, must manage your time. Talk to other people and to other students before you take the class, and during the class

84. Monroe Community College 84 Student Advice Practical Use of Online Student Advice Integrate into brochures, orientation seminars Do mock testimonies for video displays Share with Counseling and Advising staff Share with Faculty

85. Monroe Community College 85 Research to Inform Practice: Managing Services Manage the Expectations  Additional pre-start information sharing  Additional orientations offered Manage the Support Services  Greater integration with support team  Additional points of access to services Manage the Academics  Faculty course design issues and training  Emphasis on multiple interactions

86. Monroe Community College 86 Managing Services Manage the Expectations Welcome letter from Academic Vice President has been expanded Face-to-face student orientations offered* MCC Online Orientation CD distributed Phone calls made to all Fall 2002 online students and 1 st -time Spring 2003 online students *additional analysis conducted

87. Monroe Community College 87 Managing Services Manage the Support Services Student Services web page expansion Revisions made to the service gateway to online learning web page Brochures at Records and Registration and at the Counseling Center

88. Monroe Community College 88 Research to Inform Practice: Managing Services Manage the Academics Retention strategies integrated into faculty trainings Discipline-specific retention studies Discussions on other academic support— online tutoring, writing centers, etc. Pilot test CourseSpace (use of online template for web course enhancement)

89. Monroe Community College 89 Research to Inform Practice: Defining Retention Question: How do you define retention? It depends: Research–two basic presentations  As an enrollment statistic  As a performance indicator Policy  Be consistent, clarify your statistics

90. Monroe Community College 90 Defining Retention and Withdrawal Rates All SLN CoursesFall 2002 Annual 2002-03 First day enrollment18514154 First week enrollment18574225 Census day enrollment17143847 Last day enrollment17163832 Retention as an enrollment statistic

91. Monroe Community College 91 Online Retention Rates Fall 20021 st WeekCensus DayLast Day 1 st Day1.02.94.95 1 st Week.92 Census1.00 All 2002-031 st WeekCensus DayLast Day 1 st Day1.02.93.92 1 st Week.91 Census1.00

92. Monroe Community College 92 Defining Retention and Withdrawal Rates Retention as a performance measure (All SLN Courses) GradeFall 2002 Annual 2002-2003 A to D1,2532,755 F, W, Other4631,077

93. Monroe Community College 93 Withdrawal Rates Fall 2002 F&W Grades Spring 2003 F&W Grades 2002-2003 F&W Grades All MCC20.5%19.6%19.5% Online Only30.0%29.5%28.1%

94. Monroe Community College 94 Next Steps: Things to Consider Services Policies Definition of “Retention” for data collection Replicate studies at other institutions

95. Monroe Community College 95 Contact Information Dr. Jeffrey P. Bartkovich jbartkovich@monroecc.edu jbartkovich@monroecc.edu Marie J. Fetzner mfetzner@monroecc.edu mfetzner@monroecc.edu  MCC Web Page: www.monroecc.eduwww.monroecc.edu  MCC Online Learning page: http://www.monroecc.edu/depts/distlearn/index.htm http://www.monroecc.edu/depts/distlearn/index.htm