Online vs Face to Face: Is There a Significant Difference in Learning Outcomes? Elaine Gerber & AJ Kelton, Montclair State University & Tracy Chu, Brooklyn College, CUNY

Abstract Our aim is to better understand whether and how instructional modality (online vs face-to-face) impacts student learning. More specifically, this presentation compares survey results and final grades from an in-person face-to-face class with those from an online class, where the professor, course, and semester are the same. While much has been written about online learning, it is rare for the same course to be taught by the same professor in the same semester at the same institution. Thus, these circumstances provided a unique opportunity for comparison. Consistent with extant research, our preliminary findings suggest that there is no significant difference in learning outcomes, according to modality. What did appear significant, however, is that the online students worked more hours and experienced more life obstacles. Despite working more hours and having more obstacles, the online students performed as well in the course overall, as students who didn’t experience as many challenges or spend as many hours in paid employment. This suggests that certain attributes of online learners, such as being highly organized, a “self-starter,” or previously performing well in school, may be better predictors of success than the modality used to deliver course content. This research is ongoing in order to gather a larger sample with greater predictive power. Our presentation at the RAUL Showcase 2014 will highlight findings from the first round of data collection (Spring 2013).

A “natural experiment” Research Design A “natural experiment” Same course Same professor Same school Same semester RQ: Is there a significant difference in learning outcomes by modality? Learning outcome = final grade for course COURSE: Anthropology of Multicultural America (ANTH 110) was taught face-to-face and online, during the Spring 2013 semester at Montclair State University (MSU) in New Jersey. The course satisfies a GEN ED requirement on the campus of a large 4-year, mainly undergraduate public institution of higher education, and therefore, attracts students from across the campus.   The course description, objectives, reading list, and full syllabus can be provided……

Same Course?! Similar / shared the same: course objectives, readings enrollment size (35 students each) Instructor, semester, university Differences: “lecture” content nature, amount, and quality of peer-based discussion presence of the instructor amount of writing involved weights attributed to various assignments type of student taking each modality (below) Selection Bias re enrollments / type of students: the online version of the course only became available shortly before the semester was about to begin, and therefore, the students enrolling in it were those, who for one reason or another, enrolled “late” and may have needed a last-minute course to complete their schedule.  

Other limitations? How similar is the course really? (see above) Planning / Intentional course structure for research design vs retro-fitting to work with existing course offerings Intellectual property issues Self-reports, and the problems that go with it Small sample size Student characteristics / attributes

Sample Bias? WE THOUGHT THAT STUDENTS WHO WERE INTERESTED IN THE TOPIC MIGHT HAVE MORE INTERNAL MOTIVATION TO LEARN….

Data Collection Subjects were recruited from the full class roster of all students who were enrolled in ANTH 110-01 (face-to-face) and ANTH 110-03 (online) at Montclair State University, Spring 2013 Students were given 2 points on their final exam for participating; they also had the option to complete a short assignment in lieu of participating, in order to receive the same extra credit.   The professor did not know which students had completed the survey and which had done the optional assignment, only which students were to receive the extra two points on their final exam, until after grades were submitted to the Registrar.  

Data Analysis Data were then exported from the Limesurvey database into SPSS for analysis.  Most questions were close-ended and could therefore be analyzed using this program to measure frequencies and other descriptive statistics.  We used T-tests for correlations and a multiple regression analysis, which I’ll talk more about below. The few open-ended questions were analyzed “by hand,” by reading and re-reading the data in either MS Word or Excel.  There was not sufficient open-ended content to warrant using a qualitative software analysis program, such as The Ethnograph.

Preliminary Findings Still preliminary, ongoing Collecting “round 2” data Spring 2014 Nonetheless…….. What did we find?

Statistical Findings: Only significant difference between the online and F2F samples is that: online students worked more hours, and experienced more obstacles! But, they did just as well in the course overall…

Grades by modality Final Exam Course Grade F2F 85.23 84.5 Online 84.13 85.5 Note:  grades were slightly lower for students who did not participate in the survey. We also see that the greatest number of lowest scores in the course overall, came from the F2F class.   Twelve of the lowest 14 scores (86%) were all from the F2F section.

T-test (variables by modality) Age -- ns But .075, so almost sig Academic Credits - Current Semester   ns But .055, so almost sig

T-test (variables by modality)

Obstacles Are there any obstacles to learning that you faced this semester? (check all that apply): Working too many hours Commute to class and/or parking often caused me to miss class I had family emergencies I had personal issues I had housing problems I was too busy Access to computers & Internet were not reliable Course content too intellectually difficult or presented too poorly to learn Other (please explain):  ______________________________

Predictors of Learning Outcome When looking at the entire sample overall (n=41) , the only variables that correlate with FINAL GRADE are RACE and GPA. When looking at each modality separately, in the online class (n=17), RACE was correlated to FINAL GRADE, but GPA was not. However in the F2F class (n=24), GPA was correlated to Final Grade, and RACE was not.

When GPA is controlled for, race is no longer a significant predictor of course grade (Model 1 & Model 2). Moreover, modality does NOT predict course grade, even when controlling for race and GPA (Model 3). Controlling for modality and race, only significant predictor of course grade is previous GPA (Model 3).

Discussion What is going on here?

Other Predictors of Outcome? Do you consider yourself… (select all that apply): very organized highly motivated to learn an independent worker able to multi-task good with time management focused & goal-oriented comfortable with technology other: _________________

Implications …And, thanks! Not all students are the same, no universal student Implications for academic advising: Retention Time to graduation Questions? Suggestions? …And, thanks!