“No Significant Distance” Between Face to Face and Online Instruction: Evidence from Principles of Economics Brad Humphreys UMBC Department of Economics

Motivation Increasing emphasis on web-based instruction, including on-line courses Explosion of on-line courses in economics across the country (Coates and Humphreys, 2001) Interest in research on the impact of technology on student outcomes

An Experiment Matched face-to-face and fully on-line principles of economics sections offered in Fall 2000 Each section taught be the same instructor Instructors had taught on-line before Offered at three institutions: UMBC, SUNY- Oswego, Christopher Newport University Surveyed students about preparation, background Administered Test of Understanding College Economics (TUCE) as pre- and post-test

Research Hypotheses 1.Students enrolled in on-line courses have the same characteristics as students enrolled in face-to-face sections 2.Students in on-line sections and students in face-to-face sections perform similarly on TUCE

Selection Issues If the characteristics of students in the on-line sections differ from those of students in the classroom sections, this can bias the statistical analysis of the effect of on-line instruction This problem is well-known in the economics literature and several statistical techniques have been developed to deal with it –James Heckman was awarded the Nobel Prize in economics recently for work on this issue

Differences in Students On-line –75% have jobs –Work 35.5 hrs/week –11.5 credits taken –Junior/Senior –30+ more likely –Transfers more likely –Fin Aid less likely –Familiarity Face-to-face –50% have jobs –Work 21 hrs/wk –13.5 credits taken –Freshmen/Sophomore –17-22 more likely

Selection-corrected Results Students enrolled in on-line sections answered between 4 and 6 fewer questions (out of 30) correctly on post-test TUCE Underclassmen in on-line sections answered between 2 and 3 fewer questions correctly Every 4 correct answers on pre-test raises post-test score by 1 correct answer Each 100 additional points on SAT raises score on post-test by.8

“Switching” Results Allows us to assess the impact of self selection on student outcomes –Would students who enrolled in the classroom sections have performed better in either setting or –Did students choose the section that best suited their learning style? Results suggest that the latter is true

Conclusions On-line students performed worse on the TUCE than students enrolled in face-to-face sections Underclassmen preformed worse than upper classmen –Teaching principles of economics to underclassmen on- line is probably a bad idea Better prepared students perform better in on-line classes Students select themselves into the section that best suits their learning style