Inside, Outside, Upside Down: New Directions in Teaching and Learning An Interdisciplinary Assessment of Online Learning in Higher Education Lena Paulo Kushnir, Ph.D., Kenneth Berry, MSc., Don Boyes, Ph.D.,

 Advancements in technology and innovations in education allow universities to entertain new ways of teaching and learning  Use of technology in education enables such innovation  use technology in ways that brings students together, empowers and engages them  Use of podcasts to deliver lectures is increasingly popular and often viewed as an effective way to enhance undergraduate education.  Since 2005, there has been a tremendous increase in the use of podcasts. Background: …What are some of the “changing needs” of today’s students? …and who are today’s students?

 Online learning initiatives are popular and help meet a variety of goals  But what’s the evidence that online tools and teaching strategies supported by such tools are effective for positive learning outcomes?  In a study that we describe today, we compared online learning to face-to-face learning  What does the literature say?  Generally positive, and for the most part, no significant differences between online vs face-to-face learning  though online is usually a bit lower (though not significant) and very few studies that show online results in significantly higher learning outcomes Background continued:

Objectives of our research:  To evaluate the impact of various online tools and teaching strategies on student learning outcomes, student engagement, satisfaction of learning experience, and workload and overload. …and are there any differences between online courses versus face-to-face courses? Or are students having similar experiences? What are their experiences?

172 students in 2 courses, 2 2 institutions Our Study: Intro to Geographic Information Systems (GIS) Intro to Psychology 29 online (2 nd yr Geography course) (2nd yr course) What parts of the course contributed most to their learning? How they used certain tools in the course to help them learn? What was the workload like and did they feel overloaded? If they could do it again, would they do it online or F2F? experience with course specific aspects that differed for GGR and PSY Learning outcome measures, plus… no significant differences first year average cumulative average high school entrance average 31 face-to-face 15 online 97 face-to-face

 Teaching tools and strategies  Learning modules What we did and how we did it:

Learning modules

 Teaching tools and strategies  Learning modules  Learning outcomes What we did and how we did it:

Learning outcomes

 Teaching tools and strategies  Learning modules  Learning outcomes  Lecture Podcasts What we did and how we did it:

Lecture podcasts

scriptswin.rcat.utoronto.ca/GGR_Online/5_1/index.htm

Lecture slides (PDF)

 Teaching tools and strategies  Learning modules  Learning outcomes  Lecture Podcasts  Online readings What we did and how we did it:

Online readings

 Teaching tools and strategies  Learning modules  Learning outcomes  Lecture Podcasts  Online readings  “Demonstration” videos  “Presence” videos What we did and how we did it:

Demonstration Videos

 Teaching tools and strategies  Learning modules  Learning outcomes  Lecture Podcasts  Online readings  “Demonstration” videos  “Presence” videos  Online assignments using Blackboard Test tool What we did and how we did it:

Assignments using Blackboard test tool

 Teaching tools and strategies  Learning modules  Learning outcomes  Lecture Podcasts  Online readings  “Demonstration” videos  “Presence” videos  Online assignments using Blackboard Test tool  Quizzes What we did and how we did it:

Self-assessment quizzes

 Teaching tools and strategies  Learning modules  Learning outcomes  Lecture Podcasts  Online readings  “Demonstration” videos  “Presence” videos  Online assignments using Blackboard Test tool  Quizzes  Online discussions and Peer Instruction  Rapid feedback  Strategies affect tools, and vice versa What we did and how we did it:

What’s the evidence:  Measures of how “things were going”…any differences between the groups (online versus face-to-face? Or are they having similar experiences?  Half-way checking-in survey (formative type feedback – GGR and PSY)  End of term survey (summative type feedback – only PSY)  Learning outcome measures (only PSY)  6 Quizzes (10%),  2 Term tests (40%),  1 Final Exam (30%),  1 Assignment (10%),  Peer Instruction and Participation (10%),  overall Final Course Grade

What aspects of the course are most successfully contributing to your learning? (Choose all that apply.) GGR PSY

On a week-by-week basis, what did you find took the most amount of time in the course? GGRPSY

In what part of the course, or what course activities did you feel most engaged? GGR PSY

If you could take this course again, which would you pick, online or face-to-face? GGR PSY

Why? GGR PSY 0 Online/f2f f2f/f2f Online/online f2f/online

Generally speaking, how are things going with the course and do you have anything else you’d like to share? (e.g., what surprised you most, what you particularly liked or didn’t like, what you think should be changed.) GGR PSY Online Face-to-Face Online Face-to-Face

Is English your first language (your native language)? GGR PSY = 54% ESL students f2f = 26% ESL students online = 47% ESL students online = 31% ESL students f2f

How are you using the Lecture Podcasts / How would you use Lecture Podcasts if they were available? GGR (F2F)PSY (F2F)

Did you feel isolated in the course? GGRPSY (ONLINE)

You indicated that you would have liked more interactions with your classmates. What sort of interactions would you have wanted? GGRPSY (ONLINE)

Learning outcome measures: GGR PSY  6 Quizzes (10%),  2 Term tests (40%), 54% ON vs 55% f2f, and 58% ON vs 62% f2f  1 Final Exam (30%), 55% ON vs 59% f2f  1 Assignment (10%), 87% ON vs 92% f2f  Peer Instruction and Participation (10%),  Overall Final Course Grade * 2 significantly higher for online, all others slightly lower * significantly higher for f2f * NOT significantly higher for f2f, but close!!

Peer Instruction and Participation online PIf2f PI Mean Final Course raw scores online FINAL GRADEf2f FINAL GRADE Mean Interesting given that students reported they were most engaged in Peer Instruction!

Thank You! Kenneth Berry, MSc., Don Boyes, Ph.D., Lena Paulo Kushnir, Ph.D., Visit for Lena’s top 10 tips for online Teaching & Learninghttp://uoft.me/top10tips