Evaluation of the implementation of blended learning models in large enrollment courses EDUCAUSE – 2011 Norma I. Scagnoli, PhD Jill Moore-Reynen College of Business – University of Illinois at U-C

Abstract & Agenda This presentation introduces the result of an evaluation study done on the design and implementation of blended learning models in large enrollment ( +500 students) courses. Agenda Design for large courses From traditional to blended Evaluation questions User perceptions Research information Strengths of both cases Cases (I) and (II) Recommendations

Designing BL for small towns Large enrollment courses > 600 students Challenge: How to establish a presence as the instructor Current design of large courses (lecture only) Relevant issues: One small change means a lot in large numbers Change in schedule? Or change in activities? Video recording, a need? Blend of activities, what will be done online and what will be done face to face?

Reasons and RQs Reason for evaluation: To determine the direction for the development of blended learning for large enrollment business courses Questions that guided the data collection and analysis How does the blended learning format change a traditional lecture course in a large class? How is that change perceived by students and faculty? What are the strengths of these two models for teaching large enrollment groups? Which of the two models, or what combination of the two, would present a better alternative for implementing a blended learning format in large enrollment courses?

Student population & Data collection Course Intro to Finance Intro to Mktg Students 635 605 Level So 21%; Jr 45%; Sr. 32% Jr 61%; Sr. 35% Major Bus: 38% Bus 55% Data collection FIN MKTG Surveys 3 4 Focus Group 2 Interviews 1 Observation

Instructional design model (I) Online textbook resources Video (micro) lectures (10) per week) Organized by module Screen demos Centralized conversations

Instructional design model (II) Video instructions Organized by module News quizzes Social bookmarking Wiki for paper (extra credit)

From traditional to blended RQ: How does blended learning format change a traditional lecture course in a large class? From traditional to blended Stages Traditionally BL -FIN BL-MKTG F2F meetings 3hrs/wk lecture 1 ½ h/wk face-to-face Weekly Video Lectures No change Few mini lectures Tracking / feedback No formative tracking Weekly tracking Lecture Focused on topic Focused on student questions/comments Student engagement Contact instructor or TA in office hrs, or email Conversation among themselves and with instructor/TA via discussion board Review=own notes Review Videos Online activities Reading before class Read & Quiz before class / Post questions Read & Quiz before class Instructor engagement Direct contact with Teaching Assts More aware about issues because of discussion board Course materials Textbook, course pack & clickers Textbook, publisher’s online materials, other online resources, use of other tools

How is BL perceived by students? RQ: How is that change perceived? How is BL perceived by students? YES YES YES YES NO NO NO NO

How is BL perceived by students? RQ: How is that change perceived? How is BL perceived by students? LIKE DISLIKE Convenience/flexibility Quizzes help prepare for f2f class/track progress Access to materials Instructional design and organization of site Integration with f2f activities Repetition Too many videos/too long Work overload Forced to develop self discipline and time management skills

How is BL perceived by students? RQ: How is that change perceived? How is BL perceived by students? % of Comments About Positive Negative Reasons FIN 40% Quizzes 63% 37% Helped prepare for class or test MKT 5% 92% 8% 30% Format 33% 67% Related to video or to face time 11% 50% 27% Video 36% 64% Too long, too many, repetition 3% 12% 88% 18% Course overall 98% 2% 77% 23% 14% Traditional lecture Preference for lecture in class, or interaction 1%

Students quotes about new format RQ: How is that change perceived? Students quotes about new format About the use of online videos “The (class) offers a variety of ways to recap the lecture” “The videos allow me to stop, take notes and continue…” “The videos give us an idea of what the professor wants us to get out of each week’s lesson” “ I learn more in other classes because I am being taught by someone, in this class it is (pretty much) me teaching myself” “not worth the money we pay, the teacher should teach and not make movies” About the online activities “Online activities force me to stay involved” “Pre-class activities make me read” “I like how we can come together and go over everything once a week yet be able to work at our own pace the rest of the week!”

Strengths of two models RQ: Which are the strengths of these two models for teaching large enrollment groups?? Strengths of two models Element… Strengths…. Face time Affective / personal connection to instructor / TA /class Provides instructors’ personal insights on a topic Online activities Encourages students to develop time management and self learning strategies Provides better tracking of student progress Students come to class more prepared to engage in lecture or discussion Interaction Discussion board provides good alternative ways of communication among students and with TAs or instructor Discussion board works as a ‘personalized’ FAQ for the class because it pertains only to this class (cannot be imported from previous courses) Instructor participation via video or audio Good way to introduce a new idea that will follow up in class, or to review content Provide instructor insight on a topic Enhance teaching presence Free class time for other activities

From our learning, we suggest… RQ: Which of the two models seems a better alternative for implementing a blended learning format in large enrollment courses? From our learning, we suggest… For… Strategies Face time Plan class time to introduce what will be reinforced in online activities; or to reinforce what they did online Share the online environment on screen, and provide general feedback in class Make the connection between activities in the online and face-to-face environments Avoid duplication by lecturing the same in two formats Focus on interaction

From our learning, we suggest… RQ: Which of the two models seems a better alternative for implementing a blended learning format in large enrollment courses? From our learning, we suggest… For… Strategies Engagement Include pre or post quizzes but not only on textbook content. Use them as an opportunity to help students think and find insights in other materials Use tracking information as an opportunity to reach laggards Use quiz or discussion board results to customize lecture, focusing on flaws or clarifying students’ comments

From our learning, we suggest… RQ: Which of the two models seems a better alternative for implementing a blended learning format in large enrollment courses? From our learning, we suggest… For… Strategies Interaction Design a communication system for different audiences so that it is easier to manage (student lounge, team forums, class forums) Device a plan to gear students towards community development (centralizing class communication as opposed to using email) Use as opportunity for students to ask questions that can be addressed in face time (i.e. “Questions for Town Meeting”)

From our learning, we suggest… RQ: Which of the two models seems a better alternative for implementing a blended learning format in large enrollment courses? From our learning, we suggest… For… Strategies Enhancing Instructor’s connection Use video or audio as opportunity to clarify or expand on a subject (i.e.no time to deal with it in class), or as feedback. Use short videos (micro lectures) or audio to focus on a particular issue. Don’t repeat yourself in the class lecture. If there are weekly or frequent videos, make clear what are the goals of having videos.

What direction do we follow? We are developing courses following an instructional design model for large enrollment courses that encourages: Pre or post class assignments Some short (micro) lectures in video or audio (produced by instructor or with guest speakers) Further use of online resources from different sources Spaces for communication and discussion inside and/or outside LMS Creativity and experimentation with new tools Rethinking how f2f time is spent – change in class activities and/or reduction in class time Emphasis on formative vs. summative assessment

Evaluation of the implementation of blended learning models in large enrollment courses Norma I. Scagnoli scagnoli@illinois.edu Jill Moore-Reynen j-moore6@illinois.edu