1. A New Use for Multimedia Learning in Introductory Physics Gary Gladding University of Illinois July 22, 2008 AAPT Meeting Edmonton, Alberta

2. Introductory Physics at Illinois About 10 years ago, we began a major reform of the way we teach introductory physics. Big Idea: Integrate all aspects of a course using interactive engagement methods based on physics education research in a team teaching environment. –For more information on the reform, come to Tim Stelzer’s talk this afternoon at 3:15 in this same room ! Instructional Cycle: Textbook Lecture Peer Instruction JiTT Online Homework Interactive Examples Discussion Section Collaborative Group Learning How Important is this Component in Helping You Understand the Material? Not Very Useful Useful Very Useful

3. What’s Wrong With This Picture? Textbook Lecture Online Homework Discussion Section Peer Instruction JiTT Interactive Examples Collaborative Group Learning How Important is this Component in Helping You Understand the Material? Not Very Useful Useful Very Useful Students Do Not Value the Text They don’t read the text either: 70% of students say that they “rarely” or “never” read the text before class. The Initial Exposure to Material occurs in Lecture Encourages Traditional “Teaching by Telling” Lecture Activity Lectures can be more effective if students are prepared Less time “telling” and more time “interacting” JiTT informs design of peer instruction questions

4. What to Do?? Why? The Big Idea: Provide initial exposure to content through a web-based “multimedia learning activity” prior to lecture More interactive “Lecture” (students better prepared) Multimedia Learning Works (research base) Prelecture activity is logged (improves compliance) (data-based development)

5. Multimedia Learning What is Multimedia Learning? Learning from Dual Channel messages Representations: Words and Pictures Modalities: Auditory and Visual What is Underlying Cognitive Theory? Dual Channel (independent) Limited Capacity (working memory) Active Processing (coordinates information to create mental models) What is the Empirical Basis? Psychology Experiments Instructional Message (brief explanation (500 words) of causal chain) examples: lightning, brakes, and pumps… how do they work? Two presentations designed to focus on some principle example: words only vs words plus pictures (multimedia principle) Post-Tests: Retention: “write an explanation for how brakes work, etc..” Transfer: “why do brakes get hot?” “what could be done to make the brakes stop the car more quickly?”

6. 6/9 2/2 6/8 11/11 4/42/2 9/9 5/5 8/8 10/11 4/42/2 Empirical Results Effect Size: Principles: “Students learn better:” Multimedia: from words & pictures than from words alone Contiguity: when words & pictures are placed near each other (in space or time) Coherence: when extraneous material is excluded rather than included Modality: from animation and narration rather than animation and on-screen text Redundancy: from animation & narration rather than animation, narration, & on-screen text. Effect Size ~ 1 is Large !

7. Our Implementation Multimedia Learning Modules (MLMs) Module contains essential content of one lecture (~ 15 minutes) Module ≈ 10 “slides” “Slide” = Flash movie Narration plus Animations Student controls progress of movie (play, pause, repeat, ff, etc..) Questions: occasionally (2-3 times/module) student is presented with a question Student answers correctly: given explanation and can proceed Student answers incorrectly: given feedback and a related question Once related question answered correctly, original question is represented. Design of modules guided by priciples of multimedia learning Animations coordinated with Narration No extraneous visual/auditory material Minimal text on slide DEMO

8. Initial Experiment “Clinical” test (limited statistics but controlled environment) Population: Students near end of semester course in calculus-based mechanics Content: Four initial units in E&M course Coulomb’s Law Electric Fields Electric Flux Gauss’ Law Treatment Groups: (45 students completed all segments) Multimedia Learning Module (MLM): 16 students Scripts (from MLM): 13 students Text (standard): 16 students Protocol: Students do materials and then take test ~45 minutes per unit (study + ~16 question test) Students return two weeks later to take retention test (37 questions)

9. Results Raw Performance Data: Combined (Units 1-4) Retention MLM (N=16) 74.7 ± 2.669.7 ± 4.0 Script (N=13) 71.7 ± 3.065.1 ± 3.7 Text (N=16) 63.2 ± 3.156.8 ± 4.1

10. Effect Size Analysis Correlate with Exam Scores from Mechanics Course MLMScript Text ES = 0.74 p < 0.01 ES = 0.36 p = 0.06 Script ES = 0.46 p = 0.03 Combined (Units 1 – 4) MLMScript Text ES = 0.67 p < 0.01 ES = 0.31 p = 0.16 Script ES = 0.45 p = 0.04 Retention Paper submitted to AmJournPhys -- now available at: http://arxiv.org/abs/0806.0405

11. The REAL WORLD Clinical test looks good! BUT What about a real course?? PHYS 212 Before SPRING 2008 MLMs implemented as “Prelectures” (initial exposure to material) JiTT Preflights (few read text) LECTURE 75 minutes Narrative + Peer Instruction CHANGES PHYS 212 SPRING 2008 LECTURE 50 minutes Highly Interactive (8 – 10 clicker questions) JiTT Preflights (formative assessment)

12. Preflight Assessment Repeated Preflight Questions allow for assessment Spring06-07 Spring08 Spring08 vs Spring06-07 Spring07 Spring06 Spring07 vs Spring06 50 questions (77%) show improvement 27 questions (42%) show > 2  improvement 7 questions (11%) show > 2  decrease

13. Prelecture Participation We see overall impovement in Preflight performance (as expected), but how many students are really playing the game?? WHY?? Student logs indicate some students are “clicking” through prelecture without viewing movies. Percentage of students who do this increases as semester goes on… Prelecture Due one hour before Midterm Demands on students increase with time Early Deadline (6pm before morning of lecture) Animation access decreased with time We can (and will) fix these

14. Prelecture Analysis Analysis in Progress: Compare Participant/Non-Participant/Previous Performance on each question for 3 groups of different “ability” (i.e., grades of A, B, or C in previous mechanics course. Some Examples AB C Improvement only for Better Students AB C No Improvement, Confusion for Poorer Students who Participate Improvement for all Participants participants previous semester non-participants AB C

15. Exam Analysis We have modified the lecture experience No changes have been made to highly valued homework & discussion components Do we see any improvement in exam performance?? Hour Exam 1: Repeat exam from Spring 1998 = 73.8 ± 0.8% = 76.8 ± 0.7% Effect Size = 0.21 p =.002 Low expectations: Time (Prelecture+Lecture) ~ 30% Time (Homework+Discussion) Average Score Spring 1998 Average Score Spring 2008

16. Hour Exam Analysis Hour Exam 2: Repeat exam from Spring 2001 = 76.1 ± 0.9% = 79.4 ± 0.8% Effect Size = 0.22 p =.003 Hour Exam 3: Repeat exam from Spring 2002 = 70.9 ± 0.7% = 71.3 ± 0.7% Effect Size = 0.05 p =.36 interesting 70.4 72.9 0.17 0.006

17. Final Exam Analysis interesting 70.4 72.9 0.17 0.006 Final Exam A: Repeat exam from Spring 2007 = 70.9 ± 1.1% = 73.4 ± 1.1% Final Exam B: Repeat exam from Spring 2007 = 73.0 ± 1.1% = 71.3 ± 1.2% intentional Methods of combining currently under study, but it looks like a small positive effect

18. Exam Analysis Conclusions interesting 70.4 72.9 0.17 0.006 intentional Old Lecture (75 min) Prelecture + New Lecture (50 min) Small Improvement (ES ~ 0.2) in Exam Performance BUT Not everyone participated in Prelecture ! Is Effect Really Bigger? Do Students Who Participate Perform Better? = 80.4 ± 1.1% = 71.7 ± 1.3% 12% Increase Performance on Sum of 212 Hour Exams Better Students Choose to Participate? BUT Effect is Not All Due to Better Student Population Performance on Sum of 211 Hour Exams = 83.3 ± 0.9% = 78.2 ± 1.0% 7% Increase

19. Real World Conclusions We have changed “Lecture” Component of the Course Added web-based Multimedia Learning Module as a “Prelecture” Reduced Lecture from 75 min to 50 min and made more interactive We see dramatic improvement in students’ preparation for lecture Evidence from Preflight data We see a “small” improvement in exam performance Expect increase to “moderate” improvement when participation increases Students value Prelectures ! Text Prelecture Lecture HomeworkDiscussion Not Very Useful Useful Very Useful How Important is this Component in Helping You Understand the Material?

20. The Future?? PHYS 212/Fall 2008 1000 students revised scripts/animations improved implementation BEYOND.. EyeTracker experiments to understand/improve animation “The future’s so bright we gotta wear shades ! “