Streaming Video Tutorials for Complex Laboratory Techniques: Lessons Learned in Sophomore and Junior Biomechanics Courses Kristen L. Billiar, Ph.D. Dept of Biomedical Engineering, Worcester Polytechnic Institute Dept of Surgery, University of Massachusetts Medical School

Department of Biomedical Engineering Goals Laboratories in all BME courses Making and interpreting measurements Making and interpreting measurements Challenge-based, self-guided, self-paced Challenge-based, self-guided, self-paced Particular challenge Rapid growth of BME (40-50 per course) Rapid growth of BME (40-50 per course) Limited resources Limited resources Instructor time (and TA) Volume of sophisticated equipment Laboratory space (scheduling time) Funds Practical vs. theoretical Expanding access, not assessing educational theories or comparing methods Expanding access, not assessing educational theories or comparing methods Share approach to problem and assessment Discuss time and effort involved Discuss time and effort involved Describe assessment methods and findings Describe assessment methods and findings Initiate dialog – chance to get your feedback Initiate dialog – chance to get your feedback

Department of Biomedical Engineering Approach and goal of study Approach - teaching technology Web-based streaming video tutorials Web-based streaming video tutorials Provide the “toolbox” Provide the “toolbox” Economical, sustainable Economical, sustainable Allow students to perform labs Allow students to perform labs Self-paced, flexible schedule Without constant instructor or TA supervision or TA supervision Goal - formative study Assess student learning from the tutorials Assess student learning from the tutorials Gather student feedback on their usefulness and quality as a guide for future improvements Gather student feedback on their usefulness and quality as a guide for future improvements

Department of Biomedical Engineering Tutorials – web-based Tutorials How? rather than What and Why How? rather than What and Why Large number (>20), short ( 20), short (< 5 minutes) Easy to create and modify 1 Easy to create and modify 1 Format Format MS Powerpoint, Camtasia, video Slides, video, audio Design team Two professors Two professors Biomedical and Mechanical Engineering Academic Technology Center Staff Academic Technology Center Staff Video, editing, streaming servers Two sophomore-level students Two sophomore-level students 20 hrs/wk x 8wks x 2 = 320 hrs Funding Two small internal grants, paid for student time and provided access Two small internal grants, paid for student time and provided access 1 Billiar and Shonat, PEBEL II, 2004

Department of Biomedical Engineering Example tutorial - safety Posted on myWPI (Blackboard) PPT file complete (link) link 23 MB (w/o video clip) 23 MB (w/o video clip) PPS file 1.2 MB 1.2 MB Streaming audio Streaming audio Streaming video Streaming video Requires fast internet connection Requires fast internet connection X

Department of Biomedical Engineering Assessment Sophomore-level “Foundations of Biomechanics” large number (n=47), limited use, single laboratory large number (n=47), limited use, single laboratory Junior-level “Experimental Biomechanics” small number (n=17), extensive use as they were meant to be, challenge-based small number (n=17), extensive use as they were meant to be, challenge-based Questionnaires (sophomore-level only) Pre-tutorial Pre-tutorial Post-tutorial – with answers and explanation Post-tutorial – with answers and explanation Controlled environment – 2 one-hour sessions Controlled environment – 2 one-hour sessions Automation with Blackboard Automation with Blackboard Safety practicum minutes/group, Password given minutes/group, Password given Grade comparison (sophomore-level only) Quiz assessing knowledge of concept of stress-strain analysis Quiz assessing knowledge of concept of stress-strain analysis Feedback – usefulness and quality Likert-type Likert-type Written responses Written responses

Department of Biomedical Engineering Example questionnaire

Department of Biomedical Engineering Assessment of tutorials Questionnaires Pre-tutorial score34% ± 9% Pre-tutorial score34% ± 9% Post-tutorial score71% ± 12% Post-tutorial score71% ± 12% p < 0.05 paired t-test in all groups (except M5)

Department of Biomedical Engineering Assessment of tutorials Compliance with questionnaires Completed 100%: n = 27 of 47 (57% compliance) Completed 100%: n = 27 of 47 (57% compliance) Completed >90%: n = 36 of 47 (77% compliance) Completed >90%: n = 36 of 47 (77% compliance)

Department of Biomedical Engineering Feedback survey

Department of Biomedical Engineering Feedback survey What about the Tutorials would you change? What did you especially like about the Tutorials and your experience using them? Please identify aspects of particular Tutorials that were especially problematic.

Department of Biomedical Engineering Feedback – Likert scale SophomoreJunior completion of survey 74% (35/47) 94% (16/17) general quality either “good,” “very good,” or “excellent.” 70% to 90% 80% to 93% “easy” or “very easy” to use 60%100% “quite” or “extremely” useful (all but one) 54% to 72% 73% to 100% “as helpful,” “more helpful,” or “much more helpful” than written materials 80%100% tutorials alone were adequate (or better) in preparing them for their lab 60%75% “as helpful” or “more helpful” than one-on- one time with the professor 30%75%

Department of Biomedical Engineering Quiz grade comparison

Department of Biomedical EngineeringDiscussionMotivation Provide labs to students with resource savings in the long term Provide labs to students with resource savings in the long term Available tutorials (MTS, Microsoft) not well suited, much longer Available tutorials (MTS, Microsoft) not well suited, much longer General enough for multiple types of users at WPI and beyond General enough for multiple types of users at WPI and beyondAssessment Effectiveness of tutorials and aspects to improve Effectiveness of tutorials and aspects to improve Not video vs. written or professor Not video vs. written or professor Not lab vs. no lab Not lab vs. no lab Controls for summative evaluations not practical Controls for summative evaluations not practical Resource requirement High initial investment (team, time) High initial investment (team, time) Written materials require effort also Written materials require effort also Advantage - technical staff and students Advantage - technical staff and students

Department of Biomedical EngineeringDiscussion Overall success – short term Students passed practicum, completed laboratories Students passed practicum, completed laboratories Sophomore, junior, and grad as well Sophomore, junior, and grad as well Instructor time in lab minimal Instructor time in lab minimalQuestionnaires Controlled viewing, but lower scores than desired Controlled viewing, but lower scores than desired Too many (12 x 2), too much time commitment Too many (12 x 2), too much time commitmentFeedback Too broad for one lab in sophomore-level course Too broad for one lab in sophomore-level course Well received when used as designed Well received when used as designed Increased use of equipment - time logged Sophomore-level course > 50 hours; ~ 1hr/student Sophomore-level course > 50 hours; ~ 1hr/student Junior-level course >20 hours/student, relatively few scheduling conflicts Junior-level course >20 hours/student, relatively few scheduling conflicts Facilitated challenge-based laboratories True time and resource savings - future

Department of Biomedical Engineering Acknowledgements Allen Hoffman tutorial content tutorial content Megan Murphy and Dan Filipe sophomores sophomores tutorial creation tutorial creation Paula Quinn assessment consultant assessment consultant Adam Jacobi and David Taranto (WPI ATC) production support production support Elias Wilson (TA) questionnaire and practicum administration Ross Shonat (NIH) introduction to formal CBIFunding WPI Educational Development Council Grant WPI Teaching Technology Fellowship

Department of Biomedical Engineering Questions?

Educational framework Challenge-based learning (self-guided) Aid in the development of Aid in the development of critical thinking – authentic problems Building on HPL framework 1 Learner-, community-, knowledge-, and assessment-centered Learner-, community-, knowledge-, and assessment-centeredSelf-paced Allow for differences Allow for differences Allow greater number of student-hours in lab Allow greater number of student-hours in lab The “Challenge” & problem identification Generate ideas Literature & interview Testing & analysis Communication of findings Learning Cycle Research & revise 1 Bransford, Brown, and Cocking, 2000