Jacob Prunuske, MD, MSPH STFM Predoc 2009 Asynchronous web-based orientation for a required rotation: creating time to teach complex concepts Jacob Prunuske, MD, MSPH STFM Predoc 2009

Objectives Educational potential Effectiveness: live vs web Approved by the University of Wisconsin Educational Research IRB

UW 4th Year Preceptorship Required 171 students in ‘08-’09 6 weeks 45 sites, 31 Communities Community Health Focus Rotation Structure Clinical Modules

Rationale for change Orient & drive Didactics & geography Peer learning Complex topics

Live vs WBL Multiple studies showing equivalency in knowledge assessment following WBL or live lecture Wofford, Spickard et al. 2001 Williams, Aubin et al. 2001 Chumley-Jones, Dobbie et al. 2002 Solomon, Ferenchick et al. 2004 Jenkins, Goel et al. 2008

Web Based Learning Advantages Convenience Revisit* Travel Standardization Disadvantages Questions Group interaction Discussion Course Changes * 70% of students revisited previous online courses to review lecture presentations, notes or quizzes. (Masters and Gibbs 2007)

Ho = No difference live vs web

Orientation Orientation Video Course management software

150 Evaluations Month 2007-2008 Paper 2008-2009 Electronic July 3 August 11 September 25 November 16 21 January 19 12 February March 22 Total 78 72

Results Average difference = 0.33 * * * * * * * * * * Question * P < 0.05

Time of Year Month 2007-2008 Paper 2008-2009 Electronic July 3 August 11 September 25 November 16 21 January 19 12 February March 22 Total 78 72

Nov 07 vs Nov 08 No difference in 7 out of 13 questions… * * * * * * * P < 0.05

Time of Year Month 2007-2008 Paper 2008-2009 Electronic July 3 August 11 September 25 November 16 21 January 19 12 February March 22 Total 78 72

Jan 08 vs Jan 09 No difference in any of the questions… Question P > 0.05 for all questions

Learner Comments Live: 21 of 78 Web: 54 of 72

Live = 21 comments 7 critical of length 3 critical of content 7 positive review of speaker/content

Web = 54 Comments 4 critical of length 2 critical of content 3 tech problems 24 appreciate electronic format 11 positive review speakers/content

Few Tech Problems “Downloading the videos was INCREDIBLY slow.” “I have a mac laptop and was able to access all 3 videos, however none of the 3 played to completion”

Not a barrier Students had some technical difficulties using the modules, but agreed or strongly agreed that modules could serve as adequate replacement for live lectures Solomon, Ferenchick et al. 2004

Positive Comments

“The orientation was probably the best one I have had before a clerkship.  I really liked that we could watch the video on our own time and go back to things that were maybe not clear the first time through.”

“Overall, this was an excellent orientation to the 6 week rotation “Overall, this was an excellent orientation to the 6 week rotation.  The first lecture provided all of the basic information needed for the clerkship, and did so succinctly and while providing useful information regarding meaningful participation/goals for the rotation.”

“It was really awesome how you allowed us to watch videos & go thru orientation on our own.  This is a tremendous help in the interview season as well as for travelling between sites.  If we would have had to come back to Madison for doing the same thing that we were able to do online, it would not have been a good use of our time.  Thank you.”

Limitations Perception vs Reality Students preferred live 1/3 point on 5-pt scale Did not measure actual knowledge acquisition Students attending the live lecture rated their knowledge & skills higher than students who used computers, despite no measurable difference Williams, Aubin et al. 2001

Limitations Did not adjust for student demographics Knowledge performance live = on-line & unrelated to Age level of computer skill attitudes toward computer tasks beliefs in role of computers in medical education time between intervention & post-test Spickard, Alrajeh et al. 2002

Hurdles “Probably my own fault, but it was not clear to me that I was to have this orientation before the start of the preceptorship and it would have been helpful if I had. Perhaps another reminder or two? I know, I should have been more on top of it.” - Adult Learner

Benefits Efficiency: Lecturers free to do other tasks 60 min live vs 25 min on-line 12 ½ min if watch 2x speed Lecturers free to do other tasks Addition of workshop Students participating in on-line lecture spent significantly less time at the task Spickard, Alrajeh et al. 2002

Plans Month 2007-2008 Paper 2008-2009 Electronic July 3 August 11 September 25 November 16 21 January 19 12 February ? March 22 Total 78 72

Plans Evaluate workshop Use evaluation data to improve video

Conclusions Live & web orientations comparable More learner comments online Learner preference for web delivery Benefits of efficiency & flexibility outweigh 1/3 point difference Minimal tech problems Faculty time: Workshop vs orientation

Discussion

Thanks! 40