1. Medical Education Research Methods and Innovative Designs (MERMAID) How To Turn Your Educational Presentation into an On-Line Module Gary Tabas March 13, 2015

2. Goals Why do it? How to do it What works ? A look at some of what we all have done

3. A Case You are a faculty member who runs a workshop for a rotating group of residence each month. The workshop runs about 2 hours and the faculty members that use to help you are no longer available. Your own workload has recently increased. Additionally not all of the residents have been able to get to the workshop because of other demands. You are thinking that you would like to ease the burden on yourself but make sure that all the residents continue to receive the necessary education.

4. Why Do It? Delivery repetition Multiple sites Increasing faculty demands Increasing demands on learners’ time Differing learning styles - adaptive learning Flipped classroom – Shorter group meeting time – Targeted education

5. Why Do It? Learners can: – Work at their own pace (individualized learning) – Practice using repetition – Complete the learning at work or at home – Work independently or in small groups – Work with or without a facilitator

6. Types of On-Line Learning Recorded slides with audio Purely online - no face-to-face meetings Blended Learning - combination of online and face-to-face Synchronous (chat, video conferencing) Asynchronous (email, listserv, blog) Self-study or group - Instructor-led Adapted from WorldWideLearn

7. Simulation in Medical Education Medical Simulation in Medical Education: Results of an AAMC Survey

10. On-line Module Structure Linear – All learners take the same path Cases, tutorials, quizzes, graphics Branching – Two or more paths through the module Cases, management options, tutorials, quizzes

11. Linear Case Vignette Quiz (diagnosis, treatment, management) Tutorial Vignette

12. Branching Vignette Management Option 1 Option 2 Option 3 Result of option 1 Result of option 2 Result of option 3 new mgt option 1 new mgt option 2 new mgt option 3

13. How to do it? How to get started:  Needs analysis, learning goals and objectives  What is the problem you want to address?  What is the gap between current knowledge and ideal?  Will your on-line instruction be primary or supplemental?  Instructor involvement?

14. Design Shovelware Use the power and flexibility of the web to enhance learning, not just put on-line what can be done without the web

15. Principles 1.Develop clinical scenarios (vignettes) that actively engage learners This and subsequent slide principles 1-13 From: Posel, McGee, Fleiszer

17. 2. Practice data acquisition

18. 3. Emphasize critical analysis 4. Require decision-making (with consequences), hypothesis generation, treatment and management

19. A week later, Joshua calls the office and requests to be seen. He complains about feeling completely "wiped out" and is concerned about the new drooling out of his mouth. You agree to see him at noon. In the office he complains he has been too exhausted to work for 3 days, which is costing him income. He admits to feeling intermittently warm and achy all the time. He denies chills, night sweats, sore throat, cough, nausea, vomiting, diarrhea, dysuria, joint swelling and pain.

20. 5. Consider uncommon situations that learners might not otherwise see. 6. Make cases longitudinal 7. Link to EBM explanations, guidelines, references

21. 8. Emphasize the importance of communication Kathy suddenly becomes very angry, saying: "WHAT?? What do you mean you're not going to let me get a mammogram until I turn 50? Do you want me to get breast cancer? Everyone I know gets mammograms as soon as they're 40. Forget it, I'll find a doctor who wants to take care of me." She storms out of your office before you can reply.

22. You call Kathy the following day, and start the conversation by saying: "I am sorry that I upset you yesterday. I know you're understandably very scared since your friend found out that she has breast cancer. My intention was to explain the risks and benefits of mammograms, because we know that unfortunately, when to start mammograms is a complicated issue. But I know that I did not make this clear yesterday, and I'm hoping we can start the conversation over."

23. 9.Provide continuous and immediate individualized feedback 10. Provide opportunity to learn from errors: Consequences of suboptimal choices can be: – Suboptimal outcome – Less than high value care – Patient upset

24. Provide Summary A gluten-free diet should be initiated only after the diagnosis of celiac disease is confirmed by endoscopy/biopsy as it involves lifelong adherence and is an expensive and socially inconvenient diet. Asymptomatic family members can obtain genetic testing to assess their risk for celiac disease (and determine whether periodic screening should be initiated). While lifelong therapy is indicated, it is unclear what the real long-term risks to non-adherence are. Dermatitis herpetiformis is associated exclusively with celiac disease and should be treated in the same manner.

25. 12.Use your case in other settings PCC 13. Encourage learners to author a case – Learn by doing, think how learners think (learning and teaching at the same time) – Search for own evidence – Scholarship opportunity

26. Web design Consistent page organization Good use of space Concise text – Bulleted, short phrases – Limit sentences to 20 words – Limit scrolling (3 screens to a page max) Easy navigation

27. Software Easier to purchase commercial software Program from scratch Combination of above

28. Case Authoring Software Campus System for Virtual Patients Casus DecisionSim (VpSim) OpenLabyrinth Tusk Web-SP http://www.medbiq.org/virtual_patient/implementers

29. VpSim

32. What Works

33. What does the Literature Say? Web-based Learning in Clinic: An RCT Cook, DA Academic Med 2005 Residents randomized to paper vs web for ambulatory topics: asthma, depression, DM, nicotine dependence 97 residents were randomly assigned 2 units on paper and 2 on web.

34. Results No difference in knowledge scores – Pre to post test 68 to 75% for web 66 to 73% for paper (another study showed higher scores for multimedia module on physical dx compared to paper) 78% preferred web format (95% CI, 67-86%) – 10 items (efficient, convenient, easier, effective)

35. Less time spent on web, 47 min web, 59 min paper (learning efficiency) < 50% used links to additional learning materials

36. Self-Study Web Vs Print Bell DS. Ann Int Med 2000 In proctored sessions, residents assigned to review post acute MI care Both had MCQ’s and identical guideline text Web had tutorials, links to guideline passages and graphic evidence animations

37. Results Immediate post-test scores same Less time spent w web (27 min vs 38 min) So better learning efficiency w web – Score gain 8.6/hr vs 6.7/hr Greater satisfaction w learning w web – 17 vs 15 on 20 pt scale (P< 0.001) After 4-6 mo, knowledge decrease to same extent (similar scores, P = 0.12)

38. Internet-based Learning in the Health Professions: A Meta-analysis (Cook 2008) Internet vs non-internet learning was superior for knowledge gains, but similar for satisfaction, skills and behaviors. Practice exercises, tutorials, on-line peer discussions and longer duration courses had positive effects. JAMA 9/10/08; Vol 300 No. 10

39. Cook Meta-analysis In this landmark article, Cook concluded: Some methods of implementing CAI would likely be more effective than others and recommended that future research clarify which features of CAI are most effective.

40. Self-Assessment Questions: An RCT 149 residents randomized to web-based learning with or with out self-assessment questions imbedded in the module Topics = cervical cancer screening, dementia, osteoporosis, dyspepsia Cook; Academic Medicine March 2006

41. Results Higher post-test scores when using question format: – 78.9% vs 76.2% (p=0.006) Residents preferring question format scored higher than those preferring standard format: – 79.7% vs 69.5% (p<0.001) 83% preferred question format and 71% said it was more efficient (took longer: 60 vs 44 min)

42. Adaptive Web-based Instruction An intervention to adapt web-based instruction to learners’ prior knowledge 122 IM residents 4 ambulatory medicine modules Learners who correctly answer case-based questions skip corresponding content Cook; JGIM 23(7):985-90

43. Methods 17-21 case-based questions per module Didactic info appearing as feedback to each question Standard: 1-sentence explanation + didactics (1-4 paragraphs of text, tables, pictures, links) Adaptive: if correct response to MCQ, learner is directed to go to next MCQ w option to view didactics. If incorrect, no option to skip didactics

44. Results AdaptiveStandardP Knowledge76%77%.34 Time Spent (min)29.335.6.0003 75% preferred adaptive format Conclusion: adaptive format is associated with greater learning efficiency

45. Educating Residents Using Virtual Case- Based Simulation Improves Diabetes Management: An RCT N=341; 9 Primary care residencies Intervention Gp: 18 VPs that respond to provider actions CG: Sham non-DM case with instructions Assessment: 4 VPs, pre-post test, survey Results: Achieve clinical goals on VP2 15-48% vs 1.8-18% (p =.02 - <.001) Academic Medicine vol 89, No 12/Dec 2014

46. Findings in the Literature Learning efficiency improved Knowledge about the same Case-based MCQ’s improved knowledge Links may not be used – limit them

47. Our Use of an On-line Teaching Case - DKA 51 Third year medical students – AIMC 84 pharmacy students – Clinical pharmacology course 11 Endocrine fellows

48. Pretest VP (path score, time-on-task) Posttest Survey

49. Pre-Posttest Scores * P =.026 † P<.001 vs all posttest †

50. Selected Posttest Questions NS * † * P =.01 † P<.001 Scores (%)

51. VP Time-On-Task * P =.004 vs other groups

52. Survey Data

53. The ability to see and react to the consequences of my decisions in this module was more effective in teaching me clinical decision making than with other learning methods All differences NS

54. The module was effective in helping me learn how to adjust therapy in patients with DKA who are not responding appropriately * P =.02 vs Med students *

55. The module improved my confidence in managing DKA. * P =.01 vs other groups *

56. The module was of high educational value

57. Summary It is feasible to create a interactive branched- narrative VP to teach the management of a complex medical illness One way to fill an educational gap Effectiveness was significant as evidenced by pre-posttest score improvement Students felt that the VP was effective in teaching management of DKA and preferred the VP over other teaching methods

58. A look at some local uses of on-line learning A group of MS looking at which aspects of curriculum can be taught with on-line cases Individual MS and residents developing cases Integrated Case Studies (MS 2) General Medicine Modules Johns Hopkins Modules Yale Ambulatory Curriculum Annals Virtual Patients

60. April Release  Pulmonary Hypertension  Avoiding Problems in Transitions of Care  Hyperthyroidism  A Woman with Dyspnea and Edema (HFpEF)  Parkinson Disease  Migraine Fall ’15 Release  Approach to the Patient with HIV  Obstructive Sleep Apnea  Care of the Adult Cancer Survivor  Concussion  Stable Ischemic Heart Disease  Neurocognitive Decline Spring ’16 Release  Insomnia  Generalized Anxiety Disorder  Back Pain

61. Michael BarnettMaggie Benson Mamta BhatnagarRene Claxton Jennifer CorbelliDave Demoise Anna DonovanAndrea Elliott Kristian FeterikMichelle Freeman Alda Maria GonzagaRosanne Granieri Brian HeistScott Herrle Harish JastiAmar Kohli Mary KorytkowskiMelissa McNeil Alexandra MieczkowskiAnuradha Munshi Ruth PreisnerJohn Ragsdale Harsha RaoAdam Sawatsky Gaetan SgroJamie Stern Jo-Anne SuffolettoGary Tabas Sarah TilstraDianne Zalenski Annals VP Authors

62. Virtual Patient Simulation Approach to Teaching Appropriate Use of Cardiac Stress Testing Daniel Nguyen MD, Amar Kohli MD, Kathryn Berlacher MD