A Web-based Approach to Enhance Preventive Medicine Education Outcomes for Third Year Medical Students 35th Annual Predoctoral Education Conference 2009 Society of Teachers of Family Medicine Savannah, GA Pablo Joo MD, Sharon Krackov EdD, Michelle V. Hall, Edgar Figueroa MD MPH, Devayani Tirthali, and Susan Lin DrPH Columbia P&S Center for Family and Community Medicine New York, New York Contact E-mail: pjoo@aecom.yu.edu (Albert Einstein College of Medicine)

Session goals: Illustrate a framework for learning objectives that will assist in building a preventive medicine curriculum. Demonstrate a dynamic and interactive web-based intervention to teach medical students preventive medicine.

Background Liaison Committee on Medical Education requires preventive medicine education in medical schools (ED-11, ED-13). 1 The lack of prevention education in U.S. medical schools is well documented.2 LCME Accreditation Standards. http://www.lcme.org/functionslist.htm Accessed January 18, 2009 Pomrehn, P. et al: Prevention for the 21st Century: Setting the Context through Undergraduate Medical Education. Acad Med. 2000; 75 (7 suppl): S5-13

A Framework of Learning Objectives for Preventive Medicine In 1997, the Association of Teachers of Preventive Medicine (ATPM)* outlined core preventive medicine competencies for students. 2 *Now, Association of Prevention Teaching and Research (APTR) Pomrehn, P. et al: Prevention for the 21st Century: Setting the Context through Undergraduate Medical Education. Acad Med. 2000; 75 (7 suppl): S5-13

APTR Domains of Preventive Medicine*2 Clinical Prevention Quantitative Skills Community Dimensions Health Services Organization and Delivery *Please refer to handout Pomrehn, P. et al: Prevention for the 21st Century: Setting the Context through Undergraduate Medical Education. Acad Med. 2000; 75 (7 suppl): S5-13

Challenges Students still report inadequate time dedicated to the subject.3 Educators still report gaps in curriculum and a lack of assessment of program outcomes.3 Similar challenges at our institution. Garr, D. et al. Prevention Education and Evaluation in U.S. Medical Schools: A Status Report. Acad Med. 2000;75 (7 suppl):S14-21

Our Clerkship Setting In our third year primary care clerkship, 150 students spend five weeks at one of 18 diverse clinical practices across the United States. Concord, New Hampshire Cooperstown, NY Stamford, Connecticut New York, NY Shiprock, New Mexico Zuni, New Mexico Whiteriver, Arizona

Our Intervention Create a web-based module to reach all of the students Interactive case-based format

Module development Curriculum based on the APTR competencies Content driven by baseline evaluation of prevention education at our institution Baseline evaluation linked to the first three levels of Kirkpatrick’s Hierarchy of Outcome Measures

Learner assessment Students complete the online module Prepare a required 6 – 10 page paper on a prevention topic Clinical preceptors assess students on formulation of prevention plans and behavioral counseling during active patient care

organizational practice Kirkpatrick’s Hierarchy of Outcome Measures Adaptation by Barr H, Freeth D, Hammick M et al., 2000 Level 4b: Benefits to patients or clients Level 4a:Change in organizational practice Level 3: Change in behavior Level 2b: Acquisition of knowledge/skills Level 2a: Modification of attitudes/perceptions Level 1: Learners’ reactions Kirkpatrick’s Four Levels of Evaluation, 1967

Post-module assessment Outcomes were reassessed after introduction of the module on the clerkship.

Level 1: Learners’ reactions Kirkpatrick’s Hierarchy of Outcome Measures Adaptation by Barr H, Freeth D, Hammick M et al., 2000 Level 4b: Benefits to patients or clients Level 4a:Change in organizational practice Level 3: Change in behavior Level 2b: Acquisition of knowledge/skills Level 2a: Modification of attitudes/perceptions Level 1: Learners’ reactions Kirkpatrick’s Four Levels of Evaluation, 1967

Level 1: Students’ reactions to their preventive medicine education before our intervention 2006 AAMC Medical Student Graduation Survey: percentages of our students who believed inadequate time was devoted to: health promotion and disease prevention 14.9% immunization 36.2% chemoprevention 52.1% health surveillance strategies 39.4%.

Level 1: Students’ reactions about our intervention Please rate the materials in this web module in adding to your understanding of health promotion and disease prevention. % of students responses N=71

Level 1: How competent do you feel in the subject of health promotion and disease prevention before / after completing this web module? % of students responses N=71

Modification of attitudes/perceptions Kirkpatrick’s Hierarchy of Outcome Measures Adaptation by Barr H, Freeth D, Hammick M et al., 2000 Level 4b: Benefits to patients or clients Level 4a:Change in organizational practice Level 3: Change in behavior Level 2b: Acquisition of knowledge/skills Level 2a: Modification of attitudes/perceptions Level 1: Learners’ reactions Kirkpatrick’s Four Levels of Evaluation, 1967

before our web module existed after completing our web module Level 2a: Assessment of third year medical students attitudes about preventive medicine before our web module existed after completing our web module

Level 2a: To what extent do you feel health promotion and patient education should be left to the physician or other the health care team members? % of responses

Level 2a: To what extent do you feel the responsibility for providing preventive health care belongs to primary care physicians or specialist physicians? % of responses

Acquisition of knowledge/skills Kirkpatrick’s Hierarchy of Outcome Measures Adaptation by Barr H, Freeth D, Hammick M et al., 2000 Level 4b: Benefits to patients or clients Level 4a:Change in organizational practice Level 3: Change in behavior Level 2b: Acquisition of knowledge/skills Level 2a: Modification of attitudes/perceptions Level 1: Learners’ reactions Kirkpatrick’s Four Levels of Evaluation, 1967

Level 2b: Prevention Knowledge Pre / Post Examination All students completed a 10 item pre/post preventive medicine knowledge examination on the clerkship without the module in 2007 – 2008 academic year with the module in 2008 – 2009 academic year

Level 2B: Preventive Medicine Knowledge

Level 2B: Clinical Prevention Domain Knowledge

Level 2B: Community Dimensions Domain Knowledge

Findings Our development process allowed us to tailor the online module to students’ learning needs Our intervention received positive feedback from most students increased student self-rated competence improved attitudes towards an interdisciplinary role in preventive care enhanced prevention knowledge

New Challenges Locating skilled partners in education technology to successfully build and maintain the module. Logistic and financial obstacles to the assessment of Kirkpatrick’s Hierarchy of Outcome Measures at Level 3 (change in practice behavior) or Level 4 (change in patient outcomes).

Acknowledgements Columbia University Center for New Media Teaching and Learning Columbia University Medical Center / Center for Education Research and Evaluation Columbia P&S Primary Care Education Fund Columbia University IRB

Bibliography LCME Accreditation Standards. http://www.lcme.org/functionslist.htm , accessed January 18, 2009 Pomrehn, P. Davis, M., Chen, DW, Barker, W. Prevention for the 21st Century: Setting the Context through Undergraduate Medical Education. Acad Med. 2000; 75 (7 suppl): S5-13 Garr, D. et al. Prevention Education and Evaluation in U.S. Medical Schools: A Status Report. Acad Med. 2000;75 (7 suppl):S14-21

Thank You!