1. A Web-based Approach to Enhance Preventive Medicine Education Outcomes for Third Year Medical Students
35th Annual Predoctoral Education Conference 2009
Society of Teachers of Family Medicine
Savannah, GA
Pablo Joo MD, Sharon Krackov EdD, Michelle V. Hall,
Edgar Figueroa MD MPH, Devayani Tirthali, and Susan Lin DrPH
Columbia P&S
Center for Family and Community Medicine
New York, New York
Contact (Albert Einstein College of Medicine)

2. Session goals:
Illustrate a framework for learning objectives that will assist in building a preventive medicine curriculum.
Demonstrate a dynamic and interactive web-based intervention to teach medical students preventive medicine.

3. Background
Liaison Committee on Medical Education requires preventive medicine education in medical schools (ED-11, ED-13). 1
The lack of prevention education in U.S. medical schools is well documented.2
LCME Accreditation Standards.
Accessed January 18, 2009
Pomrehn, P. et al: Prevention for the 21st Century: Setting the Context through Undergraduate Medical Education. Acad Med. 2000; 75 (7 suppl): S5-13

4. A Framework of Learning Objectives for Preventive Medicine
In 1997, the Association of Teachers of Preventive Medicine (ATPM)* outlined core preventive medicine competencies for students. 2
*Now, Association of Prevention Teaching and Research (APTR)
Pomrehn, P. et al: Prevention for the 21st Century: Setting the Context through Undergraduate Medical Education. Acad Med. 2000; 75 (7 suppl): S5-13

5. APTR Domains of Preventive Medicine*2
Clinical Prevention
Quantitative Skills
Community Dimensions
Health Services Organization and Delivery
*Please refer to handout
Pomrehn, P. et al: Prevention for the 21st Century: Setting the Context through Undergraduate Medical Education. Acad Med. 2000; 75 (7 suppl): S5-13

6. Challenges
Students still report inadequate time dedicated to the subject.3
Educators still report gaps in curriculum and a lack of assessment of program outcomes.3
Similar challenges at our institution.
Garr, D. et al. Prevention Education and Evaluation in U.S. Medical Schools: A Status Report. Acad Med. 2000;75 (7 suppl):S14-21

7. Our Clerkship Setting
In our third year primary care clerkship, 150 students spend five weeks at one of 18 diverse clinical practices across the United States.
Concord, New Hampshire
Cooperstown, NY
Stamford, Connecticut
New York, NY
Shiprock, New Mexico
Zuni, New Mexico
Whiteriver, Arizona

8. Our Intervention
Create a web-based module to reach all of the students
Interactive case-based format

10. Module development Curriculum based on the APTR competencies
Content driven by baseline evaluation of prevention education at our institution
Baseline evaluation linked to the first three levels of Kirkpatrick’s Hierarchy of Outcome Measures

11. Learner assessment Students complete the online module
Prepare a required 6 – 10 page paper on a prevention topic
Clinical preceptors assess students on formulation of prevention plans and behavioral counseling during active patient care

12. organizational practice
Kirkpatrick’s Hierarchy of Outcome Measures
Adaptation by Barr H, Freeth D, Hammick M et al., 2000
Level 4b:
Benefits to
patients or clients
Level 4a:Change in
organizational practice
Level 3: Change in behavior
Level 2b:
Acquisition of knowledge/skills
Level 2a:
Modification of attitudes/perceptions
Level 1: Learners’ reactions
Kirkpatrick’s Four Levels of Evaluation, 1967

13. Post-module assessment
Outcomes were reassessed after introduction of the module on the clerkship.

14. Level 1: Learners’ reactions
Kirkpatrick’s Hierarchy of Outcome Measures
Adaptation by Barr H, Freeth D, Hammick M et al., 2000
Level 4b:
Benefits to
patients or clients
Level 4a:Change in
organizational practice
Level 3: Change in behavior
Level 2b:
Acquisition of knowledge/skills
Level 2a:
Modification of attitudes/perceptions
Level 1: Learners’ reactions
Kirkpatrick’s Four Levels of Evaluation, 1967

15. Level 1: Students’ reactions to their preventive medicine education before our intervention
2006 AAMC Medical Student Graduation Survey: percentages of our students who believed inadequate time was devoted to:
health promotion and disease prevention 14.9%
immunization 36.2%
chemoprevention 52.1%
health surveillance strategies 39.4%.

16. Level 1: Students’ reactions about our intervention
Please rate the materials in this web module in adding to your understanding of health promotion and disease prevention.
% of students
responses
N=71

17. Level 1: How competent do you feel in the subject of health promotion and disease prevention before / after completing this web module?
% of students
responses
N=71

18. Modification of attitudes/perceptions
Kirkpatrick’s Hierarchy of Outcome Measures
Adaptation by Barr H, Freeth D, Hammick M et al., 2000
Level 4b:
Benefits to
patients or clients
Level 4a:Change in
organizational practice
Level 3: Change in behavior
Level 2b:
Acquisition of knowledge/skills
Level 2a:
Modification of attitudes/perceptions
Level 1: Learners’ reactions
Kirkpatrick’s Four Levels of Evaluation, 1967

19. before our web module existed after completing our web module
Level 2a: Assessment of third year medical students attitudes about preventive medicine
before our web module existed
after completing our web module

20. Level 2a: To what extent do you feel health promotion and patient education should be left to the physician or other the health care team members?
% of responses

21. Level 2a: To what extent do you feel the responsibility for providing preventive health care belongs to primary care physicians or specialist physicians?
% of
responses

22. Acquisition of knowledge/skills
Kirkpatrick’s Hierarchy of Outcome Measures
Adaptation by Barr H, Freeth D, Hammick M et al., 2000
Level 4b:
Benefits to
patients or clients
Level 4a:Change in
organizational practice
Level 3: Change in behavior
Level 2b:
Acquisition of knowledge/skills
Level 2a:
Modification of attitudes/perceptions
Level 1: Learners’ reactions
Kirkpatrick’s Four Levels of Evaluation, 1967

23. Level 2b: Prevention Knowledge Pre / Post Examination
All students completed a 10 item pre/post preventive medicine knowledge examination on the clerkship
without the module in 2007 – 2008 academic year
with the module in 2008 – 2009 academic year

24. Level 2B: Preventive Medicine Knowledge

25. Level 2B: Clinical Prevention Domain Knowledge

26. Level 2B: Community Dimensions Domain Knowledge

27. Findings
Our development process allowed us to tailor the online module to students’ learning needs
Our intervention
received positive feedback from most students
increased student self-rated competence
improved attitudes towards an interdisciplinary role in preventive care
enhanced prevention knowledge

28. New Challenges
Locating skilled partners in education technology to successfully build and maintain the module.
Logistic and financial obstacles to the assessment of Kirkpatrick’s Hierarchy of Outcome Measures at Level 3 (change in practice behavior) or Level 4 (change in patient outcomes).

29. Acknowledgements
Columbia University Center for New Media Teaching and Learning
Columbia University Medical Center / Center for Education Research and Evaluation
Columbia P&S Primary Care Education Fund
Columbia University IRB

30. Bibliography
LCME Accreditation Standards. , accessed January 18, 2009
Pomrehn, P. Davis, M., Chen, DW, Barker, W. Prevention for the 21st Century: Setting the Context through Undergraduate Medical Education. Acad Med. 2000; 75 (7 suppl): S5-13
Garr, D. et al. Prevention Education and Evaluation in U.S. Medical Schools: A Status Report. Acad Med. 2000;75 (7 suppl):S14-21

31. Thank You!