1. Evaluating the Principal Clinical Experience: A Progress Report
Ed Krupat, PhD
Medical Education
Grand Rounds
May 18, 2007

2. The PCE Pilots Cambridge Integrated Clerkship--begun July 2004
Brigham & Women’s Hospital--begun July 2005
Beth Israel Deaconess Medical Center--begun July 2005
Massachusetts General Hospital--begun July 2006

3. The Principles Longitudinality Mentoring
with faculty and house staff
with patients
of feedback
of curriculum
Mentoring
Interdisciplinary perspectives
Integration of basic and clinical sciences
Patient-centered approach
Student-centered learning

4. Plans & Methods Mentoring programs Planned curriculum
Tutorials
Students present cases
PCC (with physicians who admit to the hospital)
PD III at site
Writing/reflection

5. The Evaluation Plan Comprehensive Multi-method Scientifically rigorous
Experiences
Knowledge & knowledge retention
Skills
Attitudes/perceptions
Self-assessment
Multi-method
Quantitative
Qualitative
Scientifically rigorous
Tied directly to pilot program goals

6. The Design
Comparison/control group recruited from students doing traditional rotations
Pre-clerkship measures demonstrated no differences on
MCAT scores
Step I scores
PD II OSCE scores
Career preferences
Attitudes toward patient care

7. Characterizing the Experience

8. “How well would you say the following adjectives describe your clerkship experience: Frustrating” (1= Not At All & 6= Perfectly)
’05-’06 Clerkships

9. “How well would you say the following adjectives describe your clerkship experience: Confidence-building” (1= Not At All & 6= Perfectly)
’05-’06 Clerkships

10. “How well would you say the following adjectives describe your clerkship experience: Humanizing”
’05-’06 Clerkships

11. “How satisfied are you with: The overall quality of your clerkship experience” (1= Extremely Dissatisfied & 6= Extremely Satisfied)
’05-’06 Clerkships

12. Performance: Knowledge

13. Shelf Scores: CIC vs. Control Year 1 (2004-2005)
T-Test P-Value
Ob/Gyn
77.13
70.40
.250
Pediatrics
76.25
74.22
.689
Psychiatry
81.25
70.60
.128
Surgery
77.38
73.20
.424

14. Shelf Exam

15. Shelf Exam

16. Shelf Exam

17. Shelf Exam

18. Performance: Skills

19. Comprehensive OSCE

20. Comprehensive OSCE

21. Comprehensive OSCE

22. Comprehensive OSCE Skill Score History
’05-’06 Clerkships

23. Comprehensive OSCE Skill Score Physical Exam
’05-’06 Clerkships

24. Attitudes & Perceptions about the Year

25. “To what extent have your experiences prepared you: To have the knowledge base necessary to be a competent practitioner” (1= Very Poorly & 6= Very Well)
’05-’06 Clerkships

26. “To what extent have your experiences prepared you: To integrate basic sciences and clinical practice” (1= Very Poorly & 6= Very Well)
’05-’06 Clerkships

27. “To what extent have your experiences prepared you: To practice evidence-based medicine” (1= Very Poorly & 6= Very Well)
’05-’06 Clerkships

28. “To what extent have your experiences prepared you: To see how the social context affects patients and their problems” (1= Very Poorly & 6= Very Well)
’05-’06 Clerkships

29. “To what extent have your experiences prepared you: To relate well to a diverse patient population” (1= Very Poorly & 6= Very Well)
’05-’06 Clerkships

30. “To what extent have your experiences prepared you: To know your strengths and limitations” (1= Very Poorly & 6= Very Well)
’05-’06 Clerkships

31. Attitudes & Perceptions about Patient Care and Life on the Wards

32. Patient-Practitioner Orientation Scale (PPOS)
’05-’06 Clerkships

33. Patient-Practitioner Orientation Scale (PPOS)
Pre Post
Fall ’05 Spring ‘06
Site
Site
Site
Control

34. “During this past clerkship year, I observed residents encouraging patients’ participation in their own care.” (1= Never & 7= Always)
’05-’06 Clerkships

35. “Your team is rounding on a patient in his hospital room when one of the consulting services arrives for the patient. Your attending and the consulting attending proceed to talk about the patient’s case as if the patient weren’t there.” (1= Very Often & 5= Never)
’05-’06 Clerkships

36. “In general, when I made an effort to develop rapport with my patients, my instructors _____ me.” (1= Completely Discouraged & 7= Completely Encouraged)
’05-’06 Clerkships

37. “In general, when I made an effort to get to know patients as unique persons, my instructors _____ me.” (1= Completely Discouraged & 7= Completely Encouraged)
’05-’06 Clerkships

38. Focus Group Findings Valued-added for the PCE students
Continuity with faculty
Access to hearing how faculty think about clinical issues
Mentoring
Peer group support
Sense of belonging to something
larger than self

39. “Today I was in a public restroom at _____,and I noticed how there was paper all over the floor. For a moment I was disgusted and I thought about all the residents and physicians who work so hard to give top notch care. I wouldn’t want patients or their families to use the bathroom at ____ and feel this hospital is dirty when hospitals are supposed to be clean! So I picked up all the trash. At that moment, I realized that I had taken ownership of this hospital.”

40. Value-Added for the Faculty
Contact and meaningful cross-discipline discussion
Ability to get to know students well
Personally satisfying
Ability to identify strengths and weaknesses
Ability to provide feedback and assist in student growth and development

41. Remaining Challenges Paradox of the Pullout
Longitudinal, interdiscipinary curriculum
Attendings vs residents?? More observation & better feedback

42. Challenges (cont.) Patient contact pre-workup
Longitudinal contact with patients
Quality of the individual clerkships
Consistency
Across disciplines within site, within discipline across sites, and across PCEs