Evaluating the Principal Clinical Experience: A Progress Report Ed Krupat, PhD Medical Education Grand Rounds May 18, 2007

The PCE Pilots Cambridge Integrated Clerkship--begun July 2004 Brigham & Women’s Hospital--begun July 2005 Beth Israel Deaconess Medical Center--begun July 2005 Massachusetts General Hospital--begun July 2006

The Principles Longitudinality Mentoring with faculty and house staff with patients of feedback of curriculum Mentoring Interdisciplinary perspectives Integration of basic and clinical sciences Patient-centered approach Student-centered learning

Plans & Methods Mentoring programs Planned curriculum Tutorials Students present cases PCC (with physicians who admit to the hospital) PD III at site Writing/reflection

The Evaluation Plan Comprehensive Multi-method Scientifically rigorous Experiences Knowledge & knowledge retention Skills Attitudes/perceptions Self-assessment Multi-method Quantitative Qualitative Scientifically rigorous Tied directly to pilot program goals

The Design Comparison/control group recruited from students doing traditional rotations Pre-clerkship measures demonstrated no differences on MCAT scores Step I scores PD II OSCE scores Career preferences Attitudes toward patient care

Characterizing the Experience

“How well would you say the following adjectives describe your clerkship experience: Frustrating” (1= Not At All & 6= Perfectly) ’05-’06 Clerkships

“How well would you say the following adjectives describe your clerkship experience: Confidence-building” (1= Not At All & 6= Perfectly) ’05-’06 Clerkships

“How well would you say the following adjectives describe your clerkship experience: Humanizing” ’05-’06 Clerkships

“How satisfied are you with: The overall quality of your clerkship experience” (1= Extremely Dissatisfied & 6= Extremely Satisfied) ’05-’06 Clerkships

Performance: Knowledge

Shelf Scores: CIC vs. Control Year 1 (2004-2005) T-Test P-Value Ob/Gyn 77.13 70.40 .250 Pediatrics 76.25 74.22 .689 Psychiatry 81.25 70.60 .128 Surgery 77.38 73.20 .424

Shelf Exam

Shelf Exam

Shelf Exam

Shelf Exam

Performance: Skills

Comprehensive OSCE

Comprehensive OSCE

Comprehensive OSCE

Comprehensive OSCE Skill Score History ’05-’06 Clerkships

Comprehensive OSCE Skill Score Physical Exam ’05-’06 Clerkships

Attitudes & Perceptions about the Year

“To what extent have your experiences prepared you: To have the knowledge base necessary to be a competent practitioner” (1= Very Poorly & 6= Very Well) ’05-’06 Clerkships

“To what extent have your experiences prepared you: To integrate basic sciences and clinical practice” (1= Very Poorly & 6= Very Well) ’05-’06 Clerkships

“To what extent have your experiences prepared you: To practice evidence-based medicine” (1= Very Poorly & 6= Very Well) ’05-’06 Clerkships

“To what extent have your experiences prepared you: To see how the social context affects patients and their problems” (1= Very Poorly & 6= Very Well) ’05-’06 Clerkships

“To what extent have your experiences prepared you: To relate well to a diverse patient population” (1= Very Poorly & 6= Very Well) ’05-’06 Clerkships

“To what extent have your experiences prepared you: To know your strengths and limitations” (1= Very Poorly & 6= Very Well) ’05-’06 Clerkships

Attitudes & Perceptions about Patient Care and Life on the Wards

Patient-Practitioner Orientation Scale (PPOS) ’05-’06 Clerkships

Patient-Practitioner Orientation Scale (PPOS) Pre Post Fall ’05 Spring ‘06 Site 1 5.14 5.13 Site 2 4.89 4.83 Site 3 5.14 5.12 Control 4.90 4.57

“During this past clerkship year, I observed residents encouraging patients’ participation in their own care.” (1= Never & 7= Always) ’05-’06 Clerkships

“Your team is rounding on a patient in his hospital room when one of the consulting services arrives for the patient. Your attending and the consulting attending proceed to talk about the patient’s case as if the patient weren’t there.” (1= Very Often & 5= Never) ’05-’06 Clerkships

“In general, when I made an effort to develop rapport with my patients, my instructors _____ me.” (1= Completely Discouraged & 7= Completely Encouraged) ’05-’06 Clerkships

“In general, when I made an effort to get to know patients as unique persons, my instructors _____ me.” (1= Completely Discouraged & 7= Completely Encouraged) ’05-’06 Clerkships

Focus Group Findings Valued-added for the PCE students Continuity with faculty Access to hearing how faculty think about clinical issues Mentoring Peer group support Sense of belonging to something larger than self

“Today I was in a public restroom at _____,and I noticed how there was paper all over the floor. For a moment I was disgusted and I thought about all the residents and physicians who work so hard to give top notch care. I wouldn’t want patients or their families to use the bathroom at ____ and feel this hospital is dirty when hospitals are supposed to be clean! So I picked up all the trash. At that moment, I realized that I had taken ownership of this hospital.”

Value-Added for the Faculty Contact and meaningful cross-discipline discussion Ability to get to know students well Personally satisfying Ability to identify strengths and weaknesses Ability to provide feedback and assist in student growth and development

Remaining Challenges Paradox of the Pullout Longitudinal, interdiscipinary curriculum Attendings vs residents?? More observation & better feedback

Challenges (cont.) Patient contact pre-workup Longitudinal contact with patients Quality of the individual clerkships Consistency Across disciplines within site, within discipline across sites, and across PCEs