Angela Buffington, PhD, MA Milestone Evaluation of Physician-Patient Communication: A New Tool to Assess Resident Skill Angela Buffington, PhD, MA Keith Stelter, MD, MMM Erin Westfall, DO

The Issue ACGME Milestone Project gave educators a framework through which to assess development of resident physicians Challenge: Educators are tasked with understanding Milestone language and application while simultaneously fostering skill development

Our Idea: Minnesota Evaluation of Relationship Interaction Tool (MERIT) For the competency domain of Communication Tool to assess resident skill and enhance educational goals Used while observing resident physician-patient encounters to track progress and set goals to improve communication

What we’ve done thus far Each of the 4 Milestones pertaining to Communication was embellished Generated detailed behavioral anchors that indicate mastery of a given Milestone skill Exhibiting proficiency with a set of behavioral anchors provides evidence that a resident mastered a Milestone skill

The behavioral anchors Taken from numerous existing scientific papers addressing physician communication Motivational Interviewing PCOF AIDET 4 Habits Kalamazoo Criteria CAT SPIKES Original Revised which anchors fit best with which Milestone Revised wording to be more user-friendly Benefit of eliminating numeric ratings

Novel elements of MERIT “Communication is a fluid process. Thus, inherent to any rating tool is a risk of restricting the dialog.” –CJ Peek Incorporates open-ended discussion questions Includes encounter ratings by patient & resident Can focus on one skill or the entire process

Where we’re at Done: Content validity Used standards of assessment set by the ACGME Behavioral anchors from existing measures and tools that pertain to physician communication Face validity Engaged multi-disciplinary faculty experts to select, align and judge behavioral anchors

Next: Inter-rater reliability Multiple raters view several video recordings of resident-patient encounters Target reliability is kappa ≥ 0.90

Questions that remain Does the MERIT give residents useful and relevant feedback? Does the MERIT help faculty track resident progress in the domain of Communication? Will the MERIT improve efficiency of Clinical Competency Committee meetings? Do we have the most relevant behavioral anchors for each Milestone?

References The Accreditation Council for Graduate Medical Education and The American Board of Family Medicine (2013). The Family Medicine Milestone Project. http://www.acgme.org/acgmeweb/Portals/0/PDFs/Milestones/FamilyMedicineMilestones.pdf Baile WF, Buckman R, Lenzi R, Glober G, Beale EA, Kudelka AP (2000). SPIKES—A Six-Step Protocol for Delivering Bad News: Application to the Patient with Cancer. The Oncologist, 5, 302-311. Mauksch L (2011). Patient Centered Observation Form. https://depts.washington.edu/fammed/files/PCOF%205.16.2011_0-2.pdf Miller, WR, & Rose, GS (2009). Toward a theory of motivational interviewing. American Psychologist, 64(6), 527-537. Myerholtz L (2014). Assessing Family Medicine Residents’ Communication Skills From the Patient’s Perspective: Evaluating the Communication Assessment Tool. Journal of Graduate Medical Education, September, 495-500. Peterson EB, Calhoun AW, Rider EA (2014). The reliability of a modified Kalamazoo Consensus Statement Checklist for assessing the communication skills of multidisciplinary clinicians in the simulated environment. Patient Education and Counseling, 96, 411–418. Stange KC (2009). A Science of Connectedness. Annals of Family Medicine, 7, 387-395. Stein T, Frankel RM, Krupat E (2005). Enhancing clinician communication skills in a large healthcare organization: A longitudinal case study. Patient Education and Counseling, 58; 4–12.

@UMNFamilyMed University of Minnesota Family Medicine and Community Health https://www.youtube.com/user/UMNFamilyMedicine