Angela Buffington, PhD, MA Milestone Evaluation of Physician-Patient Communication: Advances in the MERIT to Assess Resident Skill Angela Buffington, PhD, MA Erin Westfall, DO Keith Stelter, MD, MMM

The Issue ACGME Milestone Project gave educators a framework through which to assess development of resident physicians Challenge: Educators are tasked with understanding Milestone language and application while simultaneously fostering skill development

Our Idea: Minnesota Evaluation of Relationship Interaction Tool (MERIT) For competencies related to physician-patient communication Tool to assess resident skill and enhance educational goals Used while observing resident physician-patient encounters to track progress and set goals to improve communication

What we’ve done thus far Milestones pertaining to physician-patient communication were embellished Generated detailed behavioral anchors that indicate competency with a given Milestone Exhibiting proficiency with these behavioral anchors then provides evidence that a resident is competent with a Milestone

The behavioral anchors Taken from numerous existing scientific papers addressing physician communication Motivational Interviewing PCOF AIDET 4 Habits Kalamazoo Criteria CAT SPIKES Original Revised which anchors fit best with which Milestone Revised wording to be more user-friendly Benefit of eliminating numeric ratings

Novel elements of MERIT Created from Milestones (rather than fitting the Milestones into an existing tool) Incorporates open-ended discussion questions Includes encounter ratings by patient & resident Can focus on one element of communication or the entire process

Where we’re at Done: Content validity Used standards of assessment set by the ACGME Behavioral anchors from existing measures and tools that pertain to physician communication Face validity Engaged multi-disciplinary faculty experts to select, align and judge behavioral anchors

Done: Removed Level 1-5 labels so that MERIT is truly text based Made behavior anchors chronological to follow flow of encounter First attempt at electronic version

Next steps Inter-rater reliability Multiple raters view several video recordings of resident-patient encounters Target reliability is kappa ≥ 0.90 Electronic version Getting the format to be most user friendly Automating a Milestone report

Questions that remain Does the MERIT give residents useful and relevant feedback? Does the MERIT help faculty track resident progress on Milestones? Will the MERIT improve efficiency of Clinical Competency Committee meetings? Will the electronic version of the MERIT be user friendly?

References The Accreditation Council for Graduate Medical Education and The American Board of Family Medicine (2013). The Family Medicine Milestone Project. http://www.acgme.org/acgmeweb/Portals/0/PDFs/Milestones/FamilyMedicineMilestones.pdf Baile WF, Buckman R, Lenzi R, Glober G, Beale EA, Kudelka AP (2000). SPIKES—A Six-Step Protocol for Delivering Bad News: Application to the Patient with Cancer. The Oncologist, 5, 302-311. Mauksch L (2011). Patient Centered Observation Form. https://depts.washington.edu/fammed/files/PCOF%205.16.2011_0-2.pdf Miller, WR, & Rose, GS (2009). Toward a theory of motivational interviewing. American Psychologist, 64(6), 527-537. Myerholtz L (2014). Assessing Family Medicine Residents’ Communication Skills From the Patient’s Perspective: Evaluating the Communication Assessment Tool. Journal of Graduate Medical Education, September, 495-500. Peterson EB, Calhoun AW, Rider EA (2014). The reliability of a modified Kalamazoo Consensus Statement Checklist for assessing the communication skills of multidisciplinary clinicians in the simulated environment. Patient Education and Counseling, 96, 411–418. Stange KC (2009). A Science of Connectedness. Annals of Family Medicine, 7, 387-395. Stein T, Frankel RM, Krupat E (2005). Enhancing clinician communication skills in a large healthcare organization: A longitudinal case study. Patient Education and Counseling, 58; 4–12.

@mnecw (Dr. Westfall); @UMNFamilyMed University of Minnesota Family Medicine and Community Health https://www.youtube.com/user/UMNFamilyMedicine