Communicating Milestones Evaluation Data and Comments from the CCC to Residents Using an Innovative “Grade Card” Steven McDonald, MD – Program Director Nicole McGuire, MS – Education Coordinator Union Hospital Family Medicine Residency - Terre Haute, IN

Activity Disclaimer The material presented here is being made available by the American Academy of Family Physicians for educational purposes only. This material is not intended to represent the only, nor necessarily best, methods or procedures appropriate for the medical situations discussed. Rather, it is intended to present an approach, view, statement, or opinion of the faculty, which may be helpful to others who face similar situations. The AAFP disclaims any and all liability for injury or other damages resulting to any individual using this material and for all claims that might arise out of the use of the techniques demonstrated therein by such individuals, whether these claims shall be asserted by a physician or any other person. Every effort has been made to ensure the accuracy of the data presented here. Physicians may care to check specific details such as drug doses and contraindications, etc., in standard sources prior to clinical application. This material might contain recommendations/guidelines developed by other organizations. Please note that although these guidelines might be included, this does not necessarily imply the endorsement by the AAFP. Faculty Disclosure It is the policy of the AAFP that all individuals in a position to control content disclose any relationships with commercial interests upon nomination/invitation of participation. Disclosure documents are reviewed for potential conflicts of interest (COI), and if identified, conflicts are resolved prior to confirmation of participation. Only those participants who had no conflict of interest or who agreed to an identified resolution process prior to their participation were involved in this CME activity. All faculty in a position to control content for this session have indicated they have no relevant financial relationships to disclose.

Objectives Give an example of how Union Hospital FMR conducts thorough and efficient Milestones evaluations during CCC sessions Describe an innovative grading process using the Milestones evaluations Describe a method for adapting our CCC procedures and Milestones grading process for use in your residency

Background Starting in July 2014, Family Medicine Residency programs were required to start reporting on resident educational progress using the Family Medicine ACGME Milestones. As programs are creating Clinical Competency Committees (CCC’s) and testing out the Milestones, many are struggling to find a process that leads to thorough, yet efficient, evaluations for residents. The 1st CCC/Milestones meeting was held October 2013 We entered our first Milestones evaluation data into WebAds in December 2014.

Union Hospital Family Medicine Residency 7-7-7 program 345 Bed Community Hospital Single Residency Open ICU 6 core faculty NICU, Inpatient Pediatrics, OB

Union Hospital Family Medicine Residency Established in 1976 Graduated 176 residents > 20,000 patients annually > 2,500 well child visits yearly 150 monthly deliveries

CCC Sessions Each session 3 ½ hours PGY1 – December and May PGY2 – September and April PGY3 – October and May

CCC Meeting Agenda First 30 minutes – Presentation of Objective Data Discuss each resident ; assigned CCC members present the following information for consideration: Duty hour logging Didactic attendance Medical record suspensions (if applicable) Evaluations from community preceptors Staff 360 Scholarly activity Study plans ITE scores Individualized education plans (IEPs) / remediation plans Community medicine assignment completion Patient encounter logging Logging of procedures Timeliness of charting/administrative responsibilities Disciplinary action or issues OSCE performance Video precepting feedback forms

CCC Meeting Agenda (cont.) Remaining 3 hours – Milestones Milestones in a PowerPoint presentation on the projector Faculty comments are recorded - CRITICAL For each Milestone, level of achievement is recorded

CCC Meeting Agenda (cont.) Remaining 3 hours – Milestones Example: Begin with PC-1 and assign a level for each resident before moving on to PC-2 Timers are helpful to keep committee members on task Only move to next Milestone when a consensus is reached on the level assignment for each resident

After the CCC Meeting Education Coordinator enters the Milestones data into a template for each resident Data are entered into the grade card for each resident

After the CCC Meeting (cont.) The Education Coordinator shares a copy of the Milestones Evaluation and Grade Card Faculty review and sign (turnaround time - 1 week) Education Coordinator schedules evaluation meetings with faculty advisor

Scheduling the Evaluation Meeting Evaluation meetings are held between the resident and academic advisor The Education Coordinator may also be present Evaluation meetings typically last 30 minutes

Evaluation Meeting Agenda First 5 minutes: Education Coordinator and faculty advisor provide the resident with an overview of the Milestones, explanation of grade point average and expectations for current level of training

Evaluation Meeting Agenda (cont.) Next 15 minutes: Faculty advisor reviews each page of the Milestones evaluation with the resident, highlighting CCC comments, and providing examples and meaningful feedback for both high and low scores

Evaluation Meeting Agenda (cont.) Next 5 minutes: Faculty advisor asks resident if they have any questions or need further explanation of any covered information. Advisor and resident engage in open discussion of the evaluation

Evaluation Meeting Agenda (cont.) Last 5 minutes: Faculty uses an advisement tool as a guide to cover relevant topics with the resident (i.e., discuss goals , elective rotation selection, recognition of fatigue) Resident receives a copy of the Milestones evaluation and Grade Card Resident signs a copy that is placed in the resident file

PGY1 – 90 Evaluation Contains a Grade Point Average (GPA) Shows a median GPA for the class Provides an at-a-glance look at residence performance in each of the core competencies Highlights areas needing improvement

PGY1 – 6 mo Evaluation

PGY1 – 11 mo Evaluation

PGY2 – 3 mo Evaluation

Grade Card Demonstration Handout Grade Card Instructions CD with customizable Grade Card program

Discussion Since implementing our “Grade Card”… Obstacles Successes Lessons Learned Feedback from audience?

One last thing… Please complete the online evaluation.

Questions? Steven McDonald, MD – Program Director samcdonald@uhhg.org Nicole McGuire, MS – Education Coordinator nmcguire@uhhg.org