1. Delivering the Milestones Evaluation: Structuring Feedback & Comments from the CCC Dr. Eric Beachy, MD, Dr. Manju Thothala, MD, Dr. Nicole McGuire, DHSc April 4, 2016

2. AAFP Disclosure 2 It is the policy of the AAFP that all individuals in a position to control content disclose any relationships with commercial interests upon nomination/invitation of participation. Disclosure documents are reviewed for potential conflicts of interest and, if identified, conflicts are resolved prior to confirmation of participation. Only those participants who had no conflict of interest or who agreed to an identified resolution process prior to their participation were involved in this CME activity. All individuals in a position to control content for this session have indicated they have no relevant financial relationships to disclose.

3. Objectives 3  Give an example of how one residency facilitates thorough and efficient Milestones evaluations during CCC sessions.  Conduct a resident Milestones evaluation using an innovative evaluation model.  Describe a method for adapting our CCC procedures and Milestones evaluation model for use in other residency programs.

4. Union Hospital Family Medicine Residency  7-7-7 program  Single-sponsored  7 core faculty  345 Bed Community Hospital  Open ICU  NICU, Inpatient Pediatrics, OB 4

5. Union Hospital Family Medicine Residency  Established in 1976 – celebrating forty years of resident education  Graduated 183 residents Many of which practice in the Wabash Valley  > 20,000 patients annually  >2,500 well child visits yearly  150 monthly deliveries 5

6. CCC Sessions  Each session 3 ½ hours  PGY1 – December and May  PGY2 – September and April  PGY3 – October and May 6

7. CCC Meeting Agenda  First 30 minutes – Presentation of Objective Data –Discuss each resident ; assigned CCC members present the following information for consideration: Duty hour logging Didactic attendance Medical record suspensions (if applicable) Evaluations from community preceptors Inpatient Medicine Service Weekly Report Upper level field note form Staff 360 Scholarly activity Study plans ITE scores Individualized education plans (IEPs) / remediation plans Community medicine assignment completion Patient encounter logging Logging of procedures Timeliness of charting/administrative responsibilities Disciplinary action or issues OSCE performance Video precepting feedback forms 7

8. CCC Meeting Agenda (cont.) 8  Remaining 3 hours – Milestones  Milestones in a PowerPoint presentation on the projector  Faculty comments are recorded - CRITICAL  For each Milestone, level of achievement is recorded

9. CCC Meeting Agenda (cont.) 9  Remaining 3 hours – Milestones  Example: Begin with PC-1 and assign a level for each resident before moving on to PC-2  Timers are helpful to keep committee members on task  Only move to next Milestone when a consensus is reached on the level assignment for each resident

10. After the CCC Meeting (cont.) 10  Education Coordinator enters the Milestones data into a template for each resident  Data are entered into a grade card for each resident (grade card program and all supplemental materials were uploaded to fmdrl.org)

11. After the CCC Meeting (cont.) 11  The Education Coordinator shares a copy of the Milestones Evaluation and Grade Card  Faculty review and sign  Education Coordinator schedules evaluation meetings with faculty advisor

12. Scheduling the Evaluation Meeting 12  Evaluation meetings are held between the resident and academic advisor  The Education Coordinator may also be present  Evaluation meetings typically last 30 minutes

13. Prior to the Evaluation Meeting 13  Resident receives the evaluation 1 week prior to the meeting to review all scores and comments

14. Evaluation Meeting Agenda 14  First 5 minutes: Education Coordinator and faculty advisor provide the resident with an overview of the Milestones, explanation of grade point average and expectations for current level of training

15. 15 Sample Grade Card

16. Evaluation Meeting Agenda (cont.) 16  Next 15 minutes: Faculty advisor reviews each page of the Milestones evaluation with the resident, highlighting CCC comments, and providing examples and meaningful feedback for both high and low scores

17. Evaluation Meeting Agenda (cont.) 17  Next 5 minutes: Faculty advisor asks resident if they have any questions or need further explanation of any covered information. Advisor and resident engage in open discussion of the evaluation

18. Wrapping Up the Evaluation Meeting 18  Last 5 minutes: Faculty uses an advisement tool as a guide to cover relevant topics with the resident (i.e., discuss goals, elective rotation selection, recognition of fatigue)  Resident receives a copy of the Milestones evaluation and Grade Card  Resident signs a copy that is placed in the resident file

19. Resident Perspective  Feedback is structured  Evaluation process is more objective 19

20. Conclusions  Audience suggestions?  Feedback? 20

21. During the break… Discuss / think about how you might implement the information you just heard. Fill out a session evaluation. 21

22. Questions Eric Beachy, MD – Faculty edbeachy@uhhg.org Nicole McGuire, DHSc – Education Specialist nmcguire@uhhg.org Manju Thothala, MD – Resident Physician mthothala@uhhg.org 22