1. Presenters: Dr. Rae Wright, Family Medicine of Southwest Washington Dr. Zinna Johns, East Pierce Family Medicine Hosted by: Family Medicine Residency Network Webinar: October 1, 2014 CCC: Lessons Learned – Two Programs, Two Case Examples

2. Structure  Program 1: Family Medicine of Southwest Washington  Developing the CCC Team  Developing and Determining Evaluation Tools  Case: Resident Profile  Program 2: East Pierce Family Medicine  CCC Background  Case: Resident Profiles – CCC Discussion and Response  Outcome  Discussion

3. CCC and Case 1 Family Medicine of Southwest Washington Presented By: Dr. Rae Wright

4. Strategically Select CCC Members  Start with a champion  Respected and trusted by faculty and residents  Active in teaching and evaluating residents in a variety of settings  Interested in learning Milestones lingo  History of being collaborative in meetings, etc.  FMSW has 6 members, 5 full spectrum FM faculty and one BH faculty

5. Develop a Milestones Based Evaluation System  Collect aggregate date on Milestones over time  Tools should be easy to interpret  New Innovations has built-in tools  Direct vs. indirect evaluations  Shift cards  Milestones reports  Gradually integrate new evaluations

6. Periodic Meetings for CCC  Discussion of residents of concern  Use Competencies and Milestones based language for discussion  FMSW Style  CCC meets 1-2 times per month  Must have at least 3 members present for interim meeting. Usually 4-5.  Pre-biannual meeting with other faculty including advisors

7. Case 1: Concern  “Working to improve documentation – some uncertainty about what is needed.”  “Still struggling on nights to get work done by [themself], as well as learning about all the small extra tasks that are required, but once [resident] is shown will then consistently perform them. Visible improvement over the few days I was with [resident].”  “Needs some improvements in organizational skills to prioritize and perform duties as needed for care of patients.”  “Presentations are a work in progress. I encourage [resident] to avoid extraneous comments and questions during presentations. Presentations were initially difficulty to follow due to the lack of structure, but they improved in the week we had together.”  “Presentations not yet polished, can be scattered.”  “Does not consistently carry pager when on call.”

8. Discussion  Competencies/Milestones of Concern  Patient Care (PC-1)  Professionalism (Prof-1,2)  Communication (C-3,4)  Plan for Improvement  Seek out feedback to improve performance real time.  Carry pager as required.  Focus on task at hand before moving to next tasks.  Practice oral presentations as part of active precepting and with senior residents.

9. Semi-Annual Meetings  Preparation  Use support staff to gather all data beforehand  Pre-meet with advisors and other available faculty for Resident Review  Meeting  Consider splitting into 2 groups  Use a time keeper

10. Case 1: Resident Profile  “Enthusiastic, energetic, and always eager to learn.”  “Actively seeks out feedback and takes suggestions well.”

17. After the CCC Semi-Annual Review  Milestones information completed in NI  Email sent to advisors with instructions and meeting time  Advisors review all information with advisees, including Milestones info on NI  Informal vs. formal feedback to CCC after advisors meet with advisees

18. What did we do with our resident?  Interim meetings with advisor  Active precepting in clinic  Fine tune presentations when in clinic  Shadow senior residents in the inpatient setting to see the other side  Resident received all feedback well and has made some progress

19. CCC and Case 2 East Pierce Family Medicine Presented By: Dr. Zinna Johns

20. CCC Background  3 CCC meetings per class broken up into specific teams: Pine, Oak, and Maple.  Thus a total of 9 CCC meetings biannually at EPFM.  Advisor(s) for each team must be at the CCC for their advisees. Other faculty members may attend if schedule allows.  Program Director, Program Coordinator and Behavioral Health Specialist present for all CCC meetings. CCC EPFM Style

21. Case 2: Resident Profile: Above or Below the Bar?  SA is one of 6 residents in the 1 st class of residents at EPFM  At time of CCC, is half way through her 2 nd year of residency  In general, performance is “meets” or “exceeds expectations”  Had a reputation as a resident that “sets the bar”

22. Case 2: Resident Profile: Above or Below the Bar?  Spring 2014 CCC for R2s on average took about 35-42 minutes.  The outlier was the CCC evaluation for SA, which took 75 minutes.  Areas of concern were: SBP4, Prof1-4, and Com3.  Tools used for evaluation include: Faculty observation, 360 evaluations, ITEs, Rotation evaluations that were mapped to Milestones

29. Case 2: Resident Profile

30. Case 2: Concern  Through out residency, SA has had cyclical episodes of interactions that were concerning for lack professional conduct.  Behaviors such as explosive response to changes to a previously established policy; inappropriate selection of time and modality of giving negative feedback (to med students, peers, and faculty); repeated inflexibility with changes that are perceived as unfair  Resident is effectively isolating self from fellow residents because of lack of willingness to be a team player.

31. Case 2: Mock CCC Discussion

32. Case 2: Outcomes  Resident was given Milestones feedback, after completing self assessment with the Milestones packet.  On average scored 2.5. except for the areas of concern.  On self assessment, scored self at 3.5.  Areas of weakness were reviewed with resident.  Resident was informed that the behavior was problematic and needed to change.

33. Case 2: Outcomes  Reviewed the cyclic pattern with resident and outlined correlation with stressful schedules such as night float.  Resident was directed to Behavioral Health Specialist for tools and/or reading a book about professionalism and communication.  2 follow-up meetings have occurred since.  SA never met with BH (now 4-5 months later). Resident chose a book about spirituality at the workplace and felt overall improvement.  Planned pre-CCC meeting with SA to revisit areas of weakness, which persist with clarification that persistence these behaviors could lead to formal process.

34. Lessons Learned  CCC Meetings increase in value with higher number of faculty members.  The more faculty members, the longer the CCC meetings.  If there is a prolonged discussion on a specific Milestone for a certain resident, that person is possibly struggling in that area.  CCC’s task is simply to evaluate the data and assign the resident’s progress for the Milestones.  This must be separated from identification of whether a resident is in difficulty.  CCC is not intended for problem solving and the tendency to do so will limit efficiency.

35. Questions & Discussions