University of Texas at Houston Family Medicine Residency Program Creating a Resident Performance Timeline: Facilitating the Work of the Clinical Competency Committee Rolf Montalvo, M.D. University of Texas at Houston Family Medicine Residency Program

Disclosures None

The Problem Fitting resident assessment tools into… A succinct performance description… And transforming it into a milestone level

Clinical Competency Committee “Using a CCC will increase the chances of a more balanced summary evaluation of a resident”. “…advantage of a CCC is that more voices can lead to more faculty member interest and engagement in resident education”. ACGME, December 2013 Clinical Competency Committees

“…observable steps on the continuum of increasing ability…” Milestones “…observable steps on the continuum of increasing ability…” ACGME, October 2013 FM milestones/NAS presentation

Assigning a Milestone Requires Knowledge of the Milestones 22 different milestones Each milestone has 5 narratives 22 X 5 = 110 narratives

Approximately 30-32 individual documents to assess Per Resident, Per Year… 12 rotation evaluations 4 quarterly clinic evaluations 1 self evaluation 1 nursing evaluation 1 staff evaluation 1 patient evaluation 1 In-Training exam score report 4 quarterly didactic attendance records 1 procedure log/OB experience log +/- 2 unsolicited comments +/- 2 Academic letters (warning/probation/concern) “hidden landmines” +/- 2 Academic letters of excellence “hidden diamonds” Approximately 30-32 individual documents to assess per resident.

A Lot of Data 110 possible milestones + 30-90 documents per resident This much data still requires a pre-review by one or two faculty.

Visualize Compare Participate TIMELINES Visualize Compare Participate

Anatomy of a Timeline Larry Bird, M.D. End of PGY2 review CCC meeting:6/1/14

Interacting with a Timeline Displayed during a CCC meeting “Guided” by the faculty reviewer or by CCC chair TIKI-TOKI http://www.tiki-toki.com

POSITIVES Visually rich. Gives the “big picture”. Reflects changes over time. Brings more faculty into a discussion. Picks up on “incidents” or one-time events. More information in less amount of TIME.

NEGATIVES It takes TIME to create a timeline (initially). Requires WEB and a good projector set-up May oversimplify some events.

SUMMARY Milestones require review of a large amount of data on each resident. Purpose of CCC is to involve more faculty in discussions and educational activities. Milestones should be assigned based on a continuum of performance.

SUMMARY (continued) Timelines save time by reviewing same amount of information in less time. Timelines still require a faculty reviewer for each resident. Initial transfer of information is the most burdensome step.

References ACGME, December 2013 Clinical Competency Committees slide presentation, www.acgme.org 2. Tiki-Toki Web-based timeline software, www.tiki-toki.com

Please evaluate this session at: stfm.org/sessionevaluation