1. Clinical Competency Committees (CCC): 3 different perspectives Sharon Dabrow: Pediatrics PD Cuc Mai: Internal Medicine PD Todd Kumm: Radiology PD

2. ACGME requirements & CCC Required in the Next Accreditation System A trained group who can make assessments of the competency of a resident based on milestones and the evaluation tools used within the residency.

3. Questions to Answer… 1.Who serves (any role for residents, chiefs) and how large 2.Role of Program Director 3.Role of committee members and how frequent the committees are meeting 4.Faculty Development & Support 5.Administrative support 6.The phase in process

4. Diagnostic Radiology How in the world can I do this?

5. Building the CCC… DATA EDUCATION FACULTY PLAN TIME AND ORGANIZATION … ADS In March of 2012 I began to build our “CCC”

6. Diagnostic Radiology 32 residents 2 chief residents (PL5) 1 Assoc. Program Director Work at TGH, Moffitt, JAHVA, ACH & Bay Pines “Site Director” Faculty assigned as “educational leader” at each site

7. Committee Structure Program Director 3 Core Clinical Faculty (TGH, MCC & JAHVA) Appointed for minimum 2 year term Selected by site director; approved by PD Chief Residents not involved Department chair Included initially as resident advocate Program Coordinator

8. Program Directors Role Program Chair = Program Director Establish “charter” for the CCC Member responsibilities Define term of members Set date & time of CCC meetings Work with Program coordinator to ensure “upload” of milestone data to ACGME ADS Revise & update evaluation process to match MILESTONES

9. The CCC “Charter” was drafted

10. Committee “roll-out” Committee members selected Easier than I expected Confirmed commitment and “TIME” of committee members Supported by Chair & site directors Sufficient time for meaningful review of resident & data Committee informational & educational meetings Quarterly prior to December 2013

11. And then I waited...

12. Minor setback. No big deal FACULTY PLAN TIME AND ORGANIZATION … ADS ACGME EDUCATION DATA

13. Timeline for “roll-out” This is going to be a piece of cake! Educate Organize Evaluate & Submit to ADS!

14. So it’s December. Let’s get back on track… Work on the education Educate Organize Evaluate & Submit to ADS!

15. From Competencies to Milestones…

16. Don’t forget the RESIDENTS Development FacultyResidents

17. Resident Education & Development

18. Resources Remember, you are not alone. Annual Meetings (ACGME & Specialty meetings) Networking with other program directors Program Director Associations

19. SHARON DABROW M.D. PROGRAM DIRECTOR PEDIATRIC RESIDENCY PROGRAM

20. Pediatrics 54 residents 2 PL3 chief residents 2 Assoc. Program Directors (1 ACH faculty) Work at All Children’s Hospital, TGH and many out patient locations

21. Committee Structure Program Director, Chair 2 Assoc. Directors 2 Chief Residents 3-4 Additional Faculty—chosen from the Core faculty. Faculty appointed for 2-3 year term Per ACGME--(Can include non-physician faculty or PL3 if desired)

22. Program Director’s Role Determine goals and organizational structure Ensure files and data available for review Work closely with office staff in preparation Can vote Reviews and determines remediation plan when necessary and ensures compliance

23. Committee Members Meet at minimum twice per year. May increase to quarterly May be required to review and follow a subset of residents during their term; mentor them? Must understand milestones and competencies Must have time to attend meetings What else can they do? Serve as observer for SCOs, mentor for various activities, serve as core evaluators. Could be responsible for coordination all materials.

24. Faculty Devp./Support Need to learn about new requirements and NAS. Should attend fac. devp workshops, etc. PD to provide education as needed

25. Administrative Support Residency coordinators critical Present at all meetings; must take specific minutes that should be reviewed by PD and placed in res. file

26. Internal Medicine Residency Cuc Mai MD Program Director

27. Morsani College of Medicine Internal Medicine Residency Program 85 residents across 3 years Inpatient rotations available at 3 different affiliate sites and another outpatient ambulatory site 4 Associate Program Directors with potentially 3 fourth year chief residents

28. Who serves on our committee? Committee chair is assigned by program director. Chair is usually an associate program director. All associate program directors, all site directors, 4 th year chief residents, and 2 additional chair appointed faculty members. Currently 9 members. Appointed faculty members serve a two year term.

29. What is Program Director’s role? Assigns chair of committee. Serves as non-voting member and mainly resident advocate. Has final decision on remediation recommendations made by committee.

30. Committee Member Duties and Roles Currently meeting every 3 months. Meetings usually last 2 hours. Committee Chair has been presenting and reviewing files for discussion. However, ultimately will assign smaller group to review files. Assignment will be based on residency class. Also, committee is now given task of evaluating our evaluation system and making appropriate changes.

31. Faculty Development & Support Started by sending committee chair to American Board of Internal Medicine (ABIM) faculty development workshop on evaluation. In the process of developing and using the ABIM workbook for continuous development. (Holmboe and Hawkins. Practical Guide to the Evaluation of Clinical Competence. ) Plan on sending a committee member to the ABIM workshop every year.

32. Administrative Support Residency coordinator present at all meetings Documentation of minutes Gives another perspective on professionalism competency

33. What we have learned? Committee members benefit from faculty development and co-mentoring on the evaluation process. We have been able to identify residents at risk earlier and remediating residents earlier takes additional resources to improve resident’s success. Have identified areas for improvement in our evaluation system and are still working on milestones evaluations. Could use committee members as mentor or direct observers especially in scenarios where evaluation is inconsistent.

34. Conclusions from all of our experiences Committee decisions can improve accuracy and timeliness of evaluations and contribute to more accurate and appropriate remediation plans. Many ways to design your CCC. Details need to fit the needs of your program and residents. The process should be dynamic; focused on improving weaknesses in the process (evaluation system, faculty understanding of milestones, remediation process, etc…)