CCC SWOT Workshop

Learning Objectives Clarify the critical role of the CCC in the assessment system Identify strengths and weaknesses of the current CCC structure and process Capitalize on opportunities to improve the quality of the CCC structure and process Identify potential threats to the CCC structure and process Create some first steps towards quality improvement (QI) of the CCC structure and process- i.e. an “action plan” Speaker Notes: slide #2 By the end of the session, the CCC faculty should be able to do all of the items listed as “learning objectives”

What is the “Job” of the CCC? The Clinical Competency Committee should: Review all resident evaluations semi-annually; (Core) Prepare and ensure the reporting of Milestones evaluations of each resident semi-annually to ACGME; (Core) Advise the program director regarding resident progress, including promotion, remediation, and dismissal. (Detail)" (ACGME, Internal Medicine Program Requirements) Speaker Notes: slide #3 We know the CCC “job statement” listed in the internal medicine program requirements (review them if need be) BUT, how engaged are your CCC members in the process? (seque to next slide)

Role of CCC in the Assessment System “The CCC serves several purposes, for the program director, the program itself, the faculty, the residents/fellows, the ACGME, and the specialty. The ultimate purpose is to demonstrate our accountability as medical educators to the public, that our graduates will provide high quality, safe care to our patients and maintain the standards of the health care system.” (Andolsek et al. ACGME CCC Guidebook) Speaker Notes: slide #4 We know how the ACGME describes the function of the CCC Now that we’ve been running our CCCs for some time, does this description fully describe the CCC’s role? – If you have the time, get some responses from the group. The statement above suggests that the CCC has a professional responsibility to stakeholders. Do the members of your CCC accept this responsibility? Segue to the next slide----Apart from ACGME’s job description for CCC’s, what are some ways that CCCs can become more engaged in the assessment process? (segue to next slide)

Identify curriculum issues Identify assessment issues What Do CCCs Do? A Lot! MAKE JUDGEMENTS Make sense of data Identify needed data Identify curriculum issues Identify assessment issues Work as a group Advise PD Track learning Provide feedback In thinking about providing faculty development for CCCs it helps to think about what CCCs do A little schematic like this can help you think about where you want to focus your efforts. Taking the time to breakdown the work of the CCC also helps you decide if the structure of your CCC supports the work they have to do.

Engaging the CCC Beyond the ACGME’s definition, CCCs can be involved in specific tasks: e.g. Identifying curriculum gaps Curriculum mapping Providing feedback to residents Providing feedback to core faculty Annual program evaluation Apart from the ACGME’s description of the CCC, there are multiple ways that CCC members can contribute to the residency program (see slide for examples) Each program will need to think about it’s expectations of CCC members and thus define their role The process of defining expectations of CCC members is key to empowering them to meet the professional responsibility expectations described in the ACGME CCC guidebook.

CCC Membership & Structure Membership Qualifications Knowledgeable about curriculum, & next accreditation system? Core faculty? Chief residents? Medical education background? PD/APD? Non-physicians? Does your current CCC structure best support their work? Speaker Notes: slide #7 In order for the CCC to be successful at their job, you also need to think about who should be on the committee and why- if there is time, elicit input from the group about this. So, keeping in mind the role of the CCC and who makes up the committee, it is important to see how the CCC fits into the larger assessment system (segue to next slide)

Role of the CCC in the Assessment System Speaker Notes: slide #8 Please refer to the session guide for a full description of the diagram, however the following points should be made: Assessment isn’t just an event in one moment e.g. a resident’s performance on a mini-CEX or their end-of –rotation evaluation on a given experience. Yes, there is a plethora of data, but most importantly there are lots of people (stakeholders with various needs- residents, PD, core faculty, CCC) working within a unique educational milieu (institutional culture, actual versus hidden curriculum etc) who are seek to provide the best possible care to our patients. The CCC has a “bird’s eye view” i.e. a strategic position in the assessment system- i.e. it “sees” all the incoming data, synthesizes it and sends out both milestones ratings AND feedback to key stakeholders (residents, PD). Ultimately, that feedback can impact the public. Due to the importance of the CCC in the assessment system, it should engage in quality improvement (i.e. “QI”) Created by Ekpenyong, A 2015, based on schematic in ACGME CCC Guidebook 2015

Perform a CCC SWOT Analysis Group Exercise Perform a CCC SWOT Analysis Speaker Notes: slide #9 It’s time for the SWOT Explain what the SWOT is One large group or create small groups?- Depends on the number of participants

Strengths Weaknesses Opportunities Threats What is a CCC SWOT? Clearly defined CCC role Weaknesses All opinions are not heard Opportunities Work closely with the PEC Threats No time allotted for CCC faculty Speaker Notes: slide # 10 Define each of the components of the SWOT and provide an example: Strengths i.e. “What are the strengths of our CCC?” e.g. our CCC has a clearly defined role, Members with a diverse expertise Weaknesses i.e. “What are the weaknesses in our CCC (e.g. with respect to its structure, processes, output)?” e.g. All opinions are not heard Opportunities i.e. “What are some opportunities to make our CCC more effective/efficient?” e.g. Work closely with the program evaluation committee (PEC) Threats i.e. “What are the potential threats to our CCC?” e.g. Lack of time allotted to CCC to do it’s work Let the participants know that they will work as a group to fill in the SWOT (i.e. each of the 4 squares). (SWOT analysis. Wikipedia. https://en.wikipedia.org/wiki/SWOT_analysis)

Create an “Action Plan” (Large Group Activity) CCC SWOT Create an “Action Plan” (Large Group Activity) Speaker Notes: Slide #11 You will lead this activity You can divide the participants into small groups OR use a flip chart to do this with the large group Let participants know that a real action plan will occur as a result of their SWOT analysis- as such the plan needs to be feasible and ideally with a measurable outcome The whole group will come back together after the SWOT analysis (if done in small groups) to work on this It’s ok to skip the “HOW” i.e. “implementation plan” at this point (or just jot down some initial thoughts) if you are running out of time. The responsible party will work on this later. Discuss the process through which the CCC will follow up on the action plan- will the “responsible party” for each action item report to the CCC chair? Will a subcommittee need to be formed to over see the action plan? Brainstorm about this prior to the session keeping in mind what is feasible in your setting.

Strengths Weaknesses Opportunities Threats CCC SWOT (Example) Clearly defined CCC role Member diversity CCC provide feedback Residents engaged in CCC processes Weaknesses Not doing “QI” Role of CCC not clearly defined All opinions are not heard Need faculty development Opportunities Work closely with the PEC Engage in QI activities Threats No time allotted for CCC faculty CCC faculty lack confidence in voicing opinions Speaker Notes: slide # 12 After each small group/large group reports their SWOT, you can offer these examples before moving on to the Action plan. In these examples “QI” refers to various quality improvement exercises to improve the CCC/assessment system e.g. -performing curriculum mapping -giving each CCC member a specific assignment to work on with respect to identifying curriculum gaps -curriculum mapping -reviewing the milestones ratings of the CCC- are there patterns that suggest possible rating errors? -assessing the quality of feedback from CCC to key stakeholders – residents, program director (for annual program evaluation) etc. -does the current committee structure (such as use of subcommittees) foster efficiency? “PEC” = Program Evaluation Committee

Next step: Expectations for follow up on the action plan WHAT? WHO? HOW? (What does the CCC wants to work on based on the SWOT?) i.e. who is willing to take this on? Implementation Plan Speaker Notes: slide #13 Introduce the Action Plan The goal is to have a real outcome from this exercise therefore the action plan must be feasible. Better to do something small and feel good about it than do nothing at all or too much and feel overwhelmed. This is an activity for the entire group to do together Next step: Expectations for follow up on the action plan

Wrap Up Take home points: The CCC plays an important role in the assessment system with the potential to drive QI A SWOT analysis can: generate ideas for QI in assessment be a great first step in a faculty development program for CCCs Speaker Notes: slide #14 Review the take home points Emphasize the follow up on the action plan created