1. The Master Clinician: Assessing and Teaching Clinical Competence
Kim Marvel, Ph.D.
David Marchant, M.D.
Kristen L. Bené, M.S.
Fort Collins Family Medicine Residency Program

2. The Master Clinician: Assessing and Teaching Clinical Competence
Overview
Introduction and objectives
Developing the core competencies
Teaching strategies
Assessment methods
Using the data
Discussion

3. Fort Collins Family Medicine Residency Program
Community-based program
Located in northern Colorado
6-6-6 program
9 faculty physicians (7.0 FTE)
5 non-physician faculty
Pharm D
CPA
Health educator
Family therapist
Psychologist

4. The Master Clinician: Assessing and Teaching Clinical Competence
Objectives:
Identify clinical competencies appropriate for intern physicians
Describe two methods for teaching the competencies
Implement a method for observing and measuring intern clinical competencies
Describe how the data can be used for promotion decisions and program improvement

5. Evolution of the “Master Clinician” Concept
Our goals:
Define abilities of a competent family physician
Create a framework to tie together all aspects of the curriculum
Motivating forces:
We were recognizing skill deficits too late
Holes in our curriculum (“Swiss cheese”)
ACGME requiring competency-based education
Wanted to define not only “competence”, but mastery as well

6. Overview: Developing the Core Competencies
How we developed the competencies:
Began with STFM and ACGME competencies
Created 1-5 scale with word anchors (adapted from Mayo Scottsdale Program)
Competencies were expanded and revised by our faculty
“Mastery” level
Currently 43 core competencies (26 for interns, 10 for R-2s, and 7 for R-3s)

7. Overview: Developing the Core Competencies
Categories of the intern core competencies:
Communication
History taking
Physical exam
Information mastery
Medical decision-making
Professional communication (eg, presenting a case)
Pharmacotherapy
Technical skills (procedures)
Personal/Professional Growth and Development

8. Overview: Developing the Core Competencies
Example of an intern communication competency:
“Establishes focus for the visit”
Needs Improvement: Does not ask for patient’s reason for visit (e.g., begins by asking about symptoms: “Tell me about your throat”)
Entry Level: Focuses the visit solely on the patient’s initial concern
Competence: Asks for initial concern(s) (e.g., “How can I help you?”) and later in visit asks for other concerns
Proficiency: Routinely asks for initial concern and, early in the visit, if there are additional concerns (e.g., “Anything else besides the sore throat?”)
Mastery: Asks for initial concern(s) and additional concerns and prioritizes them with patient in all visits

9. Overview: Strategies for Teaching the Core Competencies
Educational conferences for interns
Early in first year
Procedure clinics
Shadowing in clinic
Verbal feedback
Competency-specific teaching resources

10. Overview: Example of a competency-specific teaching resource
Establishes the Focus for the Visit
Objective: To increase the resident’s ability to effectively establish a focus for the visit.
Rationale: Establishing a focus for the visit offers the following advantages:
Most patients have between 2 and 3 concerns and the first one voiced may not be the most important or urgent (potentially important pt data may be missed)
Efficiency is increased by reducing the chance of “doorknob” questions
Saves time for in-depth discussion of complicated problems
Patients are more satisfied
Skills:
After the patient states his/her initial concern, ask “anything else?” until all concerns are identified
Prioritize the concerns with the patient to determine which one(s) will be addressed during this visit (patient and provider agree on problem priority); that is, define reasonable goals for the available time
Resources: Videotape “Establishing Focus” (10 minutes)
Evidence:
There is no evidence that establishing a focus early in the visit improves health outcomes.
Establishing a focus early in the visit reduces the chance of a late-arising concern (doorknob question) from 35% to 15% (Marvel, et al)
Establishing focus does not lengthen the visit (Mauksch, et al)
Providers rated their patients as more satisfied; patients were more satisfied with visit when a focus was established early in visit (Mauksch, et al)
Unvoiced agenda items were associated with poorer outcomes (unwanted prescriptions and non-adherence to treatment) (Barry et al)
References:

11. Overview: Methods for Assessing the Core Competencies
Shadowing in clinic
Procedure observation/checklist
360o evaluations
Faculty evaluations
(Future plans: self-assessment, patient ratings, quality indicators pulled from EHR)
(Past attempts: standardized patients)

12. Shadowing: A Method for Teaching and Assessing Core Competencies
Faculty accompanies resident into each office visit for morning or afternoon
Advantages:
Provides sufficient number of observations
Fits our resources
Least disruptive to clinic flow

13. Shadowing in Clinic How shadowing is scheduled:
Added after clinic schedule has been made
Inserted when attending-to-resident ratio allows (e.g., 2 attendings and 5 residents)
reminders are sent to attending and resident a few days prior to shadowing date

14. Shadowing in Clinic Rating forms:
For interns: 22 items in 7 categories
For upper-level residents: 14 items in 6 categories
Forms are available in attending area and can be accessed by computer
See handout

15. Shadowing in Clinic Guidelines for faculty observers:
Prior to clinic, ask resident to identify specific competencies to prioritize for that session
Maintain observer role (avoid taking over patient care)
Example: defer patient questions back to resident
Return to preceptor role between patient visits

16. Shadowing in Clinic Guidelines for faculty observers: (cont’d)
Allow resident to maintain efficiency
Try to observe all visits during an observation session
Both physician and non-physician faculty participate
After the clinic, discuss observations with resident
invite resident self-assessment
provide verbal feedback

17. Shadowing in Clinic Activity: Watch 5 minute excerpt of interview
Rate the communication competencies
Compare our ratings
Discuss inter-rater reliability and the need for faculty development

18. Using the Shadowing Data
What we do with the completed rating forms
Evaluation of resident
Program evaluation (internal)
Program evaluation (external)

19. Using the Shadowing Data: Resident Evaluation
Summative assessment of all shadowing experiences provided to faculty advisor in advance of resident annual review.
Level of competency reached determines actions taken.

20. Using the Shadowing Data: Program Evaluation (Internal)

21. Using the Shadowing Data: Program Evaluation (External)
Intern Clinical Competencies are tracked as an item on our hospital system’s balanced scorecard.
This reviews our performance in key areas for accountability purposes.
We set our balanced scorecard goals to parallel the academic year and the progress we expect to see.

22. Summary of Shadowing Advantages: Issues to consider:
This method enables direct observation of residents in clinic
The data are helpful for resident development and program decisions
Issues to consider:
Requires “buy in” from director and all faculty
Requires extra coordination to insert into schedule and summarize the data

23. Summary of Master Clinician
Provides a comprehensive framework for the overall curriculum
Defines standards (competencies) – sets clear expectations for residents
Standardizes measurements - addresses the “black box” by directly observing resident behavior
Defines advanced skills beyond proficiency

24. Contact Information
Kim Marvel, Ph.D.
(970)
David Marchant, M.D.
(970)
Kristen L. Bené, M.S.
(970)