Feedback and Evaluation in Clinical Learning Catherine Lucey, MD Experience Examined Feedback and Evaluation in Clinical Learning Catherine Lucey, MD

Goal Increase the comfort with which participants assess, provide feedback to, and complete evaluations on their learners

“Clinical Learning is Experience Examined” Ende and Davidoff, 1992

Two Critical Roles Coach: Feedback/ Formative Evaluation Information provided during the rotation, describing performance, with the intent to guide future performance. Judge: Summative Evaluation Judgement provided at the close of a rotation assessing whether the learner met performance standards.

Feedback and Evaluation: Linked but Different

Residents : Help! How often do you get?: Isaacson et al. Cleveland Clinic Foundation, 1995

Programs : Help! 2/3s of Program Directors feel that accuracy of faculty evaluations are major concerns APDIM Precourse, April 1999 50% of Program Directors had a recent troubling feedback/eval session APDIM Plenary, October 1999

Brainstorming Session What are the Barriers? Brainstorming Session

Barriers: Fairness, Uncertainty, Time Will my evaluation be reliable? “ I don’t know what learners today are supposed to be able to do at this level...” Will my evaluation be valid? “I didn’t see them do ‘x’ enough to be able to judge fairly…” “I can’t remember …”

Barriers: Sensitivity Concerns Will my feedback be well received? “what if they cry?” “There goes my teaching award….” How will my feedback and evaluation be used? “I don’t want to ruin a promising career…”

Overcoming Barriers Knowledge Attitudes Skills Expectations of the rotation Understanding of the tools Attitudes Faith in the process and the learner Skills Targeted observation and Constructive Feedback

Road Map for A Successful Rotation Expectations Formative Eval Summative Eval

Case Study Program: University of Anywhere Learner: Terry, a PGY-1 Preceptor: Dr. Andersen, a general internist in practice on the eastern shore of Maryland. S/he knows Terry’s program director from residency. The program director remembers Dr. Andersen as a smart doctor and good teacher.

Beginning of the Month Choose Your Focus Choose The Right Tools Announce your feedback intentions

1. Choose your Focus: Rotation Specific Objectives Overall Clinical Competence

Examples Knowledge: Skills: Offer a prioritized differential diagnosis for common ambulatory complaints such as CP, SOB, Dizziness, Edema……. Skills: Use and document telephone encounters for dx and thx follow-up…...

2. Choose Your Evaluation Tools Standard Tools For Assessment Supervised Direct Patient Care Case Presentations Standard Tool for Measurement: Global Rating Scales Behaviorally Anchored Scales Developmentally Anchored Scales

BARS *: The Final Pathway *Behaviorally Anchored Rating Scales ABIM: 123 456 789 unsatisfactory satisfactory superior Dartmouth poor fair good excellent Must be improved Room for improvem Adequate Skills Superior Skills

Developmental Scales Difficulty despite Does with assistance Often does w/o assist Usually does w/o assist Always expert, w/o assist

Other Tools to Consider Medical Record Audit Chart stimulated recall Mini CEX Standardized Patients Nursing and Patient Evaluations

3. Announce Your Feedback Intentions Why will feedback be given? Be all you can be... When will feedback be given? Micro-feedback Macro-feedback or Formative Evaluation How will feedback be given?

Heart of the Rotation Use Micro-feedback skills to coach the learner: What did they do well? Where do they need to improve? What is the next step for them?

Micro-feedback shapes behavior Timely: Case by Case, 1st Hand Focused: Choose your pearls Expected: Part of the culture Applied: Next step, next patient, goal directed Documented: Progress Notes

How to give effective micro-feedback Look for behaviors to reinforce as well as correct Be specific about what you saw Avoid speculating on intentions Use nonjudgmental language Encourage self evaluation Identify next steps

Reinforcing Feedback “I was impressed that you gave Mrs.. Jones plenty of time to tell her story.” (specific and reinforcing) When you are patient like that, you often get a more accurate history (nonjudgmental and reinforcing) The next step for you is to practice using clarifying questions for symptoms such as chest pain.

Corrective Feedback “I found a very large liver on Mr. Smith, why do you think you underestimated its size?” (specific, encourages self evaluation) “When you examine for a liver, it is important to undress the patient.” (nonjudgmental, forces self evaluation) “As a next step, let’s go over how you position your hands”

Embarrassment and Humiliation Humiliation: Externally driven generally negative experience avoidable with skilled feedback techniques Shame/Embarrassment: Internally driven neither entirely avoidable nor entirely negative

Progress Notes

Feedback Progress Notes Action: Document case by case impressions Goal: Identify patterns of strengths and weaknesses in multiple encounters Outcome: More specific formative and summative evaluations

Group Exercise Using the scenarios in your handout, role play micro-feedback and progress note completion Switch roles to allow each individual to experience the role of feedback provider

Formative Evaluation= Coaching Role Continues Midpoint Formative Evaluation= Macro Feedback Coaching Role Continues ?

Macro-Feedback Use patterns identified during micro-feedback Concretely compare to the agreed upon goals Plan of action to achieve the end goals more specific = more achievable

Skill Set: Communicating Evaluations Ask: How am I doing as a teacher? Ask: What do you see as your strengths; then weaknesses? Tell: Summarize your observations and ideas Ask: What is the take home message, next steps?

Group Exercise Practice the Ask, Ask,Tell, Ask format of leading an evaluative session Check your action plans for specificity

End of the Month Summative Evaluation: Did they meet standards? Met Exceeded Failed

Skill Set: Pitfalls in Evaluation Range Restriction Hawk/Dove Halo Effect

Group Exercise Complete the Attached evaluation form, using the notes that you have taken during the role play sessions.

Conclusions Feedback and Evaluation are critical components of successful clinical learning Optimal performance of these skills requires collaboration between the program and preceptor

Conclusions Less is Frequently More: Feedback is a two way street Choose your focus for evaluation Provide feedback frequently, in small bits of manageable information Next steps should be measurable Feedback is a two way street