1. Feedback and Evaluation in Clinical Learning Catherine Lucey, MD
Experience Examined
Feedback and Evaluation in Clinical Learning
Catherine Lucey, MD

2. Goal
Increase the comfort with which participants assess, provide feedback to, and complete evaluations on their learners

3. “Clinical Learning is Experience Examined”
Ende and Davidoff, 1992

4. Two Critical Roles Coach: Feedback/ Formative Evaluation
Information provided during the rotation, describing performance, with the intent to guide future performance.
Judge: Summative Evaluation
Judgement provided at the close of a rotation assessing whether the learner met performance standards.

5. Feedback and Evaluation: Linked but Different

6. Residents : Help! How often do you get?:
Isaacson et al. Cleveland Clinic Foundation, 1995

7. Programs : Help!
2/3s of Program Directors feel that accuracy of faculty evaluations are major concerns
APDIM Precourse, April 1999
50% of Program Directors had a recent troubling feedback/eval session
APDIM Plenary, October 1999

8. Brainstorming Session
What are the Barriers?
Brainstorming Session

9. Barriers: Fairness, Uncertainty, Time
Will my evaluation be reliable?
“ I don’t know what learners today are supposed to be able to do at this level...”
Will my evaluation be valid?
“I didn’t see them do ‘x’ enough to be able to judge fairly…”
“I can’t remember …”

10. Barriers: Sensitivity Concerns
Will my feedback be well received?
“what if they cry?”
“There goes my teaching award….”
How will my feedback and evaluation be used?
“I don’t want to ruin a promising career…”

11. Overcoming Barriers Knowledge Attitudes Skills
Expectations of the rotation
Understanding of the tools
Attitudes
Faith in the process and the learner
Skills
Targeted observation and
Constructive Feedback

12. Road Map for A Successful Rotation
Expectations
Formative Eval
Summative Eval

13. Case Study Program: University of Anywhere Learner: Terry, a PGY-1
Preceptor: Dr. Andersen, a general internist in practice on the eastern shore of Maryland. S/he knows Terry’s program director from residency. The program director remembers Dr. Andersen as a smart doctor and good teacher.

14. Beginning of the Month Choose Your Focus Choose The Right Tools
Announce your feedback intentions

15. 1. Choose your Focus: Rotation Specific Objectives
Overall Clinical Competence

16. Examples Knowledge: Skills:
Offer a prioritized differential diagnosis for common ambulatory complaints such as CP, SOB, Dizziness, Edema…….
Skills:
Use and document telephone encounters for dx and thx follow-up…...

17. 2. Choose Your Evaluation Tools
Standard Tools For Assessment
Supervised Direct Patient Care
Case Presentations
Standard Tool for Measurement:
Global Rating Scales
Behaviorally Anchored Scales
Developmentally Anchored Scales

18. BARS *: The Final Pathway
*Behaviorally Anchored Rating Scales
ABIM:
unsatisfactory satisfactory superior
Dartmouth
poor fair good excellent
Must be improved
Room for improvem
Adequate Skills
Superior Skills

19. Developmental Scales Difficulty despite Does with assistance
Often does w/o assist
Usually does w/o assist
Always expert, w/o assist

20. Other Tools to Consider
Medical Record Audit
Chart stimulated recall
Mini CEX
Standardized Patients
Nursing and Patient Evaluations

21. 3. Announce Your Feedback Intentions
Why will feedback be given?
Be all you can be...
When will feedback be given?
Micro-feedback
Macro-feedback or Formative Evaluation
How will feedback be given?

22. Heart of the Rotation Use Micro-feedback skills to coach the learner:
What did they do well?
Where do they need to improve?
What is the next step for them?

23. Micro-feedback shapes behavior
Timely: Case by Case, 1st Hand
Focused: Choose your pearls
Expected: Part of the culture
Applied: Next step, next patient, goal directed
Documented: Progress Notes

24. How to give effective micro-feedback
Look for behaviors to reinforce as well as correct
Be specific about what you saw
Avoid speculating on intentions
Use nonjudgmental language
Encourage self evaluation
Identify next steps

25. Reinforcing Feedback
“I was impressed that you gave Mrs.. Jones plenty of time to tell her story.” (specific and reinforcing)
When you are patient like that, you often get a more accurate history (nonjudgmental and reinforcing)
The next step for you is to practice using clarifying questions for symptoms such as chest pain.

26. Corrective Feedback
“I found a very large liver on Mr. Smith, why do you think you underestimated its size?” (specific, encourages self evaluation)
“When you examine for a liver, it is important to undress the patient.” (nonjudgmental, forces self evaluation)
“As a next step, let’s go over how you position your hands”

27. Embarrassment and Humiliation
Humiliation: Externally driven
generally negative experience
avoidable with skilled feedback techniques
Shame/Embarrassment: Internally driven
neither entirely avoidable nor entirely negative

28. Progress Notes

29. Feedback Progress Notes
Action: Document case by case impressions
Goal: Identify patterns of strengths and weaknesses in multiple encounters
Outcome: More specific formative and summative evaluations

30. Group Exercise
Using the scenarios in your handout, role play micro-feedback and progress note completion
Switch roles to allow each individual to experience the role of feedback provider

31. Formative Evaluation= Coaching Role Continues
Midpoint
Formative Evaluation=
Macro Feedback
Coaching Role Continues ?

32. Macro-Feedback Use patterns identified during micro-feedback
Concretely compare to the agreed upon goals
Plan of action to achieve the end goals
more specific = more achievable

33. Skill Set: Communicating Evaluations
Ask: How am I doing as a teacher?
Ask: What do you see as your strengths; then weaknesses?
Tell: Summarize your observations and ideas
Ask: What is the take home message, next steps?

34. Group Exercise
Practice the Ask, Ask,Tell, Ask format of leading an evaluative session
Check your action plans for specificity

35. End of the Month Summative Evaluation: Did they meet standards? Met
Exceeded
Failed

36. Skill Set: Pitfalls in Evaluation
Range Restriction
Hawk/Dove
Halo Effect

37. Group Exercise
Complete the Attached evaluation form, using the notes that you have taken during the role play sessions.

38. Conclusions
Feedback and Evaluation are critical components of successful clinical learning
Optimal performance of these skills requires collaboration between the program and preceptor

39. Conclusions Less is Frequently More: Feedback is a two way street
Choose your focus for evaluation
Provide feedback frequently, in small bits of manageable information
Next steps should be measurable
Feedback is a two way street