1. Maria Gabriela Castro MD Archana Kudrimoti MBBS MPH David Sacks PhD
Developing a Rubric for Assessment and Feedback for Medical Students’ Clinical Reasoning
Maria Gabriela Castro MD
Archana Kudrimoti MBBS MPH
David Sacks PhD

2. Disclosures
none

3. Show of hands

4. Our goal

5. How did I do? “Great job!” “Will make an wonderful physician”
“Well-liked by patients”
“Excellent work”
“Kind and thoughtful”
“Team player”
Pleasant, A&Ox3, NAD

6. Learning objectives Describe existing tools Create a rubric
Discuss best practices
1. Describe existing tools used to assess clinical reasoning
2. Create a rubric to provide instant formative feedback to clerkship students giving case presentations
3. Demonstrate best practices for training community faculty on the use of assessment tools on giving real-time formative feedback to clerkship students

7. It is challenging to give feedback
How do get good feedback?
Feedback is the process by which the teacher provides learners with information about their performance for the purpose of improvement
Course correction to keep on track = narrows the gap between actual and desired performance
Feedback = teaching
Different from evaluation (= a grade of final judgement of abilities compared to a standard)
Without feedback …
Why do we care about feedback?
How do we get good feedback?
Let’s review what we know about how to give effective feedback
Private, real-time, non-judgemental
Allow learner to discuss his/her experience
Start with the positive
Provide information-specific and issue-focused observations
Limit the amount of feedback
Link to learner goals
Summarize essential points
Make a plan
** The success of a formative feedback session is dependent on three key areas:
structure,
Immediate, focus on knowledge, attitudes, behaviors
Self-assessment, teacher assessment, plan
format, and
Content
**Levels of feedback:
Minimal:
Approval/disapproval (verbal or nonverbal)
Specific/behavioral:
Approval/disapproval + explanation of why
Interactive:
Elicits self-assessment
Includes feedback on self-assessment
Develops action plan
Allow reaction to feedback

8. How do your receive feedback?
Goal is to help community faculty give students effective formative feedback on clincial reasoning skills, so let’s review challenges and best practices for giving and getting feedback

9. Think/Pair/Share Role play (30 seconds) Give your student feedback
React to the feedback given
Feedback
Maybe save this for later – after we build the rubric, see how we apply then watch the feedback given
What makes good feedback?
Two videos
ARCH and sandwich

10. Overview of tools for medical student assessment

11. Clinical reasoning
Clinical reasoning is a context-dependent way of thinking and decision making in professional practice to guide practice actions. (Higgs, 2010)
Educational strategies to promote clinical reasoning Judith Bowen MD NEJM :

12. Clinical reasoning Patient’s story Data Acquisition
Accurate “Problem representation”
Generation of hypotheses
Search for and selection of Illness script
Diagnosis
a thorough but directed clinical history, with initial hypothesis generation and subsequent testing through specific questioning
selection of a primary diagnosis and differential diagnosis in order of likelihood
physical examination directed at gathering further data necessary to confirm or refute the hypotheses
thoughtful and critical selection of investigations to gather additional data
implementation of a targeted and rationalized management plan.

13. EPA for medical students
Gather a history and perform a physical examination
Prioritize a differential diagnosis following a clinical encounter
Recommend and interpret common diagnostic and screening tests
Enter and discuss orders/prescriptions
Document a clinical encounter in the patient record
Provide an oral presentation of a clinical encounter
Form clinical questions and retrieve evidence to advance patient care

14. EPA for medical students
Give or receive a patient handover to transition care responsibility
Collaborate as a member of an inter-professional team
Recognize a patient requiring urgent or emergent care, and initiate evaluation and management
Obtain informed consent for tests and/or procedures
Identify system failures and contribute to a culture of safety and improvement
Perform general procedures of a physician

16. Levels of skill
Bloom’s taxonomy of cognitive skills (1956)Knowledge, Comprehension, Application, Analysis, Synthesis, Evaluation
Drefyrus level of performance( 1986) Novice, Advanced beginner, Competent, Proficient, Expert , Master
Miller’s learning pyramid ( 1990) Knows, Shows, Shows How, Does
Pangaro’s rime scheme (1999) Reporter, Interpreter, Manager, Educator

17. Methods for student assessment
Written Exercises
MCQ, problem solving questions, Essays,
reflection
Assessment by supervising clinicians
Global, Direct observation, oral exams
Clinical simulations
Standardized patients, high-tech simulations
Multi-source /360- degree evaluation
Peer assessment, patient/ staff assessment self assessment, portfolios
Assessment in Medical Education Ronald M. Epstein, M.D. N Engl J Med 2007; 356:
Learn language and theory of clinical reasoning CR
Gain Exp. CR
Understand CR – prelude to real Exp.
Itten exc
Reflect/reason

19. Direct observation assessment tools
Various tools are available
Assess history taking ability, interpersonal and communication counselling, skills specific like examination, scaled with behavioral anchors
Tools for Direct Observation and Assessment of Clinical Skills of Medical Trainees A Systematic Review - Jennifer R. Kogan et al , MD JAMA, September 2009—Vol 302, No. 12 (Reprinted)

20. Teaching strategies STFM-Teaching physician
SNAPP model
Five step Microskills- One- Minute Preceptor ( POWER precepting)
Reverse Presentation – ASOAP Format
What if Model
Summarize briefly the history and physical
Narrow the differential to 2-3 relevant possibilities
Analyze the differential by comparing and contrasting the possibilities
Probe the preceptor by asking questions about uncertainties, difficulties, or
alternative approaches
Plan management for the patient’s medical issues
Select a case-related issue for self-directed learning
(1) Get a commitment
(2) Probe for supporting evidence
(3) Teach general rules
(4) Reinforce what was done right
(5) Correct mistakes

21. Feedback strategies STFM-Teaching physician
Sandwich Model
ARCH model
Five step microskills
RIME model
Ask /Teach /Ask model
Allow/Ask for self-assessment
Reinforce things being done well (attitudes, skills, and knowledge)
Confirm areas needing Correction/improvement
Help learner with improvement plan
Reporter – understands what is wrong with a patient.
Interpreter – understands why something is wrong
Manager – understands how to address the problem
Educator  - understands deeply, informs the team and is committed to continued, self-learning

22. Rubric Development
“And during the few moments that we have left,… We want to talk right down to earth in a language that everybody here can easily understand.” Malcolm X

23. Clear Expectations

24. Efficiency Increase grading efficiency
A bright idea for generating efficiency

25. Consistency

26. Reduce Subjectivity

27. Description of Dimensions
Task Description
Scale
Dimensions
Description of Dimensions
Share presentation rubric to discuss each part and explain appropriate time to share with students during the semester
Task description
Scale
Dimensions
Description of Dimensions

28. Videos What If Model - oral pres until 047 Sandwich Model
One Minute Preceptor
Feedback
Maybe save this for later – after we build the rubric, see how we apply then watch the feedback given
What makes good feedback?
Two videos
ARCH and sandwich

29. Implementation Best practices:
Face to face session to cover objectives and learning outcomes
Demo (video) of how to give feedback
Demo of rubric with video as a reminder of feedback elements and structure
Orientation with students – self grade prior to use and/or practice grading a student using a video so they understand how they will receive feedback

30. Challenges specific to working with community faculty
Delicate balance of asking and giving with community faculty
Those that take fewer students are less likely to practice feedback
Face time with preceptors is scare and followup on implementation of “new methods” is another challenge
It takes practice to get good at this

31. Solutions
for implementing with community faculty / activating the learner
Reverse precepting: your student will ask you about this and will ask you to review his/her performance
Student: would this be a good time for you to observe my (interview, physical exam, patient counseling)
Student: can we spent a minute reviewing my performance?
Preceptor:

33. Please evaluate this presentation using the conference mobile app
Please evaluate this presentation using the conference mobile app! Simply click on the "clipboard" icon on the presentation page.