1. Introduction to Evaluation
Part Three: Evaluating Learners Objectively

2. Learning Objectives
By the end of this session, participants will be able to:
Evaluate clinical learners objectively
Provide meaningful comments for evaluation
Note: This is part three of a three part series. Part one provides an overview of feedback and evaluation and part two covers how to give effective feedback.

3. Recall from Part One: Evaluation
A measurement judging performance given at the completion of a learning experience
Comparison with other learners’ performances
Often mistaken for feedback
Similar to scoring in gymnastics or ice skating

4. Formative vs. Summative Evaluations
Similar to feedback, but at the end of an experience
Informs the learner about him/herself (internal)
Informs learner of progress
Advises learner of observed learning needs
Guides self-reflection and motivates improvement
Summative
A final assessment of overall performance
Informs others about the learner (external)
A measure of outcome
May be used verbatim in medical student performance evaluation (MSPE)/ “Dean’s letter”

5. Making Evaluations Objective
To be meaningful, evaluations need to have context
Level and experience of learner
What are the expectations (learning objectives)
Situations of the environment
You should be provided with the learning objectives that offer this context

6. Example Evaluation Form: Providing Context
CLINICAL SKILLS
The medical student demonstrated the clinical skills necessary, such as performing patient interviews and/or patient histories, to participate in clinical experiences during preceptorship.
Does Not Meet Expectations
Meets Some Expectations
Meets Expectations
Exceeds Expectations
Expectations: Ability to perform a history and physical exam; present in a clear and concise way; garner appropriate clinical information from a variety of sources; document daily records appropriately; perform routine procedures; adapt to a variety of clinical environments
Performance/
behavior that does not meet expected levels, performance/
behavior that may be poor and is significantly less than expected levels. (i.e. lateness, inappropriate attire or language, or difficulty with communication)
behavior that meets some expectations but not all of the criteria
behavior that meets expectations
behavior that significantly exceeds expectations for a first year medical student. For example, students who exhibit abilities, performance, qualities, values and characteristics that go above and beyond the standard expectations
References to give context to your evaluation
“Based on the statement and definition below for a first year medical student. Please refer to your Preceptor Handbook and course objectives for additional information.”

7. Example Evaluation Form: Context Resident Competencies & Milestones
Competency Domain: Patient Care Competency: Gather essential and accurate information about the patient Developmental Milestones:
Level 1
Level 2
Level 3
Level 4
Level 5
Either gathers too little information or exhaustively gathers information
Linkage of signs and symptoms with prior experiences
Some pertinent positives and negatives
Uses broad diagnostic categories
Pattern-recognition
Real-time development of a differential
Precise diagnoses for common problems reached with ease and efficiency
Still limited on more complex problems
Unconscious gathering of essential and accurate information
Targeted and efficient with most clinical problems
In this example, the domain is Patient Care, the Competency is “Gather essential and accurate information about the patient.” The resident should be observed taking a history to see if he is able to demonstrate these skills.

8. Tools: The Evaluation Form
Effective tools help make an evaluation objective
Good evaluation forms have “anchors”
Anchors allow for measurement of observable behaviors:
Level 1
Level 2
Level 3
Level 4
Level 5
Either gathers too little information or exhaustively gathers information
Linkage of signs and symptoms with prior experiences
Some pertinent positives and negatives
Uses broad diagnostic categories
Pattern-recognition
Real-time development of a differential
Precise diagnoses for common problems reached with ease and efficiency
Still limited on more complex problems
Unconscious gathering of essential and accurate information
Targeted and efficient with most clinical problems
anchors

9. Tools: Use the Timeline
Review the evaluation form with the learner at the start of a rotation
Relating this form to learning objectives is helpful
Keep notes of feedback given to the learner; refer to these notes while completing the evaluation form
Orientation
Feedback
Feedback
Evaluation

10. For Written Comments
Use objectives and/or competencies for the basis of each comment
Use specific examples
Avoid generic “feel good” comments

11. Tools: Relating to Learning Objectives & Competencies
Neurological exam (patient care)
Teaches the team/brings articles (practice-based learning)
Receptive to/incorporates feedback (professionalism)
Interrupts other team members (interpersonal skills)
Overconfident (patient care, knowledge)
Texts during rounds (professionalism)
Use of resources (systems-based practice)

12. Tools: Specific Examples
“Well organized patient presentation of a complicated patient with _____.” “Was able to apply the principles of acid-base physiology.” “Demonstrated proper use of sterile technique.” “Thoroughly documented the history and physical, but needs to increase the breadth of the differential diagnosis and expand the assessment to better demonstrate their clinical problem solving abilities.”

13. Exercise: Practice Writing Formative & Summative Comments
Using the phrases below, write a few evaluative sentences: one set formative
(directed to the student) and one set summative (appropriate for a final evaluation).
“Brought articles”
“Impressed by his DDx”
“Needs to read more about renal Pathophysiology”
“Often late for rounds”
“Thorough history and physical examinations”
“Will make a great house officer”
Formative comments
Summative comments

14. Examples: Formative & Summative Comments
Notice we left out the “will be a good house officer” which does not describe this learner’s performance.
This is a judgment call: if the student
was given feedback and did not improve,
could include in summative comments.
Formative comment
Summative comment
“Dilbert, your DDx were very good and related to the patient’s history and physical examination. You need to plan ahead to make sure you are on time for rounds.
You need to learn more about renal pathophysiology as this was identified as an area of weakness on rounds.”
“Dilbert’s overall knowledge base is good, his H&Ps are thorough, and his DDX were well thought out and appropriate. He made important contributions to the team by researching articles helpful to the care of complicated patients.”

15. Questions? Contact: Jennifer Hagen, MD Office of Faculty Development
April Heiselt, PhD
Office for Community Faculty