1. EVALUATION: Making it Work Borrowed from the “Mahec” preceptor program. http://www.mahec.net/pdp/ e-Learning_Tools.asp M odified for the UBC, Family Practice, Postgraduate Program http://www.mahec.net/pdp/ e-Learning_Tools.asp http://www.mahec.net/pdp/ e-Learning_Tools.asp

2. Presentation based on materials from the: Produced by : Supported by HRSA Family Medicine Training Grant # 1 D15 PE50119-01

3. Evaluation : Making it Work Goals: 1. Review characteristics of evaluation. 2. Discuss pitfalls in evaluation. 3. Outline a system for effective evaluation.

4. What is the difference between feedback and evaluation? Feedback Feedback Take 3 minutes to discuss. Evaluation

5. Evaluation: What is it? Feedback vs. Evaluation

6. Evaluation: What is it? Feedback vs. Evaluation Timing:Timely Scheduled

7. Evaluation: What is it? Feedback vs. Evaluation Timing:Timely Scheduled Setting:Informal Formal

8. Evaluation: What is it? Feedback vs. Evaluation Timing:Timely Scheduled Setting:Informal Formal Basis: Observation Observation

9. Evaluation: What is it? Feedback vs. Evaluation Timing:Timely Scheduled Setting:Informal Formal Basis: Observation Observation Content:Objective Objective

10. Evaluation: What is it? Feedback vs. Evaluation Timing:Timely Scheduled Setting:Informal Formal Basis: Observation Observation Content:Objective Objective Scope:Specific Global Action Performance

11. Evaluation: What is it? Feedback vs. Evaluation Timing:Timely Scheduled Setting:Informal Formal Basis: Observation Observation Content:Objective Objective Scope:Specific Global Action Performance Purpose:Improvement “Grading”

12. Evaluation: What is it? Characteristics of Evaluation Characteristics of Evaluation Scheduled Scheduled Formal Formal Objective Objective Observation Based Observation Based Global Performance Global Performance Required for grading Required for grading

13. Evaluation Why is it important?

14. Evaluation: why is it important? A Requirement of the Residency Program A Requirement of the Residency Program Professional Responsibility Professional Responsibility A Teaching/Learning Opportunity A Teaching/Learning Opportunity

15. Evaluation: Why is it difficult What do you find difficult or stressful about doing an evaluation? What do you find difficult or stressful about doing an evaluation? In groups of 2 or 3 discuss the aspects of evaluation which give problems. In groups of 2 or 3 discuss the aspects of evaluation which give problems.

16. Why is Evaluation difficult Time Time Damage to relationship between the teacher and the learner Damage to relationship between the teacher and the learner Implications of a negative evaluation on the learner Implications of a negative evaluation on the learner Unsure of the “Benchmark of comparison” Unsure of the “Benchmark of comparison” OTHER SUGGESTIONS? OTHER SUGGESTIONS?

17. Pitfalls “Halo Effect” “Halo Effect” “Oops”-Insufficient Evidence “Oops”-Insufficient Evidence “You never told me that!” “You never told me that!” “But I NEED Honors!” “But I NEED Honors!” “Uh-Oh”-Should they pass? “Uh-Oh”-Should they pass? “Lake Wobegon” Effect “Lake Wobegon” Effect

18. Evaluation: Making it Work STRATEGY G …Get Ready G …Get Ready R … Review Expectations with Learner R … Review Expectations with Learner A … Assess: Observe and Record A … Assess: Observe and Record D … Discuss Evaluation at Mid-Point D … Discuss Evaluation at Mid-Point E … End with a “Grade” E … End with a “Grade”

19. Evaluation: Making It Work Get Ready Get Ready Review the objectives the resident has for the rotation. Review the objectives the resident has for the rotation. Review the Evaluation Form for your rotation http://www.familypractice.ubc.ca/residency/forms/ Review the Evaluation Form for your rotation http://www.familypractice.ubc.ca/residency/forms/ Consider unique Opportunities/Challenges of your site Consider unique Opportunities/Challenges of your site What are your Expectations for the learner? What are your Expectations for the learner?

20. Evaluation: Making It Work Review Expectations with the Learner Review Expectations with the Learner Schedule a meeting with your resident before the rotation begins to discuss objectives. Schedule a meeting with your resident before the rotation begins to discuss objectives. Review the skill level expected of the resident at his or her level of training: http://www.familymed.ubc.ca/residency/facultydev elopment/BordageModel.htm Review the skill level expected of the resident at his or her level of training: http://www.familymed.ubc.ca/residency/facultydev elopment/BordageModel.htm Review your expectations. Review your expectations. Describe the Evaluation Process Describe the Evaluation Process

21. Evaluation: Making It Work Assess Assess Observe and record. Observe and record. http://www.familypractice.ubc.ca/resi dency/forms http://www.familypractice.ubc.ca/resi dency/forms Provide Feedback Regularly Provide Feedback Regularly

22. Self Assessment Encourage the learner to self assess: Encourage the learner to self assess: How did that encounter feel to you? How did that encounter feel to you? What did you do well, what did you have trouble with? What did you do well, what did you have trouble with? How do you think you could change? How do you think you could change? What would you do differently next time? What would you do differently next time?

23. Mid-rotation Evaluation In groups of two or three come up with at least 3 reasons why you think mid-rotation evaluations are important. In groups of two or three come up with at least 3 reasons why you think mid-rotation evaluations are important.

24. Evaluation: Making It Work Mid- rotation evaluations Mid- rotation evaluations Formal Meeting Formal Meeting Learner and evaluator fill out form in advance Learner and evaluator fill out form in advance Compare evaluations together. Compare evaluations together. Discuss differences and how expectations are being met Discuss differences and how expectations are being met Plan for Remainder of Rotation Plan for Remainder of Rotation

25. Evaluation: Making It Work Complete evaluation in advance Complete evaluation in advance Schedule sufficient time. Schedule sufficient time. Support your evaluation with examples Support your evaluation with examples Highlight items that can be worked on in the future. Highlight items that can be worked on in the future.

26. Grading The resident: The resident: Meets expectations Meets expectations Is Borderline or Is Borderline or Fails to meet expectations. Fails to meet expectations. Take 5 minutes to discuss what these categories mean to you. Take 5 minutes to discuss what these categories mean to you.

27. UBC Family Practice Residency Program Page 1 Resident Evaluation – Family Practice Rotation Resident: ___________________________________Faculty Supervisor: ______________________________ Rotation: ___________________________________Dates on Service: ________________________________ A. Discussion of Program Objectives (see program website: http://www.familypractice.ubc.ca) Date: _________http://www.familypractice.ubc.ca B. Resident’s personal objectives for this rotation:Were they met/completed? 1. ________________________________________ 2. ________________________________________ 3. ________________________________________ 4. ________________________________________ C. Mid-Rotation Evaluation Date: ______________________________ Comments: D. Has Resident been observed (please circle): Taking a HistoryYesNoUsing Technical SkillsYesNo Doing a Physical ExamYesNoGiving Oral PresentationYesNo

28. UBC Family Practice Residency Program Page 2 Resident Evaluation – Family Practice Rotation E. The resident has demonstrated a level of competence appropriate to level of training in the following areas: Unable to Judge Needs Improvement Averag e Above Average ExceptionalComments 1 2 34 5 6 7 8 9 10 Clinical Skills 1.Has knowledge appropriate to level of training 1.Uses knowledge effectively 1.Performs appropriate history/physician assessment 1.Demonstrates skilled focused clinical exam 1.Successfully integrates knowledge and assessment 1.Plans appropriate investigation 1.Formulates appropriate management plan 1.Demonstrates awareness of own clinical limitations 1.Demonstrates appropriate procedural skills

29. Remember If you have any difficulty with web-eval or with completing your evaluations contact your Site Director or your Site Faculty for Evaluation; If you have any difficulty with web-eval or with completing your evaluations contact your Site Director or your Site Faculty for Evaluation; Yours site faculty can be found at: http://www.familymed.ubc.ca/residency/a bout/facstaff1.htm Yours site faculty can be found at: http://www.familymed.ubc.ca/residency/a bout/facstaff1.htm http://www.familymed.ubc.ca/residency/a bout/facstaff1.htm http://www.familymed.ubc.ca/residency/a bout/facstaff1.htm Be Prepared to Explain your Assessment Be Prepared to Explain your Assessment Review evaluation forms in advance Review evaluation forms in advance

30. Remember Written comments are a very valuable part of the evaluation and very helpful to the Program Written comments are a very valuable part of the evaluation and very helpful to the Program Cite specific examples of areas of weakness or concern that need to be addressed. Cite specific examples of areas of weakness or concern that need to be addressed.

31. Evaluation: Making it Work G …Get Ready G …Get Ready Before the rotation starts. Before the rotation starts. R … Review Expectations and objectives with Learner at the beginning and throughout the rotation R … Review Expectations and objectives with Learner at the beginning and throughout the rotation A … Assess A … Assess Observe and Record. Observe and Record. D … Discuss Evaluation at Mid-Point D … Discuss Evaluation at Mid-Point Avoid surprises at the end. Avoid surprises at the end. E … End with fair and “useful” grade E … End with fair and “useful” grade Objective and Future-Oriented. Objective and Future-Oriented.