1. The program will start promptly at 12:00 PM For technical assistance please contact Tech Support at 404-969-0387 or email support@healthtecdl.org

2. Faculty Development: Part 2 Preceptor Principles of Evaluating Learners and Giving Feedback Augusta, GA December 11, 2008 T. Andrew Albritton, MD Christopher White, MD Medical College of Georgia For technical assistance please contact Tech Support at 404-969-0387 or email support@healthtecdl.org

3. Objectives At the end of this part of the presentation, faculty will know: At the end of this part of the presentation, faculty will know: –the difference between formative and summative evaluation –the goals of evaluation –the RIME approach to evaluation If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

4. Objectives At the end of this part of the presentation, faculty will know: At the end of this part of the presentation, faculty will know: –the stages of the learning of a new skill –the characteristics of effective feedback –how to give positive and negative feedback effectively –the importance of observation in providing effective feedback If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

5. Stages of Learning a New Skill Unconscious Incompetence Unconscious Incompetence Conscious Incompetence Conscious Incompetence Conscious Competence Conscious Competence Unconscious Competence Unconscious Competence Howell WC. Human Performance and Productivity. Vol 2: Information Processing and Decision Making. Hillsdale, NJ: Erlbaum; 1982. If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

6. Physicians Are Unconsciously Competent in Many Skills History-Taking History-Taking –Verbal –Non-verbal –Organization Physical Examination Physical Examination Technical Skills Technical Skills –Reading X-Rays –Interpreting most lab studies –Surgical scrub –Knot tying If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

7. It’s hard to teach someone a skill that you do automatically! If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

8. Therefore, physicians who teach medical students have to think differently, going beyond unconscious competence to a higher stage of competence If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

9. Stages of Learning a New Skill Reflective Competence Reflective Competence –Although “second nature, the practitioner can also articulate the fine details of the skill to others. –Having fully understood all necessary steps and components of the skill to be learned and the manner how they are dynamically integrated... he is thereby able to teach the skill to others in a manner that is effective and expedient.” http://www.cognitivedesignsolutions.com/Instruction/TestingEvaluation.htm If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

10. Learning a New Skill: A Practical Example Non-driver: Non-driver: Beginner: Beginner: Just passed driving test: Just passed driving test: Driver who gets to work without remembering the drive Driver who gets to work without remembering the drive Consciously safe driver Consciously safe driver Unconscious incompetence Unconscious incompetence Conscious incompetence Conscious incompetence Conscious competence Conscious competence Unconscious competence Unconscious competence Reflective competence Reflective competence If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

11. Evaluation Formative: Provide observations to help the learner assess strengths and learning needs to improve future performance. Formative: Provide observations to help the learner assess strengths and learning needs to improve future performance. Summative: Assessing a learner’s performance using specific criteria to determine competency. Summative: Assessing a learner’s performance using specific criteria to determine competency. If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

12. Goals of Evaluation Clear and articulate description of performance. Clear and articulate description of performance. More objective evaluation of core competencies. More objective evaluation of core competencies. Consistent evaluation of core competencies. Consistent evaluation of core competencies. If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

13. Preceptor’s Responsibilities Treat the patient Treat the patient Diagnose the learner Diagnose the learner Teach the learner Teach the learner If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

14. Diagnosing the Learner It’s easy: Ask 3 simple questions! It’s easy: Ask 3 simple questions! WHAT do you think is going on? WHAT do you think is going on? WHY do you think so? WHY do you think so? WHAT do you want to do? WHAT do you want to do? If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

15. RIME* Approach to Evaluation Reporter Reporter Interpreter Interpreter Manager/Educator Manager/Educator * Based on the work of Louis N. Pangaro, MD If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

16. Reporter Accurately gathers information. Accurately gathers information. Basic skills to perform a history and physical examination. Basic skills to perform a history and physical examination. Distinguishes normal from abnormal. Distinguishes normal from abnormal. Reports information without attempting integration or synthesize. Reports information without attempting integration or synthesize. If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

17. Interpreter Demonstrates the integration and synthesis of information reported. Prioritizes clinical problems at a basic level. Develops a more patient-oriented assessment including a differential diagnosis. Actively participates in patient care. If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

18. Manager Reports, interprets, and applies information to the patient. Reports, interprets, and applies information to the patient. Makes diagnostic or management suggestions. Makes diagnostic or management suggestions. Uses textbooks and the primary literature to better understand the patient. Uses textbooks and the primary literature to better understand the patient. If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

19. Educator Goes beyond the basics of self-directed learning. Goes beyond the basics of self-directed learning. Critical appraisal skills. Critical appraisal skills. Demonstrates insight and explores important questions in more depth. Demonstrates insight and explores important questions in more depth. Participates in the education of others. Participates in the education of others. If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

20. RIME Reporter: Passing but requires additional work to address deficiencies. “C” Student. Reporter: Passing but requires additional work to address deficiencies. “C” Student. Interpreter: Meeting Expectations. “B” Student. Interpreter: Meeting Expectations. “B” Student. Manager/Educator: Exceeding Expectations. “A” Student. Manager/Educator: Exceeding Expectations. “A” Student. If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

21. Giving Feedback: The Most Important Skill Needed to be an Effective Teacher If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

22. Feedback If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

23. What is Feedback? Feedback occurs when a learner is offered insight into what he or she actually did as well as the consequences of his or her actions. This insight.... Highlights the dissonance between the intended result and the actual result, thereby providing impetus for change. Ende, JAMA 1983; 250:777-781. If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

24. Overview Feedback can be either: Feedback can be either: –Positive: reinforcing what was done right –Constructive: correcting mistakes or misconceptions Feedback goals: Feedback goals: –Ensuring the learner improves –While keeping his/her self-respect intact If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

25. Important Elements of Feedback Rapport Rapport Clear expectations Clear expectations Appropriate location Appropriate location Timely and regular Timely and regular If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

26. Effective Feedback Descriptive and nonjudgmental Descriptive and nonjudgmental Based on direct observations Based on direct observations Focused on areas that can be changed Focused on areas that can be changed Limited to what the learner can use Limited to what the learner can use The learner understands the feedback and knows the next step The learner understands the feedback and knows the next step If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

27. Feedback ≠ Evaluation Feedback –Presents information enabling the student to remain on course to reach a given goal. Evaluation –Presents a judgment about how well the student met a given goal If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

28. Feedback ≠ Evaluation Feedback –Presents information enabling the student to remain on course to reach a given goal. Evaluation –Presents a judgment about how well the student met a given goal –Formative (coaching) – Summative (a grade) If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

29. Feedback ≠ Evaluation Feedback –Presents information enabling the student to remain on course to reach a given goal. Evaluation –Presents a judgment about how well the student met a given goal –Formative (coaching) –During or after the fact –Summative (a grade) –Occurs after the fact If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

30. Key Principle for Providing Feedback....

31. You only see what you look for.... If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

32. Key Principle for Providing Feedback.... You only see what you look for.... You only look for what you know.... If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

33. Feedback: What Should Teachers Look For? Attitudes: –Verbal & Non-verbal –Punctuality Skills: –Presentations –Written Notes –History & Physical Exam Knowledge: –Differential diagnosis –Therapy –Management Observable Behaviors & Skills If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

34. Why Is It So Hard To Give Feedback? Failure to obtain data (observations) If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

35. Why Is It So Hard To Give Feedback? “Observations are the currency of feedback, and without them the process becomes ‘feedback’ in name only” Ende, JAMA 1983; 250:777-781. Failure to obtain data (observations) If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

36. It’s hard to teach someone a skill that you do automatically! If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

37. Why Is It So Hard To Give Feedback? We readily recognize strong learners and weak learners by looking at how they perform common clinical tasks (Hx, PE, oral presentations, etc) “Unconscious Competence” We have difficulty “deconstructing” the individual components of the task in order to identify areas for feedback.... If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

38. Examples of Effective Feedback Ende, JAMA 1983; 250:777-781. Dr. Dee Meaning Dr. Anne Spire If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

39. Examples of Effective Feedback Ende, JAMA 1983; 250:777-781. “Your differential diagnosis for this patient is inadequate” Dr. Dee Meaning Dr. Anne Spire

40. Examples of Effective Feedback Ende, JAMA 1983; 250:777-781. “Your differential diagnosis for this patient is inadequate” “The differential diagnosis did not include the possibility of TB” Dr. Dee Meaning Dr. Anne Spire

41. Examples of Effective Feedback Ende, JAMA 1983; 250:777-781. “Your differential diagnosis for this patient is inadequate” “You looked uncomfortable talking about your patient’s cancer” “The differential diagnosis did not include the possibility of TB” Dr. Dee Meaning Dr. Anne Spire

42. Examples of Effective Feedback Ende, JAMA 1983; 250:777-781. “Your differential diagnosis for this patient is inadequate” “You looked uncomfortable talking about your patient’s cancer” “The differential diagnosis did not include the possibility of TB” “While observing you, I began to feel that you were not comfortable talking about your patient’s cancer” Dr. Dee Meaning Dr. Anne Spire

43. Giving Positive Feedback If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

44. Giving Positive Feedback Not general praise.... Not general praise.... “You did a good job with that patient.” “You are doing fine.” If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

45. Giving Positive Feedback Not general praise.... Not general praise.... “You did a good job with that patient.” “You are doing fine.” Inform the student about a specific behavior or action that was done well Inform the student about a specific behavior or action that was done well “You gave a very good description of your patient’s general appearance. Successful clinicians are good observers, and you have demonstrated that you are developing good observation skills as you evaluate patients. “You gave a very good description of your patient’s general appearance. Successful clinicians are good observers, and you have demonstrated that you are developing good observation skills as you evaluate patients. If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

46. Giving Constructive Feedback Label the feedback Label the feedback Identify a specific behavior or action that was done wrong Identify a specific behavior or action that was done wrong Tell the student how to avoid or correct the error in the future Tell the student how to avoid or correct the error in the future Make sure the leaner understands the feedback Make sure the leaner understands the feedback If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

47. Helpful Hints Ask for the learner’s thoughts about his/her performance first Ask for the learner’s thoughts about his/her performance first Avoid vague, harsh, or judgmental statements Avoid vague, harsh, or judgmental statements Feedback sandwich Feedback sandwich –Tell what was done well –Tell what was done wrong –Discuss to do next time If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

48. Why Is It So Hard To Give Feedback? “Observations are the currency of feedback, and without them the process becomes ‘feedback’ in name only” Ende, JAMA 1983; 250:777-781. Failure to obtain data (observations) If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

49. Take Home Messages Assessment drives learning Assessment drives learning Objective evaluation of core competencies helps the student become a better physician Objective evaluation of core competencies helps the student become a better physician Using the “3 W’s” helps the student in developing their interpretation and management skills Using the “3 W’s” helps the student in developing their interpretation and management skills Working hard does not necessary mean an “A” performance Working hard does not necessary mean an “A” performance If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org

50. Questions? If you have any technical issues, please contact 404-969-0387 or email support@healthtecdl.orgsupport@healthtecdl.org Please Type your questions in the chat box provided on the screen.

51. Thank you for attending our program! Please take the time to fill out the survey: http://www.surveymonkey.com/s.aspx?sm=JawKsSls6lJA5_2fvqZ13m3Q_3d_3d For more information on HealthTecDL events please visit www.healthtecdl.orgwww.healthtecdl.org For more information on becoming a Preceptor, please contact your local Area Health Education Center: http://www.healthtecdl.org/network-organizations/network-overview.cfm