1. Diagnostic errors in medical education: Where wrongs can make rights Kevin W. Eva Centre for Health Education Scholarship University of British Columbia

2. No one likes to make mistakes

3. But they aren’t all avoidable

5. Conclusions Mistakes are necessary when trying to enable learningMistakes are necessary when trying to enable learning Diagnostic strategies that avoid one type of error may increase the rate of another typeDiagnostic strategies that avoid one type of error may increase the rate of another type

6. Caveats Promoting errors is not about reducing patient safetyPromoting errors is not about reducing patient safety Mistakes are not simply the result of individuals’ decision-makingMistakes are not simply the result of individuals’ decision-making Nor are they indicative of lack of dedication/ability of health professionalsNor are they indicative of lack of dedication/ability of health professionals

7. Outline Learner Material Teacher 1 2 3

8. Considerations 1.Errors that arise while learners are learning new material 2.Errors in understanding how well said material was learned 3.Errors educators can use/induce to facilitate learning

10. Overconfidence Premature closurePremature closure Lack of insightLack of insight God complexGod complex Lake WoebegoneLake Woebegone See Berner and Graber (2008); Crosskerry (2002) Learner Material Why?

11. Self-regulated learning Requires deciding:Requires deciding: What to studyWhat to study How long to studyHow long to study When to stopWhen to stop See Nelson and Narens (1994) Surprisingly challenging Learner Material

12. Jowett, et al. (2007) Learner Material

13. Jowett, et al. (2007) Proficiency is inferred from the rate of learning rather than the amount learnedProficiency is inferred from the rate of learning rather than the amount learned See Kornell and Metcalfe (2006) Learner Material

14. The fluency heuristic Massed TrainingSpaced Training SessionsFew, IntenseMany, Spread out SpeedFaster ConfidenceHigher SatisfactionGreater RetentionLonger PerformanceBetter Which is better?

15. Desirable difficulties Teaching strategies that elicit more errors (and can make learning seem more difficult) are often more beneficialTeaching strategies that elicit more errors (and can make learning seem more difficult) are often more beneficial See Bjork (1999); Simon and Bjork (2001) Learner Material

16. Test-enhanced learning Roediger and Karpicke (2006)

17. Summary Errors are more likely to be made over the long term if they are not induced during learningErrors are more likely to be made over the long term if they are not induced during learning There is a clear need for external prompts and guidanceThere is a clear need for external prompts and guidance Learner Material

18. Outline Learner Material Teacher 1 2 3

19. Three maxims Feedback is goodFeedback is good The more immediate, often, and accurate, the betterThe more immediate, often, and accurate, the better Errors in medicine proliferate because feedback is often unavailableErrors in medicine proliferate because feedback is often unavailable Learner Teacher

21. Boehler, et al. (2006)

22. What predicts generation of learning goals? Overall R 2 =0.45; Eva et al., 2010 NOTE: Absolute values illustrated for ease of comparison

23. The ironic nature of self-assessment We can’t trust it … but we should encourage itWe can’t trust it … but we should encourage it We need to influence it … without changing itWe need to influence it … without changing it

24. The need for difficulties Promoting “self-assessment” should be about … (a)creating situations that will enable learners to discover the limits of their knowledge/ability (b)role modeling the habit of seeking and using data to guide continuous quality improvement (see Simon and Bjork, 2001; Eva, 2009)

25. Credibility may be the key “With your clinical partner … they know your strengths and weaknesses and they’re probably going to give you quite useful feedback. Maybe a consultant watched you one time. They might not have the whole picture of what you’re capable of and what you normally do” (Sargeant, et al., 2010)

26. Credibility Of relationships: Engagement and awareness Of the process: Validity and authenticity Of intent: Beneficence and non-maleficence (Sargeant, et al., 2010)

27. Summary Learners rarely induce in themselves the sorts of errors that maximize learningLearners rarely induce in themselves the sorts of errors that maximize learning It’s the educator’s role to facilitate growth from errors and empower learners to make errors in a safe placeIt’s the educator’s role to facilitate growth from errors and empower learners to make errors in a safe place Learner Teacher

28. Outline Learner Material Teacher 1 2 3

30. The source of all evil The use of pattern recognition by novices is not advocated for fear that “[i]nadequate experience might lead to potentially grim consequences”The use of pattern recognition by novices is not advocated for fear that “[i]nadequate experience might lead to potentially grim consequences” See Coderre, et al. (2003)See Coderre, et al. (2003) Material Teacher

31. What else could it be? Is there anything that doesn’t fit?

32. Overcoming the evil “Cognitive forcing strategies … are designed to prevent clinicians from pursuing a pattern recognition path that will typically lead to error”“Cognitive forcing strategies … are designed to prevent clinicians from pursuing a pattern recognition path that will typically lead to error” See Crosskerry (2002)See Crosskerry (2002) Material Teacher

33. Norman, et al. (1989)

34. Non-exclusivity Analytic processes Analytic processes Non-analytic processes Non-analytic processes Combined See Eva (2005)

35. Ark, Brooks, and Eva (2006)

37. Summary We need to structure experiences for students that help them recognize that exclusive reliance on non-analytic processes does create error …We need to structure experiences for students that help them recognize that exclusive reliance on non-analytic processes does create error … … but so does absolute aversion to those same processes… but so does absolute aversion to those same processes Material Teacher

38. Strategies for doing so Manipulating order of examplesManipulating order of examples Inducing learners to compare and contrast casesInducing learners to compare and contrast cases Actively engaging learners in problem solvingActively engaging learners in problem solving See Eva, et al. (1998); Eva (2009)See Eva, et al. (1998); Eva (2009) Material Teacher

39. The grand finale Dominant discourse around diagnostic error: Heuristics badDominant discourse around diagnostic error: Heuristics bad Problems arising from that discourse:Problems arising from that discourse: This view overlooks the fact that any strategy can lead to errors though they are of different typesThis view overlooks the fact that any strategy can lead to errors though they are of different types Reinforces the tendency for students to want to be right rather than to learn howReinforces the tendency for students to want to be right rather than to learn how

40. “An expert is a person who has made all the mistakes that can be made in a very narrow field.” (Niels Bohr)

41. “Education is learning what you didn’t even know you didn’t know” (Boorstin, 1914) “Education is learning what you didn’t even know you didn’t know” (Boorstin, 1914)

42. Thanks kevin.eva@ubc.ca See Eva, 2009 (AHSE) for more detail