Metacognition: Applying Intuition and Intellect to Clinical Reasoning Clinton Pong, MD Tufts/Cambridge Health Alliance Family Medicine, Master Teacher Fellow, PGY-5 FMEC October 2014

Aims and Objectives Teaching Aims: Learning Objectives: Give clinicians simple tools they can use to routinely pause and reflect on their clinical reasoning (CR) process Engage in discussion on teaching/assessment of CR Learning Objectives:  by the end of the session, participants will be able to: Recognize common heuristics and biases Define the dual-process cognitive model Apply a systematic, routine method for catching clinical errors before they occur Reflect on current milestone level for critical thinking Raise of hands: how many of the audience are: Medical students? Residents? Faculty? Taylor time of sections to presentation

Clinical reasoning as Detective work Clinical Method Detective Fiction Started by Sir William Osler in 1890s Moved medicine out of the classroom and invented “bedside rounds” “Listen to your patient, he is telling you the diagnosis” Started by Sir Arthur Conan Doyle in 1890s Faith in Logic, keen observations and “deductive” reasoning “You See, but you do not Observe. The distinction is clear.” Analogies between detective fiction and clinical method: Born in the same historical period, Second half of the 19th century, Shared faith in logic, Similar working patterns

Thinking about thinking Metacognition, Heuristics, and Seeing Patterns http://differentialdiagnosi.proboards.com/index.cgi? Primer Thinking about thinking

Managing One’s Own Thinking Metacognition  the act of “thinking about thinking (and feeling)” Of one’s own and another’s  Heuristic Greek: "Εὑρίσκω", "find" or "discover" strategies using readily accessible, though loosely applicable, information to control problem solving Rules of Thumb e.g. “Better safe than sorry” Heuristic: mental short cuts to ease the cognitive load of making a decision. Examples of this method include using a rule of thumb, an educated guess, an intuitive judgment, or common sense.

Medical Heuristic Aphorisms When you hear hoof beats, think of horses, not zebras Geography is Destiny. If it looks like a duck, walks like a duck, quacks like a duck, it’s a duck. When all you have is a hammer, everything looks like a nail Here are some common medical quotes that we use as rules of thumb: do you know which heuristics are being activated with these aphorisms?

Unpacking principle & Search satisficing Failure to elicit key history/physical findings or neglecting to pursue pertinent negatives Ignoring your toolkit (Satisfy + Suffice) Calling off the search for other diagnoses when something is found (but other things are missed) We have a toolkit that we typically use and sometimes we need to unpack the wrench and screwdriver instead of the hammer. SS is a Portmanteau of “Satisfy and Suffice.” We need to be bloodhounds in hunting things down and keep sniffing for clues.

Anchoring, Adjustment & Context Errors First Impressions: initially locking-on to early features without later revision can be corrected by extending the chain, link by link but we are subconsciously “primed” by our starting point Work by Daniel Kahneman and Amos Tversky has shown that there is a “mental priming” that subconsciously contaminates our initial starting points. Context errors are most noticeable when they occur at the beginning of a case presentation with novice learners. Critical signals are distorted by background noise Hanging weight on the wrong evidence “Red herrings” “getting thrown off the trail”

Available recall & Base-Rate Neglect Judgment of what is likely based on: previous experience, recent events or recalling what easily comes to mind Exaggerating or minimizing the prevalence of a disease Sometimes with flashback to prior situations. B-R neglect typically comes to my mind when I think about specialists and tertiary care centers. They have a selection bias from referrals.

Premature closure “Locking-in” a diagnosis too early; closing a mental door “When the diagnosis is made, the thinking stops.” This is the most common heuristic people think about with misdiagnosis and it typically occurs “post-mortem” during an M&M.

Representativeness restraint Typified pattern recognition without considering atypical presentations “Forcing a fit” Disregard for pattern recognition by thinking its “good enough” Dr. Centor, of strep throat fame, says it best in his blog: Example of Representative Restraint: Many diagnostic errors occur because we try to fit the data to our hypothesis rather than fitting the hypothesis to our data. -Dr Robert Centor http://www.medrants.com/archives/4917

On dispositions, biases and error When our thin slices lead us astray On Problems On dispositions, biases and error

When Heuristics Fail Heuristics are Cognitive Dispositions to Respond Cognitive biases Predictable patterns of deviation in judgment Sometimes lead to perceptual distortion, inaccurate judgment, illogical interpretation, or irrationality Cognitive Errors

Heuristic Distribution Dispositions > Biases > Error Most of the time, heuristics work in our favor. That’s why a great clinic day seems to “just click” and flows. This is a graph I just made up to illustrate a point: our dispositions are right most of the time, become ruts some of the time and that sometimes contributes to errors. Researchers fall prey to B-R Neglect and blame ALL Heuristics for a MINORITY of Errors

Biases / Error M&M discussion Anchoring Available recall Base-rate neglect Premature closure Representativeness restraint Unpacking principle & Search satisficing Context errors First Impressions Previous experience “Making mountains out of molehills” or vice versa Lock-it in. Prejudice Typical vs atypical patterns Calling off the search too early Red herrings Now that you’re all primed, we’ll pause here and take some time reflecting on a recent M&M, misdiagnosis or patient error that you’ve experienced recently.  Take a few minutes in a group of two and look at the Heuristics and Biases checklist and have someone share a brief example with a partner. We’ll spend five minutes doing this.

Positive Dispositions: the “Anti M&M” Observe and deduce Recognize inconsistencies Exhibit Tenacity Avoid protocols and procedures when their hunch takes them in another direction Stop and think when necessary Adjustment, Experiential Availability Representativeness Alert Avoid Unpacking Principle and Search satisficing Representativeness restraint Switch to System II Let’s contrast the Biases/Error point of view with a positive checklist, generated from fictional detectives. Now switch and have your partner think about how these steps SAVED you from an M&M, misdiagnosis or error. How many of you have the opposite of an M&M? What would you call THAT? Rapezzi, C. et al. (2005) White coats and fingerprints: diagnostic reasoning in medicine and investigative methods of fictional detectives. BMJ December 2005. 331;24-31.

Mental Model for clinical reasoning Type I vs Type II: Work by Croskerry, Graber, Klein, Kahneman & Tversky On Theory Mental Model for clinical reasoning

Two Approaches to Clinical Reasoning Cognitive processing What goes on in our brains when we are making a decision and how do we arrive at our final diagnosis/plan? We use two types of thinking: Type I Heuristic Go / Fast / “Hot” / Intuition Type II Analytic Stop / Slow / “Cold” / Intellect

General properties of Type I and Type II Property System 1 System 2 Reasoning style Intuitive Analytical (“two/too” intellectual) Heuristic Normative Associative Deductive Concrete Abstract Cost/Effort Low/Minimal High/Considerable Awareness/Automaticity Low/Automatic High/Deliberate Speed Fast Slow Channels Multiple, parallel Single, linear Propensities Causal Statistical Action Reflexive, skilled Deliberate, rule-based Prototypical Yes No, based on sets Errors Common Few Reliability Low, variable High, consistent Vulnerability to bias Less so Affective valence Often Rarely Context importance High Low Predictive power Scientific rigour Type I process: the automatic hard-wired, specialized parts of the brain that deal with specific needs Type 2 processes: deliberate training used to solve reasoning and decision problems in a systematic, analytical way. They follow rules of logic and science and generally deliver error-free solutions providing that everything works appropriately, but this isn’t always the case. Consider the computer. When patients use the internet to find a diagnosis, that’s error-prone. When doctors use their EMR, “suggestions” often hinder our care.

Croskerry’s Dual-Process Model for Diagnostic Thinking (modified) Type I intuition Diagnosis/ Management Decision Pattern processing When we consider a patient case, our brain processes it and asks “does this match a pattern?” If so, we quickly use Type I intuition or “Kirk” thinking. If not, we slowly ponder using our Type II Intellect or “Spock” thinking. They duke it out, either consciously or unconsciously, and we arrive at a diagnosis. REFLECTION is a key point to make sure that the right commander takes the helm. Type II intellect Modified from Croskerry P. A universal model of diagnostic reasoning. Acad Med. 2009 Aug;84(8):1022-8.

Croskerry’s Dual-Process Model for Diagnostic Thinking (modified) Type I intuition Rational Override Pattern recognition Diagnosis/ Management Decision Pattern processing Practice Dysrationalia Override The goal of residency with practice and reflection is to build more and more of the heuristics from more complex decisional algorithms, shifting from Type II slow to type I fast thinking. That’s a good thing! Well, as long as we balance it out with an internal “Spock vs Kirk” debate. Type II intellect Modified from Croskerry P. A universal model of diagnostic reasoning. Acad Med. 2009 Aug;84(8):1022-8.

How the Two Systems Interact Rational Override A Type 1 response is inappropriately triggered The pattern isn’t matching the data bank The decision maker stops and resets with a Type 2 analytical approach We monitor Type I in most situations Rational Override

Dysrationalia override aka residency, being on-call, etc. “I should have known better!” Sleep, Hunger Fear, Irritability Inattentiveness, Distractions Prior experience Dysrationalia Override

Balancing snap Judgments and deliberative rumination Using Internalities, Checks and Balances On Solutions Balancing snap Judgments and deliberative rumination

Clustering of approaches on an intuitive-analytical continuum Hypothetico-deductive reasoning Inductive reasoning Thin Slices Exhaustion strategy Blink = thin slicing and heuristics/biases methods are what medical educators/researchers focus on Inductive vs Deductive reasoning for Detectives Medical students and interns under stress, may revert to an exhaustion strategy Disposition / Bias / Error Type I Type II Croskerry, P. (2009) Clinical cognition and diagnostic error: applications of a dual process model of reasoning. Advances in Health Sciences Education. 11 Aug 2009. 14:27–35

Reliability and Cost in Decision Making We are training ourselves and our medical students and residents to become expert thinkers. Graber M, et al. Reducing diagnostic errors in medicine: what's the goal? Acad Med. 2002 Oct;77(10):981-92.

Checklists Checks and balances Croskerry’s General Diagnostic Time-Out Heuristics & Biases Differential Diagnosis Be a Detective I have a few simple solutions.

Croskerry's General Obtain your own complete medical History Perform a focused and purposeful Physical Exam Generate initial hypotheses and differentiate these with additional history, physical exam and diagnostic tests Take a “Diagnostic Timeout” Am I being comprehensive? Am I considering the inherent flaws of heuristic thinking? Was my judgment affected by any other bias? Do I need to make the diagnosis now, or can I wait? What is the worst-case scenario? Embark on a plan, but acknowledge uncertainty and ensure a pathway for follow-up “Crystal Ball Analysis” If things goes awry, why? What can you change? For attendings and senior residents, this checklist is about proper oversight. Step back and think of yourself as a General on a battlefield rather than the frontline soldier. Pausing will help with (Dysrationalia Overrides & allow for Calibration)

“Pre-mortem” M&M Halting Dysrationalia: Effortful deliberation “Crystal Ball Analysis” Assume your plan goes awry. What happened? Why? What can you modify to avoid adverse events? Klein, G. (2003) The Power of Intuition, 2003, p. 98–101

Croskerry’s Cognitive Biases and Failed Heuristics Checklist Anchoring Availability Base-rate neglect Premature closure Representativeness Restraint Unpacking Principle Search Satisficing Context Errors Consider diagnoses other than the initially favored one Consider diagnoses other than those that come readily to mind Consider the relative prevalence of diseases in primary care that fit the patient’s complaint Re-open the diagnostic process and consider alternative diagnoses before discharging the patient Consider causes for symptoms other than the ones that readily fit the pattern Ask questions that might confirm or rule out alternative diagnoses Consider additional causes of the complaint after something is found Re-think assumptions and maintain objectivity

http://links.lww.com/ACADMED/A38 http://pie.med.utoronto.ca/DC/DC_content/DC_checklist.html

Be a Detective Steps Heuristics Activated 1. Observe and deduce Adjustment, Experiential Availability 2. Recognize inconsistencies Representativeness Alert 3. Exhibit Tenacity Avoid Unpacking Principle and Search Satisficing 4. Avoid protocols and procedures when a hunch leads in another direction Ponder Representativeness restraint   5. Stop and think when necessary Switch to System II Rapezzi, C. et al. (2005) White coats and fingerprints: diagnostic reasoning in medicine and investigative methods of fictional detectives. BMJ December 2005. 331;24-31.

Mapping to ACGME Milestones for Critical Thinking On Assessment Formative feedback & Summative assessments of clinical reasoning in Medical School and residency

Milestones of Critical Thinking (Papp)   Stage 1: Unreflective Thinker Stage 2: Beginning Critical Thinker Stage 3: Practicing Critical Thinker Stage 4: Advanced Critical Thinker Stage 5: Accomplished Critical Thinker Devolved state: the Challenged Thinker Meta- cognition Lacking knowledge about cognition Unaware of different approaches to thinking Requires external motivation to sustain reflection Familiar with metacognitive theories Applies conscious effort in his own critical thinking Has a solid repertoire of approaches to thinking Identifies different cognitive approaches Consciously displays critical thinking Uses theories of metacognition Habitually monitors, revises, and rethinks approaches for continual improvement Resists others’ perspectives, flouting prior knowledge of metacognition Fails to recognize personal cognitive biases Skills Has a single approach to gathering and processing information based on crude scripts (e.g., rote memory) Sporadically uses different approaches Recognizes decision making principles but, disconnecting theory from practice, does not apply them in action Articulates multiple approaches to problem solving Uses established principles to make sense of observations and guide decisions Uses intuitive and analytical strategies interchangeably Avoids cognitive biases Fundamental principles and concepts explicitly applied/demonstrated throughout approach to problem solving Role-models critical thinking for others Able to “toggle” adeptly between approaches Elaborates on connections between principles to explain observed phenomena Has the ability to create new knowledge or understanding by reasoning inductively Entrenched in a singular approach Does not adjust when appropriate or when aspects do not exactly fit the clinical situation Adapted from Papp, K. et al. (2014) Milestones of Critical Thinking: A Developmental Model for Medicine and Nursing. Acad Med, 89(5), 715–720.

Milestones of Critical Thinking (Papp)   1: Unreflective Thinker 2: Beginning Critical Thinker 3: Practicing Critical Thinker 4: Advanced Critical Thinker 5: Accomplished Critical Thinker Devolved state: the Challenged Thinker Meta- cognition Lacking knowledge about cognition Unaware of different approaches to thinking Requires external motivation to sustain reflection Familiar with metacognitive theories Applies conscious effort in his own critical thinking Has a solid repertoire of approaches to thinking Identifies different cognitive approaches Consciously displays critical thinking Uses theories of metacognition Habitually monitors, revises, and rethinks approaches for continual improvement Resists others’ perspectives, flouting prior knowledge of metacognition Fails to recognize personal cognitive biases Skills Has a single approach to gathering and processing information based on crude scripts (e.g., rote memory) Sporadically uses different approaches Recognizes decision making principles but, disconnecting theory from practice, does not apply them in action Articulates multiple approaches to problem solving Uses established principles to make sense of observations and guide decisions Uses intuitive and analytical strategies interchangeably Avoids cognitive biases Fundamental principles and concepts explicitly applied/demonstrated throughout approach to problem solving Role-models critical thinking for others Able to “toggle” adeptly between approaches Elaborates on connections between principles to explain observed phenomena Has the ability to create new knowledge or understanding by reasoning inductively Entrenched in a singular approach Does not adjust when appropriate or when aspects do not exactly fit the clinical situation Maps to the following ACGME FM Milestones: MK-2 Applies critical thinking skills in patient care PBLI-1 Locates, appraises, and assimilates evidence from scientific studies related to the patients’ health problems PBLI-2 Demonstrates self-directed learning Also supports PROF-4 Pursues continual personal and professional growth Which Milestone have I reached? Knowledge on metacognition, skills and attitudes (but I left this section which includes more PROF-4 items Adapted from Papp, K. et al. (2014) Milestones of Critical Thinking: A Developmental Model for Medicine and Nursing. Acad Med, 89(5), 715–720.

Take Home Points Tricks: Traps: Checklists For education: Heuristics are “Type I” shortcuts that “thin-slice” intuitive first impressions Heuristics are excellent tools when checks and balances are in place Traps: Heuristics are prone to cognitive biases and therefore, subject to error Use checklists to use avoid these pitfalls Checklists Rapezzis’ Call to “Be a Detective” Croskerry’s General Diagnostic Time-Out Heuristics and Biases Differential Diagnosis Checklist For education: Consider the positive dispositions case-discussion (“anti-M&M”) For Assessment: Papp’s Milestones

References Croskerry, P. Clinical cognition and diagnostic error: applications of a dual process model of reasoning. Adv in Health Sci Educ (2009) 14:27–35 Croskerry P. The importance of cognitive errors in diagnosis and strategies to minimize them. Acad Med. 2003 Aug;78(8):775-80. Croskerry P. A universal model of diagnostic reasoning. Acad Med. 2009 Aug;84(8):1022-8. Ely JW, et al. Checklists to reduce diagnostic errors. Acad Med. 2011 Mar;86(3):307-13. Graber M, et al. Reducing diagnostic errors in medicine: what's the goal? Acad Med. 2002 Oct;77(10):981-92. Kahneman, D. Slovic P, Tversky A. (1982) Judgment under uncertainty: Heuristics and biases. Cambridge, UK: Cambridge University Press. Klein, G. (2003) The Power of Intuition, 2003, p. 98–101 Papp, K. et al. (2014) Milestones of Critical Thinking: A Developmental Model for Medicine and Nursing. Acad Med, 89(5), 715–720. Rapezzi, C. et al. (2005) White coats and fingerprints: diagnostic reasoning in medicine and investigative methods of fictional detectives. BMJ December 2005. 331;24-31.