Representativeness, Similarity, & Base Rate Neglect Psychology 466: Judgment & Decision Making Instructor: John Miyamoto 10/27/2015: Lecture 05-1 Note:

Slides:



Advertisements
Similar presentations
Tests of Hypotheses Based on a Single Sample
Advertisements

Testing Theories: Three Reasons Why Data Might not Match the Theory.
1 Intuitive Irrationality: Reasons for Unreason. 2 Epistemology Branch of philosophy focused on how people acquire knowledge about the world Descriptive.
Psychology 290 Special Topics Study Course: Advanced Meta-analysis April 7, 2014.
Research Methods in Psychology
Solved the Maze? Start at phil’s house. At first, you can only make right turns through the maze. Each time you cross the red zigzag sign (under Carl’s.
Inductive Reasoning Bayes Rule. Urn problem (1) A B A die throw determines from which urn to select balls. For outcomes 1,2, and 3, balls are picked from.
Developing Ideas for Research and Evaluating Theories of Behavior
Psych 548, Miyamoto, Win '15 1 Set Up for Students Your computer should already be turned on and logged in. Open a browser to the Psych 548 website ( you.
Methodology: How Social Psychologists Do Research
Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc. Chapter 8 Tests of Hypotheses Based on a Single Sample.
Lecture 9: p-value functions and intro to Bayesian thinking Matthew Fox Advanced Epidemiology.
The problem of sampling error in psychological research We previously noted that sampling error is problematic in psychological research because differences.
Chapter 6 Thinking & Intelligence 2 of 28 Topics to Explore 1.Problem Solving 2.Thinking Under Uncertainty 3.Intelligence.
Today’s Topic Do you believe in free will? Why or why not?
Theory testing Part of what differentiates science from non-science is the process of theory testing. When a theory has been articulated carefully, it.
“While nothing is more uncertain than a single life, nothing is more certain than the average duration of a thousand lives” -Elizur Wright By: Gail Larsen.
EVAL 6970: Cost Analysis for Evaluation Dr. Chris L. S. Coryn Nick Saxton Fall 2014.
Testing Theories: Three Reasons Why Data Might not Match the Theory Psych 437.
Interpersonal (Personality) Psychology CSCW January 26, 2004.
Week 1 Introduction to Psychology. Chapter 1 Overview Exploring psychology’s roots Exploring psychology’s roots Schools of thought in psychology Schools.
Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Lecture 25 Wednesday, 20 October.
Various topics Petter Mostad Overview Epidemiology Study types / data types Econometrics Time series data More about sampling –Estimation.
Testing Theories: The Problem of Sampling Error. The problem of sampling error It is often the case—especially when making point predictions—that what.
1 Reasoning Under Uncertainty Artificial Intelligence Chapter 9.
Bayesian vs. frequentist inference frequentist: 1) Deductive hypothesis testing of Popper--ruling out alternative explanations Falsification: can prove.
Sampling and Probability Chapter 5. Sampling & Elections >Problems with predicting elections: Sample sizes are too small Samples are biased (also tied.
Uncertainty Management in Rule-based Expert Systems
Uncertainty in Expert Systems
Bayesian Inference, Review 4/25/12 Frequentist inference Bayesian inference Review The Bayesian Heresy (pdf)pdf Professor Kari Lock Morgan Duke University.
Populations III: evidence, uncertainty, and decisions Bio 415/615.
Artificial Intelligence CIS 342 The College of Saint Rose David Goldschmidt, Ph.D.
Experimental Research Methods in Language Learning Chapter 5 Validity in Experimental Research.
Base Rate Neglect (continued) Psychology 466: Judgment & Decision Making Instructor: John Miyamoto 10/29/2015: Lecture 05-2 Note: This Powerpoint presentation.
The Psychology of Prediction and Uncertainty Jason Baer.
Unit 1 Sections 1-1 & : Introduction What is Statistics?  Statistics – the science of conducting studies to collect, organize, summarize, analyze,
Research Methods Chapter 2.
Linear Judgment Models: What Do They Suggest About Human Judgment?
When “Less Is More” – A Critique of the Heuristics & Biases Approach to Judgment and Decision Making Psychology 466: Judgment & Decision Making Instructor:
1 DECISION MAKING Suppose your patient (from the Brazilian rainforest) has tested positive for a rare but serious disease. Treatment exists but is risky.
Representativeness Heuristic Then: Framing Effects Psychology 355: Cognitive Psychology Instructor: John Miyamoto 6/2 /2015: Lecture 10-2 This Powerpoint.
Expected Value, Expected Utility & the Allais and Ellsberg Paradoxes
Introduction to Preference and Decision Making
Allais Paradox, Ellsberg Paradox, and the Common Consequence Principle Then: Introduction to Prospect Theory Psychology 466: Judgment & Decision Making.
Classroom Experiment Answer the questions on the handout labeled: “Four Famous Reasoning Problems” Try not to remember what you may have read about these.
Bayesian Statistics, Modeling & Reasoning What is this course about? P548: Intro Bayesian Stats with Psych Applications Instructor: John Miyamoto 01/04/2016:
INDUCTIVE ARGUMENTS The aim of this tutorial is to help you learn to recognize, analyze and evaluate inductive arguments.
Bayes Theorem, a.k.a. Bayes Rule
Markov-Chain-Monte-Carlo (MCMC) & The Metropolis-Hastings Algorithm P548: Intro Bayesian Stats with Psych Applications Instructor: John Miyamoto 01/19/2016:
A. Judgment Heuristics Definition: Rule of thumb; quick decision guide When are heuristics used? - When making intuitive judgments about relative likelihoods.
Phonological Loop, Visuospatial Sketchpad & Articulatory Suppression Psychology 355: Cognitive Psychology Instructor: John Miyamoto 04/25/2016: Lecture.
Outline Historical note about Bayes’ rule Bayesian updating for probability density functions –Salary offer estimate Coin trials example Reading material:
The Psychology of Inductive Inference Psychology 355: Cognitive Psychology Instructor: John Miyamoto 5/26/2016: Lecture 09-4 Note: This Powerpoint presentation.
Naive Bayes Classifier. REVIEW: Bayesian Methods Our focus this lecture: – Learning and classification methods based on probability theory. Bayes theorem.
The Representativeness Heuristic then: Risk Attitude and Framing Effects Psychology 355: Cognitive Psychology Instructor: John Miyamoto 6/1/2016: Lecture.
Computing with R & Bayesian Statistical Inference P548: Intro Bayesian Stats with Psych Applications Instructor: John Miyamoto 01/11/2016: Lecture 02-1.
Review of Probability.
MCMC Output & Metropolis-Hastings Algorithm Part I
Heuristics & Biases: The Availability Heuristic and The Representativeness Heuristic Psychology 355: Cognitive Psychology Instructor: John Miyamoto 5/31/2016:
Set Up for Instructor MGH Display: Try setting your resolution to 1024 by 768 Run Powerpoint. For most reliable start up: Start laptop & projector before.
1st: Representativeness Heuristic and Conjunction Errors 2nd: Risk Attitude and Framing Effects Psychology 355:
These slides are preview slides
Unilateral Neglect, Spatial Attention, Object-Based Attention
Brief History of Cognitive Psychology
Bayesian Reasoning & Base-Rate Neglect
My Office Hours I will stay after class on both Monday and Wednesday, i.e., 1:30 Mon/Wed in MGH 030. Can everyone stay if they need to? Psych 548, Miyamoto,
Introduction to Bayesian Model Comparison
Set Up for Instructor Classroom Support Services (CSS), 35 Kane Hall,
Set Up for Instructor Classroom Support Services (CSS), 35 Kane Hall,
STAT 515 Statistical Methods I Chapter 1 Data Types and Data Collection Brian Habing Department of Statistics University of South Carolina Redistribution.
Presentation transcript:

Representativeness, Similarity, & Base Rate Neglect Psychology 466: Judgment & Decision Making Instructor: John Miyamoto 10/27/2015: Lecture 05-1 Note: This Powerpoint presentation may contain macros that I wrote to help me create the slides. The macros aren’t needed to view the slides. You can disable or delete the macros without any change to the presentation.

Outline Base-rate neglect:  What is base-rate neglect?  How is base-rate neglect related to Bayesian reasoning Three types of evidence that people tend to ignore base-rate information. Similarity Heuristic: Base-rate neglect occurs because people often substitute a judgment of similarity for a judgment of probability (attribute substitution). Psych 466, Miyamoto, Aut '15 2 Representatitiveness Heuristic is an Example of Attribute Substitution

Representativeness Heuristic as Attribute Substitution Attribute Substitution: When trying to evaluate a hard-to-judge attribute, people often substitute a judgment of a related attribute that is easy-to-judge. ♦ E.g., when trying to judge a probability, people may substitute a judgment of availability or similarity. ♦ This is a common mental habit. People are often unaware that they are making this substitution. Psych 466, Miyamoto, Aut '15 3 Same Slide with Addition of Another Bulllet Point

Representativeness Heuristic as Attribute Substitution Attribute Substitution: When trying to evaluate a hard-to-judge attribute, people often substitute a judgment of a related attribute that is easy-to-judge. Similarity Thesis: People substitute a judgment of similarity for a judgment of probability. ♦ The conjunction fallacy – a consequence of similarity-based reasoning ♦ Insensitivity to sample size ♦ Insensitivity to regression effects ♦ Base rate neglect ♦ Misperceptions of randomness Psych 466, Miyamoto, Aut '15 4 Discussed Last Thursday. Discuss Today. Background to Study of Base Rate Neglect

What is "base-rate neglect"? ♦ "Base-rate": Another name for the prior probability of an individual event. ♦ "Base-rate neglect" has occurred if a person's judgments of probability are insufficiently influenced by variations in base-rate across different problems. Remainder of this Lecture: Bayesian Reasoning – what is it? What is a base-rate? Medical example of Bayesian reasoning Experimental evidence that people do not generally follow Bayes Rule when reasoning about probabilities Psych 466, Miyamoto, Aut '15 5 What is Bayesian Reasoning?

Psych 466, Miyamoto, Aut '15 6 What is Bayesian Reasoning? Bayesian Hypothesis: People reason as if uncertainty is measured by subjective probability. Possibly this is a normative claim. Possibly this is a descriptive claim. Bayesian theory accomplishes two things: 1.Tells us how our existing beliefs should adjust to new information. Tells us how our probabilities should change as we acquire new information. 2.Tells us how to adopt internally consistent gambling strategies. Tells us how to make optimal decisions relative to our values (utilities) and our subjective probabilities. Bayes Rule - What Is It? Why Is It Important? Bayes Rule

Bayes Rule – What Is It? Why Is It Important? Psych 466, Miyamoto, Aut '15 7 Reverend Thomas Bayes, 1702 – 1761 British clergyman & mathematician Bayes Rule is fundamentally important to: o Bayesian statistics o Bayesian decision theory o Bayesian models in psychology Explanation of Bayes Rule

Bayes Rule – Explanation (Look at Handout) Psych 466, Miyamoto, Aut '15 8 Derivation of the Odds Form of Bayes Rule Normalizing Constant Likelihood of the Data Posterior Probability of the Hypothesis Prior Probability of the Hypothesis a.k.a. base rate of the hypothesis

Psych 466, Miyamoto, Aut '15 9 Bayes Rule – Odds Form Bayes Rule for H given D Bayes Rule for not-H given D Odds Form of Bayes Rule Explanation of Odds form of Bayes Rule

H = a hypothesis, e.g., H = hypothesis that the patient has cancer = the negation of the hypothesis, e.g., ~H = hypothesis that the patient does not have cancer D = the data, e.g., D = + test result for a cancer test Bayes Rule (Odds Form) Psych 466, Miyamoto, Aut '15 10 Memorable Form of the Bayes Rule (Odds Version) Posterior Odds Likelihood Ratio (diagnosticity) Prior Odds

H = a hypothesis, e.g., H = hypothesis that the patient has cancer = the negation of the hypothesis, e.g., ~H = hypothesis that the patient does not have cancer D = the data, e.g., D = + test result for a cancer test Bayes Rule (Odds Form) Psych 466, Miyamoto, Aut '15 11 Interpretation of Medical Test Result Posterior Odds = (Likelihood Ratio) x (Prior Odds)

Psych 466, Miyamoto, Aut '15 12 Three Types of Empirical Violations of Bayes Rule Given statistical information about prior probabilities and likelihoods, people's judgments of posterior probability seriously violate Bayes Rule. ♦ Example: Physicians judgments of P( +Cancer | +Test Result) are much too high Judging probability in the context of widely-known base rates. ♦ Preview of finding: Judgments ignore well-known base rates. Judging probability when base rates are experimentally manipulated between subjects. ♦ Preview of finding: Judgments are approximately the same in conditions where the base rate is low or where it is high. Medical Diagnosis Problem Next.

Psych 466, Miyamoto, Aut '15 13 Bayesian Analysis of a Medical Test Result (Look at Handout) QUESTION: A physician knows from past experience in his practice that 1% of his patients have cancer (of a specific type) and 99% of his patients do not have the cancer. He also knows the probabilities of a positive test result (+ result) given cancer and given no cancer. These probabilities are: P(+ test | Cancer) =.792andP(+ test | no cancer) =.096 Suppose Mr. X has a positive test result. What is the probability that Mr. X has cancer? P(Cancer) = Prior probability of cancer =.01 P(No Cancer) = Prior probability of no cancer = 1 - P(Cancer) =.99 Frequency-Based Solution to this problem Have class write down their answers

Psych 466, Miyamoto, Aut '15 14 Frequency-Based Analysis of the Medical Test Result Imagine that we have data for 1,000 patients. A 1% cancer rate means: 990 do not have cancer, and 10 have cancer (on the average) 8 of 10 patients with cancer will have a + test result (8  0.792*10). 95 of 990 patients without cancer will have a + test result (95  0.092*990). 103 = : Out of 103 patients with + test results, 8 have cancer (8/103 = 0.077) and 95 do not have cancer (95/103 = 0.92). So if Mr. X has a + test result, he is much more likely to not have cancer than to have cancer. Math Analysis of the Medical Test Problem

Psych 466, Miyamoto, Aut '15 15 Mathematical Analysis of the Medical Test Result Mr. X has a + test result. What is the probability that Mr. X has cancer? P(+ test | Cancer) =.792andP(+ test | no cancer) =.096 P(Cancer) = Prior probability of cancer =.01 P(No Cancer) = Prior probability of no cancer = 1 - P(Cancer) =.99 P(Cancer | + test) = 1 / (12 + 1) = David Eddy: Fallacious Reasoning by Physicians re Interpretation of Test Result Bayes Rule

Psych 466, Miyamoto, Aut '15 16 Fallacious Reasoning by Physicians Correct answer given the data (based on Eq. (4)) : P(Cancer | + Result) = (.792)(.01)/(.103) =.077 Notice: The test is very diagnostic but still P(cancer | + result) is low because the base rate is low. David Eddy found that about 95% of physicians think that P(cancer | +result) is about 75% in this case (very close to the 79% likelihood of a + result given cancer). Physicians sometimes overlook base rates! Bad Advice from a Medical Textbook

Psych 466, Miyamoto, Aut '15 17 Fallacious Probabilistic Reasoning in Medicine DeGowin & DeGowin. Bedside diagnostic examination. "When a patient consults his physician with an undiagnosed disease, neither he nor the doctor knows whether it is rare until the diagnosis is finally made. Statistical methods can only be applied to a population of thousands. The individual either has a rare disease or doesn't have it; the relative incidence of two diseases is completely irrelevant to the problem of making his diagnosis." Eddy documents many examples in medical literature of confusion between P(cancer | +result) and P(+ result | cancer). General Characteristics of Bayesian Inference DeGowin & DeGowin say: Ignore the prior odds!

Psych 466, Miyamoto, Aut '15 18 General Characteristics of Bayesian Inference The decision maker (DM) is willing to specify the prior probability of the hypotheses of interest. DM can specify the likelihood of the data given each hypothesis. Using Bayes Rule, we infer the probability of the hypotheses given the data. Main Point of Cognitive Critique: People tend to ignore base rates. People do not use Bayes Rule to update their probabilities. Three Types of Empirical Violations of Bayes Rule Given Information

Psych 466, Miyamoto, Aut '15 19 Three Types of Empirical Violations of Bayes Rule Given statistical information about prior probabilities and likelihoods, people's judgments of posterior probability seriously violate Bayes Rule. ♦ Example: Physicians judgments of P( +Cancer | +Test Result) are much too high Judging probability in the context of widely-known base rates. ♦ Preview of finding: Judgments ignore well-known base rates. Judging probability when base rates are experimentally manipulated between subjects. ♦ Preview of finding: Judgments are approximately the same in conditions where the base rate is low or where it is high. Medical Diagnosis Problem Previous Topic. Next Topic.

Psych 466, Miyamoto, Aut '15 20 Problem #4: Tom W (K&T, 1973) PERSONALITY DESCRIPTION OF TOM W: Tom W. is currently a graduate student at the University of Washington. Tom W. is of high intelligence, although lacking in true creativity. He has a need for order and clarity, and for neat and tidy systems in which every detail finds its appropriate place. His writing is rather dull and mechanical, occasionally enlivened by somewhat corny puns and by flashes of imagination of the sci-fi type. He has a strong drive for competence. He seems to have little feel and little sympathy for other people and does not enjoy interacting with others. Self-centered, he nonetheless has a deep moral sense. Response Sheet

Psych 466, Miyamoto, Aut '15 21 Analysis of Problem #4: Tom W (cont.) Between-subjects design. Separate groups of subjects judged (i) base rate, (ii) similarity ranks, (iii) probability ranks. EstimatedSimilarityProbability Base RateRankRankGrad specialization _______________Bus AD _______________Computer Science _______________Engineering _______________Humanities _______________Law _______________Library Science _______________Medicine _______________Physical Science _______________Social Science Same Slide without Emphasis Rectangles Condition 3 Condition 2 Condition 1

Psych 466, Miyamoto, Aut '15 22 Analysis of Problem #4: Tom W (cont.) Between-subjects design. Separate groups of subjects judged (i) base rate, (ii) similarity ranks, (iii) probability ranks. EstimatedSimilarityProbability Base RateRankRankGrad specialization _______________Bus AD _______________Computer Science _______________Engineering _______________Humanities _______________Law _______________Library Science _______________Medicine _______________Physical Science _______________Social Science Use Bayes Rule to Analyze Tom W Problem

Relative Prevalence of Different Grad Specializations People fail to take this into account. Tom W & Bayes Rule Psych 466, Miyamoto, Aut '15 23 D = Data = the personality description of Tom W H = Hypothesis that Tom W is a ____ student, e.g., an engineering student Results for Tom W Probability of Grad Specialization H Given Tom W's Description Similarity of H to D i.e., similarity of stereotype H to Tom W's Description

Psych 466, Miyamoto, Aut '13 24 Tom W Problem - Results Correlation between judged base rate and likelihood rank = .63. Correlation between similarity rank and likelihood rank = "Likelihood" is just another word for "probability" and here it refers to the posterior probability of each of the grad specializations. Repeat: Tom W & Bayes Rule

Relative Prevalence of Different Grad Specializations People fail to take this into account! Tom W & Bayes Rule Psych 466, Miyamoto, Aut '15 25 D = Data = the personality description of Tom W H = Hypothesis that Tom W is a ____ student, e.g., an engineering student Conclusions re Tom W Probability of Grad Specialization H Given Tom W's Description Similarity of H to D i.e., similarity of stereotype H to Tom W's Description

Similarity heuristic is one part of the representativeness heuristic. The judgment of probability ought to be base in part on a judgment of similarity. ( Similarity determines the diagnosticity of the evidence. ) Focusing on similarity should not cause one to overlook the relevance of base rate. Psych 466, Miyamoto, Aut '15 26 Similarity Heuristic Predicts Tom W Results Repeat: Bayes Rule Odds Form with Annotation for Components People substitute a judgment of similarity,.... How much is Tom like the stereotype of a computer science student?... for a judgment of probability What is the probability that Tom is a computer science student?

Relative Prevalence of Different Grad Specializations People fail to take this into account! Tom W & Bayes Rule Psych 466, Miyamoto, Aut '15 27 D = Data = the personality description of Tom W H = Hypothesis that Tom W is a ____ student, e.g., an engineering student Return to ThreeTypes of Empirical Violations of Bayes Rule Probability of Grad Specialization H Given Tom W's Description Similarity of H to D i.e., similarity of stereotype H to Tom W's Description

Psych 466, Miyamoto, Aut '15 28 Tuesday, October 27, 2015 : The Lecture Ended Here

Psych 466, Miyamoto, Aut '15 29 Three Types of Empirical Violations of Bayes Rule Given statistical information about prior probabilities and likelihoods, people's judgments of posterior probability seriously violate Bayes Rule. ♦ Example: Physicians judgments of P( +Cancer | +Test Result) are much too high Judging probability in the context of widely-known base rates. ♦ Tom W: Judgments ignore well-known base rates. Judging probability when base rates are experimentally manipulated between subjects. ♦ Preview of finding: Judgments are approximately the same in conditions where the base rate is low or where it is high. Lawyer/Engineer Problem Previous Topic. Next Topic.