Rational analysis of the selection task

Slides:



Advertisements
Similar presentations
Part II Sigma Freud & Descriptive Statistics
Advertisements

 Cognitive Modules › Background  Wason Selection Task › Purpose › Puzzles vs Social Contract problems  Fiddick & Erlich’s Paper › Introduction › Methods.
PROBABILITY. Uncertainty  Let action A t = leave for airport t minutes before flight from Logan Airport  Will A t get me there on time ? Problems :
Stupid Bayesian Tricks Gregory Lopez, MA, PharmD SkeptiCamp 2009.
1 G Lect 2a G Lecture 2a Thinking about variability Samples and variability Null hypothesis testing.
Reasoning Lindsay Anderson. The Papers “The probabilistic approach to human reasoning”- Oaksford, M., & Chater, N. “Two kinds of Reasoning” – Rips, L.
Cognitive Processes PSY 334 Chapter 10 – Reasoning.
Introduction  Bayesian methods are becoming very important in the cognitive sciences  Bayesian statistics is a framework for doing inference, in a principled.
Solved the Maze? Start at phil’s house. At first, you can only make right turns through the maze. Each time you cross the red zigzag sign (under Carl’s.
Experimental Design, Statistical Analysis CSCI 4800/6800 University of Georgia Spring 2007 Eileen Kraemer.
Reasoning and Rationality Emily Slusser February 13 th 2006 Charter, N. &Oaksford, M. (1999). Ten years of the rational analysis of cognition. Trends in.
C82MCP Diploma Statistics School of Psychology University of Nottingham 1 Overview of Lecture Independent and Dependent Variables Between and Within Designs.
Theoretical Models Economists use models to describe economic activities While most economic models are abstractions from reality, they provide aid in.
Fig Theory construction. A good theory will generate a host of testable hypotheses. In a typical study, only one or a few of these hypotheses can.
The Characteristics of an Experimental Hypothesis
Methods of Observation PS 204A, Week 2. What is Science? Science is: (think Ruse) Based on natural laws/empirical regularities. Based on natural laws/empirical.
Research in Psychology
1 © Lecture note 3 Hypothesis Testing MAKE HYPOTHESIS ©
© 2008 McGraw-Hill Higher Education The Statistical Imagination Chapter 9. Hypothesis Testing I: The Six Steps of Statistical Inference.
POSC 202A: Lecture 1 Introductions Syllabus R Homework #1: Get R installed on your laptop; read chapters 1-2 in Daalgard, 1 in Zuur, See syllabus for Moore.
Chapter 8 Introduction to Hypothesis Testing
Perfection and bounded rationality in the study of cognition Henry Brighton.
1 Today Null and alternative hypotheses 1- and 2-tailed tests Regions of rejection Sampling distributions The Central Limit Theorem Standard errors z-tests.
Reasoning Top-down biases symbolic distance effects semantic congruity effects Formal logic syllogisms conditional reasoning.
Biostatistics Class 6 Hypothesis Testing: One-Sample Inference 2/29/2000.
Evolution of Logical Reasoning
Reasoning.
Logic. What is logic? Logic (from the Ancient Greek: λογική, logike) is the use and study of valid reasoning. The study of logic features most prominently.
IT 와 인간의 만남 KAIST 지식서비스공학과 Experimental Research KSE966/986 Seminar Uichin Lee Sept. 21, 2012.
Uncertainty Management in Rule-based Expert Systems
HOW TO CRITIQUE AN ARGUMENT
Uncertainty in Expert Systems
 Descriptive Methods ◦ Observation ◦ Survey Research  Experimental Methods ◦ Independent Groups Designs ◦ Repeated Measures Designs ◦ Complex Designs.
©2010 John Wiley and Sons Chapter 2 Research Methods in Human-Computer Interaction Chapter 2- Experimental Research.
Artificial Intelligence CIS 342 The College of Saint Rose David Goldschmidt, Ph.D.
Research Methodology and Methods of Social Inquiry Nov 8, 2011 Assessing Measurement Reliability & Validity.
Textbook Basics of an Expert System: – “Expert systems: Design and Development,” by: John Durkin, 1994, Chapters 1-4. Uncertainty (Probability, Certainty.
Outline of Today’s Discussion 1.The Chi-Square Test of Independence 2.The Chi-Square Test of Goodness of Fit.
Cognitive Processes PSY 334 Chapter 10 – Reasoning.
Chapter 1 Introduction to Research in Psychology.
How Psychologists Do Research Chapter 2. How Psychologists Do Research What makes psychological research scientific? Research Methods Descriptive studies.
Cognitive Adaptations for Social Exchange Leda Cosmides and John Tooby presented by Nat Twarog.
Rational analysis of the selection task Oaksford and Chater (1994) Presented by Bryan C. Russell.
Is domain-specific reasoning in conditional reasoning tasks really domain-specific? The 2 nd London Reasoning Workshop 28-29/08/2007 Akira Nakagaki (Waseda.
Survival Skills for Researchers Study Design. Typical Process in Research Design study Generate hypotheses Develop tentative new theories Analyze & interpret.
Confirmation Bias: A Methodology of Social Science
Features of science revision
Measurement Theory Michael J. Watts
Logic of Hypothesis Testing
Chapter 2 Theoretical statement:
Unit 3 Hypothesis.
Statistical Core Didactic
MODULE 2 Myers’ Exploring Psychology 5th Ed.
Chapter 1 Definition Theory Causality
Hypothesis Testing: Hypotheses
Henrik Singmann Karl Christoph Klauer Sieghard Beller
Henrik Singmann Karl Christoph Klauer
Hypothesis Testing Summer 2017 Summer Institutes.
Henrik Singmann Sieghard Beller Karl Christoph Klauer
Representing Uncertainty
Discrete Event Simulation - 4
How Psychologists Do Research
A Hierarchical Bayesian Look at Some Debates in Category Learning
Wason and Johnson-Laird (1972)
POSC 202A: Lecture 1 Introductions Syllabus R
The Scientific Method.
Wason and Johnson-Laird (1972)
Confidence Intervals.
Diploma in Human Resource Management Using Information in Human Resources 5UIN 5/22/2019.
Introduction to the design (and analysis) of experiments
Presentation transcript:

Rational analysis of the selection task Oaksford and Chater (1994) Presented by Bryan C. Russell

A K 2 7 Wason selection task Rule: if there is a vowel on one side, then there is an even number on the other side A K 2 7

Last time… Humans are stupid Cosmides et al. Social contract theory (i.e. innate ability to detect cheaters) Depends on perspective (enforcer or actor)

Outline Probabilistic account of Wason selection task Information-theoretic background Application to abstract selection task Application to thematic selection task Discussion

Another view of the task Let rule be “if p then q” Four types of cards (p,q), (not-p,q), (p,not-q), (not-p,not-q) Two hypotheses MD: p,q are dependent (rule is true) MI: p,q are independent (rule is false)

Assumptions We can assign probabilities to the cards Should reflect natural statistics of “if p then q” statements in nature P(p | MD) = P(p | MI) = a P(q | not-p,MD) = P(q | not-p,MI) = b

Card probabilities

Card probabilities Task: Select card that maximally reduces hypothesis uncertainty

Entropy/uncertainty

Entropy/uncertainty

Entropy/uncertainty

Another experiment… Suppose you observe: TTHTHHTHHHHTHHTHTHHT

Another experiment… Suppose you observe: TTHTHHTHHHHTHHTHTHHT HTHHTTHTHHTTTTTTTTTH

Another experiment… Suppose you observe: TTHTHHTHHHHTHHTHTHHT HTHHTTHTHHTTTTTTTTTH

Mutual information

Application to selection task a = Pr(p)

Application to selection task a = Pr(p)

Model behavior

Observations If Pr(q) is low, then choosing p card is informative If Pr(p) and Pr(q) is low, then choosing q card is informative If Pr(p) is high, then choosing not-q card is informative not-p card is not informative (results in zero information) P(MI) only scales information values

Model behavior R

Rarity assumption For selection task, in humans Pr(p) and Pr(q) are low Expectation over region R choose p: 0.76 choose q: 0.20 choose not-q: 0.09 choose not-p: 0

How do humans compare?

Analysis Both humans and model accounts for the following information relationship: choose p > choose q > choose not-q > choose not-p

Thematic selection task If a passenger form says “ENTERING” on one side then the other side must include cholera information “ENTERING” checked “ENTERING” not checked cholera information no cholera information

Rule types Obligations: if action (p), then must condition (q) Permissions: if condition (p), then may action (q)

Subject perspective for permission rule Enforcer Pretend you are an immigrant officer… Actor Pretend you are a traveler… Objective inquirer Check the validity of the statement…

Human performance Obligation: action (p) => must condition (q) Permission: condition (p) => may action (q)

Utility-based model Focus on rule-use, not rule-testing Associate cost with turning over a card

Utility-based model

Performance of utility-based model

Anderson’s (1990) steps for rationality Specify precisely the goals of the cognitive system Develop a formal model of the environment to which the system is adapted Make minimal assumptions about computational limitations

Anderson’s (1990) steps for rationality Derive the optimal behavior function given the previous steps Examine the empirical evidence to see whether the predictions of the behavioral function are confirmed Rinse, lather, repeat, and refine the theory

Conclusions Bayesian approach more principled than other accounts Applies to both abstract and thematic versions of the problem (and their variants) Behavior is adapted to the environment and does not necessarily follow logic/mathematic theory

Discussion questions Is the rarity assumption valid? How do we test it? Is the selection task representative of accounting for rational thought? Is it exhaustive? How is this model realized? Does one learn explicitly the utility costs and Pr(p)/Pr(q)?