Cascaded Inference David Lagnado Evidence Project

Plan of seminar Some ideas about cascaded inference Previous work on hierarchical inference

Cascaded inference A chain of probabilistic inferences  ‘Inference upon inference’ Pervasive in many domains  Legal  Medical  Criminal  Advertising  Everyday

Cascaded inference Horse race example WEATHER (Sun, rain, drizzle, frost…) TRACK CONDITION (Heavy, soft, good, firm…) WINNER (‘Waltzing Along’, ‘Persian Weaver’, ‘Ride the Storm’…)

Cascaded inference Legal example TESTIMONY Witness testifies that accused was at crime scene EVENT Accused was at crime scene CLAIM Accused is guilty of crime

Cascaded inference Used car example Salesman’s report Maintenance history Car reliability

Cascaded inference How do people do it? Very little recent research  Most done in  Using abstract problems  No comprehensive normative model Current psychological models also seem inadequate

Normative model for cascaded inference? Modified Bayes theorem (Dodson), Jeffrey’s rule, Chain rule Weighted sum across all possible paths ABC Given evidence A: P(C|A) = P(C|B).P(B|A) + P(C|~B).P(~B|A) A ~B C B ~C E.g., A = heavy rain; B = muddy track; C = ‘Ride the Storm’ wins P(C|B) P(C|~B) P(B|A) P(~B|A) P(~C|~B) P(~C|B)

Simplifying heuristics Various studies claim that people use simplifying strategies (as-if, best guess)  Gettys et al., 1973; Schum et al., 1973; Steiger & Gettys, 1972 Treat inference from first stage as-if it is true Focus on most probable path, ignore alternative less probable paths Such strategies lead to overestimation of final probability judgments

As-if or Best guess model A ~B C B ~C Select the most probable outcome at first stage of inference (from A to B) E.g., A = heavy rain; B = muddy track; C = ‘Ride the Storm’ wins

As-if or Best guess model Select the most probable outcome at first stage of inference (from A to B) Assume this is true for the second stage inference (from B to C) Judged probability of C given A =.7 This overestimates the correct value =.52 A ~B C B ~C.7 E.g., A = heavy rain; B = muddy track; C = ‘Ride the Storm’ wins

Current study Looking at both abstract and applied settings  Some studies claim that overestimation only occurs in abstract settings Vary presentation of probability information to see if this improves inferences  Frequency tables  Pie charts  Network diagrams?

Shortcomings with previous research Neglects structural assumptions  Causal models  Conditional independencies etc These determine appropriate normative model, and likely to influence people’s reasoning Current psychological models of probabilistic reasoning do not take these factors into account  Belief activation (neural networks)  Belief updating (rule-based)  Heuristic models

Influence of causal model How are cascaded inferences affected by assumptions about causal structure? Different causal models can imply different conditional independencies ABC A and C are (unconditionally) dependent BUT A and C are conditionally independent given B ABC A and C are (unconditionally) independent BUT A and C are conditionally dependent given B

Causal models and cascaded inference MeaslesSpotsItchy Evidence = Measles Infer that spots are probable Infer that itchiness is probable MeaslesSpotsChicken pox Evidence = Measles Infer that spots are probable Do not infer that Chicken pox is probable Evidence does not alter P(Chicken pox) Causal structure influences permissible inferences

Causal models and cascaded inference Schum – ‘conditional non-independence’ ABC E.g., A = Weather, B = Track condition, C = Winner Some horses may run better/worse in the rain, independent of the track condition

Moral Normative models for cascaded inference depend on causal and structural assumptions

Future research Construct cognitive model of cascaded inference that allows for causal and structural assumptions How well does this conform to normative models? Various studies suggest that people are poor at probability estimation/computation, but are quite good at qualitative causal reasoning Decision aids  To support inference when there are many variables, complex computations etc.  To correct for any systematic biases  Better presentation of probability information  E.g., graphical representations …

Interdisciplinary aid What are the appropriate normative models (statistics)? What are the plausible computational algorithms? Are there economical heuristic procedures? What are the cognitive mechanisms that people use (psychology, neuroscience)? Are there naturally occurring inference problems of this kind (epidemiology, forensic, legal, history)?

Some earlier studies on hierarchical inference Cascaded inference Hierarchically structured information Learning and judgment

Learning and judgment with category hierarchies Hierarchical structure  Pervasive feature of how we represent the world  Organizes knowledge and structures inference FLU Type B Type AType C

Inference using a hierarchy One powerful feature of a category hierarchy is that given information about categories at one level, you can make inferences about categories at another level. This allows you to exclude alternatives, or reduce the number you need to consider TabloidBroadsheet TimesGuardian Mirror Sun

Probabilistic inference using a hierarchy In many real-world situations we must base our initial category judgments on imperfect cues, degraded stimuli, or statistical data. What effect do such probabilistic contexts have on the hierarchical inferences that we are licensed to make? TabloidBroadsheet TimesGuardian Mirror Sun

Commitment heuristic Reason ‘as-if’ probable info is true (but reduce overall confidence) Commitment heuristic - When people select the most probable category at the superordinate level, they assume that it contains the most probable subordinate category (and vice-versa) This leads to the neglect of subordinates from the less probable superordinate TabloidBroadsheet TimesGuardian Mirror Sun

How effective is such a heuristic? Depends on the structure of the environment In certain environments it is advantageous  increases inferential power by focus on appropriate subcategories  reduces computational demands by avoiding complex Bayesian calculations But in some environments it leads to anomalous judgments and choices

Non-aligned hierarchy The most frequently read type of paper is a Tabloid, but the most frequently read paper is the Guardian (a Broadsheet). Non-aligned hierarchy: the most probable superordinate category does not contain the most probable subordinate category. Tabloid 60Broadsheet 40 Times 5 Guardian 35 Mirror 30 Sun 30

Cascaded inference Inference from hierarchy to a related category Different levels can support opposing predictions Tabloid 60Broadsheet 40 Times 5 Guardian 35 Mirror 30 Sun 30 Party A Party B

Experiments 1 & 2 What is effect of manipulating level of representation on subsequent probability judgment? Learning phase - participants exposed to a non- aligned hierarchical environment in which they learn to predict voting behavior from newspaper readership. 100 trials ‘reading/voting profiles’

Screen during learning phase Broadsheet Sun Tabloid MirrorGuardianTimes ○ Party A ○ Party B

Screen during learning phase Broadsheet Sun Tabloid MirrorGuardianTimes ○ Party A ○ Party B Reading profile for J. K.

Screen during learning phase Broadsheet Sun Tabloid MirrorGuardianTimes ○ Party A ○ Party B Reading profile for J. K. Outcome feedback

Structure of environment Tabloid 60Broadsheet 40 Times 5 Guardian 35 Mirror 30 Sun 30 Party A Party B 50 NB Overall each party equally frequent

Judgment phase Which paper is X most likely to read? X is selected at random What is the probability that X votes for Party B rather than A? Which kind of paper is X most likely to read? Baseline General Specific

Predictions Which paper is X most likely to read? X is selected at random What is the probability that X votes for Party B rather than A? Which kind of paper is X most likely to read? Baseline General Specific General choice Tabloid -> Party A Specific choice Guardian -> Party B Based on commitment heuristic

Results Which paper is X most likely to read? X is selected at random What is the probability that X votes for Party B rather than A? Which kind of paper is X most likely to read? Baseline General Specific General choice Tabloid -> Party A Specific choice Guardian -> Party B 47% 28% 75% Mean probability rating

Summary People allow their initial categorization to shift their inferences, even though all judgments are based on the same statistical data. Simplifying heuristic that assumes that environment is aligned Empowers inference when hierarchical structure is aligned, otherwise can lead to error Suggests tendency to reason as if a probable conclusion is true

Applied to Medical choices In medical settings treatment options (and survival rates) often grouped to facilitate understanding and communication Can this lead to errors? SURGERY DRUGS Type2 Type1 Type2 Type1

Medical choices Suppose success rates are non-aligned Will grouping affect people’s treatment choices? SURGERY 60% DRUGS 40% Type2 10% Type1 70% Type2 60% Type1 60%

Medical choices experiment Participants learn about success rates trial-by-trial SURGERY 60% DRUGS 40% Type2 10% Type1 70% Type2 60% Type1 60%

Medical choices experiment Ungrouped control – 75% chose most effective treatment Drug4 10% Drug3 70% Drug2 60% Drug1 60%

Medical choices experiment Grouped condition – 25% chose most effective treatment (first asked about best superordinate treatment) SURGERY 60% DRUGS 40% Type2 10% Type1 70% Type2 60% Type1 60%

Summary Grouping with a non-aligned hierarchy can lead to poorer choices How generalizable is this?  Does it depend on memory processes?  Will it apply to summary presentations? Consider situations where people must use large databases with various levels of hierarchy (e.g., NHS statistics)

Useful biases? Systematic biases reveal something about judgment and learning processes Need not be indictment of human reasoning Heuristic strategies might be well adapted to the inferential tasks that we commonly confront

Challenge for future Do people use simplifying heuristics in cascaded and hierarchical inference? If so, how can we work with these heuristics to improve human judgment?