1. Everyday inductive leaps Making predictions and detecting coincidences Tom Griffiths Department of Psychology Program in Cognitive Science University of California, Berkeley (joint work with Josh Tenenbaum, MIT)

2. Inductive problems Inferring structure from data Perception –e.g. structure of 3D world from 2D visual data data hypotheses cube shaded hexagon

3. Inductive problems Inferring structure from data Perception –e.g. structure of 3D world from 2D visual data Cognition –e.g. whether a process is random hypotheses fair coin two heads data HHHHH

4. Perception is optimal

5. Cognition is not

6. Everyday inductive leaps Inferences we make effortlessly every day –making predictions –detecting coincidences –evaluating randomness –learning causal relationships –identifying categories –picking out regularities in language A chance to study induction in microcosm, and compare cognition to optimal solutions

7. Two everyday inductive leaps Predicting the future Detecting coincidences

8. Two everyday inductive leaps Predicting the future Detecting coincidences

9. Predicting the future How often is Google News updated? t = time since last update t total = time between updates What should we guess for t total given t ?

10. Reverend Thomas Bayes

11. Bayes’ theorem Posterior probability LikelihoodPrior probability Sum over space of hypotheses h: hypothesis d: data

12. Bayes’ theorem h: hypothesis d: data

13. Bayesian inference p(t total |t)  p(t|t total ) p(t total ) posterior probability likelihoodprior

14. Bayesian inference p(t total |t)  p(t|t total ) p(t total ) p(t total |t)  1/t total p(t total ) assume random sample (0 < t < t total ) posterior probability likelihoodprior

15. The effects of priors Different kinds of priors p(t total ) are appropriate in different domains e.g. wealthe.g. height

16. The effects of priors

17. Evaluating human predictions Different domains with different priors: –a movie has made $60 million [power-law] –your friend quotes from line 17 of a poem [power-law] –you meet a 78 year old man [Gaussian] –a movie has been running for 55 minutes [Gaussian] –a U.S. congressman has served 11 years [Erlang] Prior distributions derived from actual data Use 5 values of t for each People predict t total

18. people parametric prior empirical prior Gott’s rule

19. Probability matching p(t total |t past ) t total Quantile of Bayesian posterior distribution Proportion of judgments below predicted value

20. Probability matching Average over all prediction tasks: movie run times movie grosses poem lengths life spans terms in congress cake baking times p(t total |t past ) t total Quantile of Bayesian posterior distribution Proportion of judgments below predicted value

21. Predicting the future People produce accurate predictions for the duration and extent of everyday events Strong prior knowledge –form of the prior (power-law or exponential) –distribution given that form (parameters) Contrast with “base rate neglect” (Kahneman & Tversky, 1973)

22. Two everyday inductive leaps Predicting the future Detecting coincidences

23. November 12, 2001: New Jersey lottery results were 5-8-7, the same day that American Airlines flight 587 crashed

24. "It could be that, collectively, the people in New York caused those lottery numbers to come up 911," says Henry Reed. A psychologist who specializes in intuition, he teaches seminars at the Edgar Cayce Association for Research and Enlightenment in Virginia Beach, VA. "If enough people all are thinking the same thing, at the same time, they can cause events to happen," he says. "It's called psychokinesis."

25. The bombing of London (Gilovich, 1991)

26. The bombing of London (Gilovich, 1991)

27. (Snow, 1855) John Snow and cholera

29. 76 years 75 years (Halley, 1752)

30. The paradox of coincidences How can coincidences simultaneously lead us to irrational conclusions and significant discoveries?

31. A common definition: Coincidences are unlikely events “an event which seems so unlikely that it is worth telling a story about” “we sense that it is too unlikely to have been the result of luck or mere chance”

32. Coincidences are not just unlikely... HHHHHHHHHH vs. HHTHTHTTHT

33. Priors: p(cause) p(chance) Data: d Hypotheses: causechance a novel causal relationship exists no such relationship exists Likelihoods: p(d|cause) p(d|chance) Bayesian causal induction

34. Likelihood ratio (evidence) Prior odds high low high low cause chance ? ?

35. Likelihood ratio (evidence) Prior odds high low high low cause chance coincidence ? Bayesian causal induction

36. What makes a coincidence? A coincidence is an event that provides evidence for causal structure, but not enough evidence to make us believe that structure exists

37. What makes a coincidence? likelihood ratio is high A coincidence is an event that provides evidence for causal structure, but not enough evidence to make us believe that structure exists

38. likelihood ratio is high prior odds are low posterior odds are middling A coincidence is an event that provides evidence for causal structure, but not enough evidence to make us believe that structure exists What makes a coincidence?

39. Coincidence and the supernatural our (physical) theory says no connection exists the data suggest a connection between events our physical theory is wrong, or there are forces outside it likelihood ratio is high prior odds are low posterior odds are middling

40. HHHHHHHHHH HHTHTHTTHT likelihood ratio is high prior odds are low posterior odds are middling

41. Bayesian causal induction Hypotheses: Likelihoods: Priors: Data: frequency of effect in presence of cause cause chance E C E C  1 -  0 < p(E) < 1 p(E) = 0.5

42. HHHHHHHHHH HHTHTHTTHT likelihood ratio is high prior odds are low posterior odds are middling likelihood ratio is low prior odds are low posterior odds are low coincidence chance

43. Empirical tests Is this definition correct? –from coincidence to evidence How do people assess complex coincidences? –the bombing of London –coincidences in date

44. Empirical tests Is this definition correct? –from coincidence to evidence How do people assess complex coincidences? –the bombing of London –coincidences in date

45. HHHHHHHHHHHHHHHHHHHHHH HHHHHHHHHH likelihood ratio is high prior odds are low posterior odds are middling coincidence likelihood ratio is very high prior odds are low posterior odds are high cause

46. From coincidence to evidence coincidenceevidence for a causal relation Transition produced by –increase in likelihood ratio (e.g., coin flipping) –increase in prior odds (e.g., genetics vs.ESP)

47. Testing the definition Provide participants with data from experiments Manipulate: –cover story: genetics vs. ESP (prior) –data: number of heads/males (likelihood) –task: “coincidence or evidence?” vs. “how likely?” Predictions: –coincidences affected by prior and likelihood –relationship between coincidence and posterior

48. 47 51 55 59 63 70 87 99 r = -0.98 47 51 55 59 63 70 87 99 Number of heads/males Proportion “coincidence” Posterior probability

49. Empirical tests Is this definition correct? –from coincidence to evidence How do people assess complex coincidences? –the bombing of London –coincidences in date

50. Complex coincidences Many coincidences involve structure hidden in a sea of noise (e.g., bombing of London) How well do people detect such structure? Strategy: examine correspondence between strength of coincidence and likelihood ratio

51. The bombing of London

53. (uniform) Spread Location Ratio Number Change in... People

54. Bayesian causal induction Hypotheses: Likelihoods: Priors:  1 -  uniform + regularity cause chance T X X X X T T T T X X X X T Data: bomb locations

55. r = 0.98 (uniform) Spread Location Ratio Number Change in... People Bayes

56. 76 years 75 years

57. May 14, July 8, August 21, December 25 vs. August 3, August 3, August 3, August 3 Coincidences in date

58. People

59. Bayesian causal induction Hypotheses: Likelihoods: Priors:  1 -  uniform uniform + regularity August Data: birthdays of those present cause chance P P P P P P P P B B B B B B B B

60. People Bayes Regularities: Proximity in date Same day of month Same month

61. Coincidences Provide evidence for causal structure, but not enough to make us believe that structure exists Intimately related to causal induction –an opportunity to revise a theory –a window on the process of discovery Guided by a well calibrated sense of when an event provides evidence of causal structure

62. The paradox of coincidences false significant discovery true false conclusion Status of current theory Consequence The utility of attending to coincidences depends upon how much you know already

63. Two everyday inductive leaps Predicting the future Detecting coincidences

64. Subjective randomness View randomness as an inference about generating processes behind data Analysis similar (but inverse) to coincidences –randomness is evidence against a regular generating process (Griffiths & Tenenbaum, 2003)

65. AB Other cases of causal induction (Griffiths, Baraff, & Tenenbaum, 2004)

66. Aspects of language acquisition (Goldwater, Griffiths, & Johnson, 2006)

67. Categorization x Probability (Sanborn, Griffiths, & Navarro, 2006)

68. Conclusions We can learn about cognition (and not just perception) by thinking about optimal solutions to computational problems We can study induction using the inferences that people make every day Bayesian inference offers a way to understand these inductive inferences

71. Magic tricks Magic tricks are regularly used to identify infants’ ontological commitments Can we use a similar method with adults? (Wynn, 1992)

72. Ontological commitments (Keil, 1981)

73. What’s a better magic trick?

74. Participants rate the quality of 45 transformations, 10 appearances, and 10 disappearances –direction of transformation is randomized between subjects A second group rates similarity Objects are chosen to lie at different points in a hierarchy milk water a brick a vase a rose a daffodil a dove a blackbird a man a girl Applicable predicates

75. milk water a brick a vase a rose a daffodil a dove a blackbird a man a girl milk water a brick a vase a rose a daffodil a dove a blackbird a man a girl milk water a brick a vase a rose a daffodil a dove a blackbird a man a girl milk water a brick a vase a rose a daffodil a dove a blackbird a man a girl What’s a better magic trick?

76. Ontological asymmetries milk water a brick a vase a rose a daffodil a dove a blackbird a man a girl milk water a brick a vase a rose a daffodil a dove a blackbird a man a girl milk water a brick a vase a rose a daffodil a dove a blackbird a man a girl milk water a brick a vase a rose a daffodil a dove a blackbird a man a girl

77. Analyzing asymmetry Build a regression model: –similarity –appearing object –disappearing object –contains people –direction in hierarchy (-1,0,1) All factors significant Explains 90.9% of variance milk water a brick a vase a rose a daffodil a dove a blackbird a man a girl Applicable predicates

78. Summary: magic tricks Certain factors reliably influence the estimated quality of a magic trick Magic tricks might be a way to investigate our ontological assumptions –inviolable laws that are otherwise hard to assess A Bayesian theory of magic tricks? –strong evidence for a novel causal force –causal force is given low prior probability

80. A reformulation: unlikely kinds Coincidences are events of an unlikely kind –e.g. a sequence with that number of heads Deals with the obvious problem... p(10 heads) < p(5 heads, 5 tails)

81. Problems with unlikely kinds Defining kinds August 3, August 3, August 3, August 3 January 12, March 22, March 22, July 19, October 1, December 8

82. Problems with unlikely kinds Defining kinds Counterexamples P(4 heads) < P(2 heads, 2 tails) P(4 heads) > P(15 heads, 8 tails) HHHH > HHHHTHTTHHHTHTHHTHTTHHH HHHH > HHTT

84. Sampling from categories Frog distribution P(x|c)

85. Markov chain Monte Carlo Sample from a target distribution P(x) by constructing Markov chain for which P(x) is the stationary distribution Markov chain converges to its stationary distribution, providing outcomes that can be used similarly to samples

86. Metropolis-Hastings algorithm p(x)p(x)

87. p(x)p(x)

88. p(x)p(x)

89. A(x (t), x (t+1) ) = 0.5 p(x)p(x)

90. Metropolis-Hastings algorithm p(x)p(x)

91. A(x (t), x (t+1) ) = 1 p(x)p(x)

92. A task Ask subjects which of two alternatives comes from a target category Which animal is a frog?

93. Collecting the samples Which is the frog? Trial 1Trial 2Trial 3

94. Sampling from natural categories Examined distributions for four natural categories: giraffes, horses, cats, and dogs Presented stimuli with nine-parameter stick figures (Olman & Kersten, 2004)

95. Choice task

96. Samples from Subject 3 (projected onto a plane)

97. Mean animals by subject giraffe horse cat dog S1S2S3S4S5S6S7S8

98. Markov chain Monte Carlo with people Rational models can guide the design of psychological experiments Markov chain Monte Carlo (and other methods) can be used to sample from subjective probability distributions –category distributions –prior distributions