1. Markov chain Monte Carlo with people
Tom Griffiths
Department of Psychology
Cognitive Science Program
University of California, Berkeley

2. Two deep questions What are the biases that guide human learning?
prior probability distribution P(h)
What do mental representations look like?
category distribution P(x|c)

3. Develop ways to sample from these distributions
Two deep questions
What are the biases that guide human learning?
prior probability distribution on hypotheses, P(h)
What do mental representations look like?
distribution over objects x in category c, P(x|c)
Develop ways to sample from these distributions

4. Outline Markov chain Monte Carlo Sampling from the prior
(with Mike Kalish, Steve Lewandowsky, Saiwing Yeung)
Sampling from category distributions
(with Adam Sanborn)

5. Outline Markov chain Monte Carlo Sampling from the prior
(with Mike Kalish, Steve Lewandowsky, Saiwing Yeung)
Sampling from category distributions
(with Adam Sanborn)

6. Markov chains Variables x(t+1) independent of history given x(t)
Transition matrix
T = P(x(t+1)|x(t))
Variables x(t+1) independent of history given x(t)
Converges to a stationary distribution under easily checked conditions (i.e., if it is ergodic)

7. Markov chain Monte Carlo
Sample from a target distribution P(x) by constructing Markov chain for which P(x) is the stationary distribution
Two main schemes:
Gibbs sampling
Metropolis-Hastings algorithm

8. x-i = x1(t+1), x2(t+1),…, xi-1(t+1), xi+1(t), …, xn(t)
Gibbs sampling
For variables x = x1, x2, …, xn and target P(x)
Draw xi(t+1) from P(xi|x-i)
x-i = x1(t+1), x2(t+1),…, xi-1(t+1), xi+1(t), …, xn(t)

9. Gibbs sampling
(MacKay, 2002)

10. Metropolis-Hastings algorithm (Metropolis et al
Metropolis-Hastings algorithm (Metropolis et al., 1953; Hastings, 1970)
Step 1: propose a state (we assume symmetrically)
Q(x(t+1)|x(t)) = Q(x(t))|x(t+1))
Step 2: decide whether to accept, with probability
Metropolis acceptance function
Barker acceptance function

11. Metropolis-Hastings algorithm
p(x)

12. Metropolis-Hastings algorithm
p(x)

13. Metropolis-Hastings algorithm
p(x)

14. Metropolis-Hastings algorithm
p(x)
A(x(t), x(t+1)) = 0.5

15. Metropolis-Hastings algorithm
p(x)

16. Metropolis-Hastings algorithm
p(x)
A(x(t), x(t+1)) = 1

17. Outline Markov chain Monte Carlo Sampling from the prior
(with Mike Kalish, Steve Lewandowsky, Saiwing Yeung)
Sampling from category distributions
(with Adam Sanborn)

18. Iterated learning (Kirby, 2001)
What are the consequences of learners learning from other learners?

19. Analyzing iterated learning
PL(h|d)
PL(h|d)
PP(d|h)
PP(d|h)
PL(h|d): probability of inferring hypothesis h from data d
PP(d|h): probability of generating data d from hypothesis h

20. Iterated Bayesian learning
PL(h|d)
PL(h|d)
PP(d|h)
PP(d|h)
Assume learners sample from their posterior distribution:

21. Analyzing iterated learningh1 d1 h2
PL(h|d)
PP(d|h) d2 h3
d PP(d|h)PL(h|d) h1 h2 h3
A Markov chain on hypotheses d0 d1
h PL(h|d) PP(d|h) d2
A Markov chain on data

22. Stationary distributions
Markov chain on h converges to the prior, P(h)
Markov chain on d converges to the “prior predictive distribution”
(Griffiths & Kalish, 2005)

23. Explaining convergence to the prior
PL(h|d)
PL(h|d)
PP(d|h)
PP(d|h)
Intuitively: data acts once, prior many times
Formally: iterated learning with Bayesian agents is a Gibbs sampler on P(d,h)
(Griffiths & Kalish, 2007)

24. Serial reproduction (Bartlett, 1932)
Participants see stimuli, then reproduce them from memory
Reproductions of one participant are stimuli for the next
Stimuli were interesting, rather than controlled
e.g., “War of the Ghosts”

25. Iterated function learning
Each learner sees a set of (x,y) pairs
Makes predictions of y for new x values
Predictions are data for the next learner
data
hypotheses
(Kalish, Griffiths, & Lewandowsky, 2007)

26. Function learning experiments
Stimulus
Response
Slider
Feedback
Examine iterated learning with different initial data

27. Initial
data
Iteration

28. Iterated predicting the future
Each learner sees values of t
Makes predictions of ttotal
The next value of t is chosen from (0, ttotal)
data
hypotheses
A movie has made $30 million so far
$60 million total
(Lewandowsky, Griffiths & Kalish, 2009)

29. Chains of predictions Movie grosses Poems ttotal ttotal Iteration
(Lewandowsky, Griffiths & Kalish, 2009)

30. Stationary distributions
(Lewandowsky, Griffiths & Kalish, 2009)

31. Iterated causal learning
Each trial shows contingency data
Elicits estimates of w0 and w1
Data on next trial sampled with these values
data
hypotheses
C present
(c+)
C absent
(c-) w0 w1
E present (e+)
E absent (e-)
(Yeung & Griffiths, in press)

32. Eliciting strength judgments
Generative:
“Of 100 people who did not express the effect, how many
would express the effect in presence of the cause?”
Preventive:
“Of 100 people who expressed the effect, how many
would express the effect in presence of the cause?”
Model using
(Lu, Yuille, Liljeholm, Cheng, & Holyoak, 2008)

33. Priors on causal strength
(Lu, Yuille, Liljeholm, Cheng, & Holyoak, 2008)
“Sparse and Strong”

34. Comparing models to data
(Lu, Yuille, Liljeholm, Cheng, & Holyoak, 2008)

36. Experiment design (for each subject)
four iterated learning chains
one independent learning “chain”
(Yeung & Griffiths, in press)

37. Sparse and Strong prior
Empirical prior
Sparse and Strong prior
(Yeung & Griffiths, in press)

38. Comparing models to data
(Yeung & Griffiths, in press)

39. Empirical fit vs. SS prior fit
(Yeung & Griffiths, in press)

40. Other empirical results…
Concept learning
(Griffiths, Christian, & Kalish, 2008)
Reproduction from memory
(Xu & Griffiths, 2008)
Estimating linguistic frequencies
(Reali & Griffiths, 2009)
Systems of color terms
(Xu, Dowman, & Griffiths, 2010)
“DUP”

41. Outline Markov chain Monte Carlo Sampling from the prior
(with Mike Kalish, Steve Lewandowsky, Saiwing Yeung)
Sampling from category distributions
(with Adam Sanborn)

42. Sampling from categories
Frog distribution P(x|c)

43. Ask subjects which of two alternatives comes from a target category
A task
Ask subjects which of two alternatives comes from a target category
Which animal is a frog?

44. A Bayesian analysis of the task
Assume:

45. Response probabilities
If people probability match to the posterior, response probability is equivalent to the Barker acceptance function for target distribution p(x|c)

46. Collecting the samples
Which is the frog?
Which is the frog?
Which is the frog?
Trial 1
Trial 2
Trial 3

47. Verifying the method

48. Training
Subjects were shown schematic fish of different sizes and trained on whether they came from the ocean (uniform) or a fish farm (Gaussian)

49. Between-subject conditions

50. Choice task
Subjects judged which of the two fish came from the fish farm (Gaussian) distribution

51. Examples of subject MCMC chains

52. Estimates from all subjects
Estimated means and standard deviations are significantly different across groups
Estimated means are accurate, but standard deviation estimates are high
result could be due to perceptual noise or response gain

53. Sampling from natural categories
Examined distributions for four natural categories: giraffes, horses, cats, and dogs
Presented stimuli with nine-parameter stick figures (Olman & Kersten, 2004)

54. Choice task

55. Samples from Subject 3 (projected onto plane from LDA)

56. Mean animals by subject
giraffe
horse
cat
dog

57. Marginal densities (aggregated across subjects)
Giraffes are distinguished by neck length, body height and body tilt
Horses are like giraffes, but with shorter bodies and nearly uniform necks
Cats have longer tails than dogs

58. Relative volume of categories
Convex Hull
Minimum Enclosing Hypercube
Convex hull content divided by enclosing hypercube content
Giraffe
Horse
Cat
Dog

59. Discrimination method (Olman & Kersten, 2004)

60. Parameter space for discrimination
Restricted so that most random draws were animal-like

61. MCMC and discrimination means

62. Conclusion
Markov chain Monte Carlo provides a way to sample from subjective probability distributions
Many interesting questions can be framed in terms of subjective probability distributions
inductive biases (priors)
mental representations (category distributions)
Other MCMC methods may provide further empirical methods…
Gibbs for categories, adaptive MCMC, …