1. Nonparametric Bayes and human cognition Tom Griffiths Department of Psychology Program in Cognitive Science University of California, Berkeley

2. data hypothesis Statistics about the mind

3. Analyzing psychological data Dirichlet process mixture models for capturing individual differences (Navarro, Griffiths, Steyvers, & Lee, 2006) Infinite latent feature models… –…for features influencing similarity (Navarro & Griffiths, 2007; 2008) –…for features influencing decisions ()

4. Statistics in the mind data hypothesis data hypothesis Statistics about the mind

5. Flexible mental representations Dirichlet

6. Categorization How do people represent categories? cat dog ?

7. cat Prototypes (Posner & Keele, 1968; Reed, 1972) Prototype cat

8. Exemplars (Medin & Schaffer, 1978; Nosofsky, 1986) Store every instance (exemplar) in memory cat

9. Something in between (Love et al., 2004; Vanpaemel et al., 2005) cat

10. A computational problem Categorization is a classic inductive problem –data: stimulus x –hypotheses: category c We can apply Bayes’ rule: and choose c such that P(c|x) is maximized

11. Density estimation We need to estimate some probability distributions –what is P(c)? –what is p(x|c)? Two approaches: –parametric –nonparametric These approaches correspond to prototype and exemplar models respectively (Ashby & Alfonso-Reese, 1995)

12. Parametric density estimation Assume that p(x|c) has a simple form, characterized by parameters  (indicating the prototype) Probability density x

13. Nonparametric density estimation Approximate a probability distribution as a sum of many “kernels” (one per data point) estimated function individual kernels true function n = 10 Probability density x

14. Something in between mixture distribution mixture components Probability x Use a “mixture” distribution, with more than one component per data point (Rosseel, 2002)

15. Anderson’s rational model (Anderson, 1990, 1991) Treat category labels like any other feature Define a joint distribution p(x,c) on features using a mixture model, breaking objects into clusters Allow the number of clusters to vary… a Dirichlet process mixture model (Neal, 1998; Sanborn et al., 2006)

16. A unifying rational model Density estimation is a unifying framework –a way of viewing models of categorization We can go beyond this to define a unifying model –one model, of which all others are special cases Learners can adopt different representations by adaptively selecting between these cases Basic tool: two interacting levels of clusters –results from the hierarchical Dirichlet process (Teh, Jordan, Beal, & Blei, 2004)

17. The hierarchical Dirichlet process

18. A unifying rational model cluster exemplar category exemplar prototype Anderson

19. HDP +,  and Smith & Minda (1998) HDP +,  will automatically infer a representation using exemplars, prototypes, or something in between (with  being learned from the data) Test on Smith & Minda (1998, Experiment 2) 111111 011111 101111 110111 111011 111110 000100 000000 100000 010000 001000 000010 000001 111101 Category A:Category B: exceptions

20. HDP +,  and Smith & Minda (1998) Probability of A prototype exemplarHDP Log-likelihood exemplar prototype HDP

21. The promise of HDP +,+ In HDP +,+, clusters are shared between categories –a property of hierarchical Bayesian models Learning one category has a direct effect on the prior on probability densities for the next category

22. Learning the features of objects Most models of human cognition assume objects are represented in terms of abstract features What are the features of this object? What determines what features we identify? (Austerweil & Griffiths, submitted)

25. Binary matrix factorization 

26.  How should we infer the number of features?

27. The nonparametric approach  Use the Indian buffet process as a prior on Z Assume that the total number of features is unbounded, but only a finite number will be expressed in any finite dataset (Griffiths & Ghahramani, 2006)

28. (Austerweil & Griffiths, submitted)

29. An experiment… (Austerweil & Griffiths, submitted) Training Testing Correlated Factorial Seen Unseen Shuffled

30. (Austerweil & Griffiths, submitted) Results

31. Conclusions Approaching cognitive problems as computational problems allows cognitive science and machine learning to be mutually informative Machine

32. Credits Categorization Adam Sanborn Kevin Canini Dan Navarro Learning features Joe Austerweil MCMC with people Adam Sanborn Computational Cognitive Science Lab http://cocosci.berkeley.edu/