1. New Bulgarian University 9th International Summer School in Cognitive Science Simplicity as a Fundamental Cognitive Principle Nick Chater Institute for Applied Cognitive Science Department of Psychology University of Warwick Funding support: EU Training Grant; EU Training Grant HPRN-CT-1999-00065, Human Frontier Science Program, ESRC, Leverhulme Trust, Oliver, Wyman & Company

2. Plan of Lectures 1. Simplicity and the Mind 2. A Simplicity Principle For Perception 3. Simplicity in Language Acquisition: Analysis of an ‘ideal’ Learner 4. Mental mechanisms: Speculations on human causal learning and reasoning 5. Science, Simplicity and Embodied Cognition

3. Lecture 1 Simplicity and the Mind

4. Overview The Appeal of Simplicity Quantifying Simplicity Why Simple Explanations are the Most Probable Cognition as a Search for Simplicity Conclusions

5. 1. The Appeal of Simplicity Common Senseconvoluted explanations Sciences1, 2, 4, 8... Aesthetics“economy” What explains our bias towards simplicity? Claim 1: Simple explanations are the most reliable Claim 2: Cognition is the search for simplicity

6. 2. Quantifying Simplicity Proposal: Measure simplicity as length of shortest unique description. “Kolmogorov complexity theory” (a deep – simple -! mathematical theory of simplicity) …a a a a a a a a a a a a a a … simple …a b a b a b a b a b a b a b …simple …a b a a a b a b a b b b a b …complex (random) 3 vs. 1,000,000,000,000,000 vs. 6,553,694,284,875,223

8. A Paradox? “the smallest number that cannot be uniquely specified in less than 20 words of English” (16 words) Call it N But now N has a 16 word description: Contradiction!?! (the Richards-Berry Paradox)

9. Escaping the paradox… Only allow unique descriptions that allow the object to be calculated i.e., “computer” programs This rules out: “the smallest number that cannot be uniquely specified in less than 20 words of English” (16 words) And the paradox is avoided (But a similar “paradox” proves Godel’s theorem)

10. In psychology, simplicity will be determined by length of “mental” code – i.e., code in “language of thought” But, fortunately, between any two languages, description length for all objects will be the same up to a constant: i.e., Language Invariance So we can define absolute simplicity of an object x, K(x), as the length of the shortest program that calculates x K(x) is the “Kolmogorov complexity of x” Simplicity depends on the description language (i.e., depends on the nature of mental representation)

11. Objection! How can this be relevant to people? 1. But surely simplicity is in the eye of the beholder? True: remember that simplicity does depend on the description language; and that may differ between individuals 2. But there will be simple patterns that we don’t notice (e.g., ) True: cognition may prefer the simplest pattern; but only amongst those it can find 3. But all this talk of “programs” sounds more relevant to computers than people All that is required is that people have ways of representing the world; and specifying things that they have in mind; call a specification for a thing a “program”

12. 3. Why simple explanations are the most reliable Consider science. The aim is to figure out the most probable hypothesis, H, given the data, D: P(H\D) By standard probability theory (Bayes’ theorem) - P(H\D) P(D\H)P(H)

13. Now the H that maximises: P(H\D) P(D\H)P(H) … is the H that minimizes -log 2 P(D\H) -log 2 P(H) and it turns out that (almost always) -log 2 P(H) K(H) -log 2 P(D\H) K(D\H)

14. So the H that is the most probable minimizes: K(H) + K (D\H) = description length of H + description length of D, given H i.e., H which gives shortest description of D Conclusion: the simplest explanations are the most probable

15. Simple patterns, explanations, ideas, theories are the most likely to be true and They give the most reliable predictions Solomonoff’s prediction theorem (See Lecture 3) So any intelligent system (including people!) should search for simplicity…

16. 4.Cognition as a Search for Simplicity Case 1: Simplicity as a principle in perception

18. Case Study 2: Similarity Similarity = simplicity of transformation (Hahn, Chater & Richardson, 2002) Decreasing similarity; increasing number of transformations

19. Case study 3: Simplicity in learning from experience

20. 5. Conclusions People have strong preferences for simplicity Simplicity provides the most reliable explanations of the world The mind may be searching for simplicity in many areas of thought: perception, similarity, learning from experience The ability to find simplicity may be a core aspect of human intelligence