1. Building and evaluating models of human-level intelligence
Josh Tenenbaum
MIT
Department of Brain and Cognitive Sciences
Computer Science and AI Lab (CSAIL)
Game plan: biology
If 5 minutes left at the end, do words and theory acquisition.
If no time left, just do theory acquisition.

2. The big problem of intelligence
How do we go beyond the information given – draw inferences, make generalizations, and build models of the world from the sparse, noisy data of our experience?
Ken: Human-scale learning includes:
- Learning in small number of trials
- Learning with rich, relational representations
Nick: You can’t know whether you have a good
computational model of how people solve some task
unless you have a computational model that actually
solves that task (i.e., AI and cog. sci. are not separable).

3. Word learning on planet Gazoob
“tufa”

4. The big problem of intelligence
The development of intuitive theories in childhood.
Psychology: How do we learn to understand others’ actions in terms of beliefs, desires, plans, intentions, values, morals?
Biology: How do we learn that people, dogs, bees, worms, trees, flowers, grass, coral, moss are alive, but chairs, cars, tricycles, computers, the sun, Roomba, robots, clocks, rocks are not?

5. The big problem of intelligence
Common sense reasoning.
Consider a man named Boris.
Is the mother of Boris’s father his grandmother?
Is the mother of Boris’s sister his mother?
Is the son of Boris’s sister his son?
(Note: Boris and his family were stranded on a desert island when he was a young boy.)

6. Approaches to cognitive modeling
Micro approach:
Build a processing model of a specific laboratory task.
Pro: Can test precise behavioral predictions. Can establish clearly how the model works and why.
Con: May not learn much of general value about the hardest, real-world problems of human cognition.
model 1
model 2 …
model n
task 1
task 2 …
task n

7. Approaches to cognitive modeling
Macro approach:
Build an architecture for modeling human cognition.
Pro: Can build end-to-end human simulations. The same model applies to many tasks.
Con: Hard to test. Many degrees of freedom, hard to know what is doing the work. Hard to export insights outside of the immediate modeling community.
Cognitive architecture
task 1
task 2
task n

8. Approaches to cognitive modeling
Principle-based approach:
Propose a small number of general-purpose principles for representation, learning, and inference; then use those principles to build models of many specific tasks. Examples: Connectionist, Bayesian, ….
Pro: Unifying explanations of human cognition, supported by rigorously testable models. Principles supported by success of specific models they instantiate. Principles export well.
Con: Principles not directly testable. No end-to-end human simulation.
Principles
model 1
model 2 …
model n
task 1
task 2 …
task n

9. The “Bayesian” approach
Probabilistic inference in generative models
Hierarchical probabilistic models, with inference at all levels of abstraction
Probabilities defined over structured representations: graphs, grammars, predicate logic, schemas
Flexible representations, growing in complexity or changing form in response to the observed data.
Approximate methods of learning and inference, such as Markov chain Monte Carlo (MCMC), to scale up to large problems.

10. Inductive reasoning Which argument is stronger?
Gorillas have T9 hormones.
Seals have T9 hormones.
Anteaters have T9 hormones.
“Similarity” “Typicality”
“Diversity”
Gorillas have T9 hormones.
Seals have T9 hormones.
Horses have T9 hormones.
Gorillas have T9 hormones.
Chimps have T9 hormones.
Monkeys have T9 hormones.
Baboons have T9 hormones.
Horses have T9 hormones.

11. Beyond similarity-based induction
Reasoning based on dimensional thresholds: (Smith et al., 1993)
Reasoning based on causal relations: (Medin et al., 2004; Coley & Shafto, 2003)
Poodles can bite through wire.
German shepherds can bite through wire.
Dobermans can bite through wire.
German shepherds can bite through wire.
Salmon carry E. Spirus bacteria.
Grizzly bears carry E. Spirus bacteria.
Grizzly bears carry E. Spirus bacteria.
Salmon carry E. Spirus bacteria.

12. Theory-based Bayesian model (structural form + process)
Property type
“has T9 hormones” “can bite through wire” “carry E. Spirus bacteria”
Theory-based Bayesian model (structural form + process)
taxonomic tree directed chain directed network
+ diffusion process drift process noisy transmission
Class D
Class C
Class G
Class F
Class E
Class D
Class B
Class A
Class D
Class A
Class A
Class F
Class E
Class C
Class C
Class B
Class G
Class E
Class B
Class F
Hypotheses
Class G
Class A
Class B
Class C
Class D
Class E
Class F
Class G
. . .
. . .
. . .

13. “can bite through wire”
“has T9
hormones”
“can bite through wire”
“carry
E. Spirus bacteria”
“is found near
Minneapolis”
(Kemp & Tenenbaum)

14. Integrating multiple forms of reasoning
2) Causal relations
between features
… Parameters of causal
relations vary smoothly
over the category hierarchy.
1) Taxonomic
relations between categories
T9 hormones cause elevated heart rates.
Elevated heart rates cause faster metabolisms.
Mice have T9 hormones. …?
(Kemp, Shafto et al.)

15. Integrating multiple forms of reasoning

16. Learning domain structures
P(structure | form)
P(data | structure)
P(form)
F: form
Tree with species at leaf nodes
mouse
squirrel
chimp
gorilla
S: structure
hormones
Has T9 F1 F2 F3 F4
mouse
squirrel
chimp
gorilla ?
D: data …
(Kemp & Tenenbaum)

17. Structural forms as graph grammars
Process
Form
Process

18. Learning the abstract principles organizing a domain

19. Learning multiple structures within a domain
(Shafto, Kemp, et al.)

20. Learning relational theories
concept
predicate
concept
Biomedical predicate data from UMLS (McCrae et al.):
134 concepts: enzyme, hormone, organ, disease, cell function ...
49 predicates: affects(hormone, organ), complicates(enzyme, cell function), treats(drug, disease), diagnoses(procedure, disease) …
(Kemp, Tenenbaum, Griffiths et al.)

21. Learning relational theories
e.g., Diseases affect
Organisms
Chemicals interact
with Chemicals
Chemicals cause
Diseases

22. Learning abstract relational structures
Dominance
hierarchy Tree Cliques Ring
Primate troop Bush administration Prison inmates Kula islands
“x beats y” “x told y” “x likes y” “x trades with y”

23. Causal learning and reasoning
Principles
Structure
Data
(Griffiths, Tenenbaum, et al.)

24. The “blessing of abstraction”
Task: learning a causal
graphical model G 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
# of samples:
Graph G
edge
(G)
Data D
edge
(G)
Abstract theory Z
Graph G
class
(z)
Data D

25. “Universal Grammar” Grammar Phrase structure Utterance Speech signal
Hierarchical phrase structure grammars (e.g., CFG, HPSG, TAG)
P(phrase structure | grammar)
P(utterance | phrase structure)
P(speech | utterance)
(c.f. Chater and Manning, 2006)
P(grammar | UG)
Grammar
Phrase structure
Utterance
Speech signal

26. Vision as probabilistic parsing
(Han & Zhu, 2006;
Yuille & Kersten, 2006 )

27. Goal-directed action (production and comprehension)
(Wolpert et al., 2003)

28. Payoffs
model 1
model 2
model n
task 1
task 2
task n …
Principles
Ultimately, a framework for studying human knowledge: its nature, use, and acquisition, at multiple levels of abstraction, across domains and tasks.
Export principles and insights to cognitive scientists outside the modeling community.
Studying how principles interact takes us beyond traditional cog. sci. dichotomies (“Twenty questions”)
A bridge to state-of-the-art AI.
Future work: a bridge to neuroscience.