Development and Disintegration of Conceptual Knowledge: A Parallel-Distributed Processing Approach James L. McClelland Department of Psychology and Center for Mind, Brain, and Computation Stanford University

Representation is a pattern of activation distributed over neurons within and across brain areas. Bidirectional propagation of activation underlies the ability to bring these representations to mind from given inputs. The knowledge underlying propagation of activation is in the connections. language Parallel Distributed Processing Approach to Semantic Cognition

A Principle of Learning and Representation Learning and representation are sensitive to coherent covariation of properties across experiences.

What is Coherent Covariation? The tendency of properties of objects to co- occur in clusters. e.g. –Has wings –Can fly –Is light Or –Has roots –Has rigid cell walls –Can grow tall

Some Phenomena in Development Progressive differentiation of concepts Overgeneralization Illusory correlations

The Rumelhart Model

The Training Data: All propositions true of items at the bottom level of the tree, e.g.: Robin can {grow, move, fly}

Target output for ‘robin can’ input

ajaj aiai w ij net i =  a j w ij w ki Forward Propagation of Activation

 k ~ (t k -a k ) w ij  i ~   k w ki w ki ajaj Back Propagation of Error () Error-correcting learning: At the output layer:w ki =  k a i At the prior layer: w ij =  j a j … aiai

ExperienceExperience Early Later Later Still

Waves of differentiation reflect coherent covariation of properties across items. Patterns of coherent covariation are reflected in the principal components of the property covariance matrix. Figure shows attribute loadings on the first three principal components: –1. Plants vs. animals –2. Birds vs. fish –3. Trees vs. flowers Same color = features covary in component Diff color = anti-covarying features What Drives Progressive Differentiation?

Overgeneralization of Frequent Names to Similar Objects “dog” “goat” “tree”

Illusory Correlations Rochel Gelman found that children think that all animals have feet. –Even animals that look like small furry balls and don’t seem to have any feet at all. A tendency to over-generalize properties typical of a superordinate category at an intermediate point in development is characteristic of the PDP network.

A typical property that a particular object lacks e.g., pine has leaves An infrequent, atypical property

Sensitivity to Coherence Requires Convergence A A A

Main Points of the Talk Sensitivity to coherent covariation in an appropriately structured Parallel Distributed Processing system underlies the development of conceptual knowledge. Gradual degradation of the representations constructed through this developmental process underlies the pattern of semantic disintegration seen in semantic dementia.

Disintegration of Conceptual Knowledge in Semantic Dementia Progressive loss of specific knowledge of concepts, including their names, with preservation of general information Overgeneralization of frequent names Illusory correlations

Picture naming and drawing in Sem. Demantia

Grounding the Model in What we Know About The Organization of Semantic Knowledge in The Brain There is now evidence for specialized areas subserving many different kinds of semantic information. Semantic dementia results from progressive bilateral disintegration of the anterior temporal cortex. Rapid acquisition of new knowledge depends on medial temporal lobes, leaving long-term semantic knowledge intact. language

Proposed Architecture for the Organization of Semantic Memory color form motion action valance Temporal pole name Medial Temporal Lobe

Rogers et al (2005) model of semantic dementia Gradually learns through exposure to input patterns derived from norming studies. Representations in the temporal pole are acquired through the course of learning. After learning, the network can activate each other type of information from name or visual input. Representations undergo progressive differentiation as learning progresses. Damage to units within the temporal pole leads to the pattern of deficits seen in semantic dementia. nameassocfunction temporal pole vision

Severity of DementiaFraction of Neurons Destroyed omissionswithin categ. superord. Patient Data Simulation Results Errors in Naming for As a Function of Severity

Simulation of Delayed Copying Visual input is presented, then removed. After several time steps, pattern is compared to the pattern that was presented initially. Omissions and intrusions are scored for typicality nameassocfunction temporal pole vision

Omissions by feature typeIntrusions by feature type IF’s ‘camel’ DC’s ‘swan’ Simulation results

Main Points of the Talk Sensitivity to coherent covariation in an appropriately structured Parallel Distributed Processing system underlies the development of conceptual knowledge. Gradual degradation of the representations constructed through this developmental process underlies the pattern of semantic disintegration seen in semantic dementia.

Relationship between Semantic and Lexical Knowledge Patients with semantic dementia are typically ‘surface dyslexic’ –They make regularization errors reading low-frequency exception words like PINT They also make regularization errors in past- tense inflection, again especially with low- frequency exceptions like SWIM Tendency to regularize in both cases correlates with severity of semantic deficit.

Case Series Study by Patterson et al Tested 14 SD patients Assigned ‘Semantic Score’ based three tests. Reading HF&LF Reg. and Exc. Words Spelling HF&LF Reg. and Exc. Words Past Tense Inflection, HF&LF R&E Words Lexical Decision: fruit/frute, flute/fluit Object Decision (at right) Delayed Copying Test

Reg. Exc. Reg. Exc.

Words and Objects: Similar characteristics, similar degredation, similar mechanism Words and objects both have typical, idiosyncratic, and atypical properties. Connectionist networks that learn about such items pick up on the typical or regular properties and exhibit sensitivity to them. –Typical items and typical properties are more robustly represented than atypical items and properties. –Damage interferes with knowledge of idiosyncratic properties while sparing knowledge of more general properties. –Damage enhances a bias toward typicality, seen in both object and lexical decision with LF atypical items, and in other tasks.

Integrated Model of Semantic and Lexical Processing (Dilkina & McClelland, in progress) spelling sound function temporal pole vision Temporal pole mediates lexical as well as semantic knowledge. Lesions to the temporal pole produce corresponding deficits in semantic and lexical tasks. Lesions affecting one set of connections more than others can account for partial dissociations. Individual differences in network parameters and experience with particular tasks may also contribute to differences among patients. Ongoing work is addressing how well we can account for data from a wide range of tasks and patients within this framework.