Hao Wang, Toben Mintz Department of Psychology University of Southern California

The Problem of Learning Syntactical Categories Grammar includes manipulations of lexical items based on their syntactical categories. Learning syntactical categories are fundamental to the acquisition of language.

The Problem of Learning Syntactical Categories Nativist approach Children are innately endowed with the possible syntactical categories. How to map a lexical item to its syntactical category or categories? Empirical approach Children have to figure out the syntactical categories in their target language, and assign categories to lexical items. There is no or little help from syntactical constraints.

Approaches Based on Semantic Categories Grammatical Categories correspond to Semantic/Conceptual Categories (Macnamara, 1972; Bowerman, 1973; Bates & MacWhinney, 1979; Pinker, 1984) object  noun action  verb But what about action, noise, love to think, to know (Maratsos & Chalkley, 1980)

Grammatical Categories from Distributional Analyses Structural Linguistics Grammatical categories defined by similarities of word patterning ( Bloomfield, 1933; Harris, 1951) Maratsos & Chalkley (1980): Distributional learning theory lexical co-occurrence patterns (and morphology and semantics) the cat is on the mat cat, mat

Grammatical Categories from Distributional Analyses Patterns across whole utterances (Cartwright & Brent, 1997) My cat meowed. Your dog slept. Det N X/Y. Bigram co-occurrence patterns (Mintz, Newport, & Bever, 1995, 2002; Redington, Chater & Finch, 1998) the cat is on the mat

Probabilistic Bigram Co-Occurrence Patterns X was - 20% there - 5% is - 30% wants - 15% seeing - 7% … the - 57% a - 25% blue - 0% silly - 5% your - 7% … Y was - 20% has - 30% is - 30% belongs - 2% around - 5% … the - 45% a - 35% blue - 5% silly - 0% your - 10% …

Frequent Frames (Mintz, 2003) Frames are defined as “two jointly occurring words with one word intervening”. “would you put the cans back ?” “you get the nuts.” “you take the chair back. “you read the story to Mommy.” Frame: you_X_the

Sensitivity to Frame-like Units Frames lead to categorization in adults (Mintz, 2002) Fifteen-month-olds are sensitive to frame-like sequences (Gómez & Maye, 2005)

Other Motivation for Frames Verb learning in children can be facilitated by frequent frames (Childers & Tomasello, 2001) Aspects of verb meaning carried by verb frame, linguistically defined (Gleitman, 1991; Gillette, Gleitman, Gleitman, Lederer, 1999; etc.)

Distributional Analyses Using Frequent Frames (Mintz, 2003) Six corpora from CHILDES (MacWhinney, 2000). Analyzed utterances to children under 2;6. Accuracy results averaged over all corpora.

Limitation of the Frequent Frame Analyses Requires two passes through the corpus Step 1, identify the frequent frames by tallying the frame frequency. Step 2, categorizing words using those frames. Tracks the frequency of all frames E.g., approximately frame types in one of the corpora in Mintz (2003).

Goal of current study Provides a psychological plausible model of word categorization Children possesses limited memory and cognitive capacity. Human memory is imperfect. Children may not be able to track all the frames he/she has encountered.

Features of current model It processes input and updates the categorization frames dynamically. Frame is associated with and ranked by a activation value. It has a limited memory buffer for frames. Only stores the most activated 150 frames. It implements a forgetting function on the memory. After processed a new frame, the activation of all frames in the memory decreased by

Child Input Corpora Six corpora from CHILDES (MacWhinney, 2000). Analyzed utterances to children under 2;6. Peter (Bloom, Hood, Lightbown, 1974; Bloom, Lightbown, Hood, 1975) Eve (Brown, 1973) Nina (Suppes, 1974) Naomi (Sachs, 1983) Anne (Theakston, Lieven, Pine, Rowland, 2001) Aran (Theakston et al., 2001) Mean Utterance/Child: ~17,200 MIN: 6,950 ; MAX: 20,857

Procedure The child-directed utterances from each corpus was processed individually Utterances were presented to the model in the order of appearance in the corpus Each utterance was segmented into frames “you read the story to Mommy” you read the read the story the story to story to Mommy

Procedure continued… you read the read the story the story to story to Mommy Memory ActivationFrame you_X_the read_X_story the_X_to story_X_Mommy

Procedure continued… The memory buffer only stores most activated 150 frames. It becomes full very quickly after processing several utterances. Memory ActivationFrame you_X_the read_X_story the_X_to story_X_Mommy to_X_it the_X_on ……

Procedure continued… “you put the” Frame: you_X_the Look up you_X_the frame in the memory Increase the activation of you_X_the frame by 1 Re-rank the memory by activation Memory ActivationFrame you_X_the read_X_story the_X_to story_X_Mommy to_X_it the_X_on ……

Procedure continued… “you have a” Frame: you_X_a Look up you_X_a frame in the memory story_X_Mommy < 1 Remove story_X_Mommy Add you_X_a to memory, set the activation to 1 Re-rank the memory by activation Memory ActivationFrame you_X_the read_X_story the_X_to to_X_it the_X_on story_X_Mommy ……

Procedure continued… A new frame not in memory The activation of all frames in memory are greater than 1 There is no change to the memory. Memory ActivationFrame you_X_the read_X_story the_X_to to_X_it the_X_on story_X_Mommy ……

Evaluating Model Performance Hit: two words from the same linguistic category grouped together False Alarm: two words from different linguistic categories grouped together Upper bound of 1

V ADV V Accuracy Example Hits: 10 False Alarms: 5 Accuracy:

Ten Categories for Accuracy Noun, pronoun Verb, Aux., Copula Adjective Preposition Adverb Determiner Wh-word Negation -- “not” Conjunction Interjection

Averaged accuracy across 6 corpora Accuracy Eve Peter Anne Aran Nina Naomi Average

The Development of Accuracy Accuracy are very high and stable in the entire process

Compare to Frequent Frames After processing about half of the corpus, 70% of frequent frames are in the most activated 45 frames in memory.

#w2 typew2 tokenActivationFrame what_X_you you_X_to you_X_it you_X_a you_X_the are_X_doing what_X_that you_X_me to_X_it would_X_like why_X_you Memory of Final Step of Eve Corpus

Stability of Frames in Memory Big changes of frames in memory in early stage, but become stable after processing 10% of the corpus

Summary After processed the entire corpus, the learning algorithm has identified almost all of the frequent frames by highest activation. Consequently, high accuracy of word categorization is achieved. After processing fewer than half of the utterances, the 45 most activated frames included approximately 70% of frequent frames.

Summary Frames are a robust cue for categorizing words. With limited and imperfect memory, the learning algorithm can identify most frequent frames after processing a relatively small number of utterances. Thus yield a high accuracy of word categorization.