PDP Models of Morphology Psych 419/719 April 3, 2001

What is Morphology? Derives from MORPH, meaning “to change” Defines the rules of a language governing how words can be changed into new words.

Different Kinds of Morphology Inflectional –Plural (DOG->DOGS) –Past tense (BUG->BUGGED) –3rd person singular (LIKE->LIKES) Derivational: –GOVERN->GOVERNMENT –POMPOUS->POMPOSITY Compound Words: –Houseboat, boathouse Infixing

The General Character... Sound Meaning Cat Meows, purrs s + plural +

Exceptions are Legion.. Plural: MICE, MEN, GEESE, KNIVES Past tense: RAN, LIT Derivational: there’s GOVERN in GOVERNMENT, but no DEPART in DEPARTMENT Compounds: A bookcase is a container for books, a suitcase is a container for suits, but a staircase isn’t really a container for stairs, a pocketbook isn’t a book, a bookworm isn’t a worm, etc.

Despite This, We Can Generalize WUGS are either more than one WUG, or the act of the verb WUG (e.g., “Bob wugs his car”) GLORPED is doing GLORP in the past ESTRANGEMENTALITY is the processes of ESTRANGEMENT

The Standard Account Part of your knowledge of your language is knowing the rules that govern such transformations Rules operate over standard linguistic units –Stems, affixes, suffixes, etc. Knowledge of rules is independent of knowledge of mapping from sound to meaning The exceptions must be memorized by a separate system

What PDP Networks Are Like They can memorize exceptions, but attend to statistical regularities as well. They’re good at such tasks, but not as good at partitioning training into qualitatively different categories, like rules and exceptions

The PDP Story You learn to associate word forms to their meanings Regularities are easy to learn –So you pick up on them, –And languages evolve to use them. “Morphology” is simply attending to these regularities

Non-Morphological Regularities Onomatopoeia - where the sound of a word has something to do with its meaning –whistle, whir, whip, whiz –boom, bang, clang Words with similar sounds mean similar things –glitter glisten, gleam –sparkle, sputter, sprite Common Latin root –include, exclude, preclude

.. Leads to Different Interpretations On the standard account, these kinds of regularities are not “morphological” and as such are not handled by the morphological system On the PDP account, they’re just another regularity for the sound to meaning system to learn. Less predictable, but still there.

Other Effects in Morphology Claim: for some items, you regularize the inflection if the meaning is far from the original meaning –The batter flied out to left field –Maple Leafs vs. Timber Wolves Sometimes the reverse: –The suspect has been held up in the apartment for 6 hours (c.f. HOLED/HOLD)

U-Shaped Learning Initially, children seem to learn word forms whether they are regular or exception At some point, performance on exceptions drops.. Children regularize them (saying “eated” for “ate”) Ultimately, performance recovers

The Standard Account of U- Shaped learning Initially children are memorizing word forms. Then, they infer the rule for morphology This results in interference between memorized forms and rule-generated forms –Competition between two systems Eventually, rule is learned, exceptions are put in “exception box”

The PDP Account of U-Shaped Learning A single homogenous system is learning the task Interference between regular and irregular forms is not competition between two separate, atomic systems … but rather, results from the normal dynamics of learning in a single system

The Facts of U-Shaped Learning When you look closely at children’s performance, there isn’t a global “switch” from good to poor performance on all exceptions –As might be predicted by standard account Rather, there are micro-U trends by item over development And by the way, it doesn’t happen that often...

Micro-U Shaped Learning This falls out naturally from the PDP account. Not quite as straightforward for the standard account.

Review: Rumelhart & McClelland’s Past Tense Model Created a two layer network to map uninflected forms to inflections. –Initially, introduced high frequency (mostly exception) words in training. –Then, switched to whole training set One system learned exceptions and regulars Demonstrated U-Shaped learning at a global scale

Criticisms of R&M Past Tense Model The frequency shift was bogus The representation was poor The errors it made were implausible Didn’t account for semantic effects The U-Shaped learning it modeled isn’t what actually happens Wrong theory: people aren’t just inflecting base forms when making the past tense in normal language

Another Try: Plunkett & Marchman ‘91 Used more reasonable phonological representation Did not introduce explicit frequency shift Found that parameters for type/token frequency to induce best match to people was that of actual English … But, had very few items (500), and did not master vocabulary early in training

One More Time: Plunkett and Marchman ‘93 Increased training set size gradually, one item at a time. Two conditions: –Add new item when existing ones mastered. Result: Got Stuck –Add new item when certain amount of time passed. Result: Much better

Stem To Inflected Word: Summary Models were able to reproduce U-Shaped learning with a high degree of fidelity to what children do Provided an account of effect of vocabulary size on interference and generalization that is absent in standard account

A New Attack: Marslen-Wilson and Tyler Looked at priming effects in lexical decision –You get a prime such as BAKE, then have to make lexical decision on BAKER Crossed semantic overlap with phonological overlap

Marslen-Wilson’s Manipulation Semantic, Phonological, Morphological related –bake / baker Phonological but not semantic –corn / corner Semantic but not phonological –cook / baker Result: only morphological condition primed reliably Conclusion: Morphology is special

The Counter to Marslen-Wilson: Gonnerman ‘98 Marslen-Wilson observed weak (ns) priming in other conditions Maybe just phonological or semantic isn’t enough; need both M-W’s semantic primes weren’t very closely related Should get graded priming if items sufficiently related, and enough subjects

Gonnerman’s Results If semantic relatedness is high, get priming even with no morphological relationship –Jubilee - jubilant, fork - spoon Priming effects are in fact graded: semantic overlap gives some priming, semantic and phonological gives more No effect of morphology independent of semantic and phonological overlap!

The Gonnerman & Devlin Model Maps forms onto meaning Manipulated phonological and semantic overlap Results broadly replicated that of empirical study Word Form Meaning

Brain Damage Can Yield Double Dissociations Some patients exhibit an impairment in generating irregular past tenses –Say runned instead of ran While others have an impairment in generating novel past tenses –Can’t say glorped for the past tense of glorp Taken as evidence (Ullman and colleagues) for two systems in morphology

The Patients Those impaired on rules: –Patients with Parkinson’s disease, or left inferior cortex (including Broca’s area) Those impaired on exceptions: –Patients with damage to Wernike’s area, either from Alzheimer’s disease or lesion

The Two Accounts The “rules” are in the left inferior frontal area; Broca’s or the basal ganglia The “exceptions” are part of declarative memory, near Wernike’s area Phonological knowledge is needed more for rule-like items Semantic knowledge is needed more for exception processing Standard Account PDP Account

The Joanisse & Seidenberg Model Four tasks: –Speaking –Hearing –Repeating –Past tense formation Semantics Speech in Speech Out Modeled rule impairment with phonological damage And exception impairment with semantic damage

Results of Simulation Phonological damage impaired rule performance more than exceptions Semantic damage impaired exceptions more than rule performance

Another Angle: Why Do We Generalize At All? In English, the regulars are by far more common than exceptions. Easy to decide it’s the default. But: in German, there are several forms of the past tense. The “default” is low in frequency –But: Other forms are phonologically conditioned!

The Case of the Rats Eaters People can form compound nouns, by glueing two words together (rat eater, pig farmer, etc). It has been observed that people don’t like to use plurals as the head of a compound –rat eater (ok) –mice eater (ok) –rats eater (bad)

The Standard Account: The Level-Ordering Hypothesis Rules are applied at different levels of representation Each level doesn’t have access to input to previous levels –Irregular inflections stored in lexicon –Compounding applies to items in lexicon –Regular inflection happens after compounding

Problems With This Account People use regular compounds in plurals all the time –parks department, weapons inspector, pilots union, communications industry, compounds research If compounds are generated by rule, acceptability shouldn’t be frequency sensitive –But it is

The PDP-Inspired Alternative There are cues to what is acceptable Account hinges on what makes a good modifier –Children learn that modifiers generally aren’t semantically or phonologically plural –Consider adjectives: red balloons, not reds balloons –Irregulars are semantically but not phonologically plural, so not great, but better than pure plurals

Next Time: Reading and Dyslexia Optional reading on the class web page No class April 10