1. Morphology: Words and their Parts
CS 4705
Slides adapted from Jurafsky, Martin Hirschberg and Dorr.

2. English Morphology
Morphology is the study of the ways that words are built up from smaller meaningful units called morphemes
We can usefully divide morphemes into two classes
Stems: The core meaning bearing units
Affixes: Bits and pieces that adhere to stems to change their meanings and grammatical functions

3. Nouns and Verbs (English)
Nouns are simple (not really)
Markers for plural and possessive
Verbs are only slightly more complex
Markers appropriate to the tense of the verb

4. Regulars and Irregulars
Ok so it gets a little complicated by the fact that some words misbehave (refuse to follow the rules)
Mouse/mice, goose/geese, ox/oxen
Go/went, fly/flew
The terms regular and irregular will be used to refer to words that follow the rules and those that don’t.

5. Regular and Irregular Nouns and Verbs
Regulars…
Walk, walks, walking, walked, walked
Table, tables
Irregulars
Eat, eats, eating, ate, eaten
Catch, catches, catching, caught, caught
Cut, cuts, cutting, cut, cut
Goose, geese

6. Why care about morphology?
Spelling correction: referece
Morphology in machine translation
Spanish words quiero and quieres are both related to querer ‘want’
Hyphenation algorithms: refer-ence
Part-of-speech analysis: google, googler
Text-to-speech: grapheme-to-phoneme conversion
hothouse (/T/ or /D/)
Allows us to guess at meaning
‘Twas brillig and the slithy toves…
Muggles moogled migwiches
Gesagt: past tense of said in German (pres: sagen)
Arabic: v: to worship, to build, to enslave
n: religious service, worship, bondage, slavery, servant, slave, worshipper,

7. Concatenative Morphology
Morpheme+Morpheme+Morpheme+…
Stems: often called lemma, base form, root, lexeme
hope+ing hoping hop hopping
Affixes
Prefixes: Antidisestablishmentarianism
Suffixes: Antidisestablishmentarianism
Infixes: hingi (borrow) – humingi (borrower) in Tagalog
Circumfixes: sagen (say) – gesagt (said) in German

8. What useful information does morphology give us?
Different things in different languages
Spanish: hablo, hablaré/ English: I speak, I will speak
English: book, books/ Japanese: hon, hon
Languages differ in how they encode morphological information
Isolating languages (e.g. Cantonese) have no affixes: each word usually has 1 morpheme
Agglutinative languages (e.g. Finnish, Turkish) are composed of prefixes and suffixes added to a stem (like beads on a string) – each feature realized by a single affix, e.g. Finnish

9. epäjärjestelmällistyttämättömyydellänsäkäänköhän
‘Wonder if he can also ... with his capability of not causing things to be unsystematic’
Inflectional languages (e.g. English) merge different features into a single affix (e.g. ‘s’ in likes indicates both person and tense); and the same feature can be realized by different affixes
Polysynthetic languages (e.g. Inuit languages) express much of their syntax in their morphology, incorporating a verb’s arguments into the verb, e.g. Western Greenlandic
Aliikusersuillammassuaanerartassagaluarpaalli. aliiku-sersu-i-llammas-sua-a-nerar-ta-ssa-galuar-paal-li entertainment-provide-SEMITRANS-one.good.at-COP-say.that-REP-FUT-sure.but-3.PL.SUBJ/3SG.OBJ-but 'However, they will say that he is a great entertainer, but ...'
So….different languages may require very different morphological analyzers

10. What we want
Something to automatically do the following kinds of mappings:
Cats cat +N +PL
Cat cat +N +SG
Cities city +N +PL
Merging merge +V +Present-participle
Caught catch +V +past-participle

11. Morphology Can Help Define Word Classes
AKA morphological classes, parts-of-speech
Closed vs. open (function vs. content) class words
Pronoun, preposition, conjunction, determiner,…
Noun, verb, adverb, adjective,…
Identifying word classes is useful for almost any task in NLP, from translation to speech recognition to topic detection…very basic semantics

12. (English) Inflectional Morphology
Word stem + grammatical morpheme  different forms of same word
Usually produces word of same class
Usually serves a syntactic or grammatical function (e.g. agreement)
like  likes or liked
bird  birds
Nominal morphology
Plural forms
s or es
Irregular forms (goose/geese)

13. Mass vs. count nouns (fish/fish(es), email or emails?)
Possessives (cat’s, cats’)
Verbal inflection
Main verbs (sleep, like, fear) relatively regular
-s, ing, ed
And productive: ed, instant-messaged, faxed, homered
But some are not:
eat/ate/eaten, catch/caught/caught
Primary (be, have, do) and modal verbs (can, will, must) often irregular and not productive
Be: am/is/are/were/was/been/being
Irregular verbs few (~250) but frequently occurring

14. Derivational Morphology
Word stem + syntactic/grammatical morpheme  new words
Usually produces word of different class
Incomplete process: derivational morphs cannot be applied to just any member of a class
Verbs --> nouns
-ize verbs  -ation nouns
generalize, realize  generalization, realization
synthesize but not synthesization

15. Verbs, nouns  adjectives
embrace, pity embraceable, pitiable
care, wit  careless, witless
Adjective  adverb
happy  happily
Process selective in unpredictable ways
Less productive: nerveless/*evidence-less, malleable/*sleep-able, rar-ity/*rareness
Meanings of derived terms harder to predict by rule
clueless, careless, nerveless, sleepless

16. Compounding Two base forms join to form a new word
Bedtime, Weinerschnitzel, Rotwein
Careful? Compound or derivation?

17. Morphotactics
What are the ‘rules’ for constructing a word in a given language?
Pseudo-intellectual vs. *intellectual-pseudo
Rational-ize vs *ize-rational
Cretin-ous vs. *cretin-ly vs. *cretin-acious

18. Semantics: In English, un- cannot attach to adjectives that already have a negative connotation:
Unhappy vs. *unsad
Unhealthy vs. *unsick
Unclean vs. *undirty
Phonology: In English, -er cannot attach to words of more than two syllables
great, greater
Happy, happier
Competent, *competenter
Elegant, *eleganter
Unruly, ?unrulier

19. Morphological Parsing
These regularities enable us to create software to parse words into their component parts

20. Morphology and FSAs
We’d like to use the machinery provided by FSAs to capture facts about morphology
Ie. Accept strings that are in the language
And reject strings that are not
And do it in a way that doesn’t require us to in effect list all the words in the language

21. What do we need to build a morphological parser?
Lexicon: list of stems and affixes (w/ corresponding p.o.s.)
Morphotactics of the language: model of how and which morphemes can be affixed to a stem
Orthographic rules: spelling modifications that may occur when affixation occurs
in  il in context of l (in- + legal)
Most morphological phenomena can be described with regular expressions – so finite state techniques often used to represent morphological processes

22. Start Simple Regular singular nouns are ok
Regular plural nouns have an -s on the end
Irregulars are ok as is

23. Simple Rules

24. Now Add in the Words

25. Derivational morphology: adjective fragment
adj-root1
-er, -ly, -est
un- q0 q1 q2
adj-root1 q5 q3 q4 
-er, -est
adj-root2
What will happen if we use only the FSA defined by the purple nodes?
Will allow unbig, unred,…
Solution: define classes of adjective stems
NFSA: easier and more intuitive to define
Adj-root1: clear, happi, real (clearly)
Adj-root2: big, red (*bigly)

26. Parsing/Generation vs. Recognition
We can now run strings through these machines to recognize strings in the language
Accept words that are ok
Reject words that are not
But recognition is usually not quite what we need
Often if we find some string in the language we might like to find the structure in it (parsing)
Or we have some structure and we want to produce a surface form (production/generation)
Example
From “cats” to “cat +N +PL”

27. Finite State Transducers
The simple story
Add another tape
Add extra symbols to the transitions
On one tape we read “cats”, on the other we write “cat +N +PL”

28. Applications
The kind of parsing we’re talking about is normally called morphological analysis
It can either be
An important stand-alone component of an application (spelling correction, information retrieval)
Or simply a link in a chain of processing

29. FSTs Kimmo Koskenniemi’s two-level morphology
Idea: word is a relationship between lexical level (its morphemes) and surface level (its orthography)

30. Transitions c:c a:a t:t +N:ε
+PL:s
c:c means read a c on one tape and write a c on the other
+N:ε means read a +N symbol on one tape and write nothing on the other
+PL:s means read +PL and write an s

31. Typical Uses
Typically, we’ll read from one tape using the first symbol on the machine transitions (just as in a simple FSA).
And we’ll write to the second tape using the other symbols on the transitions.
In general, FSTs can be used for
Translators (Hello:Ciao)
Parser/generators (Hello:How may I help you?)
As well as Kimmo-style morphological parsing

32. Ambiguity
Recall that in non-deterministic recognition multiple paths through a machine may lead to an accept state.
Didn’t matter which path was actually traversed
In FSTs the path to an accept state does matter since differ paths represent different parses and different outputs will result

33. Ambiguity What’s the right parse (segmentation) for
Unionizable
Union-ize-able
Un-ion-ize-able
Each represents a valid path through the derivational morphology machine.

34. Ambiguity There are a number of ways to deal with this problem
Simply take the first output found
Find all the possible outputs (all paths) and return them all (without choosing)
Bias the search so that only one or a few likely paths are explored

35. The Gory Details Of course, its not as easy as
“cat +N +PL” <-> “cats”
As we saw earlier there are geese, mice and oxen
But there are also a whole host of spelling/pronunciation changes that go along with inflectional changes
Cats vs Dogs
Fox and Foxes

36. Multi-Tape Machines
To deal with this we can simply add more tapes and use the output of one tape machine as the input to the next
So to handle irregular spelling changes we’ll add intermediate tapes with intermediate symbols

37. Generativity Nothing really privileged about the directions.
We can write from one and read from the other or vice-versa.
One way is generation, the other way is analysis

38. Multi-Level Tape Machines
We use one machine to transduce between the lexical and the intermediate level, and another to handle the spelling changes to the surface tape

39. Lexical to Intermediate Level

40. Intermediate to Surface
The add an “e” rule as in fox^s# <-> foxes#

41. Foxes

42. Note
A key feature of this machine is that it doesn’t do anything to inputs to which it doesn’t apply.
Meaning that they are written out unchanged to the output tape.

43. Overall Scheme
We now have one FST that has explicit information about the lexicon (actual words, their spelling, facts about word classes and regularity).
Lexical level to intermediate forms
We have a larger set of machines that capture orthographic/spelling rules.
Intermediate forms to surface forms

44. Overall Scheme

45. Cascades This is a scheme that we’ll see again and again.
Overall processing is divided up into distinct rewrite steps
The output of one layer serves as the input to the next
The intermediate tapes may or may not wind up being useful in their own right

46. Porter Stemmer (1980)
Used for tasks in which you only care about the stem
IR, modeling given/new distinction, topic detection, document similarity
Lexicon-free morphological analysis
Cascades rewrite rules (e.g. misunderstanding --> misunderstand --> understand --> …)
Easily implemented as an FST with rules e.g.
ATIONAL  ATE
ING  ε
Not perfect ….
Doing  doe

47. Policy  police
Does stemming help?
IR, little
Topic detection, more

48. Summing Up
FSTs provide a useful tool for implementing a standard model of morphological analysis, Kimmo’s two-level morphology
But for many tasks (e.g. IR) much simpler approaches are still widely used, e.g. the rule-based Porter Stemmer
Next time:
Read Ch 4
HW1 assigned; see web page: