November 2003Computational Morphology VI1 CSA4050 Advanced Topics in NLP Non-Concatenative Morphology – Reduplication – Interdigitation

November 2003Computational Morphology VI2 Reference Ken Beesely and Lauri Karttunen, Finite State Non-Concatenative Morphotactics, Proceedings of SIGPHON-2000

November 2003Computational Morphology VI3 Koskenniemi 1983 "Only restricted infixation and reduplication can be handled adequately with the present system. Some extensions or revisions will be necessary for an adequate description of languages possessing extensive infixation or reduplication"

November 2003Computational Morphology VI4 Non-Concatenative Languages Most languages build words by stringing together morphemes like beads on a string. The word-building processes of prefixation and suffixation can be straightforwardly modeled in finite state terms by concatenation. But some languages also exhibit non- concatenative morphotactics.

November 2003Computational Morphology VI5 Non-Concatenative Phenomena 1. Reduplication In Malay bagi (bag) bagi-bagi (bags) Although this may appear concatenative, it does not involve concatenating a predictible morpheme – like "s". Instead the entire stem is copied no matter what its length. In general language class (ww | w  L) is context sensitive, but if L is finite, we can construct an FS network that encodes it.

November 2003Computational Morphology VI6 General Solution for Reduplication Therefore, assuming the number of words subject to reduplication is finite, it is possible to construct a lexical transducer for languages like Malay. To handle reduplication, a new operator ^n is introduced: A^n denotes n concatenations of A.

November 2003Computational Morphology VI7 Remarks from Beesley on Context Sensitivity finite-state grammars (cannot handle unlimited nesting or non-nested terminal dependencies) context-free (can handle unlimited nesting, such as matched parentheses in arithmetic expressions, but cannot handle non-nested dependencies between terminals) context-sensitive (can also handle non-nested dependencies between terminals, as in dogdog where terminal elements 1 and 4 have to be the same, 2 and 5 have to be the same, and 3 and 6 have to be the same. These dependencies cross, so they're not nested.

November 2003Computational Morphology VI8 Non-Concatenation 2. Interdigitation In Arabic and Maltese, prefixes and suffixes attach to stems in the usual concatenative way, but stems themselves are formed by a process known as interdigitation. An example of occurs with the Arabic stem "katab" (wrote). This stem is composed of three elements 1.the all consonant root ktb 2.an abstract consonant-vowel template CVCVC 3.a vocalisation aa (in this case signifying perfect tense and active voice)

November 2003Computational Morphology VI9 Interdigitation The same root ktb can combine with the same template CVCVC and a different vocalism ui (signifying imperfect aspect and passive voice) to produce "kutib" (was written). The same root ktb can combine with a different template CVVCVC and the vocalism ui to produce "kuutib" – another form of the verb.

November 2003Computational Morphology VI10 Intermediate Result: Template + Root d v v r v s

November 2003Computational Morphology VI11 Final Result: Intermediate Result + Vocalism d u u r i s

November 2003Computational Morphology VI12 Merge In this case the filler language contains an infinite set of strings (i, ui, uui …) but only one path can be constructed because all strings end in i. Hence the earlier vowels must be "u". This need not always be the case (eg if the filler language were u*i*).

November 2003Computational Morphology VI13 Merge Operators To introduce the merge operation into the Xerox calculus new operators,.. have been introduced. These differ only in the order of arguments. [T.. T] represent the same merge operation with F and T as filler and template respectively.

November 2003Computational Morphology VI14 The Composite Transducer With these operators the network above can be compiled by using the following expression: [d r s].m>. [C V V C V C].<m. [u* i]

November 2003Computational Morphology VI15 Merge c v v c v c d r s i u template vocalism root

November 2003Computational Morphology VI16 Compile-Replace Regular expressions are compiled into networks as usual, but in addition, the compiler is then applied to its own output. Central idea: –transduce to a language that has the format of regular expressions. –The compile-replace algorithm then replaces the regular expression with the result of its own compilation.

November 2003Computational Morphology VI17 Compile Replace Simple Example 0:^[ a * 0:^] This network maps the string a* to ^[ a* ^] (i.e. the same RE but with special delimiters) Application of CR to the lower side of the network eliminates the markers, compile the RE a* and maps the upper side to to the language resulting from the compilation.

November 2003Computational Morphology VI18 The result of compiling ^[ a* ^] a *:a a:0 *:0 0:a To answer the question: what does this network do? Figure out what it does in upward and downward directions

November 2003Computational Morphology VI19 The result of compiling ^[ a* ^] a *:a a:0 *:0 0:a When applied in the upward direction, this transducer maps any string of the infinite a* language into the regular expression from which it was compiled. When applied in the downward direction, it maps from a* to all the strings in the language a*, {0, a, aa,...}

November 2003Computational Morphology VI20 Compile-Replace: 1 Copy input path to output path until ^[ is encountered on indicated (in our case lower) side of the network. Extract path until closing delimiter ^]. 0:^[ a * 0:^] a:a *:*

November 2003Computational Morphology VI21 Compile-Replace: 2 Symbols along indicated side are concatenated into a string and eliminated from the path leaving just the symbols on the opposite side. The remaining net is The extracted string is compiled into a second network using the standard network compiler a * a

November 2003Computational Morphology VI22 Compile-Replace: 3 The 2 networks are combined together using the cross product operator. The result is spliced between the origin and destination states of the regular expression path. a * a a *:a a:0 *:0 0:a

November 2003Computational Morphology VI23 Reduplication Revisited Applying compile-replace to this transducer Lexical: b a g i +Noun +Plural Surface: ^[ [b a g i] ^ 2 ^] yields this one Lexical: b a g i +Noun +Plural Surface: b a g i b a g i

November 2003Computational Morphology VI24 Interdigitation Revisited Applying compile-replace to this transducer Up: k i t e b +Verb +Past +3Sg Do:[k t b].m>. [C V C V C].<m. [i e] yields this one Up: k i t e b +Verb +Past +3Sg Do: k i t e b

November 2003Computational Morphology VI25 Remember: Two Central Problems Morphotactics: constraints on combinations of morphemes governing the formation of valid words. unbelievable vs. believeunable Phonological/Orthographical Alternation (spelling rules): how morphemes are realised in particular environments fly + s = flies

November 2003Computational Morphology VI26 Xerox Perspective Morphotactics: handle with lexc Phonological/Orthographical Alternation (spelling rules): handle with xfst Morphotactics Rules FST Lexicon FST Lexical Transducer Alternations.o. lexc xfst