Data Elicitation for AVENUE Lori Levin Alison Alvarez Jeff Good (MPI Leipzig) Bob Frederking Erik Peterson Language Technologies Institute Carnegie Mellon University

Outline Elicitation The Functional-Typological Corpus Corpus Creation Feature Detection Corpus Navigation

The Elicitation Tool

Input to the Elicitation Tool: corpus of minimal pairs Eliciting from Spanish # 1,2,3 {Sg,pl} person pronouns newpair srcsent: Canto context: comment: newpair srcsent: Canté context: comment: newpair srcsent: Estoy cantando context: comment: newpair srcsent: Cantaste context: comment: Eliciting from English # 1,2,3 {Sg,pl} person pronouns newpair srcsent: I sing context: comment: newpair srcsent: I sang context: comment: newpair srcsent: I am singing context: comment: newpair srcsent: You sang context: comment:

Output of the elicitation process newpair srcsent: Tú caíste tgtsent: eymi ütrünagimi aligned: ((1,1),(2,2)) context: tú = Juan [masculino, 2a persona del singular] comment: You (John) fell newpair srcsent: Tú estás cayendo tgtsent: eymi petu ütünagimi aligned: ((1,1),(2 3,2 3)) context: tú = Juan [masculino, 2a persona del singular] comment: You (John) are falling newpair srcsent: Tú caíste tgtsent: eymi, ütrunagimi aligned: ((1,1),(2,2)) context: tú = María [femenino, 2a persona del singular] comment: You (Mary) fell

Elicitation Corpus Elicitation Corpus refers to the list of sentences in the major language.  Not yet translated or aligned Field workers call it a questionnaire.

The elicitation corpus is useful as Input to automatic rule learning Test suite for machine translation (at ARL) Fieldwork questionnaire  The consultant can do some of the tedious parts by himself/herself.

AVENUE Elicitation Corpora The Functional-Typological Corpus  Designed to elicit elements of meaning that may have morpho-syntactic realization The Structural Elicitation Corpus  Based on sentence structures from the Penn TreeBank

The Functional Typological Corpus c-my-polarity polarity-positive polarity-negative Stick to the two obvious values of polarity for now. Feature Name: c-my-polarity Values: positive, negative Note: Stick to the two obvious values of polarity for now.

Functional Typological Corpus In XML XSLT scripts can format it into human-readable text or into data structures. Currently contains around 50 features and a few hundred values.  Based on the Lingua checklist (Comrie and Smith, 1977), other fieldwork checklists, other typological taxonomies. Still under development

Functional Typological Corpus: Representation of “Who is at the meeting” ((subj ((np-my-general-type pronoun-type)(np-my-person person-unk) (np-my-number num-sg)(np-my-animacy anim-human) (np-my-function fn-predicatee) (np-d-my-distance-from-speaker distance-neutral) (np-my-emphasis emph-no-emph) (np-my-info-function info-neutral) (np-pronoun-exclusivity exclusivity-n/a) (np-pronoun-antecedent-function antecedent-n/a) (np-pronoun-reflexivity reflexivity-n/a))) (predicate ((loc-roles loc-general-at))) Continued on next slide

Continued: “Who is at the meeting” (c-my-copula-type locative)(c-my-secondary-type secondary-copula) (c-my-polarity polarity-positive) (c-my-function fn-main-clause)(c-my- general-type open-question)(gap-function gap-copula-subject)(c-my- sp-act sp-act-request-information)(c-v-my-grammatical-aspect gram- aspect-neutral)(c-v-my-absolute-tense present) (c-v-my-phase- aspect durative)(c-my-headedness-rc rc-head-n/a)(c-my-minor-type minor-n/a)(c-my-restrictivess-rc rc-restrictive-n/a)(c-my-answer-type ans-n/a)(c-my-imperative-degree imp-degree-n/a)(c-my-actor's- status actor-neutral)(c-my-focus-rc focus-n/a)(c-my-gaps-function gap-n/a)(c-my-relative-tense relative-n/a)(c-my-ynq-type ynq-n/a)(c- my-actor's-sem-role actor-sem-role-neutral)(c-v-my-lexical-aspect state))

Why is the corpus represented as a set of feature structures? Multiple elicitation languages  Generate the English and Spanish elicitation corpora from the same internal representation  Easy to add a new elicitation language Write a GenKit grammar to generate sentences from the same internal representation

Why is the corpus represented as a set of feature structures? Feature structure represents things that are not expressed in the major language  These things show up as comments in the elicitation corpus “I am singing” (comment: female)  May eventually use pictures and discourse context  We actually want to elicit the meaning associated with the feature structure. English and Spanish are just vehicles for getting at the meaning.

Corpus Creation Tools The elicitation corpus can be changed and new corpora can be created.

Motivation for Corpus Creation Tools Make new corpora easily  Add a new tense (e.g., remote past) and automatically get all the combinations with other features  Make a specialized corpus for a limited semantic domain or a specific language family

Motivation for Corpus Creation Tools Combinatorics  For example, all combinations of person, number, gender, tense, etc.  Too much bookkeeping for a human corpus creator, and too time consuming

Where do the feature structures come from? A linguist formulates a Multiply The multiply specifies a set of feature structures

A Multiply ((subj ((np-my-general-type pronoun-type common-noun-type) (np-my-person person-first person-second person-third) (np-my-number num-sg num-pl) (np-my-biological-gender bio-gender-male bio-gender-female) (np-my-function fn-predicatee))) {[(predicate ((np-my-general-type common-noun-type) (np-my-definiteness definiteness-minus) (np-my-person person-third) (np-my-function predicate))) (c-my-copula-type role)] [(predicate ((adj-my-general-type quality-type))) (c-my-copula-type attributive)] [(predicate ((np-my-general-type common-noun-type) (np-my-person person-third) (np-my-definiteness definiteness-plus) (np-my-function predicate))) (c-my-copula-type identity)]} (c-my-secondary-type secondary-copula) (c-my-polarity #all) (c-my-function fn-main-clause)(c-my-general-type declarative) (c-my-speech-act sp-act-state) (c-v-my-grammatical-aspect gram-aspect-neutral) (c-v-my-lexical-aspect state) (c-v-my-absolute-tense past present future) (c-v-my-phase-aspect durative)) This multiply expands to 288 feature structures.

There is a GUI for making Multiplies Demo may be available

GenKit Grammar Use GenKit for generation ;;declarative ( ==> ( ) (((x0 c-my-general-type) =c declarative) ((x2 verb-form) = fin) ((x3 c-my-copula-type) = (x0 c-my-copula-type)) ((x4 d-speaker-gender) = (x0 d-speaker-gender)) ((x4 d-hearer-gender) = (x0 d-hearer-gender)) ((x4 d-my-formality) = (x0 d-my-formality)) ((x3 np-my-number) = (x0 np-my-number)) ((x3 np-my-animacy) = (x0 np-my-animacy)) ((x3 np-my-biological-gender) = (x0 np-my-biological-gender)) (x3 = (x0 predicate)) (x1 = (x0 subj)) (x2 = x0)))

GenKit Lexicon ;;Pronouns (word ((cat n) (root you) (pred pro) (np-my-person person-second) (np-my-animacy anim-human) (np-my-general-type pronoun-type))) (word ((cat n) (root I) (pred pro) (np-my-person person-first) (np-my-number num-sg) (np-my-animacy anim-human) (np-my-general-type pronoun-type))) (word ((cat n) (root we) (pred pro) (np-my-person person-first) (np-my-number num-pl) (np-my-animacy anim-human) (np-my-general-type pronoun-type))) (word ((cat n) (root we) (pred pro) (np-my-person person-first) (np-my-number num-dual) (np-my-animacy anim-human) (np-my-general-type pronoun-type))) (word ((cat n) (root she) (pred pro) (np-my-person person-third) (np-my-number num-sg) (np-my-biological-gender bio-gender-female) (np-my-animacy anim-human) (np-my-general-type pronoun-type)))

Comments are also generated I & one female & sang Use comments for things that are not expressed in English.

Convert to Elicitation Format (input to Elicitation Tool) original: WHO & IS AT THE BOX & full comment: Sentence: WHO IS AT THE BOX original: I &ONE-WOMAN & AM PN_FEMALE &ONE-WOMAN & & full comment: NP1: ONE-WOMAN Sentence: I AM PN_FEMALE original: WILL I &ONE-WOMAN & BE THE TEACHER & full comment: NP1: ONE-WOMAN Sentence: WILL I BE THE TEACHER

Eight Basic Steps for Corpus Creation 1. Write FVD and format into data structure 2. Gather Exclusions (restrictions on co- occurrence of features 3. Design the Multiply 4. Get a full set of Feature Structures 5. Design Grammar and Comments 6. Design Lexicon 7. Generate Sentences from Feature Structures 8. Convert to Elicitation Format

Can make other types of corpora The Elicitation Corpus does not have to be functional-typological

Alternative Corpora: The Medical Corpus Feature: Body-Parts Values part-hand Restrictions: part-finger Restrictions: part-tooth Restrictions: symptom_redness symptom_scratch symptom_numbness symptom_cut symptom_lump symptom_rash symptom_puncture symptom_bruise symptom_frozen part-eye Restrictions: symptom_rash part-arm Restrictions: … ((subj ((body-parts #all) (Poss ((np-my-general-type pronoun-type) (np-my-person #all) (np-my-number num-sg num-pl) (np-my-animacy anim-human) (np-my-use possessive))) (Pred ((symptoms #all)) (c-my-general-type declarative) (c-my-speech-act sp-act-state) (c-v-my-grammatical-aspect gram-aspect-neutral) (c-v-my-lexical-aspect state) (c-v-my-absolute-tense present)); The Result: YOUR ARM IS RED YOUR ARM IS SCRATCHED YOUR ARM IS NUMB YOUR ARM IS NIL YOUR ARM HAS A/N INFECTION…

Feature Detection Identify meaning components that have morpho-syntactic consequences in the language that is being elicited.  The gender of the subject is marked on the verb in Hebrew.  The gender of the subject has no morpho- syntactic realization in Mapudungun.

Feature Detection: Spanish The girl saw a red book. ((1,1)(2,2)(3,3)(4,4)(5,6)(6,5)) La niña vió un libro rojo A girl saw a red book ((1,1)(2,2)(3,3)(4,4)(5,6)(6,5)) Una niña vió un libro rojo I saw the red book ((1,1)(2,2)(3,3)(4,5)(5,4)) Yo vi el libro rojo I saw a red book. ((1,1)(2,2)(3,3)(4,5)(5,4)) Yo vi un libro rojo Feature: definiteness Values: definite, indefinite Function-of-*: subj, obj Marked-on-head-of-*: no Marked-on-dependent: yes Marked-on-governor: no Marked-on-other: no Add/delete-word: no Change-in-alignment: no

Feature Detection: Chinese A girl saw a red book. ((1,2)(2,2)(3,3)(3,4)(4,5)(5,6)(5,7)(6,8)) 有 一个 女人 看见 了 一本 红色 的 书 。 The girl saw a red book. ((1,1)(2,1)(3,3)(3,4)(4,5)(5,6)(6,7)) 女人 看见 了 一本 红色的 书 Feature: definiteness Values: definite, indefinite Function-of-*: subject Marked-on-head-of-*: no Marked-on-dependent: no Marked-on-governor: no Add/delete-word: yes Change-in-alignment: no

Feature Detection: Chinese I saw the red book ((1, 3)(2, 4)(2, 5)(4, 1)(5, 2)) 红色的 书， 我 看见 了 I saw a red book. ((1,1)(2,2)(2,3)(2, 4)(4,5)(5,6)) 我 看见 了 一本 红色的 书 。 Feature: definitenes Values: definite, indefinite Function-of-*: object Marked-on-head-of-*: no Marked-on-dependent: no Marked-on-governor: no Add/delete-word: yes Change-in-alignment: yes

Feature Detection: Hebrew A girl saw a red book. ((2,1) (3,2)(5,4)(6,3)) ילדה ראתה ספר אדום The girl saw a red book ((1,1)(2,1)(3,2)(5,4)(6,3)) הילדה ראתה ספר אדום I saw a red book. ((2,1)(4,3)(5,2)) ראיתי ספר אדום I saw the red book. ((2,1)(3,3)(3,4)(4,4)(5,3)) ראיתי את הספר האדום Feature: definiteness Values: definite, indefinite Function-of-*: subj, obj Marked-on-head-of-*: yes Marked-on-dependent: yes Marked-on-governor: no Add-word: no Change-in-alignment: no

Feature detection feeds into Corpus Navigation: which minimal pairs to pursue next.  Don’t pursue gender in Mapudungun  Do pursue definiteness in Hebrew Morphology Learning:  Morphological learner identifies the forms of the morphemes  Feature detection identifies the functions Rule learning:  Rule learner will have to learn a constraint for each morpho-syntactic marker that is discovered E.g., Adjectives and nouns agree in gender, number, and definiteness in Hebrew.

Other uses of Feature Detection A human-readable reference grammar can be generated from fact records.  A human analyst knows Northern Ostyak, and then has to translate a document in Eastern Ostyak. The only reference grammar of Eastern Ostyak is written in Hungarian, which the analyst does not speak. An Eastern Ostyak consultant who speaks Russian translates the Elicitation Corpus from Russian to Eastern Ostyak. The analyst learns about Eastern Ostyak from automatically generated fact records.

Other uses of Feature Detection A human-readable reference grammar can be generated from fact records.  A human analyst knows Northern Ostyak, and then has to translate a document in Eastern Ostyak. The only reference grammar of Eastern Ostyak is written in Hungarian, which the analyst does not speak. An Eastern Ostyak consultant who speaks Russian translates the Elicitation Corpus from Russian to Eastern Ostyak. The analyst learns about Eastern Ostyak from the automatically generated fact records. I’m not really sure whether the only grammar of Eastern Ostyak is written in Hungarian. There is one reference grammar of Northern Ostyak written in English (by Irina Nikolaeva). All other Ostyak materials are in Hungarian, Russian, and German. The Ostyaks are subsistence hunters, and Eastern Ostyak is nearly extinct, so there is no real need for government translators. Other Siberian and Central Asian languages with similar scarcity of resources may be important.

Other uses of Feature Detection Help a field worker  Instead of “Elicit by day; analyze by night” (in order to know what to elicit the next day), go to sleep and look at the automatically generated analysis in the morning.  We have been working with people at EMELD and MPI Leipzig.

Corpus Navigation While the Elicitation Corpus for any one target language (TL) can be kept to a reasonable size, the universal Elicitation Corpus must check for all phenomena that might occur in any langauge. Since the universal corpus cannot be kept to a reasonable size, Corpus Navigation is necessary. Facts discovered about a particular TL early in the process constrain what needs to be looked for later in the process for that TL. Thus this is a dynamic process, different for each TL.

Corpus Navigation: search Search process, with the informant in the inner loop, expanding search states he/she is given as SL sentences by supplying the corresponding TL sentence and alignments. Analogously to game search, there is an "opening book" of moves (SL sentences to check for all languages), until enough inforrmation has been gathered to make intelligent search choices. The hueristic function driving the search process is Relative Info Gain: RIG(Y|X) = [H(Y) - H(Y|X)]/H(Y) The system reduces the remaining entropy in its knowledge of the language as much as possible. There should also be a cost factor, estimating the human effort required to expand the node. To make the process efficient enough, we will create "decision graphs", similar to RETE networks, that cache information so only the information that changes needs to be recomputed.