1. Semi-Automatic Learning of Transfer Rules for Machine Translation of Minority Languages Katharina Probst Language Technologies Institute Carnegie Mellon University Pittsburgh, PA, U.S.A.

2. Introduction and Motivation Minority languages: little or no online data available, few linguists One approach to preservation: build tools that allow bilingual education, communication in the minority language Our project: build a tool that facilitates the production of a machine translation system between a major and a minority language Close cooperation with indigenous groups Examples: source language English, target language German

3. Approach - Overview 1.Elicitation Corpus: Bilingual data is acquired from a specifically engineered corpus 2.Feature Detection: Gather information about features and their values in the minority language 3.Rule Learning: Infer syntactic transfer rules 1.Seed Generation: First guesses at rules 2.Compositionality: Use previously learned rules 3.Seeded Version Space Learning: Refine rules

4. Elicitation Corpus Translated, aligned by bilingual informant Corpus consists of linguistically diverse constructions Based on work of field linguists (e.g. Comrie 1977, Bouquiaux 1992) Organized in minimal pairs, i.e. pairs of sentences that differ in only one linguistic feature (e.g. The man slept. vs. The men slept). Organized compositionally: elicit simple structures first, then use them as building blocks Goal: minimize size, maximize coverage

5. Transfer Rule Formalism Type information Part-of-speech/constituent information Alignments x-side constraints y-side constraints xy-constraints, e.g. ((Y1 AGR) = (X1 AGR)) ;SL: the man, TL: der Mann NP::NP [DET N] →[DET N] ( (X1::Y1) (X2::Y2) ((X1 AGR) = *3-SING) ((X1 DEF = *DEF) ((X2 AGR) = *3-SING) ((X2 COUNT) = +) ((Y1 AGR) = *3-SING) ((Y1 DEF) = *DEF) ((Y2 AGR) = *3-SING) ((Y2 GENDER) = (Y1 GENDER)) )

6. Transfer Rule Formalism (II) Value constraints Agreement constraints ;SL: the man, TL: der Mann NP::NP [DET N] →[DET N] ( (X1::Y1) (X2::Y2) ((X1 AGR) = *3-SING) ((X1 DEF = *DEF) ((X2 AGR) = *3-SING) ((X2 COUNT) = +) ((Y1 AGR) = *3-SING) ((Y1 DEF) = *DEF) ((Y2 AGR) = *3-SING) ((Y2 GENDER) = (Y1 GENDER)) )

7. Rule Learning - Overview Goal: Syntactic Transfer Rules 1) Flat Seed Generation: produce rules from word- aligned sentence pairs, abstracted only to POS level; no syntactic structure 2) Add compositional structure to Seed Rule by exploiting previously learned rules 3) Seeded Version Space Learning group seed rules by constituent sequences and alignments, seed rules form s-boundary of VS; generalize with validation

8. Flat Seed Generation - Overview Create a transfer rule that is specific to the sentence pair, but abstracted to the pos level. No syntactic structure. ElementSource SL POS sequencef-structure TL POS sequenceTL dictionary, aligned SL words Type informationCorpus, same on SL and TL AlignmentsInformant x-side constraintsf-structure y-side constraintsTL dictionary, aligned SL words (list of projecting features)

9. Flat Seed Generation - Example The highly qualified applicant visited the company. Der äußerst qualifizierte Bewerber besuchte die Firma. ((1,1),(2,2),(3,3),(4,4),(5,5),(6,6)) S::S [det adv adj n v det n]→ [det adv adj n v det n] (;;alignments: (x1::y1) (x2::y2) (x3::y3) (x4::y4) (x5::y5) (x6::y6) (x7::y7) ;;constraints: ((x1 def) = *+) ((x4 agr) = *3-sing) ((x5 tense) = *past) …. ((y1 def) = *+) ((y3 case) = *nom) ((y4 agr) = *3-sing) … )

10. Compositionality - Overview Traverse the c-structure of the English sentence: for each sub-tree, translate the sentence chunk rooted at it, compare to the corresponding TL chunk. If correct translation, introduce compositional structure. Adjust constituent sequences, alignments Adjust constraints: use f-structure of correct translation vs. f-structure of incorrect translations to introduce context constraints Remove unnecessary constraints, i.e. those that are contained in the lower-level rule

11. Compositionality - Example S::S [det adv adj n v det n]→ [det adv adj n v det n] (;;alignments: (x1::y1) (x2::y2) (x3::y3) (x4::y4) (x5::y5) (x6::y6) (x7::y7) ;;constraints: ((x1 def) = *+) ((x4 agr) = *3-sing) ((x5 tense) = *past) …. ((y1 def) = *+) ((y4 agr) = *3-sing) … ) S::S [NP v det n]→ [NP v det n] (;;alignments: (x1::y1) (x2::y2) (x3::y3) (x4::y4) ;;constraints: ((x2 tense) = *past) …. ((y1 def) = *+) ((y1 case) = *nom) ((y1 agr) = *3-sing) … ) NP::NP [det adv adj n] [det adv adj n] ((x1::y1)… ((y4 agr) = *3-sing) ((x4 agr = *3-sing) ….)

12. Seeded Version Space Learning - Overview Seed rules are grouped by their constituent sequences, alignments, and types A version space represents a partial ordering of a set of instances The seed rules in a group form the specific boundary of a version space The general boundary is the (implicit) transfer rule with the same type, constituent sequences, and alignments, but no constraints

13. Seeded Version Space Learning – Overview (II) Make use of partial order in version space: Definition: A transfer rule tr1 is strictly more general than another transfer rule tr2 if all f- structures that are satisfied by tr2 are also satisfied by tr1. Generalize rules by merging them: –Deletion of constraint –Raising two value constraints to an agreement constraint, e.g. ((x1 num) = *pl), ((x3 num) = *pl) → ((x1 num) = (x3 num)

14. Seeded Version Space Learning – Overview (III) Merging algorithm proceeds in three steps. To merge tr 1 and tr 2 into tr merged : 1.Copy all constraints that are both in tr 1 and tr 2 into tr merged 2.Consider tr 1 and tr 2 separately. For the remaining constraints in tr 1 and tr 2, perform all possible instances of raising value constraints to agreement constraints. 3.Repeat step 1.

15. Seeded Version Space Learning – Overview (IV) The Seeded Version Space algorithm itself is the repeated generalization of rules by merging them A merge is successful if the set of sentences that can correctly be translated with the merged rule is a superset of the union of sets that can be translated with the unmerged rules Merge until no more successful merges

16. Seeded Version Space Learning - Example S::S [NP v det n]→ [NP v det n] (;;alignments: (x1::y1) (x2::y2) (x3::y3) (x4::y4) ;;constraints: ((x2 tense) = *past) …. ((y1 def) = *+) ((y1 agr) = *3-sing) … ((y3 agr) = *3-sing) ((y4 agr) = *3-sing)… ) S::S [NP v det n]→ [NP v det n] (;;alignments: (x1::y1) (x2::y2) (x3::y3) (x4::y4) ;;constraints: ((x2 tense) = *past) … ((y1 def) = *+) ((y1 agr) = *3-plu) … ((y3 agr) = *3-plu) ((y4 agr) = *3-plu)… ) S::S [NP v det n]→ [NP v det n] ( ;;alignments: (x1::y1) (x2::y2) (x3::y3) (x4::y4) ;;constraints: ((x2 tense) = *past) …. ((y1 def) = *+) ((y4 agr) = (y3 agr) … )

17. Future Work Baseline evaluation Adjust generalization step size Revisit generalization operators Introduce specialization operators to retract from overgeneralizations (including seed rules) Learn from an unstructured bilingual corpus Evaluate merges to pick the optimal one at any step: based on cross-validation, number of sentences it can translate