1. Dependency Tree-to-Dependency Tree Machine Translation November 4, 2011 Presented by: Jeffrey Flanigan (CMU) Lori Levin, Jaime Carbonell In collaboration with: Chris Dyer, Noah Smith, Stephan Vogel 1

2. Problem Swahili: Watoto ni kusoma vitabu. Gloss: children aux-pres read books English: Children are reading books. MT (Phrase-based): Children are reading books. 2 Why? Phrase Table: Pr(reading books| kusoma vitabu) Pr(books | kusoma vitabu) Language model: Children are three new reading books. Children are reading books three new. Swahili: Watoto ni kusoma vitabu tatu mpya. Gloss: children aux-pres read books three new English: Children are reading three new books. MT (Phrase-based): Children are three new books.

3. Problem: Grammatical Encoding Missing Swahili: Nimeona samaki waliokula mashua. Gloss: I-found fish who-ate boat English: I found the fish that ate the boat. MT System: I found that eating fish boat. 3 Predicate-argument structure was corrupted.

4. Grammatical Relations I found the fish that ate the boat. 4 SUBJ OBJ ⇒ Dependency trees on source and target! ROOT DET RCMOD DOBJ DET REF

5. Approach Source Dependency Tree Target Dependency Tree Source Sentence Target Sentence Undo grammatical encoding (parse) Translate Grammatical encoding (choose surface form, linearize) 5 All stages statistical

6. Extracting the rules: Extract all consistent tree fragment pairs 6 Children are reading books three new Abaana barasoma ibitabo bitatubishya NSUBJ NUM AUX DOBJ NUM AMOD are reading NSUBJ AUX [1] [2] DOBJ barasoma NSUBJ [1] [2] DOBJ are reading NSUBJ AUX [1] books DOBJ ibitabo barasoma NSUBJ DOBJ [1] NUM three new AMOD bishya [1] NUM AMOD bitatu Children Abaana Example Extracted Pairs SOURCE SIDE TARGET SIDE Abaana [1] Children [1] NUM

7. Translating Extension of phrase-based SMT Linear strings → Dependency trees Phrase pairs → Tree fragment pairs Language model → Dependency language model Search is top down on the target side using beam search decoder 7

8. Translation Example 8 umwaana [3] arasoma Ibitabo [4] is reading NSUBJ AUX [1] [2] DOBJ [2] arasoma NSUBJ DOBJ [1] child umwaana P(e|f)=.5 P(e|f)=.8 NSUBJ DOBJ Inventory of Rules the DET NSUBJ [1] NSUBJ child umwaana P(e|f)=.1 a DET NSUBJ [1] NSUBJ ibitabobooks P(e|f)=.7 Input The child is reading books

9. Translation Example 9 is reading [4] umwaana [3] arasoma Ibitabo [4] is reading NSUBJ AUX [1] [2] DOBJ [2] arasoma NSUBJ DOBJ [1] child umwaana P(e|f)=.5 P(e|f)=.8 NSUBJ DOBJ NSUBJ DOBJ AUX Inventory of Rules the DET NSUBJ [1] NSUBJ child umwaana P(e|f)=.1 a DET NSUBJ [1] NSUBJ Score = w 1 ln(.5)+w 2 ln(Pr(reading| ROOT ))+w 2 ln(Pr(is|(reading, AUX ))) ibitabobooks P(e|f)=.7 [3] Input Language model on target dependency tree

10. Translation Example 10 is reading books umwaana [3] arasoma ibitabo is reading NSUBJ AUX [1] [2] DOBJ [2] arasoma NSUBJ DOBJ [1] child umwaana P(e|f)=.5 P(e|f)=.8 NSUBJ DOBJ NSUBJ DOBJ AUX Inventory of Rules the DET [3] NSUBJ [1] NSUBJ child umwaana P(e|f)=.1 a DET NSUBJ [1] NSUBJ Score = w 1 ln(.5)+w 1 ln(.7)+w 2 ln(Pr(reading|ROOT)) +w 2 ln(Pr(is|(reading,AUX)))+w 2 ln(Pr(books|(reading,DOBJ))) ibitabobooks P(e|f)=.7 Input

11. Translation Example 11 is reading books umwaana arasoma ibitabo is reading NSUBJ AUX [1] [2] DOBJ [2] arasoma NSUBJ DOBJ [1] child umwaana P(e|f)=.5 P(e|f)=.8 NSUBJ DOBJ NSUBJ DOBJ AUX Inventory of Rules the DET child the DET NSUBJ [1] NSUBJ child umwaana P(e|f)=.1 a DET NSUBJ [1] NSUBJ Score(Translation) = w 1 ln(.5)+w 1 ln(.7)+w 1 ln(.8)+w 2 ln(Pr(reading|ROOT)) +w 2 ln(Pr(is|(reading,AUX)))+w 2 ln(Pr(books|(reading,DOBJ))) +w 2 ln(Pr(child|(reading,NSUBJ)))+w 2 ln(Pr(the|(child,DET),(reading,ROOT))) ibitabobooks P(e|f)=.7 Input

12. Linearization Generate projective trees A* Search Left to right with target LM Admissible Heuristic: Highest scoring completion without LM 12 enough is strong NSUBJ COP ADVMOD He

13. Linearization 13 enough is strong NSUBJ COP ADVMOD He He is enough strong Score=Pr(He| START )∙Pr( |is)∙Pr( |strong) Generate projective trees A* Search Left to right with target LM Admissible Heuristic: Highest scoring completion without LM

14. Linearization 14 enough is strong NSUBJ COP ADVMOD He He is enough strong Score=Pr(He |START) ∙Pr(is|He,START)∙Pr( |is)∙Pr( |strong) Generate projective trees A* Search Left to right with target LM Admissible Heuristic: Highest scoring completion without LM

15. Linearization 15 enough is strong NSUBJ COP ADVMOD He He is strong enough Score=Pr(He |START) ∙Pr(is|He,START) ∙Pr(strong|He,is)∙ Pr( |is)∙Pr( |strong) Generate projective trees A* Search Left to right with target LM Admissible Heuristic: Highest scoring completion without LM

16. Linearization 16 enough is strong NSUBJ COP ADVMOD He He is strong enough Score=Pr(He)∙Pr(is|He)∙Pr(strong|He, is)∙Pr(enough|strong, is)∙ Pr( |is)∙Pr( |strong) Generate projective trees A* Search Left to right with target LM Admissible Heuristic: Highest scoring completion without LM

17. Comparison To Major Approaches ApproachSimilaritiesDifference Old Style Analysis-Transfer-GenerateSeparate analysis, generation, transfer models Statistical, rules learned Synchronous CFGs [Chiang 2005] [Zollman et al. 2006] Model of grammatical encodingAllows adjunction and head switching Tree-Transducers [Graehl & Knight 2004] Model of grammatical encodingDifferent decoding Quasi-Synchronous Grammars [Gimpel & Smith 2009] Dependency trees on source and target Different rules, decoding Synchronous Tree Insertion Grammars [DeNeefe & Knight 2009] Allows adjunctsAllows head switching Dependency Treelets [Quirk et al 2005] [Shen et al 2008] Dependency trees on source and target Word order not in rules, linearization procedure String-to-Dependency MT [Shen et al 2008] Target dependency language model Dependency trees on both source and target Dependency tree to dependency tree (JHU Summer Workshop 2002) [Čmejrek et al 2003] [Eisner 2003] Dependency trees on source and target. Linearization step. Different learning of rules, different decoding procedure 17

18. Conclusion Separate translation from reordering Dependency trees capture grammatical relations Can extend phrase-based MT to dependency trees Complements ISI’s approach nicely Work in progress! 18

19. Backup Slides 19

20. Allowable Rules Nodes consistent w/ alignments All variables aligned Nodes ∪ variables ∪ arcs ∪ alignments = connected graph Optional Constraints Nodes on source connected Nodes on target connected Nodes on source and target connected Decoding Constraint Target tree connected 20

21. Head Switching Example 21 bébé Le vient de tomber child fell just The NSUBJ DET PREP ADVMOD POBJ ADVMOD [1] [2] just NSUBJ ADVMOD [1] vient de [2] NSUBJ PREP POBJ [1] fell [2] NSUBJ ADVMOD [1] [2] de tomber NSUBJ PREP POBJ

22. Moving Up the Triangle Propositional Semantic Dependencies Deep Syntactic Dependencies Surface Syntactic Dependencies 22

23. Comparison to Synchronous Phrase Structure Rules Training dataset: Test sentence: Synchronous decoders (SAMT, Hiero, etc) produce: The children are reading book ’s Charles new all of. The children are reading book Charles ’s all of new. Problem: Grammatical encoding tied to word order. 23 Kinyarwanda: Abaana baasoma igitabo gishya kyose cyaa Karooli. English: The children are reading all of Charles ’s new book. Kinyarwanda: Abaana baasoma igitabo gishya kyose cyaa Karooli. English: The children are reading all of Charles ’s new book. Kinyarwanda: Abaana baasoma igitabo cyaa Karooli gishya kyose.