Semantic Representation and Formal Transformation kevin knight usc/isi MURI meeting, CMU, Nov 3-4, 2011.

Semantic Representation and Formal Transformation kevin knight usc/isi MURI meeting, CMU, Nov 3-4, 2011

Machine Translation Phrase-based MT Syntax-based MT Meaning-based MT source string meaning representation target string source string target string source string source tree target tree target string source tree target tree NIST 2009 c2e

Meaning-based MT Too big for this MURI: – What content goes into the meaning representation? linguistics, annotation – How are meaning representations probabilistically generated, transformed, scored, ranked? automata theory, efficient algorithms – How can a full MT system be built and tested? engineering, language modeling, features, training

Meaning-based MT Too big for this MURI: – What content goes into the meaning representation? linguistics, annotation – How are meaning representations probabilistically generated, transformed, scored, ranked? automata theory, efficient algorithms – How can a full MT system be built and tested? engineering, language modeling, features, training Language-independent theory. But driven by practical desires.

Automata Frameworks How to represent and manipulate linguistic representations? Linguistics, NLP, and Automata Theory used to be together (1960s, 70s) – Context-free grammars were invented to model human language – Tree transducers were invented to model transformational grammar They drifted apart Renewed connections around MT (this century) Role: greatly simplify systems!

Finite-State Transducer (FST) k n i g h t q k  q2 *e* q2 n  q N q i  q AY q g  q3 *e* q4 t  qfinal T q3 h  q4 *e* Original input:Transformation: q k n i g h t FST q q2 qfinal q3q4 k : *e* n : N h : *e* g : *e* t : T i : AY

Finite-State (String) Transducer q2 n i g h t q k  q2 *e* q2 n  q N q i  q AY q g  q3 *e* q4 t  qfinal T q3 h  q4 *e* Original input:Transformation: k n i g h t FST q q2 qfinal q3q4 k : *e* n : N h : *e* g : *e* t : T i : AY

Finite-State (String) Transducer N q i g h t q k  q2 *e* q2 n  q N q i  q AY q g  q3 *e* q4 t  qfinal T q3 h  q4 *e* Original input:Transformation: k n i g h t FST q q2 qfinal q3q4 k : *e* n : N h : *e* g : *e* t : T i : AY

Finite-State (String) Transducer q g h t q k  q2 *e* q2 n  q N q i  q AY q g  q3 *e* q4 t  qfinal T q3 h  q4 *e* AY N Original input:Transformation: k n i g h t FST q q2 qfinal q3q4 k : *e* n : N h : *e* g : *e* t : T i : AY

Finite-State (String) Transducer q3 h t q k  q2 *e* q2 n  q N q i  q AY q g  q3 *e* q4 t  qfinal T q3 h  q4 *e* AY N Original input:Transformation: k n i g h t FST q q2 qfinal q3q4 k : *e* n : N h : *e* g : *e* t : T i : AY

Finite-State (String) Transducer q4 t q k  q2 *e* q2 n  q N q i  q AY q g  q3 *e* q4 t  qfinal T q3 h  q4 *e* AY N Original input:Transformation: k n i g h t FST q q2 qfinal q3q4 k : *e* n : N h : *e* g : *e* t : T i : AY

Finite-State (String) Transducer q k  q2 *e* q2 n  q N q i  q AY q g  q3 *e* q4 t  qfinal T q3 h  q4 *e* T qfinal AY N k n i g h t Original input:Transformation: FST q q2 qfinal q3q4 k : *e* n : N h : *e* g : *e* t : T i : AY

Transliteration Angela Knight a n ji ra na i to transliteration Frequently occurring translation problem for languages with different sound systems and character sets. (Japanese, Chinese, Arabic, Russian, English…) Can’t be solved by dictionary lookup.

Transliteration Angela Knight WFST 7 input symbols13 output symbols

Transliteration Angela Knight WFST B WFSA A WFST D AE N J EH L UH N AY T WFST C a n j i r a n a i t o

WFST B WFSA A WFST D WFST C a n j i r a n a i t o AE N J IH R UH N AY T AH N J IH L UH N AY T OH + millions more DECODE

General-Purpose Algorithms for String Automata N-best …… paths through an WFSA (Viterbi, 1967; Eppstein, 1998) EM trainingForward-backward EM (Baum & Welch, 1971; Eisner 2001) Determinization …… of weighted string acceptors (Mohri, 1997) IntersectionWFSA intersection Applicationstring  WFST  WFSA Transducer compositionWFST composition (Pereira & Riley, 1996) General-purpose toolkitCarmel (Graehl & Knight 97), OpenFST (Google, via AT&T),...

S NPVP PRO he VBZ enjoys NP VBG listening VP P to NP SBAR music Original input:Transformation: q S NPVP PRO he VBZ enjoys NP VBG listening VP P to NP SBAR music Top-Down Tree Transducer (W. Rounds 1970; J. Thatcher 1970)

S NPVP PRO he VBZ enjoys NP VBG listening VP P to NP SBAR music Original input:Transformation: q S NPVP PRO he VBZ enjoys NP VBG listening VP P to NP SBAR music Top-Down Tree Transducer (W. Rounds 1970; J. Thatcher 1970) q S x0:NPVP  s x0, wa, r x2, ga, q x1 x1:VBZx2:NP 0.2

S NPVP PRO he VBZ enjoys NP VBG listening VP P to NP SBAR music Original input:Transformation: s NP PRO he q VBZ enjoys r NP VBG listening VP P to NP SBAR music,, Top-Down Tree Transducer (W. Rounds 1970; J. Thatcher 1970), wa, ga

S NPVP PRO he VBZ enjoys NP VBG listening VP P to NP SBAR music Original input:Transformation: s NP PRO he q VBZ enjoys r NP VBG listening VP P to NP SBAR music,, Top-Down Tree Transducer (W. Rounds 1970; J. Thatcher 1970), wa, ga s NP PRO  kare he 0.7

S NPVP PRO he VBZ enjoys NP VBG listening VP P to NP SBAR music Original input:Transformation: q VBZ enjoys r NP VBG listening VP P to NP SBAR music, karewa, Top-Down Tree Transducer (W. Rounds 1970; J. Thatcher 1970),, ga

S NPVP PRO he VBZ enjoys NP VBG listening VP P to NP SBAR music karekikuongakuowadaisukidesugano Original input:Final output:,,,,,,,, Top-Down Tree Transducer (W. Rounds 1970; J. Thatcher 1970)

Top-Down Tree Transducer Introduced by Rounds (1970) & Thatcher (1970) “Recent developments in the theory of automata have pointed to an extension of the domain of definition of automata from strings to trees … parts of mathematical linguistics can be formalized easily in a tree-automaton setting …” (Rounds 1970, “Mappings on Grammars and Trees”, Math. Systems Theory 4(3)) Large theory literature – e.g., Gécseg & Steinby (1984), Comon et al (1997) Once again re-connecting with NLP practice – e.g., Knight & Graehl (2005), Galley et al (2004, 2006), May & Knight (2006, 2010), Maletti et al (2009)

Tree Transducers Can be Extracted from Bilingual Data (Galley et al, 04) i felt obliged to do my part 我有责任尽一份力 TREE TRANSDUCER RULES: VBD(felt)  有 VBN(obliged)  责任 VB(do)  尽 NN(part)  一份 NN(part)  一份力 VP-C(x0:VBN x1:SG-C)  x0 x1 VP(TO(to) x0:VP-C)  x0 … S(x0:NP-C x1:VP)  x0 x1 S NP-C VP VP-C VBD SG-C VP VBN TO VP-C VB NP-C NPB PRP PRP$ NN

这 7 人中包括来自法国和俄罗斯的宇航员. Syntax-Based Decoding

这 7 人中包括来自法国和俄罗斯的宇航员. RULE 1: DT(these)  这 RULE 2: VBP(include)  中包括 RULE 6: NNP(Russia)  俄罗斯 RULE 4: NNP(France)  法国 RULE 8: NP(NNS(astronauts))  宇航, 员 RULE 5: CC(and)  和 RULE 9: PUNC(.) . “these”“Russia”“astronauts”“.”“include”“France”“and” Syntax-Based Decoding

RULE 1: DT(these)  这 RULE 2: VBP(include)  中包括 RULE 6: NNP(Russia)  俄罗斯 RULE 4: NNP(France)  法国 RULE 8: NP(NNS(astronauts))  宇航, 员 RULE 5: CC(and)  和 RULE 9: PUNC(.) . 这 7 人中包括来自法国和俄罗斯的宇航员. RULE 13: NP(x0:NNP, x1:CC, x2:NNP)  x0, x1, x2 “France and Russia” “include”“these”“France”“and”“Russia”“astronauts”“.” Syntax-Based Decoding

RULE 1: DT(these)  这 RULE 2: VBP(include)  中包括 RULE 6: NNP(Russia)  俄罗斯 RULE 4: NNP(France)  法国 RULE 8: NP(NNS(astronauts))  宇航, 员 RULE 5: CC(and)  和 RULE 9: PUNC(.) . 这 7 人中包括来自法国和俄罗斯的宇航员. RULE 13: NP(x0:NNP, x1:CC, x2:NNP)  x0, x1, x2 RULE 11: VP(VBG(coming), PP(IN(from), x0:NP))  来自, x0 “France and Russia” “coming from France and Russia” “these”“Russia”“astronauts”“.”“include”“France”“&” Syntax-Based Decoding

RULE 1: DT(these)  这 RULE 2: VBP(include)  中包括 RULE 6: NNP(Russia)  俄罗斯 RULE 4: NNP(France)  法国 RULE 8: NP(NNS(astronauts))  宇航, 员 RULE 5: CC(and)  和 RULE 9: PUNC(.) . 这 7 人中包括来自法国和俄罗斯的宇航员. RULE 13: NP(x0:NNP, x1:CC, x2:NNP)  x0, x1, x2 RULE 11: VP(VBG(coming), PP(IN(from), x0:NP))  来自, x0 RULE 16: NP(x0:NP, x1:VP)  x1, 的, x0 “astronauts coming from France and Russia” “France and Russia” “coming from France and Russia” “these”“Russia”“astronauts”“.”“include”“France”“&” Syntax-Based Decoding

RULE 1: DT(these)  这 RULE 2: VBP(include)  中包括 RULE 6: NNP(Russia)  俄罗斯 RULE 4: NNP(France)  法国 RULE 8: NP(NNS(astronauts))  宇航, 员 RULE 5: CC(and)  和 RULE 9: PUNC(.) . 这 7 人中包括来自法国和俄罗斯的宇航员. RULE 13: NP(x0:NNP, x1:CC, x2:NNP)  x0, x1, x2 RULE 16: NP(x0:NP, x1:VP)  x1, 的, x0 RULE 11: VP(VBG(coming), PP(IN(from), x0:NP))  来自, x0 RULE 14: VP(x0:VBP, x1:NP)  x0, x1 “include astronauts coming from France and Russia” “France and Russia” “coming from France and Russia” “astronauts coming from France and Russia” “these”“Russia”“astronauts”“.”“include”“France”“&”

RULE 1: DT(these)  这 RULE 2: VBP(include)  中包括 RULE 6: NNP(Russia)  俄罗斯 RULE 4: NNP(France)  法国 RULE 8: NP(NNS(astronauts))  宇航, 员 RULE 5: CC(and)  和 RULE 9: PUNC(.) . 这 7 人中包括来自法国和俄罗斯的宇航员. RULE 10: NP(x0:DT, CD(7), NNS(people)  x0, 7 人 RULE 13: NP(x0:NNP, x1:CC, x2:NNP)  x0, x1, x2 RULE 15: S(x0:NP, x1:VP, x2:PUNC)  x0, x1, x2 RULE 16: NP(x0:NP, x1:VP)  x1, 的, x0 RULE 11: VP(VBG(coming), PP(IN(from), x0:NP))  来自, x0 RULE 14: VP(x0:VBP, x1:NP)  x0, x1 “These 7 people include astronauts coming from France and Russia” “France and Russia” “coming from France and Russia” “astronauts coming from France and Russia” “these 7 people” “include astronauts coming from France and Russia” “these”“Russia”“astronauts”“.”“include”“France”“&”

These7peopleincludeastronautscomingfromFranceandRussia. DTCDVBPNNSIN NNPCCNNPPUNC NP VP NP VP S NNSVBG PP NP Derived English Tree

General-Purpose Algorithms for Tree Automata String Automata Algorithms Tree Automata Algorithms N-best …… paths through an WFSA (Viterbi, 1967; Eppstein, 1998) … trees in a weighted forest (Jiménez & Marzal, 2000; Huang & Chiang, 2005) EM trainingForward-backward EM (Baum/Welch, 1971; Eisner 2003) Tree transducer EM training (Graehl & Knight, 2004) Determinization …… of weighted string acceptors (Mohri, 1997) … of weighted tree acceptors (Borchardt & Vogler, 2003; May & Knight, 2005) IntersectionWFSA intersectionTree acceptor intersection Applying transducersstring  WFST  WFSAtree  TT  weighted tree acceptor Transducer compositionWFST composition (Pereira & Riley, 1996) Many tree transducers not closed under composition (Maletti et al 09) General-purpose toolsCarmel, OpenFSTTiburon (May & Knight 10)

Machine Translation Phrase-based MT Syntax-based MT Meaning-based MT source string meaning representation target string source string target string source string source tree target tree target string source tree target tree

Five Equivalent Meaning Representation Formats (w / WANT :agent (b / BOY) :patient (g / GO :agent b))) w, b, g : instance(w, WANT) ^ instance(g, GO) ^ instance(b, BOY) ^ agent(w, b) ^ patient(w, g) ^ agent(g, b) E WANT BOY GO instance agent patient agent ((x0 instance) = WANT ((x1 instance) = BOY ((x2 instance) = GO ((x0 agent) = x1 ((x0 patent) = x2 ((x2 agent) = x1 instance: WANT agent: patient: instance: GO agent: instance: BOY 1 1 LOGICAL FORM PATH EQUATIONS FEATURE STRUCTURE DIRECTED ACYCLIC GRAPH PENMAN “The boy wants to go.”

Example “Government forces closed on rebel outposts on Thursday, showering the western mountain city of Zintan with missiles and attacking insurgents holed up near the Tunisian border, according to rebel sources.” (s / say :agent (s2 / source :mod (r / rebel)) :patient (a / and :op1 (c / close-on :agent (f / force :mod (g / government)) :patient (o / outpost :mod (r2 / rebel)) :temporal-locating (t / thursday)) :op2 (s / shower :agent f :patient (c / city :mod (m / mountain) :mod (w / west) :name "Zintan") :instrument (m2 / missile)) :op3 (a / attack :agent f :patient (i / insurgent :agent-of (h / hole-up :pp-near (b / border :gpi (c2 / country :name "Tunisia"] Slogan: “more logical than a parse tree”

General-Purpose Algorithms for Feature Structures (Graphs) String Automata Algorithms Tree Automata Algorithms Graph Automata Algorithms? N-best …… paths through an WFSA (Viterbi, 1967; Eppstein, 1998) … trees in a weighted forest (Jiménez & Marzal, 2000; Huang & Chiang, 2005) EM trainingForward-backward EM (Baum/Welch, 1971; Eisner 2003) Tree transducer EM training (Graehl & Knight, 2004) Determinization…… of weighted string acceptors (Mohri, 1997) … of weighted tree acceptors (Borchardt & Vogler, 2003; May & Knight, 2005) IntersectionWFSA intersectionTree acceptor intersection Applying transducers string  WFST  WFSAtree  TT  weighted tree acceptor Transducer composition WFST composition (Pereira & Riley, 1996) Many tree transducers not closed under composition (Maletti et al 09) General toolsCarmel, OpenFSTTiburon (May & Knight 10)

Automata Frameworks Hyperedge-replacement graph grammars – (Drewes et al) DAG acceptors – (Hart 75) DAG-to-tree transducers – (Kamimura & Slutski 82)

Mapping Between Meaning and Text the boy wants to see WANT BOY SEE instance agent patient agent foreign text

Mapping Between Meaning and Text the boy wants to be seen WANT BOY SEE instance agent patient foreign text

Mapping Between Meaning and Text the boy wants the girl to be seen WANT BOY SEE instance agent patient GIRL instance foreign text

Mapping Between Meaning and Text the boy wants to see the girl WANT BOY SEE instance agent patient GIRL instance agent foreign text

Mapping Between Meaning and Text the boy wants to see himself WANT BOY SEE instance agent patient agent foreign text

Bottom-Up DAG-to-Tree Transduction (Kamimura & Slutski 82) PROMISE BOY GO instance agent patient recipient GIRL instance agent

Bottom-Up DAG-to-Tree Transduction (Kamimura & Slutski 82) PROMISE BOY GO instance agent patient recipient GIRL instance

Bottom-Up DAG-to-Tree Transduction (Kamimura & Slutski 82) PROMISE BOY GO instance agent patient recipient instance q.NN | girl

Bottom-Up DAG-to-Tree Transduction (Kamimura & Slutski 82) PROMISE GO instance agent patient recipient instance q.NN | girl q.NN | boy

Bottom-Up DAG-to-Tree Transduction (Kamimura & Slutski 82) PROMISE instance agent patient recipient instance q.VBZ | goes q.NN | girl q.NN | boy

Bottom-Up DAG-to-Tree Transduction (Kamimura & Slutski 82) PROMISE instance agent patient recipient instance q.VB | go q.NN | girl q.NN | boy

Bottom-Up DAG-to-Tree Transduction (Kamimura & Slutski 82) instance agent patient recipient q.NN | girl instance q.VB | go q.VBD | promised q.NN | boy

General-Purpose Algorithms for Feature Structures String Automata Algorithms Tree Automata Algorithms Graph Automata Algorithms? N-best …… paths through an WFSA (Viterbi, 1967; Eppstein, 1998) … trees in a weighted forest (Jiménez & Marzal, 2000; Huang & Chiang, 2005) EM trainingForward-backward EM (Baum/Welch, 1971; Eisner 2003) Tree transducer EM training (Graehl & Knight, 2004) Determinization…… of weighted string acceptors (Mohri, 1997) … of weighted tree acceptors (Borchardt & Vogler, 2003; May & Knight, 2005) IntersectionWFSA intersectionTree acceptor intersection Applying transducers string  WFST  WFSAtree  TT  weighted tree acceptor Transducer composition WFST composition (Pereira & Riley, 1996) Many tree transducers not closed under composition (Maletti et al 09) General toolsCarmel, OpenFSTTiburon (May & Knight 10)

Automata for Statistical MT Used in SMT Automata Framework Devised 19601970198019902000 1990 2000 2010 2020 2010 FST TDTT DAG

Semantic Representation and Formal Transformation kevin knight usc/isi MURI meeting, CMU, Nov 3-4, 2011.

Similar presentations

Presentation on theme: "Semantic Representation and Formal Transformation kevin knight usc/isi MURI meeting, CMU, Nov 3-4, 2011."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Semantic Representation and Formal Transformation kevin knight usc/isi MURI meeting, CMU, Nov 3-4, 2011.

Similar presentations

Presentation on theme: "Semantic Representation and Formal Transformation kevin knight usc/isi MURI meeting, CMU, Nov 3-4, 2011."— Presentation transcript:

Similar presentations

About project

Feedback