Machine Translation: Challenges and Approaches Nizar Habash Post-doctoral Fellow Center for Computational Learning Systems Columbia University Invited Lecture CS 4705: Introduction to Natural Language Processing Fall 2004

Sounds like Faulkner? It lay on the table a candle burning at each corner upon the envelope tied in a soiled pink garter two artificial flowers. Not hit a man in glasses. It was once a shade, which was in all beautiful weather under a tree and varied like the branches in the wind. William Faulkner, "The sound and the fury“ Es war einmal ein Schatten, der lag bei jedem schönen Wetter unter einem Baum und schwankte wie die Zweige im Wind. Helmut Wördemann, "Der unzufriedene Schatten“ (Translated by Systran)  Faulkner  Machine Translation  Faulkner  Machine Translation

Progress in MT Statistical MT example Form a talk by Charles Wayne, DARPA Human Translation insistent Wednesday may recurred her trips to Libya tomorrow for flying Cairo 6-4 ( AFP ) - an official announced today in the Egyptian lines company for flying Tuesday is a company " insistent for flying " may resumed a consideration of a day Wednesday tomorrow her trips to Libya of Security Council decision trace international the imposed ban comment. Egyptair Has Tomorrow to Resume Its Flights to Libya Cairo 4-6 (AFP) - said an official at the Egyptian Aviation Company today that the company egyptair may resume as of tomorrow, Wednesday its flights to Libya after the International Security Council resolution to the suspension of the embargo imposed on Libya. Egypt Air May Resume its Flights to Libya Tomorrow Cairo, April 6 (AFP) - An Egypt Air official announced, on Tuesday, that Egypt Air will resume its flights to Libya as of tomorrow, Wednesday, after the UN Security Council had announced the suspension of the embargo imposed on Libya.

Road Map Why Machine Translation (MT)? Multilingual Challenges for MT MT Approaches MT Evaluation

Why (Machine) Translation? Languages in the world 6,800 living languages 600 with written tradition 95% of world population speaks 100 languages Translation Market $8 Billion Global Market Doubling every five years (Donald Barabé, invited talk, MT Summit 2003)

Why Machine Translation? Full Translation –Domain specific Weather reports Machine-aided Translation –Translation dictionaries –Translation memories –Requires post-editing Cross-lingual NLP applications –Cross-language IR –Cross-language Summarization

Road Map Why Machine Translation (MT)? Multilingual Challenges for MT –Orthographic variations –Lexical ambiguity –Morphological variations –Translation divergences MT Paradigms MT Evaluation

Multilingual Challenges Orthographic Variations –Ambiguous spelling كتب الاولاد اشعارا كَتَبَ الأوْلادُ اشعَاراً – Ambiguous word boundaries Lexical Ambiguity –Bank  بنك (financial) vs. ضفة(river) –Eat  essen (human) vs. fressen (animal)

Multilingual Challenges Morphological Variations Affixation vs. Root+Pattern write  writtenكتب  مكتوب kill  killedقتل  مقتول do  doneفعل  مفعول conj noun pluralarticle Tokenization And the cars  and the cars والسيارات  w Al SyArAt Et les voitures  et le voitures

Multilingual Challenges Translation Divergences How languages map semantics to syntax 35% of sentences in TREC El Norte Corpus (Dorr et al 2002) Divergence Types –Categorial (X tener hambre  X be hungry) [98%] –Conflational (X dar puñaladas a Z  X stab Z) [83%] –Structural (X entrar en Y  X enter Y)[35%] –Head Swapping (X cruzar Y nadando  X swim across Y)[8%] –Thematic (X gustar a Y  Y like X)[6%]

لست هنا I-am-not here be Ihere I am not here not ليس ا ناهنا Translation Divergences conflation Je ne suis pas ici I not be not here etre Jeicinepas

* ا نابردان * קרל انا بردان I cold be Icold I am coldקר לי cold for-me אני Translation Divergences categorial, thematic and structural tener Yofrio tengo frio I-have cold

swim I quickly across river I swam across the river quickly Translation Divergences head swap and categorial اسرع اناسباحةعبور نهر اسرعت عبور النهر سباحة I-sped crossing the-river swimming

swim I quickly across river I swam across the river quickly Translation Divergences head swap and categorial חצה אניבאת נהר ב שחיהמהירות חציתי את הנהר בשחיה במהירות I-crossed obj river in-swim speedily

Translation Divergences head swap and categorial חצה אניבאת נהר ב שחיהמהירות اسرع اناسباحةعبور نهر swim I quickly across river no un pre p ver b no un adve rb ver b no un ver b no un

Translation Divergences Orthography+Morphology+Syntax 妈妈的车 mama de che car mom possessed-by mom’s car سيارة ماما sayyArat mama la voiture de maman

Road Map Why Machine Translation (MT)? Multilingual Challenges for MT MT Approaches –Gisting / Transfer / Interlingua –Statistical / Symbolic / Hybrid –Practical Considerations MT Evaluation

MT Approaches MT Pyramid Source word Source syntax Source meaningTarget meaning Target syntax Target word AnalysisGeneration Gisting

MT Approaches Gisting Example Sobre la base de dichas experiencias se estableció en 1988 una metodología. Envelope her basis out speak experiences them settle at 1988 one methodology. On the basis of these experiences, a methodology was arrived at in 1988.

MT Approaches MT Pyramid Source word Source syntax Source meaningTarget meaning Target syntax Target word AnalysisGeneration GistingTransfer

MT Approaches Transfer Example Transfer Lexicon –Map SL structure to TL structure  poner X mantequilla en Y :obj :mod:subj :obj butter X Y :subj:obj X puso mantequilla en YX buttered Y

MT Approaches MT Pyramid Source word Source syntax Source meaningTarget meaning Target syntax Target word AnalysisGeneration GistingTransferInterlingua

MT Approaches Interlingua Example: Lexical Conceptual Structure (Dorr, 1993)

MT Approaches MT Pyramid Source word Source syntax Source meaningTarget meaning Target syntax Target word AnalysisGeneration Interlingua Gisting Transfer

MT Approaches MT Pyramid Source word Source syntax Source meaningTarget meaning Target syntax Target word AnalysisGeneration Interlingual Lexicons Dictionaries/Parallel Corpora Transfer Lexicons

MT Approaches Statistical vs. Symbolic Source word Source syntax Source meaningTarget meaning Target syntax Target word AnalysisGeneration

MT Approaches Noisy Channel Model Portions from

MT Approaches IBM Model (Word-based Model)

Source word Source syntax Source meaningTarget meaning Target syntax Target word AnalysisGeneration MT Approaches Statistical vs. Symbolic vs. Hybrid

Source word Source syntax Source meaningTarget meaning Target syntax Target word AnalysisGeneration MT Approaches Statistical vs. Symbolic vs. Hybrid

MT Approaches Hybrid Example: GHMT Generation-Heavy Hybrid Machine Transaltion Lexical transfer but NO structural transfer  poner Maria mantequilla en pan :obj :mod:subj :obj lay locate place put render set stand Maria butter bilberry on in into at bread loaf :obj :mod:subj :obj Maria puso la mantequilla en el pan.

MT Approaches Hybrid Example: GHMT LCS-driven Expansion Conflation Example Goal BUTTER V MARIA BREAD Agent Goal PUT V BUTTER N Theme Agent MARIA BREAD [CAUSE GO] CategorialVariation

MT Approaches Hybrid Example: GHMT Structural Overgeneration put Maria butteron bread lay Maria butterat loaf render Maria butter into loaf butter Maria bread Maria butter …

Structural N-gram Model –Long-distance –Lexemes Surface N-gram Model –Local –Surface-forms John buy MT Approaches Hybrid Example: GHMT Target Statistical Resources car ared Johnboughtcarared

MT Approaches Hybrid Example: GHMT Linearization &Ranking Maria buttered the bread Maria butters the bread Maria breaded the butter Maria breads the butter Maria buttered the loaf Maria butters the loaf Maria put the butter on bread

MT Approaches Practical Considerations Resources Availability –Parsers and Generators Input/Output compatability –Translation Lexicons Word-based vs. Transfer/Interlingua –Parallel Corpora Domain of interest Bigger is better Time Availability –Statistical training, resource building

MT Approaches Resource Poverty No Parser? No Translation Dictionary? Parallel Corpus Align with rich language Extract dictionary Parse rich side Infer parses Build a statistical parser

Road Map Why Machine Translation (MT)? Multilingual Challenges for MT MT Approaches MT Evaluation

More art than science Wide range of Metrics/Techniques –interface, …, scalability, …, faithfulness,... space/time complexity, … etc. Automatic vs. Human-based –Dumb Machines vs. Slow Humans

MT Evaluation Metrics System-based Metrics Count internal resources: size of lexicon, number of grammar rules, etc. –easy to measure –not comparable across systems –not necessarily related to utility (Church and Hovy 1993)

MT Evaluation Metrics Text-based Metrics –Sentence-based Metrics Quality: Accuracy, Fluency, Coherence, etc. 3-point scale to 100-point scale –Comprehensibility Metrics Comprehension, Informativeness, x-point scales, questionnaires most related to utility hard to measure

MT Evaluation Metrics Text-based Metrics (cont’d) –Amount of Post-Editing number of keystrokes per page not necessarily related to utility Cost-based Metrics –Cost per page –Time per page

Human-based Evaluation Example Accuracy Criteria

Human-based Evaluation Example Fluency Criteria

Fluency vs. Accuracy Accuracy Fluency conMT FAHQ MT Prof. MT Info. MT

Automatic Evaluation Example Bleu Metric Bleu –BiLingual Evaluation Understudy (Papineni et al 2001) –Modified n-gram precision with length penalty –Quick, inexpensive and language independent –Correlates highly with human evaluation –Bias against synonyms and inflectional variations

Test Sentence colorless green ideas sleep furiously Gold Standard References all dull jade ideas sleep irately drab emerald concepts sleep furiously colorless immature thoughts nap angrily Automatic Evaluation Example Bleu Metric

Test Sentence colorless green ideas sleep furiously Gold Standard References all dull jade ideas sleep irately drab emerald concepts sleep furiously colorless immature thoughts nap angrily Unigram precision = 4/5 Automatic Evaluation Example Bleu Metric

Test Sentence colorless green ideas sleep furiously Gold Standard References all dull jade ideas sleep irately drab emerald concepts sleep furiously colorless immature thoughts nap angrily Unigram precision = 4 / 5 = 0.8 Bigram precision = 2 / 4 = 0.5 Bleu Score = (a 1 a 2 …a n ) 1/n = (0.8 ╳ 0.5) ½ =  Automatic Evaluation Example Bleu Metric