1. Eiji Aramaki* Sadao Kurohashi* * University of Tokyo
Example-based Machine Translation using Structural Translation Examples
The title of my talk is “Example-based Machine Translation using Structural Translation Examples”
I am Eiji Aramaki from University of Tokyo.
As shown in the title, We are developing an example-based machine translation system using structural-translation-examples.
Eiji Aramaki*
Sadao Kurohashi*
* University of Tokyo

2. Proposed System
First, I will show you a short demonstration of our system.
(%%demo (sel 英語の新聞を下さい -best1)

3. Proposed System Parses an Input Sentence Selects Combines them
Structural Translation Examples
Combines them
to generate an output tree
First the system parses an input
(And) for each part of the input, the system selects Structural translation examples which are bilingual sub-tree pairs.
Then, Our system combines them to generate an output dependency tree.
Finally the system decide the word-order and output a translation.
Decides the word-order

4. Structural Translation Examples
pos
日本語の
(Japanese)
Give me
obj
新聞を
(news paper)
Japanese
下さい
(give)
newspaper
The Advantage of High-Usability
BUT: It requires many technologies
Parsing & Tree Alignment (are still being developed)
→ A naive method without such technologies may be efficient in a limited domain
The point of our system is handling structural-translation-example.
A Structural-translation-example has the potential advantage of high-usability,
However, building such translation-examples requires many-technologies, for example, parsing-technology and tree-alignment-technology and so on, which are still being developed.
So a naive method without such technologies may be efficient in a limited domain.
In such a situation, we believe that the comparison of our system and the other approach systems is meaningful.

5. Outline Algorithm Experimental Results Conclusion Alignment Module
Translation Module
Experimental Results
Conclusion
This is outline of this talk.
First, I will explain our system Algorithm.
Then, I will reports about the experimental results.
Finally, I will conclude my talk.

6. System Frame Work Alignment module Translation module
Input
Bilingual
Corpus
Translation
Memory
Alignment
module
Translation
module
Output
Alignment module
Builds Translation Examples from Bilingual Corpus
Our system consists of 2 modules. The Alignment module and Translation module.
The Alignment-module builds translation example from bilingual corpus.
The Translation-module (which I‘ve already showed), selects translation examples and combine them into a translation.
Translation module
Selects Translation Examples
Combines them into a Translation

7. Alignment Module （1/2）
A sentence pair is analyzed by parsers [Kurohashi1994][Charniak2000]
Correspondences are estimated by Dictionary-based Alignment method [Aramaki 2001]
pos
日本語の
(Japanese)
First, Alignment module analyzes bilingual sentence pairs using these parsers.
The Japanese parser outputs a phrasal dependency structure. We use it as it is.
The English parser outputs phrase structures.
We convert it into a dependency structure by using rules which decide on a head word in a phrase.
Then, the system estimates correspondences using translation-dictionaries.
We used 4 dictionaries which have about two million entries in total.
Give
(S (VP (VBP Give)
(NP (PRP me))
(NP (DT a)
(JJ Japanese)
(NN newspaper.)))) me
obj
新聞を
(news paper)
Japanese
下さい
(give)
newspaper

8. Alignment Module （2/2） Translation example
= A combinations of correspondences which are connected to each other
With Surrounding phrases (= the parent and children phrases of correspondences)
for Selection of Translation Examples
Then,
the system generates all combinations of correspondences which are connected to each other.
For example, these 6 combinations are generated from the previous sentence-pair
We regard such a combination of correspondences as a translation example.
In this operation, we also preserve its surrounding phrases which are parent and children of correspondences
The Surrounding phrases are used for selection of translation examples

9. System Frame Work Alignment module Translation module
Input
Bilingual
Corpus
Translation
Memory
Alignment
module
Translation
module
Output
Alignment module
Builds Translation Examples from Bilingual Corpus
Then, I will explain about the Translation module.
Translation module
Selects of Translation Examples
Combines them into a Translation

10. Translation Module(1/2)
INPUT
TRANSLATION EXAMPE
pos
中国語の
(Chinese)
Give me
obj
obj
新聞を
(news paper)
新聞を
(news paper)
下さい
(give)
下さい
(give)
newspaper
Equality : The number of equal phrases
First, an input sentence is analyzed by the parser.
Then, for each phrase of the input. the system selects suitable translation examples by using 3 measures.
Equality , Context similarity and Alignment confidence.
Context Similarity:
calculated with a Japanese thesaurus
Alignment Confidence:
the ratio of content words which can be found in dictionaries

11. Translation Module(1/2)
INPUT
TRANSLATION EXAMPE
pos
中国語の
(Chinese)
Give me
obj
obj
新聞を
(news paper)
新聞を
(news paper)
下さい
(give)
下さい
(give)
newspaper
Equality : The number of equal phrases
Equality is the number of phrases which are equal to the input.
The system conducts this equal check basically in content words
So, the differences of function-words are disregarded.
Context Similarity:
calculated with a Japanese thesaurus
Alignment Confidence:
the ratio of content words which can be found in dictionaries

12. Context = surrounding phrases
INPUT
TRANSLATION EXAMPE
pos
中国語の
(Chinese)
pos
日本語の
(Japanese)
Give me
obj
obj
新聞を
(news paper)
新聞を
(news paper)
Japanese
下さい
(give)
下さい
(give)
newspaper
Equality : The number of equal phrases
A Context is also an important clue for word selection.
As I‘ve mentioned before, we regard the context as the surrounding phrases of the equal part.
The context similarity between the surrounding phrases is calculated by using a Japanese thesaurus
Context Similarity:
calculated with a Japanese thesaurus
Alignment Confidence:
the ratio of content words which can be found in dictionaries

13. Translation Module(1/2)
INPUT
TRANSLATION EXAMPE
pos
中国語の
(Chinese)
pos
日本語の
(Japanese)
Give me
obj
obj
新聞を
(news paper)
新聞を
(news paper)
Japanese
下さい
(give)
下さい
(give)
newspaper
Equality : The number of equal phrases
We also take into account the alignment confidence.
We define the alignment confidence as the ratio of content words which can be found in dictionaries
Context Similarity:
calculated with a Japanese thesaurus
Alignment Confidence:
the ratio of content words which can be found in dictionaries

14. Translation Module(2/2)
Selection
Score:= ( Equality + Similarity ) x (λ+ Confidence)
Combine
The dependency relations & the word order in the translation examples are preserved
Give
Give
By using those measures, We define the score as shown in this formula.
(And) the system adapts examples which have the highest scores.
Then, the system combines them.
In this operation, The dependency relations & the word order in the translation examples are preserved.
The dependency relations & the word order between the translation examples are decided by heuristic rules me + = me
Japanese
Japanese
newspaper
newspaper
The dependency relations & the word order between the translation examples are decided by heuristic rules

15. Exception: Shortcut
Input
Bilingual
Corpus
Translation
Memory
Alignment
module
Translation
module
Output
If a Translation Example is almost equal to the input
⇒ the system outputs its target parts as it is.
Almost equal
= Character-based DP Matching Similarity > 90%
Finally, I will explain some Exceptions.
If a translation example is almost equal to the input,
The system outputs its target parts as it is.
Almost equal is Character-based DP matching similarity > 90%

16. Outline Algorism Experimental Results Conclusion Alignment Module
Translation Module
Experimental Results
Conclusion
Then, I will reports about the experimental results.

17. Experiments We built Translation Examples from training-set
(only given in IWSLT)
Auto. Eval. Result
bleu
nist
wer
per
gtm
Dev-set
0.38
7.86
0.52
0.45
0.66
Test-set
0.39
7.89
0.49
0.42
0.67
We built Translation Examples from training-set
We used only a training-corpus given in the IWSLT,
Then, we checked our system performance by development-set & test-set
(As a result), Both score are similar
it is because the system has no tuning metrics for the development-set.
Dev-set & Test-set score are similar
← the system has no tuning metrics for the dev-set.

18. Corpus size & Performance
The system without a corpus can generate translations using only the translation dictionaries.
The score is not saturated
⇒the system will achieve a higher performance if we obtain more corpora.
Then, we investigated the relation between the corpus size and its performance.
The result is shown in this Figure.
X axis indicates corpus-size & Y-axis indicates bleu.
The system without a corpus can generate translations using only the translation dictionaries.
And, The score is not saturated at this max point.
So, the system will achieve a higher performance if we obtain more corpora.

19. Subjective Evaluation
Subjective Evaluation Result
Error Analysis
Most of the errors are classified into the following three problems:
(1) Function Words
(2) Word Order
(3) Zero-pronoun
5: "Flawless English"
4: "Good English"
3: "Non-native English"
2: "Disfluent English"
1: "Incomprehensible"
Fluency
3.650
Adequacy
3.316
These are subjective evaluation results.
Fluency 3.6 & Adequacy 3.3.
It’s performance more than non-native English.
So, the system may be more fluent than me.
Why the output is not good English.
Most of the errors are classified into the following 3 problems:
Function Words, Word Order & zero-pronoun

20. Problem1: Function words
OUTPUT
i 'd like to contact my Japanese embassy
Translation
Example
I 'd like to contact my bank
The system selects translation examples using mainly content words
⇒ it sometimes generates un-natural function words
Determiners, prepositions
Because the system selects translation examples using mainly content words, it sometimes generates un-natural function words, especially in determiners and prepositions.
For example, the system generates the output ``i 'd like to contact my Japanese embassy'' using a translation example ``I 'd like to contact my bank'‘
We have to deal with such words more carefully.

21. Problem 3: Zero-pronoun
Problem 2: Word Order
OUTPUT
is there anything a like local cuisine?
The word order between translation examples is decided by the heuristic rules.
The lack of rules leads to the wrong word order.
Problem 3: Zero-pronoun
Second problem is the word-order.
The word order between translation examples is decided by the heuristic rules.
(So) The lack of rules leads to the wrong word order.
%(A target language model may be helpful for this problem.)
Third problem is zero-pronoun The input sentence sometimes includes zero-pronoun.
So, there are many outputs without pronouns.
In the future, we are planning to incorporate the zero-pronoun resolution technology.
OUTPUT
has a bad headache.
The input includes zero-pronoun.
⇒ outputs without a pronoun.

22. Outline Algorism Experimental Results Conclusion Alignment Module
Translation Module
Experimental Results
Conclusion
Finally, let me conclude the talk.

23. Conclusions
We described an EBMT system which handles Structural translation examples
The experimental results shows the basic feasibility of this approach
In the future, as the amount of corpora increases, the system will achieve a higher performance
In this research, we describe an EBMT system which handles structural translation examples.
The experimental result shows the basic feasibility of this approach.
In the future, as the amount of corpora increases, the system will achieve a higher performance.

25. Alignment Module （1/2） 日本語の 日本語の Give 新聞を 新聞を Japanese Japanese 下さい
newspaper
newspaper
日本語の
日本語の
Give
Give me
新聞を
新聞を
Japanese
Japanese
下さい
下さい
newspaper
newspaper
The National Institute for Japanese Language
Give
日本語の
Give me me
新聞を
新聞を
Japanese
下さい
下さい
newspaper
newspaper