1. Machine Translation and MT tools: Giza++ and Moses
-Nirdesh Chauhan

2. Outline Problem statement in SMT Translation models
Using Giza++ and Moses

3. Introduction to SMT
Given a sentence in foreign language F, find most appropriate translation in English E
P(F|E) – Translation model
P(E) – Language model

4. The Generation Process
Partition: Think of all possible partitions of the source language
Lexicalization: For a give partition, translate each phrase into the foreign language
Reordering: permute the set of all foreign words - words possibly moving across phrase boundaries
We need the notion of alignment to better explain mathematic behind the generation process

5. Alignment

6. Word-based alignment
For each word in source language, align words from target language that this word possibly produces
Based on IBM models 1-5
Model 1 – simplest
As we go from models 1 to 5, models get more complex but more realistic
This is all that Giza++ does

7. Alignment A function from target position to source position:
The alignment sequence is: 2,3,4,5,6,6,6
Alignment function A: A(1) = 2, A(2) = 3 ..
A different alignment function will give the sequence:1,2,1,2,3,4,3,4 for A(1), A(2)..
To allow spurious insertion, allow alignment with word 0 (NULL)
No. of possible alignments:
(I+1)J

8. IBM Model 1: Generative Process

9. IBM Model 1: Details No assumptions. Above formula is exact.
Choosing length: P(J|E) = P(J|E,I) = P(J|I) =
Choosing Alignment: all alignments equiprobable
Translation Probability

10. Training Alignment Models
Given a parallel corpora, for each (F,E) learn the best alignment A and the component probabilities:
t(f|e) for Model 1
lexicon probability P(f|e) and alignment probability P(ai|ai-1,I)
How to compute these probabilities if all you have is a parallel corpora

11. Intuition : Interdependence of Probabilities
If you knew which words are probable translation of each other then you can guess which alignment is probable and which one is improbable
If you were given alignments with probabilities then you can compute translation probabilities
Looks like a chicken and egg problem
EM algorithm comes to the rescue

12. Expectation Maximization (EM) Algorithm
Used when we want maximum likelihood estimate of the parameters of a model when the model depends on hidden variables
In present case, parameters are Translation Probabilities, and hidden Variables are alignment probabilities
Init: Start with an arbitrary estimate of parameters
E-step: compute the expected value of hidden variables
M-Step: Recompute the parameters that maximize the likelihood of data given the expected value of the hidden variables from E-step

13. Example of EM Algorithm
Green house
Casa verde
The house
La case
Init: Assume that any word can generate any word with equal prob:
P(la|house) = 1/3

14. E-Step
E-Step:

15. M-Step

16. E-Step again
1/3
2/3
2/3
1/3
Repeat till convergence

17. Limitation: Only 1->Many Alignments allowed

18. Phrase-based alignment
More natural
Many-to-one mappings allowed

19. Generating Bi-directional Alignments
Existing models only generate uni-directional alignments
Combine two uni-directional alignments to get many-to-many bi- directional alignments

20. Hindi-Eng Alignment | छुट्टियों के लिए गोवा एक प्रमुख समुद्र-तटीय
गंतव्य है
Goa | is a
premier
beach
vacation
destination

21. Eng-Hindi Alignment | छुट्टियों के लिए गोवा एक प्रमुख समुद्र-तटीय
गंतव्य है
Goa | is a
premier
beach
vacation
destination

22. Combining Alignments + | P=4/5=.8,R=4/7=.6 P=2/3=.67, R=2/7=.3
छुट्टियों के
लिए
गोवा एक
प्रमुख
समुद्र-तटीय
गंतव्य है
Goa + is a
premier |
beach
vacation
destination
P=4/5=.8,R=4/7=.6
P=2/3=.67, R=2/7=.3
P=5/6=.83,R=5/7=.7
P=6/9=.67,R=6/7=.85

23. A Different Heuristic from Moses-Site
GROW-DIAG-FINAL(e2f,f2e):
neighboring = ((-1,0),(0,-1),(1,0),(0,1),(-1,-1),(-1,1),(1,-1),(1,1))
alignment = intersect(e2f,f2e);
GROW-DIAG(); FINAL(e2f); FINAL(f2e);
GROW-DIAG():
iterate until no new points added
for english word e = en
for foreign word f = fn
if ( e aligned with f )
for each neighboring point ( e-new, f-new ):
if (( e-new, f-new ) in union( e2f, f2e ) and
( e-new not aligned and f-new not aligned ))
add alignment point ( e-new, f-new )
FINAL(a):
for english word e-new = en
for foreign word f-new = fn
if ( ( ( e-new, f-new ) in alignment a) and
( e-new not aligned or f-new not aligned ) )
Proposed Changes:
After growing diagonal
Align the shorter sentence first
And use alignments only from
corresponding directional alignment

24. Generating Phrase Alignments
छुट्टियों के
लिए
गोवा एक
प्रमुख
समुद्र-तटीय
गंतव्य है
Goa + is a
premier
beach
vacation
destination
a premier beach vacation destination
एक प्रमुख समुद्र-तटीय गंतव्य है
premier beach vacation
प्रमुख समुद्र-तटीय

25. Using Moses and Giza++
Refer to

26. Steps Install all packages in Moses
Input - sentence aligned parallel corpus
Training
Tuning
Generate output on test corpus (decoding)

27. Example train.pr train.en h e l l o hh eh l ow hh ah l ow w o r l d
c o m p o u n d w o r d
h y p h e n a t e d
o n e
b o o m
k w e e z l e b o t t e r
train.pr
hh eh l ow
hh ah l ow
w er l d
k aa m p aw n d w er d
hh ay f ah n ey t ih d
ow eh n iy
b uw m
k w iy z l ah b aa t ah r

28. Sample from Phrase-table
b o ||| b aa ||| (0) (1) ||| (0) (1) ||| b ||| b ||| (0) ||| (0) ||| c o m p o ||| aa m p ||| (2) (0,1) (1) (0) (1) ||| (1,3) (1,2,4) (0) ||| c ||| p ||| (0) ||| (0) ||| d w ||| d w ||| (0) (1) ||| (0) (1) ||| d ||| d ||| (0) ||| (0) ||| e b ||| ah b ||| (0) (1) ||| (0) (1) ||| e l l ||| ah l ||| (0) (1) (1) ||| (0) (1,2) ||| e l l ||| eh l ||| (0) (0) (1) ||| (0,1) (2) ||| e l ||| eh ||| (0) (0) ||| (0,1) ||| e ||| ah ||| (0) ||| (0) ||| h e ||| hh ah ||| (0) (1) ||| (0) (1) ||| h ||| hh ||| (0) ||| (0) ||| l e b ||| l ah b ||| (0) (1) (2) ||| (0) (1) (2) ||| l e ||| l ah ||| (0) (1) ||| (0) (1) |||
l l o ||| l ow ||| (0) (0) (1) ||| (0,1) (2) |||
l l ||| l ||| (0) (0) ||| (0,1) |||
l o ||| l ow ||| (0) (1) ||| (0) (1) |||
l ||| l ||| (0) ||| (0) |||
m ||| m ||| (0) ||| (0) |||
n d ||| n d ||| (0) (1) ||| (0) (1) |||
n e ||| eh n iy ||| (1) (2) ||| () (0) (1) |||
n e ||| n iy ||| (0) (1) ||| (0) (1) |||
n ||| eh n ||| (1) ||| () (0) |||
o o m ||| uw m ||| (0) (0) (1) ||| (0,1) (2) |||
o o ||| uw ||| (0) (0) ||| (0,1) |||
o ||| aa ||| (0) ||| (0) |||
o ||| ow eh ||| (0) ||| (0) () |||
o ||| ow ||| (0) ||| (0) |||
w o r ||| w er ||| (0) (1) (1) ||| (0) (1,2) |||
w ||| w ||| (0) ||| (0) |||

29. Testing output h o t  hh aa t p h o n e  p|UNK hh ow eh n iy
b o o k  b uw k