Arnar Thor Jensson Koji Iwano Sadaoki Furui Tokyo Institute of Technology Development of a Speech Recognition System For Icelandic Using Machine Translated Text

Overview Research introduction Research introduction Previous work Previous work Improving language model with translated text Improving language model with translated text Experimental Scenario Experimental Scenario Data (AM, LM) Data (AM, LM) Results Results Conclusion Conclusion Future work Future work

Research Introduction Icelandic weather information speech recognition system Icelandic weather information speech recognition system Speech recogniser needs large data Speech recogniser needs large data –Acoustic Model –Language Model Text data is often very hard and expensive to obtain, i.e. spontaneous speech text Text data is often very hard and expensive to obtain, i.e. spontaneous speech text

Resource Deficient Languages Resource deficient languages – –How many languages are spoken today? – –More than languages including dialects – –Icelandic (population ??????) Around people live in Iceland –The Icelandic people are very proud of their language (it is probably the largest part of our culture) –We have been trying to save it from foreign influences Computer -> Tölva

Resource Deficient Languages How can resource deficient languages be helped?? How can resource deficient languages be helped?? Resource rich languages e.g. English, Japanese Resource deficient language

Translation Methods How can resource deficient languages be helped?? How can resource deficient languages be helped?? Using translated data may be useful !! Using translated data may be useful !! –Manual Often hard work Often hard work –Machine Translation Sentence-by-sentence Sentence-by-sentence –Large parallel corpus / rule based system is needed for MT –Previous research in this area: Nakajima, H., Yamamoto, H., Watanabe, T. “Language Model Adaptation with Additional Text Generated by Machine Translation”, Proc. COLING, 2002, vol 2, pp Nakajima, H., Yamamoto, H., Watanabe, T. “Language Model Adaptation with Additional Text Generated by Machine Translation”, Proc. COLING, 2002, vol 2, pp Word-by-word Word-by-word –Only a dictionary is needed (often easy to obtain)

Previous Work Extended –This paper extends our previous paper [Jensson, 2005] –The dictionary used to translate word-by-word is now created automatically by a simple rule based machine translation system –This paper also introduces sentence-by-sentence machine translated texts from English to Icelandic –More data was used in the experiments Evaluation: 2 hours instead of 6 minutes Evaluation: 2 hours instead of 6 minutes Manually translated text was increased Manually translated text was increased Each experiment was performed three times with randomly chosen text to increase reliability Each experiment was performed three times with randomly chosen text to increase reliability

Translation Methods Sentence-by-Sentence (SBS) machine translation can be applied to any language pairs Sentence-by-Sentence (SBS) machine translation can be applied to any language pairs –Rule based machine translation –Statistical machine translation Word-by-Word (WBW) translation is expected to be useful for closely grammatically related languages Word-by-Word (WBW) translation is expected to be useful for closely grammatically related languages –English vs. French –English vs. Icelandic SVO SVO WORKS –English vs. Japanese SVO SOV DOES NOT WORK

Translation Method (WBW) Creation of a dictionary and how it is used to translate in our system Creation of a dictionary and how it is used to translate in our system Note that large text data for the target domain often exist in other languages Note that large text data for the target domain often exist in other languages

Core structure Language models from translated corpus and original text corpus interpolated together, creating a new Language Model (LM3) Language models from translated corpus and original text corpus interpolated together, creating a new Language Model (LM3) Interpolation formula Interpolation formula All language models were built using 3-grams with Kneser-Ney smoothing All language models were built using 3-grams with Kneser-Ney smoothing TRT ST Sparse Text (ST) Translated Rich Text (TRT)

Experimental Scenario English to Icelandic English to Icelandic Speech recognition experiments done with word-by-word and sentence-by-sentence (rule based) translated text in the weather information domain Speech recognition experiments done with word-by-word and sentence-by-sentence (rule based) translated text in the weather information domain The Jupiter system (a weather information system developed by MIT) was used as the English corpus The Jupiter system (a weather information system developed by MIT) was used as the English corpus Rich language English Sparse language Icelandic Evaluation Speech recognition WER Translation Method BLEU 1-gram BLEU 2-gram WBW SBS

Icelandic Acoustic Model Training: Training: Attribute Acoustic Corpus No. male speakers 13 No. female speakers 7 Time (hours) 3.8 Read speech from a bi- phonetically balanced text corpus

Icelandic Evaluation Set Attribute Acoustic Corpus No. male speakers 10 No. female speakers 10 No. utterances 4000 Time (hours) 2.0 Weather information domain Evaluation set:

Training Text Data The text data is as follows: The text data is as follows: Corpus Set SentencesWords Unique Words ST TRT WBW TRT SBS Manually created Icelandic sparse text Machine translated text from the English Jupiter (*) corpus to Icelandic * The Jupiter system, a weather information system developed by MIT

Results Baseline OOV=14.0% 6% relative WER improvement OOV=8.4% 15.5 % relative WER improvement OOV=4.4% Convergence point for the WBW MT Still improvements

Conclusion The results presented show that an LM can be improved considerably using either WBW or SBS translation The Word by Word MT is especially important for resource deficient languages that do not have SBS machine translation tools available The results presented show that a LM can be improved considerably – –WBW improves up to 6% (WER) – –SBS improves up to 15.5% (WER)

Future Work Large vocabulary speech recognition experiments Large vocabulary speech recognition experiments –We have already collected a corpus collected from 20 people in the news domain –We plan to perform both WBW and SBS MT experiments on this corpus Create a statistical machine translator (SBS) that is trained on sparse parallel text and then translate large documents to the target language Create a statistical machine translator (SBS) that is trained on sparse parallel text and then translate large documents to the target language –Perform sentence selection and adaptation methods on the machine translated corpora –Use corpora from language A to translate to language B and then finally to the target language using the WBW method –Etc.

Thank you for your attention Questions?