1 BILC SEMINAR 2009 Speech Recognition: Is It for Real? Tony Mirabito Defense Language Institute English Language Center (DLIELC) DLIELC

2 OVERVIEW ● The technology evolution in language teaching ● What speech recognition is ● How speech recognition works ● Shortcomings of speech recognition ● Conclusions DLIELC

3 THE TECHNOLOGY EVOLUTION The Classroom

4 WHAT SPEECH RECOGNITION IS ● A formal definition: -- A system of spoken input into a computer in which software can “recognize” the input and transform it into digitized signals—that is, “react” in various ways to the spoken input ● Examples: -- Speech to text -- “Telephony”: airlines, transportation, etc. -- Commercial software for learning a language (e.g. Rosetta Stone) DLIELC

5 WHAT SPEECH RECOGNITION IS ● A computer program that takes verbal input and “matches” it against models—acoustic and language models ● A computer program that allows speech to be evaluated as “correct” or “acceptable” or “incorrect” or “unacceptable” ● A computer program that “talks to“ an authoring software and allows the software to branch in different directions based on the evaluation DLIELC

6 WHAT SPEECH RECOGNITION IS ● “Speaker independent” vs. “speaker dependent” -- Speaker independent: a speech recognition program that recognizes all speakers (used in language learning) -- Speaker dependent: a speech recognition program that is “trained” to recognize a particular speaker (speech to text) DLIELC

7 WHAT SPEECH RECOGNITION IS ● “Discreet speech input” vs. “continuous speech input” -- Discreet speech input: requires a user to pause between words (e.g. “I + want + to + leave.”) -- Continuous speech input: blending of sounds between words is allowed (e.g. “next + week” becomes “neksweek’) ● Cannot “understand” free speech; it “matches” speech input with stored data and pre-determined parameters DLIELC

8 HOW SPEECH RECOGNITION WORKS ● Speech recognition technology is based on the Markov Chain Theory, a mathematical formula that deals with probabilities and changes ● Most speech recognition engines contain common databases for a particular language: -- Grammar -- Lexicon (dictionary) -- Supra-segmental models (prosody) -- Acoustic speech samples DLIELC

9 HOW SPEECH RECOGNITION WORKS ● Speech is input through a microphone, analyzed by databases (“search aligners”), and then scored against a norm ● A developer can determine the score as being “acceptable” or “unacceptable”; appropriate feedback from the authoring software can be given to the user (text, audio, video, etc.) DLIELC

10 DLIELC Speech Input SR Engine Databases Authoring Software Feedback HOW SPEECH RECOGNITION WORKS ● Communication between the speech engine and the authoring software is essential That’s a table.

11 SHORTCOMINGS OF SPEECH RECOGNITION TECHNOLOGY ● Quiet environment needed plus noise- reduction microphones ● General problems with consonants ● Prosody and fluency problems which requires re-engineering the engine ● Perpetual issues: false positives false negatives ● Most recognizers are effective only in limited domains DLIELC

12 SHORTCOMINGS OF SPEECH RECOGNITION TECHNOLOGY ● If used as a diagnostic tool to pinpoint pronunciation problems, it must be carefully re-engineered to do so, and it must be accurate DLIELC

13 CONCLUSIONS ● Most speech recognizers have a couple of domains where they are effective—the recognizer should be used only in these domains (“low stakes” vs. “high stakes”) ● Speech recognition technology is not a “black box” or a “magic pill”. It’s a tool that has to be used very carefully. ● Research needs to be done in order to use a recognizer effectively ● We must never forget that technology is effective only if it allows people to learn DLIELC

14 BILC SEMINAR 2009 QUESTIONS? COMMENTS? DLIELC