INCORPORATING MULTIPLE-HMM ACOUSTIC MODELING IN A MODULAR LARGE VOCABULARY SPEECH RECOGNITION SYSTEM IN TELEPHONE ENVIRONMENT A. Gallardo-Antolín, J. Ferreiros, J. Macías-Guarasa, R. de Córdoba and J.M. Pardo Grupo de Politécnica de Madrid. Tecnología del Habla. Universidad Spain {jfl, macias, cordoba, SYSTEM ARCHITECTURE Previous work on this topic at EUROSPEECH'99: Flexible Large Vocabulary (up to words) Speaker independent. Isolated word Telephone speech Two stage - bottom up strategy Using Neural Networks as a novel approach to estimate preselection list length In this paper: Integrating multiple acoustic models in this system  gender specific models SUMMARY TRAINING MULTIPLE-HMM ACOUSTIC MODELS INCORPORATING MULTIPLE-HMMs IN THE RECOGNITION STAGE EXPERIMENTAL SETUP VESTEL database realistic telephone speech corpus: Training set: 5810 utterances:  46,74 % male speakers  53,26 % female speakers Test set: 1434 utterances (vocabulary independent task) Vocabulary composed of 2000, 5000 and words CONCLUSIONS The use of multiple acoustic models per phonetic unit allows increasing acoustic modeling robustness in difficult tasks. Best results are obtained using: Training stage: Independent-training. PSBU module: Single-set. LA module: Shared Costs. A relative error reduction around 25 % is achieved when using multiple-SCHMM modeling compared to the single-SCHMM system for a dictionary of words. EXPERIMENTAL RESULTS Independent-training vs. Joint-training Discrete HMMs. Dictionary of 2000 words. Combination 2 in the recognition stage. Alternatives for PSBU and LA modules Discrete HMMs. Dictionary of 2000 words. Independent-training for multiple DHMMs. Single-SCHMM vs. Multiple-SCHMM Independent-training for multiple SCHMMs. Single-set in PSBU module + Shared-costs in LA module. Dictionary of 2000, 5000 and words. Joint training (I) Each set of models is trained using all the utterances contained in the training database. A weighting function controls the influence of each utterance in the modeling of each set. We have developed two different methods for training gender-dependent models: Independent training Each set of models is trained using only the part of the training database assigned to it. Joint training (and II) P A and P B are the likelihoods for the utterance with the set of models A and B, respectively.  A and  B are the weights to be applied to the reestimation formulae in the training stage.  is an adjustment factor, which allows to assign more training data to a particular set. Alternatives for PSBU Combined-sets Phonetic strings are composed by concatenating models coming from any set.Single-set Phonetic strings are forced to be generated by only one set of models (the one that produces the best score). Alternatives for LA Shared-costs All the allophones have the same behaviour in the LA stage, even if they have been generated from different set of models. Both sets share the same confusion matrix. Set-dependent costs Cost are gender-dependent. One square confusion matrix (for “single-set” PSBU strategy) per set. A single rectangular confusion matrix (for “combined sets”).