1 USING CLASS WEIGHTING IN INTER-CLASS MLLR Sam-Joo Doh and Richard M. Stern Department of Electrical and Computer Engineering and School of Computer Science Carnegie Mellon University October 20, 2000

Carnegie Mellon Robust Speech Group 2 Outline Introduction Review Transformation-based adaptation Inter-class MLLR Application of weights For different neighboring classes Summary

Carnegie Mellon Robust Speech Group 3 Introduction We would like to achieve Better adaptation using small amount of adaptation data  Enhance recognition accuracy Current method Reduce the number of parameters Assume transformation function  Transformation-based adaptation Example: Maximum likelihood linear regression

Carnegie Mellon Robust Speech Group 4 Introduction (cont’d) Transformation-based adaptation  Transformation classes are assumed to be independent  It does not achieve reliable estimates for multiple classes using a small amount of adaptation data Better idea ?  Utilize inter-class relationship to achieve more reliable estimates for multiple classes

Carnegie Mellon Robust Speech Group 5 Transformation-Based Adaptation Estimate each target parameter (mean vector)

Carnegie Mellon Robust Speech Group 6 Transformation-Based Adaptation (cont’d) Estimate each transformation function

Carnegie Mellon Robust Speech Group 7 Transformation-Based Adaptation (cont’d) Trade-off Better estimation of transformation function More details of target parameters Number of transformation classes Quality

Carnegie Mellon Robust Speech Group 8 Previous Works Consider Correlations among model parameters Mostly in Bayesian framework Considering a few neighboring models:  Not effective Considering all neighboring models:  Too much computation It is difficult to apply correlation on multi-Gaussian mixtures: No explicit correspondence

Carnegie Mellon Robust Speech Group 9 Previous Works (cont’d) Using correlations among model parameters

Carnegie Mellon Robust Speech Group 10 Inter-Class Relationship Inter-class relationship among transformation functions ?

Carnegie Mellon Robust Speech Group 11 Inter-Class Relationship (cont’d) Two classes are independent Class 1 Class 2

Carnegie Mellon Robust Speech Group 12 Inter-Class Relationship (cont’d) If we know an inter-class transformation g 12 (.)  Now class 2 data contribute to the estimation of f 1 (.)  More reliable estimation of f 1 (.) while it keeps the characteristics of Class 1  Transform class 2 parameters f 2 (.) f 1 (.) g 12 (.)  2k (12) Class 1 Class 2 f 2 (.) can be estimated by transforming class 1 parameters

Carnegie Mellon Robust Speech Group 13 Use Linear Regression for inter-class transformation Estimate ( A 1, b 1 ) to minimize Q Where Inter-class MLLR

Carnegie Mellon Robust Speech Group 14 Application of Weights Neighboring classes have different contributions to the target class

Carnegie Mellon Robust Speech Group 15 Application of Weights (cont’d) Application of weights to the neighboring classes  We assume in neighboring class n  The error using (A 1, b 1 ) in neighboring class n  Weighted least squares estimation:  Use the variance of the error for weight  Large error  Small weight  Small error  Large weight

Carnegie Mellon Robust Speech Group 16 Number of Neighboring Classes Limit the number of neighboring classes  Sort neighboring classes  Set threshold for the number of samples  Use “closer” neighboring class first  Count the number of samples used  Use next neighboring classes until the number of samples exceed the threshold

Carnegie Mellon Robust Speech Group 17 Experiments Test data 1994 DARPA, Wall Street Journal (WSJ) task 10 Non-native speaker x 20 test sentences (Spoke 3: s3-94) Baseline System: CMU SPHINX-3 Continuous HMM, 6000 senones 39 dimensional features MFCC cepstra + delta + delta-delta + power Supervised/Unsupervised adaptation Focus on small amounts of adaptation data 13 phonetic-based classes for inter-class MLLR

Carnegie Mellon Robust Speech Group 18 ExperimentsExperiments (cont’d) Adaptation Method1 Adapt. Sent.3 Adapt. Sent. Baseline (No adapt)27.3% Conventional MLLR (one class)24.1%23.1% Inter-class MLLR without weights (full + shift) 20.4% (15.4%)19.6% (15.2%) Inter-class MLLR with weights (full + shift) 20.2% (16.2%)19.3% (16.5%) Supervised adaptation Word Error Rates

Carnegie Mellon Robust Speech Group 19 ExperimentsExperiments (cont’d) Adaptation Method1 Test Sent.10 Test Sent. Baseline (No adapt)27.3% Conventional MLLR (one class)26.7%23.9% Inter-class MLLR without weights (full + shift) 24.0 % (10.1%)20.1% (15.9%) Inter-class MLLR with weights (full + shift) 24.3 % (9.0%)19.9% (16.7%) Unsupervised adaptation Word Error Rates

Carnegie Mellon Robust Speech Group 20 Experiments (cont’d) Limit the number of neighboring classes Supervised adaptation: 10 adaptation sentences

Carnegie Mellon Robust Speech Group 21 Summary Application of weights Use weighted least square estimation Was helpful for supervised case Was not helpful for unsupervised case (with small amount of adaptation data) Number of neighboring classes Use smaller number of neighboring classes as more adaptation data are available

Carnegie Mellon Robust Speech Group 22 Summary Inter-class transformation It can have speaker-dependent information We may prepare several sets of inter-class transformations  Select appropriate set for a new speaker Combination with Principal Component MLLR Did not provide additional improvement

Carnegie Mellon Robust Speech Group 23 Thank you !