1. Qiang Huo(*) and Chorkin Chan(**)
Bayesian Adaptive Learning of the Parameters of Hidden Markov Model for Speech Recognition (Maximum a Posterior, MAP)
Qiang Huo(*) and Chorkin Chan(**)
(*)Department of Computer Science The University of Hong Kong, Hong Kong (**) Department of Radio and Electronics University of Science and Technology of China,P.R.C.
Presenter：Hsu Ting-Wei
2006/02/16

2. Outline
Introduction
Maximum a Posterior (MAP) Estimate for Discrete HMM
Maximum a Posterior (MAP) Estimate for Semi-continuous HMM
Conclusion
2019/5/17
NTNU Speech Lab

3. Introduction
The widespread popularity of the HMM framework can mainly be attributed to the existence of the efficient training procedures for HMM.
HMM parameter estimators have been derived purely from the training observation sequences without any prior information included.
Baum Welch and segmental k-means are two most commonly used procedures for the estimation of HMM parameters.
Bayesian inference approach provides a convenient method for combining sample and prior information.
2019/5/17
NTNU Speech Lab

4. Introduction (cont.)
ex: ML
Prior ML
2019/5/17
NTNU Speech Lab

5. Introduction (cont.) + = 2019/5/17 NTNU Speech Lab
當 f 函式的model的參數pi,a,b假設為獨立時, 1.在DHMM中 搭配的prior function叫Dirichelet分布 2.在SCHMM中 搭配的prior fuction為 Dirichelet + Normal-Wishart分布
HMM中每個model中的states的參數的機率組合成一分布型態, 如:常態分布,高斯分布 ML
Prior
f 函式: 給定lambda下X所成分布 +
Prior: 收集許多lambda所求得之分佈， 再取log所的分布, 其中prior的參數叫 hyperparameter
Q 輔助函式
此即ML的概念, 利用EM去對Q函式估測 但估測出來的機率不可靠 =
R 輔助函式
此即MAP的概念, 再利用EM去對R函式估測 估測出來的機率較可靠, 因為Knowledge更多
2019/5/17
NTNU Speech Lab

6. MAP Estimate for Discrete HMM
Inference :
2019/5/17
NTNU Speech Lab

7. MAP Estimate for Discrete HMM (cont.)
Definition :
2019/5/17
NTNU Speech Lab

8. MAP Estimate for Discrete HMM (cont.)
Prior :
hyperparameter
2019/5/17
NTNU Speech Lab

9. MAP Estimate for Discrete HMM (cont.)
Q-function :
E Step
2019/5/17
NTNU Speech Lab

10. MAP Estimate for Discrete HMM (cont.)
Q-function :
2019/5/17
NTNU Speech Lab

11. MAP Estimate for Discrete HMM (cont.)
R-function :
2019/5/17
NTNU Speech Lab

12. MAP Estimate for Discrete HMM (cont.)
Lagrange Multiplier
M Step
Initial probability
sum=1
2019/5/17
NTNU Speech Lab

13. MAP Estimate for Discrete HMM (cont.)
Transition probability
2019/5/17
NTNU Speech Lab

14. MAP Estimate for Discrete HMM (cont.)
Observation probability
2019/5/17
NTNU Speech Lab

15. MAP Estimate for Discrete HMM (cont.)
How to choose the initial estimate for ?
One reasonable choice of the initial estimate is the mode of the prior density.
2019/5/17
NTNU Speech Lab

16. MAP Estimate for Discrete HMM (cont.)
What’s the mode ?
So applying Lagrange Multiplier we can easily derive above modes.
Example :
2019/5/17
NTNU Speech Lab

17. MAP Estimate for Discrete HMM (cont.)
Another reasonable choice of the initial estimate is the mean of the prior density.
2019/5/17
NTNU Speech Lab

18. MAP Estimate for Semi-continuous HMM
Model 1
Model 2
Model M
2019/5/17
NTNU Speech Lab

19. MAP Estimate for Semi-continuous HMM (cont.)
Definition :
mean
precision : covarience的倒數
2019/5/17
NTNU Speech Lab

20. MAP Estimate for Semi-continuous HMM (cont.)
Prior :
independent
2019/5/17
NTNU Speech Lab

21. MAP Estimate for Semi-continuous HMM (cont.)
Q-function :
E Step
2019/5/17
NTNU Speech Lab

22. MAP Estimate for Semi-continuous HMM (cont.)
Q-function :
2019/5/17
NTNU Speech Lab

23. MAP Estimate for Semi-continuous HMM (cont.)
R-function :
2019/5/17
NTNU Speech Lab

24. MAP Estimate for Semi-continuous HMM (cont.)
Initial probability
M Step
2019/5/17
NTNU Speech Lab

25. MAP Estimate for Semi-continuous HMM (cont.)
Transition probability
2019/5/17
NTNU Speech Lab

26. MAP Estimate for Semi-continuous HMM (cont.)
Mixture weight
2019/5/17
NTNU Speech Lab

27. MAP Estimate for Semi-continuous HMM (cont.)
Differentiating w.r.t and equate it to zero.
2019/5/17
NTNU Speech Lab

28. MAP Estimate for Semi-continuous HMM (cont.)
Case1: Full Covariance matrix case
2019/5/17
NTNU Speech Lab

29. MAP Estimate for Semi-continuous HMM (cont.)
2019/5/17
NTNU Speech Lab

30. MAP Estimate for Semi-continuous HMM (cont.)
2019/5/17
NTNU Speech Lab

31. MAP Estimate for Semi-continuous HMM (cont.)
2019/5/17
NTNU Speech Lab

32. MAP Estimate for Semi-continuous HMM (cont.)
Case1: Full Covariance matrix case
The initial estimate can be chosen as the mode of the prior PDF
And also can be chosen as the mean of the prior PDF
2019/5/17
NTNU Speech Lab

33. MAP Estimate for Semi-continuous HMM (cont.)
Case2: Diagonal Covariance matrix case
2019/5/17
NTNU Speech Lab

34. MAP Estimate for Semi-continuous HMM (cont.)
2019/5/17
NTNU Speech Lab

35. MAP Estimate for Semi-continuous HMM (cont.)
2019/5/17
NTNU Speech Lab

36. MAP Estimate for Semi-continuous HMM (cont.)
2019/5/17
NTNU Speech Lab

37. MAP Estimate for Semi-continuous HMM (cont.)
Case2: Diagonal Covariance matrix case
The initial estimate can be chosen as the mode of the prior PDF
And also can be chosen as the mean of the prior PDF
2019/5/17
NTNU Speech Lab

38. Conclusion The important issue of prior density is discussed.
Some application :
Model adaptation, HMM training…..
2019/5/17
NTNU Speech Lab