1. An Iterative Approach to Discriminative Structure Learning
Peng Xu
WS’2001

2. Discriminative Structure Learning Procedure
Baseline HMM training
Viterbi alignment of training data
Bivariate MI computation
Discriminate structure selection
Parameter re-estimation of new model
WS’2001

3. Problems With the Procedure
Viterbi alignment may change after training the new model
MLE for parameter estimation
Viterbi approximation restricts the MI computation
WS’2001

4. Discriminative Model Learning
Goal: minimize
D(P(Q|O)||P’(Q|O))
the divergence between the desired posterior probability distribution and the posterior probability distribution according to the model
WS’2001

5. Geometric Illustration of the Iterative Approach
Desired Posterior Distribution
Model Family
WS’2001

6. BMM Structures
Iteration n
Iteration n+1
WS’2001

7. EM Type Iterative Structure and Parameter Learning
E-step: Viterbi alignment of training data, MI computation
M-step: discriminative conditional mutual information based BMM edge detection, model parameter learning (MMI)
WS’2001

8. Improving MI Computation
Viterbi alignment:
Label for each frame is deterministic
Posterior probability P(Qt=q|xt) is a  function
Soft alignment:
Compute P(Qt=q|xt) using forward-backward algorithm
Each data frame contributes to all labels
WS’2001

9. Proposal for the Next Year
Formal formulation of the iterative model structure and parameter learning
Theoretical study of the EM type learning procedure
Implementation of the improved MI computation
Application to different data sets: Aurora, Audio-visual, etc.
WS’2001