1. HMM Model Structure Presentation by Durga Yeluri

2. What is the problem? So far our assumptions Transitions are possible from any state to any other state Fully connected model “let the model find out itself” Result of our assumptions Bad model for any realistic problem, even with plenty of training data local maxima, but not over fitting Less constrained model  More local maxima Solution ??

3. Model Topology Construction based on Which transitions are allowed Knowledge about the problem Disable transition from state k to state l by setting a kl = 0. If the probability is zero, the number of transitions are also zero. Two types of modeling Duration Modeling Silent States Modeling

4. Duration Modeling When there is no change in the distribution for a certain length of the sequence Probability of a state transition to itself is p. Probability of leaving the state is (1-p). P(l residues) = (1-p)p l-1

5. Example This model gives a minimum of 5 residues.

6. Example This can model any distribution with length in between 2 and 6.

7. Example of Non-geometric length Distribution Array of n states, smallest sequence length n. Probability of path with length l is p l-n (1-p) n. The number of possible paths with length l is (l- 1)choose(n-1).

8. Contd.. Total probability of all possible paths is P (l) = (l-1) choose (n-1) p l-n (1-p) n. This is called negative binomial distribution.

9. Silent States States which do not emit symbols in HMM Examples are begin and end states Also called null states Very useful in reducing the number of transitions in HMMs Leads to reduction in the number of parameters

10. Example Total number of transitions are n(n+1)/2 for n states

11. Reduction in the number of transitions with silent states Total number of transitions for n states is nearly 3n.

12. Discussion Total number of transitions with a length L in a forward connected model with out silent states?? With silent states??

13. Thank You!!!