1. HMM - Basics

2. Content Hidden Markov Model (HMM) The Three Basic Problems for HMMs
Problem 1  Solution: Forward/ Backward Algorithm
Problem 2  Solution: Viterbi Algorithm
Problem 3  Solution: Baum- Welch Algorithm
An Overview: HMM in Speech Synthesis System
Content
HMM
Three Basic Problems
Speech System Overview

3. HMM URN 2 URN 1 URN 3 P(red) = 0.8 P(green) = 0.1 P(blue) = 0.1
Content
HMM
Three Basic Problems
Speech System Overview
URN 1
URN 3
P(red) = 0.8
P(green) = 0.1
P(blue) = 0.1
P(red) = 0.2
P(green) = 0.2
P(blue) = 0.6
P(red) = 0.5
P(green) = 0.4
P(blue) = 0.1

4. Elements of an HMM λ = (A, B, π)
N, number of states S = {S1,S2,S3, … , SN}
M, number of observation symbols
V = {v1,v2,v3, … , vM}
State transition probability distribution: A = {aij}
Observation symbol probability distribution in state j:
B = bj(k)
Initial state distribution: π = {πi}
T, number of observations in the sequence
O = O1 O2 O3… OT
HMM completely characterized by:
λ = (A, B, π)
Content
HMM
Three Basic Problems
Speech System Overview

5. Why HMM? No one-to-one mapping: speech – word symbol
Different symbols – same sound
Large variation in speech
Speaker variability
Mood
Environment
No explicit symbol boundary detection
 Speech waveform is NOT a concatenation of static patterns
Content
HMM
Three Basic Problems
Speech System Overview

6. The Three Basic Problems: Problem 1
Content
HMM
Three Basic Problems
Speech System Overview
 Solution: Forward - Algorithm

7. Forward - Algorithm Forward variable: Initialization: 2) Induction:
3) Termination:
Content
HMM
Three Basic Problems
Speech System Overview

8. The Three Basic Problems: Problem 2
Content
HMM
Three Basic Problems
Speech System Overview
 Solution: Viterbi - Algorithm

9. Viterbi- Algorithm (1) Highest probability along a single path:
Initialization
Recursion
Content
HMM
Three Basic Problems
Speech System Overview

10. Viterbi- Algorithm (2) 3) Termination 4) Path Backtracking Content HMM
Three Basic Problems
Speech System Overview

11. The Three Basic Problems: Problem 3
Content
HMM
Three Basic Problems
Speech System Overview
Solution: Baum – Welch Algorithm
(finds local maximum only)

12. Baum – Welch - Algorithm(1)
Define:
Forward/backward variable:
Content
HMM
Three Basic Problems
Speech System Overview

13. Baum- Welch- Algorithm(2)
Define:
Relation:
Content
HMM
Three Basic Problems
Speech System Overview

14. Baum- Welch- Algorithm(3)
Reestimation formulas (use iteratively to local maximum!)
Baum‘s auxiliary function:
 Derive reestimation formulas directly
Content
HMM
Three Basic Problems
Speech System Overview

15. HMM - Based Speech Synthesis System
Content
HMM
Three Basic Problems
Speech System Overview

16. References
[1] „A tutorial on Hidden Markov Models and Selected Applications in Speech Recognition“. Lawrence R. Rabiner (1989)
[2] „An HMM-Based Speech Synthesis System Applied to English“.
Keiichi Tokuda et al.
[3] Talk About HMM-Based Speech Synthesis. Keiichi Tokuda (2006)
[4] HTK Book. Cambridge University Engineering Department (2006)

18. Markov- Chain(1) Transition probability: Markov- property:
Initial state probability:
Content
Markov-Chain
HMM
Three Basic Problems
Speech System Overview

19. Markov- Chain: An Example
Content
Markov-Chain
HMM
Three Basic Problems
Speech System Overview

20. The Backward Variable Backward variable: Initialization: Induction:
Content
HMM
Three Basic Problems
Speech System Overview