1. 7-Speech Recognition Speech Recognition Concepts
Speech Recognition Approaches
Recognition Theories
Bayse Rule
Simple Language Model
P(A|W) Network Types

2. 7-Speech Recognition (Cont’d)
HMM Calculating Approaches
Neural Components
Three Basic HMM Problems
Viterbi Algorithm
State Duration Modeling
Training In HMM

3. Speech Recognition Concepts
Speech recognition is inverse of Speech Synthesis
Text
Speech
Speech Synthesis
NLP
Speech
Processing
Speech
Speech
Processing
NLP
Understanding
Phone
Sequence
Text
Speech Recognition

4. Speech Recognition Approaches
Bottom-Up Approach
Top-Down Approach
Blackboard Approach

5. Voiced/Unvoiced/Silence Sound Classification Rules
Bottom-Up Approach
Signal Processing
Voiced/Unvoiced/Silence
Feature Extraction
Segmentation
Sound Classification Rules
Knowledge Sources
Signal Processing
Phonotactic Rules
Feature Extraction
Lexical Access
Segmentation
Language Model
Segmentation
Recognized Utterance

6. Inventory of speech recognition units
Top-Down Approach
Inventory of speech recognition units
Word Dictionary
Task
Model
Grammar
Unit
Matching
System
Lexical
Hypo
thesis
Syntactic
Hypo
thesis
Semantic
Hypo
thesis
Feature
Analysis
Utterance
Verifier/
Matcher
Recognized Utterance

7. Blackboard Approach Acoustic Processes Lexical Processes Black board
Environmental
Processes
Semantic
Processes
Syntactic
Processes

8. Recognition Theories Articulatory Based Recognition
Use from Articulatory system for recognition
This theory is the most successful until now
Auditory Based Recognition
Use from Auditory system for recognition
Hybrid Based Recognition
Is a hybrid from the above theories
Motor Theory
Model the intended gesture of speaker

9. Recognition Problem
We have the sequence of acoustic symbols and we want to find the words that expressed by speaker
Solution : Finding the most probable of word sequence by having Acoustic symbols

10. Recognition Problem A : Acoustic Symbols W : Word Sequence
we should find so that

11. Bayse Rule

12. Bayse Rule (Cont’d)

13. Simple Language Model
Computing this probability is very difficult and we need a very big database. So we use from Trigram and Bigram models.

14. Simple Language Model (Cont’d)
Trigram :
Bigram :
Monogram :

15. Simple Language Model (Cont’d)
Computing Method :
Number of happening W3 after W1W2
Total number of happening W1W2
AdHoc Method :

16. Recognition Tasks Isolated Word Recognition (IWR)
Connected Word (CW) , And Continuous Speech Recognition (CSR)
Speaker Dependent, Multiple Speaker, And Speaker Independent
Vocabulary Size
Small <20
Medium >100 , <1000
Large >1000, <10000
Very Large >10000

17. Error Production Factor
Prosody (Recognition should be Prosody Independent)
Noise (Noise should be prevented)
Spontaneous Speech

18. P(A|W) Computing Approaches
Dynamic Time Warping (DTW)
Hidden Markov Model (HMM)
Artificial Neural Network (ANN)
Hybrid Systems

19. Dynamic Time Warping Method (DTW)
To obtain a global distance between two speech patterns a time alignment must be performed
Ex :
A time alignment path between a template pattern “SPEECH” and a noisy input “SsPEEhH”

20. Artificial Neural Network.
Simple Computation Element
of a Neural Network

21. Artificial Neural Network (Cont’d)
Neural Network Types
Perceptron
Time Delay
Time Delay Neural Network Computational Element (TDNN)

22. Artificial Neural Network (Cont’d)
Single Layer Perceptron
. . .
. . .

23. Artificial Neural Network (Cont’d)
Three Layer Perceptron
. . .
. . .
. . .
. . .

24. Hybrid Methods
Hybrid Neural Network and Matched Filter For Recognition
Acoustic Features
Speech
Output Units
Delays
PATTERN
CLASSIFIER

25. Neural Network Properties
The system is simple, But too much iteration is needed for training
Doesn’t determine a specific structure
Regardless of simplicity, the results are good
Training size is large, so training should be offline
Accuracy is relatively good

26. Hidden Markov Model Si Sj Observation : O1,O2, . . .
States in time : q1, q2, . . .
All states : s1, s2, . . .

27. Hidden Markov Model (Cont’d)
Discrete Markov Model
Degree 1 Markov Model

28. Hidden Markov Model (Cont’d)
: Transition Probability from Si to Sj ,

29. Hidden Markov Model Example
S1 : The weather is rainy
S2 : The weather is cloudy
S3 : The weather is sunny
rainy
cloudy
sunny
rainy
cloudy
sunny

30. Hidden Markov Model Example (Cont’d)
Question 1:How much is this probability:
Sunny-Sunny-Sunny-Rainy-Rainy-Sunny-Cloudy-Cloudy

31. Hidden Markov Model Example (Cont’d)
The probability of being in state i in time t=1
Question 2:The probability of staying in a state for d days if we are in state Si?
d Days

32. HMM Components N : Number Of States M : Number Of Outputs
A : State Transfer Probability Matrix
B : Output Occurrence Probability in each state
: Primary Occurrence Probability
: Set of HMM Parameters

33. Three Basic HMM Problems
Given an HMM and a sequence of observations O,what is the probability ?
Given a model and a sequence of observations O, what is the most likely state sequence in the model that produced the observations?
Given a model and a sequence of observations O, how should we adjust model parameters in order to maximize ?

34. First Problem Solution
We Know That:
And

35. First Problem Solution (Cont’d)
Account Order :

36. Forward Backward Approach
Computing
1) Initialization

37. Forward Backward Approach (Cont’d)
2) Induction :
3) Termination :
Computation Order :

38. Backward Variable Approach
1) Initialization
2)Induction

39. Second Problem Solution
Finding the most likely state sequence
Individually most likely state :

40. Viterbi Algorithm Define :
P is the most likely state sequence with this conditions : state i , time t and observation o

41. Viterbi Algorithm (Cont’d)
1) Initialization
Is the most likely state before state i at time t-1

42. Viterbi Algorithm (Cont’d)
2) Recursion

43. Viterbi Algorithm (Cont’d)
3) Termination:
4)Backtracking:

44. Third Problem Solution
Parameters Estimation using Baum-Welch Or Expectation Maximization (EM) Approach
Define:

45. Third Problem Solution (Cont’d)
: Expectation value of the number of jumps from state i
: Expectation value of the number of jumps from state i to state j

46. Third Problem Solution (Cont’d)

47. Baum Auxiliary Function
By this approach we will reach to a local optimum

48. Restrictions Of Reestimation Formulas

49. Continuous Observation Density
We have amounts of a PDF instead of
We have
Mixture
Coefficients
Average
Variance

50. Continuous Observation Density
Mixture in HMM
M1|1
M2|1
M1|2
M2|2
M1|3
M2|3
M3|1
M4|1
M3|2
M4|2
M3|3
M4|3 S1 S2 S3
Dominant Mixture:

51. Continuous Observation Density (Cont’d)
Model Parameters:
1×N
N×M
N×M×K
N×M×K×K
N×N
N : Number Of States
M : Number Of Mixtures In Each State
K : Dimension Of Observation Vector

52. Continuous Observation Density (Cont’d)

53. Continuous Observation Density (Cont’d)
Probability of event j’th state and k’th mixture at time t

54. State Duration ModelingSi Sj
Probability of staying d times in state i :

55. State Duration Modeling (Cont’d)
HMM With clear duration
…….
……. Si Sj

56. State Duration Modeling (Cont’d)
HMM consideration with State Duration :
Selecting using ‘s
Selecting using
Selecting Observation Sequence
using
in practice we assume the following independence:
Selecting next state using transition probabilities We also have an additional constraint:

57. Training In HMM Maximum Likelihood (ML)
Maximum Mutual Information (MMI)
Minimum Discrimination Information (MDI)

58. Training In HMM
Maximum Likelihood (ML) .
Observation
Sequence

59. Training In HMM (Cont’d)
Maximum Mutual Information (MMI)
Mutual Information

60. Training In HMM (Cont’d)
Minimum Discrimination Information (MDI)
Observation :
Auto correlation :