1. Dynamic Programming Search
for Continuous Speech Recognition 2004/11/3 報告者：陳怡婷

2. Concerned Problems
Could this huge search space be handled by DP in an efficient way?
How to decide good DP search
DP computes only the single best sentence??

3. Outline Describe some problems and rule about Speech recognition
One-Pass DP Search Using a Linear Lexicon
One-Pass DP Search Using a Tree Lexicon
One-Pass DP Search for Word Graph Construction

4. Boundary of Acoustic signal or words、speakers、quality of speech signal、natural-language…
Bayes Decision Rule：(圖一所示） Pr(w1…wN | x1…xT) =>Pr(w1…wN)Pr(x1..xT|w1...wN)
Language Model：syntactic、semantic… (for large-vocabulary: bigram or trigram model)
Acoustic-phonetic Model (training、HMMs、pronunciation lexicon or dictionary)

5. (圖一 Bayes Decision Rule）

6. Specification of Search Problem
Decision on the spoken words
Language model
Acoustic-phonetic model
Pronunciation
(optimization by knowledge sources)
Thinking：a super HMM for a hypothesized word sequence

7. Consider only the most probable –Viterbi approximation
∴The search has to be performed at two level：
state level ( )
word level ( )
Recombine hypotheses at both levels by DP---beam search

9. One-Pass DP Search Using a Linear Lexicon
For three-word vocabulary ,the search space：

10. Maximum approximation：
　To assign each acoustic vector observed at time t to a (state, word) index pair.
(s1, w1),…,(st, wt),…,(sT, wT)

11. HMMs-the word interior (top)
Word boundaries (bottom) Pr(w1…wt)˙Pr(x1…xt; s1…st | w1…wt)
Note that the unknown word sequence and the unknown state sequence are determined simultaneously. ---DP algorithm

12. DP Recursion
Two quantities: Q( t, s; w) score of the best path to time that ends in state s of word
B( t, s; w) start time of the best path up to time t that ends in state s of word w
DP solved that two types of transition rules for the path－
the word interior and word boundaries.

13. In the word interior：
　Q(t, s; w) = max {P(xt, s | s’; w)˙Q(t-1, s’; w)}
　B(t, s; w) = B(t-1, smax(t, s; w); w) 　　Where smax(t, s; w) is the optimum predecessor state for the hypothesis (t, s; w).
Word boundary：
H(w; t) := max {P(w|v)˙Q(t, Sv; v)}

14. The search procedure works with a time-synchronous breadth-first strategy. (table 2)
To reduce the storage requirements－
traceback array.
A strategy－Beam Search

15. Table 2

16. One-Pass DP Search Using a three Lexicon
To large-vocabulary recognition，for efficiency reasons－Using the form of a prefix tree
How to present the search algorithm for such a context ??

17. Structure the search space as follow: ( bigram )

18. DP Recursion
Qv (t, s) := score of the best partial path that ends at time t in state s of the lexical tree for predecessor v.
Bv (t, s) := start time of the best partial path that ends at time t in state s of the lexical tree for predecessor v.

19. Qv(t, s) = max {P(wt, s | s’)˙Qv(t-1, s’)}
Bv(t, s) = Bv(t-1, svmax(t, s))
Where svmax(t, s) is the optimum predecessor state for the hypothesis (t, s) and predecessor word v.
At the word boundaries
H(w; t) := max {P(w|v)˙Qv(t, Sw)}
Qv(t-1, s=0) = H(v; t-1)
Bv(t-1, s=0) = t-1

21. Extension to Trigram Language Models
The root of each tree copy is labeled with its two-word history.
The probabilities or costs of each edge depend only on the edge itself.
Size of the potential search space is increased drastically.
Pruning strategy is even more crucial.

23. Refinements and Implementation Issues
Pruning Refinements
Acoustic pruning： QAC(t) := max{Qv(t,s)}
Qv(t,s) < fAC˙QAC(t)
Language model pruning (word end pruning) QLM(t) := max{Qv(t, s=0)}
Qv(t, s=0) < fLM˙QLM(t)
Histogram pruning

24. Language Model Look-Ahead
To incorporate the language model probabilities as early as possible into the search process
where W(s) is the set of words that can be reached from tree state s.
To incorporate the anticipated LM probabilities into the three pruning operations of the search
Why? Reduce number of state hypotheses.
Problem?

25. Fig. 11

26. Implementation
To arrive at an efficient implementation for tree search：
Set representation of active hypotheses
Set of active tree
Arc
Set of active HMM states (index s ,score Q ,back pointer)
Forward DP recombination (computational cost down)
Word boundaries、phoneme boundaries、HMM states
Direct access to each new successor hypotheses.
an exception－trigram LM.

27. Implementation Traceback and Garbage Collection
Back pointer－>a special traceback array (word index, end time of the predecessor word, score, and back pointer)
Apply a garbage collection or purging method －> extend an additional component (time stamp)

28. 報告完畢
Ps. 謝謝各位學長的指導　^^

29. Table 4

30. One-Pass DP Search For Word Graph Construction
Main idea：word alternative in regions of the speech signal.
To keep track of word sequence hypotheses whose scores are very to the optimal hypothesis, but don’t survive
To represents a word sequences by a word graph, Each word sequence should be close to single best sentence.

31. Using the same principle of time synchrony for word graph:
=Conditional probability that word w produces the acoustic vectors
=Joint probability of observing the acoustic vector and a word sequence with end time t.

32. decomposition－
where is probability of the language model
=> For construct a word graph，Introduce a formal definition of the word boundary

33. Exploiting an m-gram language model we can recombine word sequence hypotheses at phrase level if they do not differ in their final (m-1) words. =>sufficient to distinguish partial word sequence hypotheses by their final words ∴
(included pruning strategy)

34. Word Pair Approximation
The crucial assumption now is that the dependence of the word boundary can be confined to the final word pair => or
Assuming the word pair approximation－
At every time frame t, consider all word pairs =(v,w)
For each triple(t ;v ,w), keep track of word boundary 、word score
At end of the speech signal word grapy is constructed by…...

35. Fig 13

36. How to computation ?

37. table5

38. A third level－the phrase level
Depending on whether the phrase-level recognition is carried out in a time-synchronous.
Extended one-pass approach
Two-pass approach : cache-based language model

39. Fig14

40. Principal properties：
There is a maximum for the number of incoming word edges in any node, namely the vocabulary size.
There is no maximum for the number of outgoing word edges.
Two refinements of word graph method：
Short words
Long words with identical ending portions may waste esearch effort.

41. table6

42. table8