1. Probabilistic reasoning over time Ch. 15, 17

2. Probabilistic reasoning over time So far, we’ve mostly dealt with episodic environments –Exceptions: games with multiple moves, planning In particular, the Bayesian networks we’ve seen so far describe static situations –Each random variable gets a single fixed value in a single problem instance Now we consider the problem of describing probabilistic environments that evolve over time –Examples: robot localization, tracking, speech, …

3. Hidden Markov Models At each time slice t, the state of the world is described by an unobservable variable X t and an observable evidence variable E t Transition model: distribution over the current state given the whole past history: P(X t | X 0, …, X t-1 ) = P(X t | X 0:t-1 ) Observation model: P(E t | X 0:t, E 1:t-1 ) X0X0 E1E1 X1X1 E t-1 X t-1 EtEt XtXt … E2E2 X2X2

4. Hidden Markov Models Markov assumption (first order) –The current state is conditionally independent of all the other states given the state in the previous time step –What does P(X t | X 0:t-1 ) simplify to? P(X t | X 0:t-1 ) = P(X t | X t-1 ) Markov assumption for observations –The evidence at time t depends only on the state at time t –What does P(E t | X 0:t, E 1:t-1 ) simplify to? P(E t | X 0:t, E 1:t-1 ) = P(E t | X t ) X0X0 E1E1 X1X1 E t-1 X t-1 EtEt XtXt … E2E2 X2X2

5. state evidence Example

6. state evidence Example Transition model Observation model

7. An alternative visualization R t = TR t = F R t-1 = T0.70.3 R t-1 = F0.30.7 U t = TU t = F R t = T0.90.1 R t = F0.20.8 Transition probabilities Observation (emission) probabilities R=TR=F 0.7 0.3 U=T: 0.9 U=F: 0.1 U=T: 0.9 U=F: 0.1 U=T: 0.2 U=F: 0.8 U=T: 0.2 U=F: 0.8

8. Another example States: X = {home, office, cafe} Observations: E = {sms, facebook, email} Slide credit: Andy White

9. The Joint Distribution Transition model: P(X t | X 0:t-1 ) = P(X t | X t-1 ) Observation model: P(E t | X 0:t, E 1:t-1 ) = P(E t | X t ) How do we compute the full joint P(X 0:t, E 1:t )? X0X0 E1E1 X1X1 E t-1 X t-1 EtEt XtXt … E2E2 X2X2

10. Review: Bayes net inference Computational complexity Special cases Parameter learning

11. Review: HMMs Transition model: P(X t | X 0:t-1 ) = P(X t | X t-1 ) Observation model: P(E t | X 0:t, E 1:t-1 ) = P(E t | X t ) How do we compute the full joint P(X 0:t, E 1:t )? X0X0 E1E1 X1X1 E t-1 X t-1 EtEt XtXt … E2E2 X2X2

12. HMM inference tasks Filtering: what is the distribution over the current state X t given all the evidence so far, e 1:t ? –The forward algorithm X0X0 E1E1 X1X1 E t-1 X t-1 EtEt XtXt … EkEk XkXk Query variable Evidence variables …

13. HMM inference tasks Filtering: what is the distribution over the current state X t given all the evidence so far, e 1:t ? Smoothing: what is the distribution of some state X k given the entire observation sequence e 1:t ? –The forward-backward algorithm X0X0 E1E1 X1X1 E t-1 X t-1 EtEt … EkEk XkXk … XtXt

14. HMM inference tasks Filtering: what is the distribution over the current state X t given all the evidence so far, e 1:t ? Smoothing: what is the distribution of some state X k given the entire observation sequence e 1:t ? Evaluation: compute the probability of a given observation sequence e 1:t X0X0 E1E1 X1X1 E t-1 X t-1 EtEt … EkEk XkXk … XtXt

15. HMM inference tasks Filtering: what is the distribution over the current state X t given all the evidence so far, e 1:t Smoothing: what is the distribution of some state X k given the entire observation sequence e 1:t ? Evaluation: compute the probability of a given observation sequence e 1:t Decoding: what is the most likely state sequence X 0:t given the observation sequence e 1:t ? –The Viterbi algorithm X0X0 E1E1 X1X1 E t-1 X t-1 EtEt … EkEk XkXk … XtXt

16. HMM Learning and Inference Inference tasks –Filtering: what is the distribution over the current state X t given all the evidence so far, e 1:t –Smoothing: what is the distribution of some state X k given the entire observation sequence e 1:t ? –Evaluation: compute the probability of a given observation sequence e 1:t –Decoding: what is the most likely state sequence X 0:t given the observation sequence e 1:t ? Learning –Given a training sample of sequences, learn the model parameters (transition and emission probabilities) EM algorithm

17. Applications of HMMs Speech recognition HMMs: –Observations are acoustic signals (continuous valued) –States are specific positions in specific words (so, tens of thousands) Machine translation HMMs: –Observations are words (tens of thousands) –States are translation options Robot tracking: –Observations are range readings (continuous) –States are positions on a map (continuous) Source: Tamara Berg

18. Application of HMMs: Speech recognition “Noisy channel” model of speech

19. Speech feature extraction Acoustic wave form Sampled at 8KHz, quantized to 8-12 bits Spectrogram Time Frequency Amplitude Frame (10 ms or 80 samples) Feature vector ~39 dim.

20. Speech feature extraction Acoustic wave form Sampled at 8KHz, quantized to 8-12 bits Spectrogram Time Frequency Amplitude Frame (10 ms or 80 samples) Feature vector ~39 dim.

21. Phonetic model Phones: speech sounds Phonemes: groups of speech sounds that have a unique meaning/function in a language (e.g., there are several different ways to pronounce “t”)

22. Phonetic model

23. HMM models for phones HMM states in most speech recognition systems correspond to subphones –There are around 60 phones and as many as 60 3 context-dependent triphones

24. HMM models for words

25. Putting words together Given a sequence of acoustic features, how do we find the corresponding word sequence?

26. Decoding with the Viterbi algorithm

27. Reference D. Jurafsky and J. Martin, “Speech and Language Processing,” 2 nd ed., Prentice Hall, 2008