Siddiqi and Moore, www.autonlab.org Fast Inference and Learning in Large-State-Space HMMs Sajid M. Siddiqi Andrew W. Moore The Auton Lab Carnegie Mellon.

Siddiqi and Moore, www.autonlab.org Fast Inference and Learning in Large-State-Space HMMs Sajid M. Siddiqi Andrew W. Moore The Auton Lab Carnegie Mellon University

Siddiqi and Moore, www.autonlab.org Sajid Siddiqi: Happy Sajid Siddiqi: Discontented

Siddiqi and Moore, www.autonlab.org Hidden Markov Models 1/3 1 q0q0 q1q1 q2q2 q3q3 q4q4

Siddiqi and Moore, www.autonlab.org Each of these probability tables is identical i P( q t+1 =s 1 |q t = s i ) P( q t+1 =s 2 |q t = s i )… P( q t+1 =s j |q t = s i )… P( q t+1 =s N |q t = s i ) 1 a 11 a 12 … a 1j … a 1N 2 a 21 a 22 … a 2j … a 2N 3 a 31 a 32 … a 3j … a 3N ::::::: i a i1 a i2 … a ij … a iN N a N1 a N2 … a Nj … a NN Hidden Markov Models 1/3 1 q0q0 q1q1 q2q2 q3q3 q4q4 Notation:

Siddiqi and Moore, www.autonlab.org Observation Model q0q0 q1q1 q2q2 q3q3 q4q4 O0O0 O1O1 O2O2 O3O3 O4O4

Siddiqi and Moore, www.autonlab.org Observation Model q0q0 q1q1 q2q2 q3q3 q4q4 O0O0 O1O1 O2O2 O3O3 O4O4 i P( O t =1 |q t = s i ) P( O t =2 |q t = s i )… P( O t =k |q t = s i )… P( O t =M |q t = s i ) 1 b 1 (1)b 1 (2) … b 1 (k) … b 1 (M) 2 b 2 (1)b 2 (2) … b 2 (k) … b 2 (M) 3 b 3 (1)b 3 (2) … b 3 (k) … b 3 (M) : :::::: i b i (1)b i (2) … b i (k) … b i (M) : :::::: N b N (1)b N (2) … b N (k) … b N (M) Notation:

Siddiqi and Moore, www.autonlab.org Some Famous HMM Tasks Question 1: State Estimation What is P(q T =S i | O 1 O 2 …O T )

Siddiqi and Moore, www.autonlab.org Question 1: State Estimation What is P(q T =S i | O 1 O 2 …O T ) Some Famous HMM Tasks

Siddiqi and Moore, www.autonlab.org Question 1: State Estimation What is P(q T =S i | O 1 O 2 …O T ) Question 2: Most Probable Path Given O 1 O 2 …O T, what is the most probable path that I took? Some Famous HMM Tasks

Siddiqi and Moore, www.autonlab.org Question 1: State Estimation What is P(q T =S i | O 1 O 2 …O T ) Question 2: Most Probable Path Given O 1 O 2 …O T, what is the most probable path that I took? Some Famous HMM Tasks Woke up at 8.35, Got on Bus at 9.46, Sat in lecture 10.05-11.22…

Siddiqi and Moore, www.autonlab.org Some Famous HMM Tasks Question 1: State Estimation What is P(q T =S i | O 1 O 2 …O T ) Question 2: Most Probable Path Given O 1 O 2 …O T, what is the most probable path that I took? Question 3: Learning HMMs: Given O 1 O 2 …O T, what is the maximum likelihood HMM that could have produced this string of observations?

Siddiqi and Moore, www.autonlab.org Some Famous HMM Tasks Question 1: State Estimation What is P(q T =S i | O 1 O 2 …O T ) Question 2: Most Probable Path Given O 1 O 2 …O T, what is the most probable path that I took? Question 3: Learning HMMs: Given O 1 O 2 …O T, what is the maximum likelihood HMM that could have produced this string of observations? Eat Bus walk a AB a BB a AA a CB a BA a BC a CC O t-1 O t+1 OtOt b A (O t-1 ) b B (O t ) b C (O t+1 )

Siddiqi and Moore, www.autonlab.org Basic Operations in HMMs For an observation sequence O = O 1 …O T, the three basic HMM operations are: ProblemAlgorithmComplexity + Evaluation: Calculating P(O| ) Forward-Backward O(TN 2 ) Inference: Computing Q * = argmax Q P(O,Q| ) Viterbi Decoding O(TN 2 ) Learning: Computing * = argmax P(O|  Baum-Welch (EM) O(TN 2 ) T = # timesteps, N = # states

Siddiqi and Moore, www.autonlab.org Basic Operations in HMMs For an observation sequence O = O 1 …O T, the three basic HMM operations are: ProblemAlgorithmComplexity + Evaluation: Calculating P(O| ) Forward-Backward O(TN 2 ) Inference: Computing Q * = argmax Q P(O,Q| ) Viterbi Decoding O(TN 2 ) Learning: Computing * = argmax P(O|  Baum-Welch (EM) O(TN 2 ) T = # timesteps, N = # states This talk: A simple approach to reducing the complexity in N

Siddiqi and Moore, www.autonlab.org Reducing Quadratic N penalty Why does it matter? Quadratic HMM algorithms hinder HMM computations when N is large Several promising applications for efficient large-state-space HMM algorithms in biological sequence analysis speech recognition real-time HMM systems such as for activity monitoring

Siddiqi and Moore, www.autonlab.org Idea One: Sparse Transition Matrix Only K << N non- zero next-state probabilities

Siddiqi and Moore, www.autonlab.org Idea One: Sparse Transition Matrix Only K << N non- zero next-state probabilities Only O(TNK)

Siddiqi and Moore, www.autonlab.org Idea One: Sparse Transition Matrix Only K << N non- zero next-state probabilities But can get very badly confused by “impossible transitions” Cannot learn the sparse structure (once chosen cannot change) Only O(TNK)

Siddiqi and Moore, www.autonlab.org Dense-Mostly-Constant Transitions  K non-constant probabilities per row  DMC HMMs comprise a richer and more expressive class of models than sparse HMMs a DMC transition matrix with K=2

Siddiqi and Moore, www.autonlab.org Dense-Mostly-Constant Transitions The transition model for state i now comprises: NC i = { j : s i  s j is a non-constant transition probability } c i = the transition probability for s i to all states not in NC i a ij = the non-constant transition probability for s i  s j, NC 3 = {2,5} c 3 = 0.05 a 32 = 0.25 a 35 = 0.6

Siddiqi and Moore, www.autonlab.org HMM Filtering P(q t = s i | O 1, O 2 … O t )

Siddiqi and Moore, www.autonlab.org HMM Filtering P(q t = s i | O 1, O 2 … O t ) = Where

Siddiqi and Moore, www.autonlab.org HMM Filtering P(q t = s i | O 1, O 2 … O t ) = Where t  t (1)  t (2)  t (3) …  t (N) 1 2… 3 4 5 6 7 8 9

Siddiqi and Moore, www.autonlab.org HMM Filtering P(q t = s i | O 1, O 2 … O t ) = Where t  t (1)  t (2)  t (3) …  t (N) 1 2… 3… 4 5 6 7 8 9

Siddiqi and Moore, www.autonlab.org HMM Filtering P(q t = s i | O 1, O 2 … O t ) = Where t  t (1)  t (2)  t (3) …  t (N) 1 2… 3… 4 5 6 7 8 9 Cost O(TN 2 )

Siddiqi and Moore, www.autonlab.org Fast Evaluation in DMC HMMs

Siddiqi and Moore, www.autonlab.org Fast Evaluation in DMC HMMs O(N), but common to all j per timestep t O(K) for each  t ( j )  This yields O(TNK) complexity for the evaluation problem.

Siddiqi and Moore, www.autonlab.org The Viterbi algorithm uses dynamic programming to calculate the globally optimal state sequence Q g =max Q P(Q,O| ). Fast Inference in DMC HMMs Define  t (i) as The  variables can be computed in O(TN 2 ) time, with the O(N) inductive step: Under the DMC assumption, this step can be carried out in O(K) time: O(N), but common to all j per timestep t O(K) for each  t (j)

Siddiqi and Moore, www.autonlab.org Learning a DMC HMM

Siddiqi and Moore, www.autonlab.org Learning a DMC HMM Idea One: Ask user to tell us the DMC structure Learn the parameters using EM

Siddiqi and Moore, www.autonlab.org Learning a DMC HMM Idea One: Ask user to tell us the DMC structure Learn the parameters using EM Simple But in general, don’t know the DMC structure

Siddiqi and Moore, www.autonlab.org Learning a DMC HMM Idea Two: Use EM to learn the DMC structure too 1.Guess DMC structure 2.Find expected transition counts and observation parameters, given current model and observations 3.Find maximum likelihood DMC model given counts 4.Goto 2

Siddiqi and Moore, www.autonlab.org Learning a DMC HMM Idea Two: Use EM to learn the DMC structure too 1.Guess DMC structure 2.Find expected transition counts and observation parameters, given current model and observations 3.Find maximum likelihood DMC model given counts 4.Goto 2 DMC structure can (and does) change!

Siddiqi and Moore, www.autonlab.org Learning a DMC HMM Idea Two: Use EM to learn the DMC structure too 1.Guess DMC structure 2.Find expected transition counts and observation parameters, given current model and observations 3.Find maximum likelihood DMC model given counts 4.Goto 2 DMC structure can (and does) change! In fact, just start with an all-constant transition model

Siddiqi and Moore, www.autonlab.org Learning a DMC HMM 2.Find expected transition counts and observation parameters, given current model and observations

Siddiqi and Moore, www.autonlab.org We wantnew estimate of

Siddiqi and Moore, www.autonlab.org We wantnew estimate of where

Siddiqi and Moore, www.autonlab.org We want where

Siddiqi and Moore, www.autonlab.org T N  T N  We want where

Siddiqi and Moore, www.autonlab.org T N  T N  Can get this in O(TN) time We want where

Siddiqi and Moore, www.autonlab.org  We wantwhere T N T N  Can get this in O(TN) time

Siddiqi and Moore, www.autonlab.org  We want where T N T N 

Siddiqi and Moore, www.autonlab.org  We want where T N T N  S N N S 24  *2  *4 Dot Product of Columns

Siddiqi and Moore, www.autonlab.org  We want where T N T N  S N N S 24  *2  *4 Dot Product of Columns O(TN 2 )

Siddiqi and Moore, www.autonlab.org  We want where T N T N  S N N S 24  *2  *4 Dot Product of Columns O(TN 2 ) Speedups: Strassen?

Siddiqi and Moore, www.autonlab.org  We want where T N T N  S N N S 24  *2  *4 Dot Product of Columns O(TN 2 ) Speedups: Strassen Approximate  by DMC

Siddiqi and Moore, www.autonlab.org  We want where T N T N  S N N S 24  *2  *4 Dot Product of Columns O(TN 2 ) Speedups: Strassen Approximate  by DMC Approximate randomized A T B

Siddiqi and Moore, www.autonlab.org  We want where T N T N  S N N S 24  *2  *4 Dot Product of Columns O(TN 2 ) Speedups: Strassen Approximate  by DMC Approximate randomized A T B Sparse structure fine?

Siddiqi and Moore, www.autonlab.org  We want where T N T N  S N N S 24  *2  *4 Dot Product of Columns O(TN 2 ) Speedups: Strassen Approximate  by DMC Approximate randomized A T B Sparse structure fine Fixed DMC is fine?

Siddiqi and Moore, www.autonlab.org  We want where T N T N  S N N S 24  *2  *4 Dot Product of Columns O(TN 2 ) Speedups: Strassen Approximate  by DMC Approximate randomized A T B Sparse structure fine Fixed DMC is fine Speedup without approximation

Siddiqi and Moore, www.autonlab.org  We want where T N T N  S N N S 24 Insight One: only need the top K entries in each row of S Insight Two: Values in rows of  and  are often very skewed

Siddiqi and Moore, www.autonlab.org  T NN   -biggies(i)  -biggies(j) For i = 1..N, store indexes of R largest values in i’th column of  For j = 1..N, store indexes of R largest values in j’th column of  There’s an important detail I’m omitting here to do with prescaling the rows of  and .

Siddiqi and Moore, www.autonlab.org  T NN   -biggies(i)  -biggies(j) For i = 1..N, store indexes of R largest values in i’th column of  For j = 1..N, store indexes of R largest values in j’th column of  R << T Takes O(TN) time to do all indexes There’s an important detail I’m omitting here to do with prescaling the rows of  and .

Siddiqi and Moore, www.autonlab.org  T NN   -biggies(i)  -biggies(j) For i = 1..N, store indexes of R largest values in i’th column of  For j = 1..N, store indexes of R largest values in j’th column of  R << T Takes O(TN) time to do all indexes R’th largest value in i’th column of  O(1) time to obtain O(1) time to obtain (precached for all j in time O(TN) ) O(R) computation There’s an important detail I’m omitting here to do with prescaling the rows of  and .

Siddiqi and Moore, www.autonlab.org S N j 123N… S ij Computing the i’th row of S… In O(NR) time, we can put upper and lower bounds on S ij for j = 1,2.. N

Siddiqi and Moore, www.autonlab.org S N j 123N… S ij Computing the i’th row of S… In O(NR) time, we can put upper and lower bounds on S ij for j = 1,2.. N Only need exact values of S ij for the k largest values within the row

Siddiqi and Moore, www.autonlab.org S N j 123N… S ij Computing the i’th row of S… In O(NR) time, we can put upper and lower bounds on S ij for j = 1,2.. N Only need exact values of S ij for the k largest values within the row Ignore j’s that can’t be the best

Siddiqi and Moore, www.autonlab.org S N j 123N… S ij Computing the i’th row of S… In O(NR) time, we can put upper and lower bounds on S ij for j = 1,2.. N Only need exact values of S ij for the k largest values within the row Ignore j’s that can’t be the best Be exact for the rest: O(N) time each.

Siddiqi and Moore, www.autonlab.org S N j 123N… S ij Computing the i’th row of S… In O(NR) time, we can put upper and lower bounds on S ij for j = 1,2.. N Only need exact values of S ij for the k largest values within the row Ignore j’s that can’t be the best Be exact for the rest: O(N) time each. If there’s enough pruning, total time is O(TN+RN 2 )

Siddiqi and Moore, www.autonlab.org Evaluation and Inference Speedup Dataset: synthetic data with T=2000 time steps

Siddiqi and Moore, www.autonlab.org Parameter Learning Speedup Dataset: synthetic data with T=2000 time steps

Siddiqi and Moore, www.autonlab.org Performance Experiments DMC-friendly dataset: 2-D gaussian 20-state DMC HMM with K=5 (20,000 train, 5,000 test) Anti-DMC dataset: 2-D gaussian 20-state regular HMM with steadily varying, well-distributed transition probabilities (20,000 train, 5,000 test) Motionlogger dataset: Accelerometer data from two sensors worn over several days (10,000 train, 4,720 test) Regular and DMC HMMs: 20 states Small HMM: 5-state regular HMM Uniform HMM: 20-state HMM with uniform transition probabilities

Siddiqi and Moore, www.autonlab.org Learning Curves for DMC-friendly data

Siddiqi and Moore, www.autonlab.org Learning Curves for Anti-DMC data

Siddiqi and Moore, www.autonlab.org Learning Curves for Motionlogger data

Siddiqi and Moore, www.autonlab.org Tradeoffs between N and K We vary N and K while keeping the number of transition parameters (N×K) constant Increasing N and decreasing K allows more states for modeling data features but fewer parameters per state for temporal structure

Siddiqi and Moore, www.autonlab.org Tradeoffs between N and K Average test-set log-likelihoods at convergence Datasets: A: DMC-friendly B: Anti-DMC C: Motionlogger Each dataset has a different optimal N-vs-K tradeoff

Siddiqi and Moore, www.autonlab.org Regularization with DMC HMMs # of transition parameters in regular 100-state HMM: 10,000 # of transition parameters in DMC 100-state HMM with K= 5 : 500

Siddiqi and Moore, www.autonlab.org Conclusions DMC HMMs are an important class of models that allow parameterized complexity-vs- efficiency tradeoffs in large state spaces The speedup can be several orders of magnitude Even for non-DMC domains, DMC HMMs yield higher scores than baseline models The DMC HMM model can be applied to arbitrary state spaces and observation densities

Siddiqi and Moore, www.autonlab.org Related Work Felzenszwalb et al. (2003) – fast HMM algorithms when transition probabilities can be expressed as distances in an underlying parameter space Murphy and Paskin (2002) – fast inference in hierarchical HMMs cast as DBNs Salakhutdinov et al. (2003) – combined EM and conjugate gradient for faster HMM learning when missing information amount is high Beam Search – widely used heuristic in word recognition for speech systems

Siddiqi and Moore, www.autonlab.org Future Work Investigate DMC HMMs as regularization mechanism Eliminate R parameter using an automatic backoff evaluation approach Devise ways to automatically set K parameter, have per-row K parameters

Siddiqi and Moore, www.autonlab.org Future Work Investigate DMC HMMs as regularization mechanism Eliminate R parameter using an automatic backoff evaluation approach Devise ways to automatically set K parameter, have per-row K parameters The End

Siddiqi and Moore, www.autonlab.org

Siddiqi and Moore, www.autonlab.org Fast Inference and Learning in Large-State-Space HMMs Sajid M. Siddiqi Andrew W. Moore The Auton Lab Carnegie Mellon.

Similar presentations

Presentation on theme: "Siddiqi and Moore, www.autonlab.org Fast Inference and Learning in Large-State-Space HMMs Sajid M. Siddiqi Andrew W. Moore The Auton Lab Carnegie Mellon."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Siddiqi and Moore, www.autonlab.org Fast Inference and Learning in Large-State-Space HMMs Sajid M. Siddiqi Andrew W. Moore The Auton Lab Carnegie Mellon.

Similar presentations

Presentation on theme: "Siddiqi and Moore, www.autonlab.org Fast Inference and Learning in Large-State-Space HMMs Sajid M. Siddiqi Andrew W. Moore The Auton Lab Carnegie Mellon."— Presentation transcript:

Similar presentations

About project

Feedback