1. Nonparametric Bayesian Learning of Switching Dynamical Processes
Laboratory for Information and Decision Systems
Nonparametric Bayesian Learning of Switching Dynamical Processes
Emily Fox, Erik Sudderth, Michael Jordan, and Alan Willsky
Nonparametric Bayes Workshop 2008
Helsinki, Finland

2. Applications

3. Priors on Modes
= set of dynamic parameters
Switching linear dynamical processes useful for describing nonlinear phenomena
Goal: allow uncertainty in number of dynamical modes
Utilize hierarchical Dirichlet process (HDP) prior
Cluster based on dynamics
Switching Dynamical Processes

4. Outline Background HDP-AR-HMM and HDP-SLDS Sampling Techniques Results
Switching dynamical processes: SLDS, VAR
Prior on dynamic parameters
Sticky HDP-HMM
HDP-AR-HMM and HDP-SLDS
Sampling Techniques
Results
Synthetic
IBOVESPA stock index
Dancing honey bee

5. Linear Dynamical Systems
State space LTI model:
Vector autoregressive (VAR) process:

6. Linear Dynamical Systems
State space LTI model:
State space models
VAR processes
Vector autoregressive (VAR) process:

7. Switching Dynamical Systems
Switching linear dynamical system (SLDS):
Switching VAR process:

8. Prior on Dynamic Parameters
Rewrite VAR process in matrix form:
Group all observations assigned to mode k
Define the following mode-specific matrices
Place matrix-normal inverse Wishart prior on:
Results in K decoupled linear regression problems

9. Sticky HDP-HMM Time Mode Dirichlet process (DP): Hierarchical:
Infinite HMM: Beal, et.al., NIPS 2002 HDP-HMM: Teh, et. al., JASA Sticky HDP-HMM: Fox, et.al., ICML 2008
Time
Dirichlet process (DP):
Mode space of unbounded size
Model complexity adapts to observations
Hierarchical:
Ties mode transition distributions
Shared sparsity
Sticky: self-transition bias parameter
Mode

10. sparsity of b is shared, increased probability of self-transition
Sticky HDP-HMM
Global transition distribution:
Mode-specific transition distributions:
sparsity of b is shared, increased probability of self-transition

11. HDP-AR-HMM and HDP-SLDS

12. Blocked Gibbs Sampler Sample parameters Approximate HDP:
Truncate stick-breaking
Weak limit approximation:
Sample transition distributions:
Sample dynamic parameters using state sequence as VAR(1) pseudo-observations:
Fox, et.al., ICML 2008

13. Blocked Gibbs Sampler Sample mode sequence
Use state sequence as pseudo-observations of an HMM
Compute backwards messages:
Block sample as:

14. All Gaussian distributions
Blocked Gibbs Sampler
Sample state sequence
Equivalent to LDS with time-varying dynamic parameters
Compute backwards messages (backwards information filter):
Block sample as:
All Gaussian distributions

15. can be set using the data
Hyperparameters
Place priors on hyperparameters and learn them from data
Weakly informative priors
All results use the same settings
hyperparameters
can be set using the data

16. Results: Synthetic VAR(1)
HDP-VAR(1)-HMM
HDP-VAR(2)-HMM
HDP-SLDS
5-mode VAR(1) data
HDP-HMM

17. Results: Synthetic AR(2)
HDP-VAR(1)-HMM
HDP-VAR(2)-HMM
3-mode AR(2) data
HDP-SLDS
HDP-HMM

18. Results: Synthetic SLDS
HDP-VAR(1)-HMM
HDP-VAR(2)-HMM
3-mode SLDS data
HDP-SLDS
HDP-HMM

19. Results: IBOVESPA Data: Sao Paolo stock index
Daily Returns
Data: Sao Paolo stock index
Goal: detect changes in volatility
Compare inferred change-points to 10 cited world events
Carvalho and Lopes, Comp. Stat. & Data Anal., 2006
sticky HDP-SLDS
non-sticky HDP-SLDS
ROC

20. Results: Dancing Honey Bee
Sequence 1
Sequence 2
Sequence 3
Sequence 4
Sequence 5
Sequence 6
6 bee dance sequences with expert labeled dances:
Turn right (green)
Waggle (red)
Turn left (blue)
x-pos
y-pos
sinq
cosq
Observation vector:
Head angle (cosq, sinq)
x-y body position
Oh et. al., IJCV, 2007
Time

21. Movie: Sequence 6

22. Results: Dancing Honey Bee
Nonparametric approach:
Model: HDP-VAR(1)-HMM
Set hyperparameters
Unsupervised training from each sequence
Infer:
Number of modes
Dynamic parameters
Mode sequence
Supervised Approach [Oh:07]:
Model: SLDS
Set number of modes to 3
Leave one out training: fixed label sequences on 5 of 6 sequences
Data-driven MCMC
Use learned cues (e.g., head angle) to propose mode sequences
Oh et. al., IJCV, 2007

23. Results: Dancing Honey Bee
Sequence 4
Sequence 5
Sequence 6
HDP-AR-HMM: 83.2% SLDS [Oh]: 93.4%
HDP-AR-HMM: 93.2% SLDS [Oh]: 90.2%
HDP-AR-HMM: 88.7% SLDS [Oh]: 90.4%

24. Results: Dancing Honey Bee
Sequence 1
Sequence 2
Sequence 3
HDP-AR-HMM: 46.5% SLDS [Oh]: 74.0%
HDP-AR-HMM: 44.1% SLDS [Oh]: 86.1%
HDP-AR-HMM: 45.6% SLDS [Oh]: 81.3%

25. Conclusion
Examined HDP as a prior for nonparametric Bayesian learning of SLDS and switching VAR processes.
Presented efficient Gibbs sampler
Demonstrated utility on simulated and real datasets