1. The Linear Dynamic System and its application in moction capture
Lei Li
Computer Science Department
Carnegie Mellon University

2. Working example
Given a sequence of motion capture data, to estimate the velocity and acceleration of each joint at every time stamp
Scenario:
observation with noise
Application:
Computer vision: object tracking
Control: estimate missile trajectory
Motion capture data:
Other time series: economics data

3. Name Convention Developed by Rudolf Kalman in 1961
Also called linear dynamic system
Could be easily interpreted using probabilistic graphical model
Similar to Hidden Markov Model

4. Prerequisit A bit probability, esp. Gaussian Distribution
A bit linear algebra and matrix computation
A bit about Expectation Maximization

5. The Linear Dynamic System
N(u0, V0)
N(A∙z1, Γ)
N(A∙zn, Γ) z1 z2 zn
zn+1
N(C∙z1, Σ)
N(C∙z2, Σ)
N(C∙zn, Σ) x1 x2 xn
xn+1
xn : observed position(e.g. x coordinate of left hand), one dimensional
zn : hidden variable(e.g. actual pos, velocity, acceleration), three dimensional

6. The Linear Dynamic System
N(u0, V0)
N(A∙z1, Γ)
N(A∙zn, Γ) z1 z2 zn
zn+1
N(C∙z1, Σ)
N(C∙z2, Σ)
N(C∙zn, Σ) x1 x2 xn
xn+1
A : transition matrix
Γ: transition variance, noises among transition

7. The Linear Dynamic System
N(u0, V0)
N(A∙z1, Γ)
N(A∙zn, Γ) z1 z2 zn
zn+1
N(C∙z1, Σ)
N(C∙z2, Σ)
N(C∙zn, Σ) x1 x2 xn
xn+1
C : transmission matrix
Σ: transmission variance, observation noises

8. Example: motion capture data

9. Inference
Given the parameters and the observation, estimate the posterior P(zi|x1, … xN;u0,V0,A, Γ,C,Σ )
Solution: forward-backward algorithm(sum-product)

10. Forward The posterior given the sequence up to current observation
Define: alpha(zn) = P(zn|x1, … xn) = assume N(un, Vn)
N(u0, V0)
N(A∙z1, Γ)
N(A∙zn, Γ) z1 z2 zn
zn+1
N(C∙z1, Σ)
N(C∙z2, Σ)
N(C∙zn, Σ) x1 x2 xn
xn+1

11. Forward Define: cn+1 = P(xn+1|x1, … xn) N(u0, V0) N(A∙z1, Γ)
N(A∙zn, Γ) z1 z2 zn
zn+1
N(C∙z1, Σ)
N(C∙z2, Σ)
N(C∙zn, Σ) x1 x2 xn
xn+1

12. Forward Cn ∙ alpha(zn) = P(xn,zn|x1, … xn-1 )
= P(xn|zn) ∫ alpha(zn-1) ∙P(zn|zn-1) dzn-1
= N(C∙zn,Σ) ∫alpha(zn-1)∙N(A∙zn-1,Γ) dzn-1
N(un, Vn) N(un, Vn)
Could get the close solution for the forward recursion:

13. Backward There is a similar backward recursion:
The posterior given the whole observation sequence
Gamma(zn) = P(zn|x1, … xN)
= ∫P(zn+1|x1, … xN) ∙ P(zn|zn+1 , x1, … xn) dzn+1
=∫ Gamma(zn+1) ∙ P(zn|zn+1 , x1, … xn) dzn+1
N(wn, Wn) N(wn+1, Wn+1)

14. Learning the Parameters
Given the observation, estimate the most likely parameters
Maximize Expected Log-likehood
J = E[ log P(z1, … zN , x1, … xN|u0,V0,A, Γ,C,Σ )]
J is a convex function, could get close solution

15. A case study: Natural Motion Stitching
Problem:
Given two motion-capture sequences that are to be stitched together, how can we assess the goodness of the stitching?
Idea:
Naive solution: Euclidean distance
Dynamics based: compute the effort by approximating the work of making the intended transition

16. Motion stitching: instantaneous stitching

17. Motion stitching: elastic stitching

18. Result:

19. Result

20. Conclusion: Review the Kalman filter
Review the forward-backward algorithm
Application to motion capture data

21. Thank you