1. Variational filtering in generated coordinates of motion
Approximate Inference in Stochastic Processes and Dynamical Systems
Abstract
This presentation reviews variational treatments of dynamic models that furnish time-dependent conditional densities on the path or trajectory of a system's states and the time-independent densities of its parameters. These obtain by maximizing a variational action with respect to conditional densities. The action or path-integral of free-energy represents a lower-bound on the model’s log-evidence or marginal likelihood required for model selection and averaging. This approach rests on formulating the optimization in generalized co-ordinates of motion. The resulting scheme can be used for on-line Bayesian inversion of nonlinear dynamic causal models and is shown to outperform existing approaches, such as Kalman and particle filtering. Furthermore, it provides for multiple inference on a models states, parameters and hyperparameters using exactly the same principles. Free-form (Variational filtering) and fixed form (Dynamic Expectation Maximization) variants of the scheme will be demonstrated using simulated (bird-song) and real data (from hemodynamic systems studied in neuroimaging).

2. Overview Dynamic causal models
Generalised coordinates (dynamical priors)
Hierarchal forms (structural priors)
Variational filtering and action (free-form)
Laplace approximation and DEM (fixed-form)
Comparative evaluations
Hemodynamics
Bird songs
notation

3. Hierarchical dynamic causal models

4. and generalised coordinates
Dynamic models
and generalised coordinates
likelihood
prior

5. Energies and generalised precisions
Instantaneous energy
General and Gaussian forms
Precision matrices in generalised coordinates and time

6. Hierarchal forms and empirical priors
Dynamic priors
Structural priors

7. Overview Dynamic causal models
Generalised coordinates (dynamical priors)
Hierarchal forms (structural priors)
Variational filtering and action (free-form)
Laplace approximation and DEM (fixed-form)
Comparative evaluations
Hemodynamics
Bird songs

8. Variational learning (steady-state)
Aim: To optimise a free-energy bound on model evidence
Free-energy:
Expected energy:
Entropy:
Lemma 1: The free energy is maximised with respect to when
Variational density:
Mean-field approximation:
Variational energy:
Internal energy:

9. Ensemble learning (steady-state)
Let the equations of motion for each particle be
Because particles are conserved, their density over parameter space is governed by the free energy Fokker-Plank equation
It is trivial to show that the stationary solution for the ensemble density is the variational density by substitution

10. Variational learning (dynamic)
In a dynamic setting, the variational density and energy become functionals of time. By analogy with Lagrangian mechanics, denote the action by the anti-derivative or path-integral of free energy
We now seek variational densities that maximise action. It is fairly easy to show that the solutions are functionals of the instantaneous energy
Where and are the prior energies 21

11. Ensemble learning (dynamic)
Lemma 2: is the stationary solution, in a moving frame of reference, for an ensemble whose equations of motion and ensemble dynamics are
Proof: Substituting the variational density; gives
This describes a stationary density under a moving frame of reference, with velocity
as seen using the co-ordinate transform

12. Ensemble dynamics in generalized coordinates20 40 60 80
100
120 -2 -1 1 2 3 4 5 -2 2 -5 5

13. Overview Dynamic causal models
Generalised coordinates (dynamical priors)
Hierarchal forms (structural priors)
Variational filtering and action (free-form)
Laplace approximation and DEM (fixed-form)
Comparative evaluations
Hemodynamics
Bird songs

14. The Laplace approximation – why is it useful?
Under the Laplace approximation, the variational density assumes a Gaussian form
The conditional precision (a function of the mode) obtains by extremising the free-energy

15. … for dynamic models
The conditional precisions obtain by extremising the action
Conditional precisions:
which leaves the variational modes that optimise variational energy and action
Variational energy and actions:
Mean-field terms:

16. Approximating the mode
Lemma 3: The path of a particle, whose motion is
converges exponentially to the mode
Where the trajectory of the conditional mode can be realised with a local linearisation (Ozaki 1992) to give the update

17. … an augmented gradient ascent
In dynamic systems, the trajectory of the conditional mode maximises variational action, which is the solution to the ansatz
Here, can be regarded as motion in a frame of reference that moves along the trajectory encoded in generalised coordinates. The stationary solution, in this moving frame of reference, maximises variational action. This can be seen easily by noting
This is sufficient for the mode to maximise variational action (by the Fundamental lemma); c.f., Hamilton's principle of stationary action

18. DEM and belief propagation
D-Step
inference
E-Step
learning
M-Step
uncertainty
A dynamic recognition system that minimises prediction error

19. Overview Dynamic causal models
Generalised coordinates (dynamical priors)
Hierarchal forms (structural priors)
Variational filtering and action (free-form)
Laplace approximation and DEM (fixed-form)
Comparative evaluations
Hemodynamics
Bird songs

20. A linear convolution model
Prediction error
Generation
Inversion

21. Variational filtering on states and causes5 10 15 20 25 30 -1
-0.5
0.5 1
1.5
hidden states
time
cause 5 10 15 20 25 30
-0.4
-0.2
0.2
0.4
0.6
0.8 1
1.2
cause
time {bins}

22. Linear deconvolution with variational filtering (SDEs)
Linear deconvolution with Dynamic expectation maximisation (ODE)

23. The order of generalised motion
Precision in generalised coordinates
The order of generalised motion 10 20 30 40
-1.5 -1
-0.5
0.5 1
1.5 2
time
Accuracy and embedding (n) 1 3 5 7 9 11 13 2 4 6
sum squared error (causal states) 5 10 15 20 25 30 35
-1.5 -1
-0.5
0.5 1
1.5 2
time

24. DEM and extended Kalman filtering
hidden states
DEM and extended Kalman filtering 5 10 15 20 25 30 35
-1.5 -1
-0.5
0.5 1
time
DEM(0)
DEM(4)
EKF
true
With convergence when
sum of squared error (hidden states)
EKF
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
DEM(0)
DEM(4) 10 20 30 40 -1
-0.5
0.5 1
time
hidden states
DEM(0)
EKF

25. A nonlinear convolution model
level
This system has a slow sinusoidal input or cause that excites increases in a single hidden state. The response is a quadratic function of the hidden states (c.f., Arulampalam et al 2002).

26. DEM and particle filtering
Comparative performance
Sum of squared error

27. Triple estimation (DEM)
Inference on states
Triple estimation (DEM)
Learning parameters

28. Overview Dynamic causal models
Generalised coordinates (dynamical priors)
Hierarchal forms (structural priors)
Variational filtering and action (free-form)
Laplace approximation and DEM (fixed-form)
Comparative evaluations
Hemodynamics
Bird songs

29. An fMRI study of attention
Stimuli 250 radially moving dots at 4.7 degrees/s
Pre-Scanning
5 x 30s trials with 5 speed changes (reducing to 1%)
Task: detect change in radial velocity
Scanning (no speed changes)
4 x 100 scan sessions; each comprising 10 scans of 4 different conditions
F A F N F A F N S
A – dots, motion and attention (detect changes)
N – dots and motion
S – dots
F – fixation
PPC
V5+
Buchel et al 1999

30. Output: a mixture of intra- and extravascular signal
A hemodynamic model
Visual input
Motion
Attention
convolution kernel
state equations
output equation
Output: a mixture of intra- and extravascular signal

31. Hemodynamic deconvolution
Inference on states
Hemodynamic deconvolution
Learning parameters

32. … and a closer look at the states

33. Overview Dynamic causal models
Generalised coordinates (dynamical priors)
Hierarchal forms (structural priors)
Variational filtering and action (free-form)
Laplace approximation and DEM (fixed-form)
Comparative evaluations
Hemodynamics
Bird songs

34. Synthetic song-birds
syrinx
hierarchy of Lorenz attractors

35. Song recognition with DEM

36. … and broken birds

37. Summary Dynamic causal models
Generalised coordinates (dynamical priors)
Hierarchal forms (structural priors)
Variational filtering and action (free-form)
Laplace approximation and DEM (fixed-form)
Comparative evaluations
Hemodynamics
Bird songs