1. Particle Filtering (Sequential Monte Carlo)
Ercan Engin Kuruoğlu,
ISTI-CNR, Pisa

2. outline Review of particle filtering
Case study: Source separation using Particle Filtering
Application: separation of independent components in astrophysical images

7. non-stationary processes
Special cases
linear observations (h)
Gaussian observation noise (n)
linear state process (f)
Gaussian process noise (v)
non-stationary processes
stationary processes
Wiener filter
Kalman filter

8. Kalman filter R. Kalman (1960), Swerling (1958)
In control theory: linear quadratic estimation (LQE).
Kalman filters are based on linear dynamical systems discretised in the time domain.
They are modelled on a Markov chain built on linear operators perturbed by Gaussian noise. A

9. Nonlinear, non-Gaussian case

10. Extended Kalman Filter
It was the classical method for non linear state-space systems
A and H are nonlinear
Perform first order Taylor expansion

11. Unscented Kalman Filter
We will not discuss it here for the time being
You can read a very clear presentation in prepared by Eric Wan
It provides a second order expansion of Taylor series
Not analytically but through sampling

17. sequentiality
We would like to avoid w each time instant and update it sequentially

21. Resampling strategy
Deterministic sampling (fixed points with equal spacing)
stratified sampling (random points between fixed intervals)
Sampling importance sampling (SIS)
Residual resampling
Roughening and editing (adds independent jitter)
For details see:
A survey of convergence results on particle filtering methods for practitioners by Crisan, D.; Doucet, A. IEEE Transactions on Signal Processing, Volume 50, Issue 3, Mar 2002 Page(s):

23. Proposal distributions
Optimal importance function:
The posterior itself
The prior distribution as the importance function:
Easy to implement
But no information from observation!
Hybrid importance functions
Somewhere in between

24. Particle Filtering-Summary
Sequential Monte Carlo technique
Generalisation of the Kalman filtering to nonlinear/non-Gaussian systems/signals.
Handles nonstationary signals/systems

25. Basic Particle Filter - Schematic
Initialisation
measurement
Resampling
step
Importance
sampling step
Extract estimate,

26. Importance Sampling step
For sample
and set
For evaluate the importance weights
Normalise the importance weights,

27. Applications Tracking (Gordon et al.)
Audio restoration (Godsill et al.)
CDMA (Punskaya et al.)
Computer vision (Blake et al.)
Genomics (Haan and Godsill)
Array processing (Reilly et al.)
Financial time series (de Freitas et al.)
sonar (Gustaffson)

28. Applications: source separation
Ahmed, Andrieu, Doucet, Rayner, “Online non-stationary ICA using mixture models”, ICASSP 2000.
Andrieu, Godsill, “A particle filter for model based audio source separation”, ICA 2000.
Source: Gaussian model
Convolutional mixing
Audio separation
Everson, Roberts, “Particle Filters for Non-stationary ICA, Advances in Independent Components Analysis, 2000.
Only the mixing is nonstationary.
Costagli, Kuruoglu, Ahmed, ICA 2004.

29. SOURCE SEPARATION Model for observations Model for the mixing matrix
Source model
Importance function
Resampling strategy

30. Model for observations
Assume linear, instantaneous mixing (extension to the convolutional case is possible)

31. Model for the mixing In general, time-varying mixing matrix
In the lack of prior knowledge, we assume

32. Source model
Gaussian mixtures
Hidden rv/state

33. Evolution of hyperparameters-1

34. Evolution of hyperparameters-2

35. Particle filtering Need to evaluate: Can be estimated by Kalman filter
We are left with:

36. Choice of importance function
To be decided on, a choice can be:
Evaluation of this requires only one step of Kalman Filtering for each particle.

37. Resampling strategy Sampling importance resampling (SIR)
Residual resampling
Stratified sampling

38. Astrophysical source separation

39. Observation Model n observation channels (30-857 GHz)
H mixing matrix (allowed to be space-varying)
m sources (non-Gaussian and non-stationary)
w space-varying Gaussian noise

40. Noise
the noise variance is known for each pixel

41. Source Model: Mixture of Gaussians
Each source distribution is modelled by a
finite mixture of Gaussians:

42. A-priori distribution as “importance function”

43. Hierarchical structure

44. Rao-Blackwellisation
It is possible to reduce the size of the parameter set
in the Sequential Importance Sampling step:
the mixing matrix H
(re-parametrized into a vector h)
is obtained subsequently through the Kalman Filter:

45. Simulation results 2

46. Conclusions
we introduced a new, general approach to solve the source separation problem in the astrophysical context
PF provides better results in comparison with ICA, especially in case of SNR < 10 dB
Non-stationary model, non-Gaussian variables, space-varying noise
it is possible to exploit the available a-priori information

47. Computer vision applications
Now let’s have a look some results obtained using particle filters in computer vision problems