Sampling Methods for Estimation: An Introduction

Slides:



Advertisements
Similar presentations
Dynamic Spatial Mixture Modelling and its Application in Cell Tracking - Work in Progress - Chunlin Ji & Mike West Department of Statistical Sciences,
Advertisements

Jose-Luis Blanco, Javier González, Juan-Antonio Fernández-Madrigal University of Málaga (Spain) Dpt. of System Engineering and Automation May Pasadena,
CSCE643: Computer Vision Bayesian Tracking & Particle Filtering Jinxiang Chai Some slides from Stephen Roth.
Markov Chain Monte Carlo Convergence Diagnostics: A Comparative Review By Mary Kathryn Cowles and Bradley P. Carlin Presented by Yuting Qi 12/01/2006.
ECE 8443 – Pattern Recognition LECTURE 05: MAXIMUM LIKELIHOOD ESTIMATION Objectives: Discrete Features Maximum Likelihood Resources: D.H.S: Chapter 3 (Part.
1 12. Principles of Parameter Estimation The purpose of this lecture is to illustrate the usefulness of the various concepts introduced and studied in.
ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition Objectives: Jensen’s Inequality (Special Case) EM Theorem.
Visual Tracking CMPUT 615 Nilanjan Ray. What is Visual Tracking Following objects through image sequences or videos Sometimes we need to track a single.
Markov Localization & Bayes Filtering 1 with Kalman Filters Discrete Filters Particle Filters Slides adapted from Thrun et al., Probabilistic Robotics.
On-Line Probabilistic Classification with Particle Filters Pedro Højen-Sørensen, Nando de Freitas, and Torgen Fog, Proceedings of the IEEE International.
Introduction to Sampling based inference and MCMC Ata Kaban School of Computer Science The University of Birmingham.
CHAPTER 16 MARKOV CHAIN MONTE CARLO
Particle Filters.
Adaptive Rao-Blackwellized Particle Filter and It’s Evaluation for Tracking in Surveillance Xinyu Xu and Baoxin Li, Senior Member, IEEE.
A brief Introduction to Particle Filters
Sérgio Pequito Phd Student
Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.
Particle filters (continued…). Recall Particle filters –Track state sequence x i given the measurements ( y 0, y 1, …., y i ) –Non-linear dynamics –Non-linear.
Particle Filters for Mobile Robot Localization 11/24/2006 Aliakbar Gorji Roborics Instructor: Dr. Shiri Amirkabir University of Technology.
Comparative survey on non linear filtering methods : the quantization and the particle filtering approaches Afef SELLAMI Chang Young Kim.
Darryl MorrellStochastic Modeling Seminar1 Particle Filtering.
Optimal Filtering of Jump Diffusions: Extracting Latent States from Asset Prices Jonathan Stroud, Wharton, U. Pennsylvania Stern-Wharton Conference on.
Particle Filtering for Non- Linear/Non-Gaussian System Bohyung Han
Estimation and the Kalman Filter David Johnson. The Mean of a Discrete Distribution “I have more legs than average”
Novel approach to nonlinear/non- Gaussian Bayesian state estimation N.J Gordon, D.J. Salmond and A.F.M. Smith Presenter: Tri Tran
An introduction to Particle filtering
Introduction to Monte Carlo Methods D.J.C. Mackay.
Markov Localization & Bayes Filtering
ECE 8443 – Pattern Recognition LECTURE 06: MAXIMUM LIKELIHOOD AND BAYESIAN ESTIMATION Objectives: Bias in ML Estimates Bayesian Estimation Example Resources:
Object Tracking using Particle Filter
Computer vision: models, learning and inference Chapter 19 Temporal models.
Model Inference and Averaging
System Identification of Nonlinear State-Space Battery Models
From Bayesian Filtering to Particle Filters Dieter Fox University of Washington Joint work with W. Burgard, F. Dellaert, C. Kwok, S. Thrun.
SIS Sequential Importance Sampling Advanced Methods In Simulation Winter 2009 Presented by: Chen Bukay, Ella Pemov, Amit Dvash.
Particle Filtering (Sequential Monte Carlo)
Computer vision: models, learning and inference Chapter 19 Temporal models.
Stochastic Linear Programming by Series of Monte-Carlo Estimators Leonidas SAKALAUSKAS Institute of Mathematics&Informatics Vilnius, Lithuania
Jamal Saboune - CRV10 Tutorial Day 1 Bayesian state estimation and application to tracking Jamal Saboune VIVA Lab - SITE - University.
Probabilistic Robotics Bayes Filter Implementations Gaussian filters.
Probabilistic Robotics Bayes Filter Implementations.
Module 1: Statistical Issues in Micro simulation Paul Sousa.
ECE 8443 – Pattern Recognition ECE 8423 – Adaptive Signal Processing Objectives: Deterministic vs. Random Maximum A Posteriori Maximum Likelihood Minimum.
Young Ki Baik, Computer Vision Lab.
Overview Particle filtering is a sequential Monte Carlo methodology in which the relevant probability distributions are iteratively estimated using the.
-Arnaud Doucet, Nando de Freitas et al, UAI
Clustering and Testing in High- Dimensional Data M. Radavičius, G. Jakimauskas, J. Sušinskas (Institute of Mathematics and Informatics, Vilnius, Lithuania)
PROBABILITY AND STATISTICS FOR ENGINEERING Hossein Sameti Department of Computer Engineering Sharif University of Technology Principles of Parameter Estimation.
Mobile Robot Localization (ch. 7)
Maximum a posteriori sequence estimation using Monte Carlo particle filters S. J. Godsill, A. Doucet, and M. West Annals of the Institute of Statistical.
An Introduction to Kalman Filtering by Arthur Pece
Ch. 14: Markov Chain Monte Carlo Methods based on Stephen Marsland, Machine Learning: An Algorithmic Perspective. CRC 2009.; C, Andrieu, N, de Freitas,
Sequential Monte-Carlo Method -Introduction, implementation and application Fan, Xin
Markov Chain Monte Carlo for LDA C. Andrieu, N. D. Freitas, and A. Doucet, An Introduction to MCMC for Machine Learning, R. M. Neal, Probabilistic.
ECE 8443 – Pattern Recognition Objectives: Jensen’s Inequality (Special Case) EM Theorem Proof EM Example – Missing Data Intro to Hidden Markov Models.
Tracking with dynamics
6. Population Codes Presented by Rhee, Je-Keun © 2008, SNU Biointelligence Lab,
1 Chapter 8: Model Inference and Averaging Presented by Hui Fang.
Introduction to Sampling Methods Qi Zhao Oct.27,2004.
The Unscented Particle Filter 2000/09/29 이 시은. Introduction Filtering –estimate the states(parameters or hidden variable) as a set of observations becomes.
Rao-Blackwellised Particle Filtering for Dynamic Bayesian Network Arnaud Doucet Nando de Freitas Kevin Murphy Stuart Russell.
CS Statistical Machine learning Lecture 25 Yuan (Alan) Qi Purdue CS Nov
École Doctorale des Sciences de l'Environnement d’Île-de-France Année Universitaire Modélisation Numérique de l’Écoulement Atmosphérique et Assimilation.
Ch 1. Introduction Pattern Recognition and Machine Learning, C. M. Bishop, Updated by J.-H. Eom (2 nd round revision) Summarized by K.-I.
Introduction to Sampling based inference and MCMC
Probabilistic Robotics
Where did we stop? The Bayes decision rule guarantees an optimal classification… … But it requires the knowledge of P(ci|x) (or p(x|ci) and P(ci)) We.
2. University of Northern British Columbia, Prince George, Canada
Parametric Methods Berlin Chen, 2005 References:
Convergence of Sequential Monte Carlo Methods
Presentation transcript:

Sampling Methods for Estimation: An Introduction Particle Filters, Sequential Monte-Carlo, and Related Topics Nahum Shimkin Faculty of Electrical Engineering Technion

Outline The nonlinear estimation problem Monte-Carlo Sampling: - Crude Monte-Carlo - Importance Sampling Sequential Monte-Carlo

The Estimation Problem: Consider a discrete-time stochastic system with state xtX (usually n), and observation ytY. The system is specified by the following prior data: p(xo): initial state distribution p(xt+1| xt): state transition law (t=0, 1, 2, ..) p(yt| xt): measurement distribution (t=1, 2, ..) Denote

The Estimation Problem (cont’d) Note: As usual, we assume the Markovian state property: The above laws are often specified through state equations: The system may be time-varying, for simplicity we omit the time variable.

The Estimation Problem (cont’d) We wish to estimate the state xt, based on the prior data and the observations y1:t. The MMSE-optimal estimator, for example, is given by with covariance More generally, compute the posterior state distribution: We can then compute

Posterior Distribution Formulas: Using the Bayes rule, we easily obtain the following two-step recursive formulas that relate p(xt|y1:t) to p(xt-1| y1:t-1) and yt: Time Update (prediction step): Measurement Update:

Posterior Distribution Formulas: (2) Unfortunately, these are hard to compute directly – especially when x is high-dimensional! Some well-known approximations: The Extended Kalman Filter Gaussian Sum Filters General/Specific Numerical Integration Methods Sampling-based methods – Particle Filters.

Monte-Carlo Sampling: Crude Monte Carlo The basic idea: Estimate the expectation integral by the following mean: where That is, are independent samples from p(x).

Crude Monte-Carlo – Interpretation We can interpret Monte-Carlo sampling as trying to approximate p(x) by the discrete distribution It is easily seen that The samples x(i) are sometimes called “particles”, with weights w(i) = N-1.

Crude Monte Carlo: Properties The following basic properties follow from standard results for i.i.d. sequences. Bias: IN (g) is an unbiased estimate of I(g), namely SLLN: CLT: whenever Advantages (over numerical integration):  Straightforward scheme.  Focuses on “important” areas of p(x).  Rate of convergence does not depend (directly) on dim(x).

The Monte-Carlo Method: Origins Conceiver: Stanislav Ulam (1946). Early developments (1940’s-50’s):  Ulam & Von-Neumann: Importance Sampling, Rejection Sampling.  Metropolis & Von-Neumann: Markov-Chain Monte-Carlo (MCMC). A surprisingly large array of applications, including  global optimization  statistical and quantum physics  computational biology  signal and image processing  rare-event simulation  state estimation …

Importance Sampling (IS) Idea: Sample x from another distribution, q(x). Use weights to obtain the correct distribution. Advantages: May be (much) easier to sample from q than from p. The covariance may be (much) smaller  Faster convergence / better accuracy.

Importance Sampling (2) Let q(x) be a selected distribution over x – the proposal distribution. Assume that Observe that: where is the likelihood ratio, or simply the weight-function.

Importance Sampling (3) This immediately leads to the IS algorithm: (1) Sample (2) Compute the weights and Basic Properties: The above-mentioned convergence properties are maintained. The CLT covariance is now This covariance is minimized by q(x)=p(x) g(x). Unfortunately this choice is often too complicated, and we settle for approximations.

Importance Sampling (4) Additional Comments: It is often convenient to use the normalized weights so that The weights w(i) are sometimes not properly normalized for example, when p(x) is known up to a (multiplicative) constant. We can still use the last formula, with We may interpret as a “weighted-particles” approximation to p(x):

Two Discrete Approximations

The Particle Filter Algorithm We can now apply these ideas to our filtering problem. The basic algorithm is as follows: 0. Initialization: sample from p0(x) to obtain with For t=1,2,…, given Time update: For i=1,…, N sample Set Measurement update: For i=1,…, N, evaluate the (unnormalized) importance weights: Resampling: Sample N points from the discrete distribution corresponding to Continue with

The Particle Filter Algorithm (2) Output: Based on we can estimate We may also use for that purpose. This algorithm is the original “bootstrap filter” introduced in Gordon et al. (1993). CONDENSATION is a similar algorithm developed independently in the computer vision context (Isard and Blake, 1998). Many improvements and variations now exist.

Particle Flow (from Doucet et. al., 2001)

The Particle Filter: Time Update Recall that approximates and is given. An approximation is therefore obtained by sampling and setting For example, for the familiar state model we get

The Particle Filter: Time Update (2) It is possible (and often useful) to sample from another (proposal) distribution In that case we need to modify the weights: The ideal choice for would by Some schemes exist which try to approximate sampling with q* - e.g., using MCMC.

The Particle Filter: Measurement Update Recall that and approximate To obtain an approximation to we only need to modify the weights accordingly: with

The Particle Filter: Resampling Resampling is not strictly required for asymptotic convergence (as ). However, without it, the distributions tend to degenerate, with few “large” particles, and many small ones. The idea is therefore to split large particles into several smaller ones, and discard very small particles. Random sampling from is one option. Another is a deterministic splitting of particles according to namely: particles at with In fact, the last option tends to reduce the variance.

Additional Issues and Comments Data Association: This is an important problem for applications such as multiple target tracking, and visual tracking. Within the Kalman Filter framework, such problems are often treated by creating multiple explicit hypothesis regarding the data origin which may lead to combinatorial explosion. Isard and Blake (1998) observed that particle filters open new possibilities for data association for tracking. Specifically, different particles can be associated to different measurements, based on some “closeness” (or likelihood) measure. Encouraging results have been reported.

Additional Issues and Comments (2) Joint Parameter and State Estimation: As is well known, this “adaptive estimation” problem can be reduced to a standard estimation problem by embedding the parameters in the state vector. This however leads to a bi-linear state equation, for which standard KF algorithms did not prove successful. The problem is currently treated using non-sequential algorithms (e.g. EM + smoothing). Particle filters have been applied to this problem with encouraging results.

Additional Issues and Comments (3) Low-noise State Components: State components which are not excited by external noise (such as fixed parameters) pose a problem since the respective particle components may be fixed in their place, and cannot correct initially bad placements. The simplest solution is to add a little noise. Additional suggestions involve intermediate MCMC steps, and related ideas.

Conclusion: Particle Filters and Sequential Monte Carlo Methods are an evolving and active topic, with good potential to handle “hard” estimation problems, involving non-linearity and multi-modal distributions. In general, these schemes are computationally expensive as the number of “particles” N needs to be large for precise results. Additional work is required on the computational issue – in particular, optimizing the choice of N, and related error bounds. Another promising direction is the merging of sampling methods with more disciplined approaches (such as Gaussian filters, and the Rao-Blackwellization scheme …).

To Probe Further: Monte-Carlo Methods (at large): R. Rubinstein, Simulation and the Monte-Carlo Method, Wiley, 1981 (and 1996). J.S. Liu, Monte-Carlo Strategies in Scientific Computing, Springer, 2001. M. Evans and T. Swartz, “Methods for approximating integrals in statistics with special emphasis on Bayesian integration problems”, Statistical Science 10(3), 1995, pp. 254-272. Particle Filters: N.J. Gordon, D. J. Salmond and A. Smith, “Novel approach to non-linear / non-Gaussian Bayesian state estimation”, IEE Proceedings-F 140(2), pp. 107-113. M. Isard and A. Blake, “CONDENSATION – conditional density propagation for visual tracking”, International Journal of Computer Vision 28(1), pp. 5-28. A Doucet, N. de Freitas and N. Gordon (eds.) Sequential Monte Carlo Method in Practice, Springer, 2001. S. Arulampalam et al., “A tutorial on particle filters, for on-line non-linear/non-Gaussian Bayesian tracking”, IEEE Trans. on Signal Processing 50(2), 2002.