Sanjay Patil 1 and Ryan Irwin 2 Intelligent Electronics Systems, Human and Systems Engineering Center for Advanced Vehicular Systems URL: HUMAN AND SYSTEMS ENGINEERING: Introduction to Particle Filtering

Page 1 of 20 Introduction to Particle Filtering Abstract Particle Filtering: Most conventional techniques used for speech analysis are based on modeling the speech signal as Gaussian mixture models. Nonlinear approaches are expected to outperform the conventional techniques because of their abilities to compensate for the mismatched channel conditions and to significantly reduce the complexity of the models. Particle filtering is one such nonlinear method based on sequential Monte Carlo technique. Particle filtering works by approximating the target probability distribution. Thus, it greatly reduces the complexities associated with the models.

Page 2 of 20 Introduction to Particle Filtering 5000 samples500 samples 200 samples Take p(x)=Gamma(4,1) Generate some random samples Plot basic approximation to pdf Each sample is called as ‘Particle’ Drawing samples from a probability distribution function Concept of samples and its weights

Page 3 of 20 Introduction to Particle Filtering Particle filtering - Different Names – Sequential Monte Carlo filters Bootstrap filters Condensation Algorithm Survival of the fittest General Problem Statement – Filtering – estimation of the states Tracking the state (parameters or hidden variables) as it evolves over time Sequentially arriving (noisy and non-Gaussian) observations Idea is to have best possible estimate of hidden variables

Page 4 of 20 Introduction to Particle Filtering Assume that pdf p(x k-1 | y 1:k-1 ) is available at time k -1. Prediction stage: This is the prior of the state at time k ( without the information on measurement). Thus, it is the probability of the the state given only the previous measurements Update stage: This is posterior pdf from predicted prior pdf and newly available measurement. Particle filtering algorithm continue… General two-stage Framework (Prediction-Update stages)

Page 5 of 20 Introduction to Particle Filtering Particle filtering algorithm step-by-step (1)

Page 6 of 20 Introduction to Particle Filtering Particle filtering step-by-step (2)

Page 7 of 20 Introduction to Particle Filtering Particle filtering step-by-step (3)

Page 8 of 20 Introduction to Particle Filtering Particle filtering step-by-step (4)

Page 9 of 20 Introduction to Particle Filtering Particle filtering step-by-step (5)

Page 10 of 20 Introduction to Particle Filtering Particle filtering step-by-step (6)

Page 11 of 20 Introduction to Particle Filtering Particle filtering - visualization Drawing samples Predicting next state Updating this state What is THIS STEP??? Resampling….

Page 12 of 20 Introduction to Particle Filtering Sampling Importance Resample algorithm (necessity)

Page 13 of 20 Introduction to Particle Filtering Applications: All the applications are mostly tracking applications in different forms…. Visual Tracking – e.g. human motion (body parts) Prediction of (financial) time series – e.g. mapping gold price, stocks Quality control in semiconductor industry Military Applications Target recognition from single or multiple images Guidance of missiles What is the application for IES NSF funded project – Time series estimation for speech signal (Java demo) Speaker Verification (details on next slide)

Page 14 of 20 Introduction to Particle Filtering Pattern Recognition Applet Java applet that gives a visual of algorithms implemented at IES Classification of Signals: PCA - Principle Component Analysis LDA - Linear Discrimination Analysis SVM - Support Vector Machines RVM - Relevance Vector Machines Tracking of Signals LP - Linear Prediction KF - Kalman Filtering PF – Particle Filtering

Page 15 of 20 Introduction to Particle Filtering Pattern Classification Different data sets need to be differentiated without looking at all the data samples Classifications distinguishes between sets of data without the samples Algorithms separate data sets with a line of discrimination To have zero error the line of discrimination should completely separate the classes These patterns are easy to classify

Page 16 of 20 Introduction to Particle Filtering Pattern Classification Toroidals are not classified very successfully with a straight line Error should be around 50% because half of each class is separated A proper line of discrimination of a toroidal would be a circle enclosing only the inside set

Page 17 of 20 Introduction to Particle Filtering Signal Tracking The input signals are now time based with the x-axis representing time All the signal tracking algorithms are implemented with interpolated data The interpolation ensures that the input samples are at regular intervals Sampling is always done on regular intervals The linear prediction algorithm is a linear way to predict signals with no noise

Page 18 of 20 Introduction to Particle Filtering Signal Tracking The Kalman filter and particle filter are based on prediction of the states of the signal States are related to the observations through the state equation The particle filtering algorithm introduces process and measurement noise At each iteration possible states are given by the black points The average of the black points is where the overall state is predicted to be

Page 19 of 20 Introduction to Particle Filtering Summary: Particle filtering promises to be one of the nonlinear techniques. More points to follow

Page 20 of 20 Introduction to Particle Filtering References: S. Haykin and E. Moulines, "From Kalman to Particle Filters," IEEE International Conference on Acoustics, Speech, and Signal Processing, Philadelphia, Pennsylvania, USA, March M.W. Andrews, "Learning And Inference In Nonlinear State-Space Models," Gatsby Unit for Computational Neuroscience, University College, London, U.K., December P.M. Djuric, J.H. Kotecha, J. Zhang, Y. Huang, T. Ghirmai, M. Bugallo, and J. Miguez, "Particle Filtering," IEEE Magazine on Signal Processing, vol 20, no 5, pp , September N. Arulampalam, S. Maskell, N. Gordan, and T. Clapp, "Tutorial On Particle Filters For Online Nonlinear/ Non-Gaussian Bayesian Tracking," IEEE Transactions on Signal Processing, vol. 50, no. 2, pp , February R. van der Merve, N. de Freitas, A. Doucet, and E. Wan, "The Unscented Particle Filter," Technical Report CUED/F-INFENG/TR 380, Cambridge University Engineering Department, Cambridge University, U.K., August S. Gannot, and M. Moonen, "On The Application Of The Unscented Kalman Filter To Speech Processing," International Workshop on Acoustic Echo and Noise, Kyoto, Japan, pp 27-30, September J.P. Norton, and G.V. Veres, "Improvement Of The Particle Filter By Better Choice Of The Predicted Sample Set," 15th IFAC Triennial World Congress, Barcelona, Spain, July J. Vermaak, C. Andrieu, A. Doucet, and S.J. Godsill, "Particle Methods For Bayesian Modeling And Enhancement Of Speech Signals," IEEE Transaction on Speech and Audio Processing, vol 10, no. 3, pp , March 2002.