Ryan Irwin Intelligent Electronics Systems Human and Systems Engineering Center for Advanced Vehicular Systems URL: Introduction To The Pattern Recognition Applet:

Page 1 of 13 Introduction to Pattern Recognition Applet General Overview oJava based applet that demonstrates various algorithms implemented at IES oEach implementation closely mirrors the code and functionality of the actual implementation in the repository oTwo types of algorithms implemented  Pattern Classification: PCA, LDA, SVM, RVM Separation of 2 or more classes  Signal Tracking/Modeling: LP, KF, UKF, PF Time based One signal/class at a time

Page 2 of 13 Introduction to Pattern Recognition Applet Pattern Classification o Algorithms separate different classes with line of discrimination o Different colored points represent different classes o Deemed successful if there are no points of different color on the same side of the line o At left, orange line separates red and green classes

Page 3 of 13 Introduction to Pattern Recognition Applet Pattern Classification – Principal Component Analysis o A covariance generally describes how two datasets relate to each other o A transform maps point from current space to a new feature space o Class-Independent PCA – One covariance and transform for all points calculated o Class-Dependent PCA – A covariance and transform for each class is calculated o Points are mapped from current space to new space with use of the transforms

Page 4 of 13 Introduction to Pattern Recognition Applet Pattern Classification – Linear Discrimination Analysis o Within-class scatter defines distribution of a set o Between-class scatters defines scatter of expected vectors around the global mean o Class Independent – Single between-class scatter o Class Dependent – Multiple between-class scatters o Goal is to minimize within-class scatters and maximize between-class scatters

Page 5 of 13 Introduction to Pattern Recognition Applet Pattern Classification – Support Vector Machine o Classification by light training o Training picks out points nearest other classes o This reduces the number of points for final classification o Final classification is takes more computation with SVM than RVM o More practical if one-time training and one-time classification

Page 6 of 13 Introduction to Pattern Recognition Applet Pattern Classification – RVM o Training is more computationally involved o A selection of points most suitable for classification is made o Only a few points are used for final classification (fewer than SVM) o More practical if training is not needed every time a classification is made

Page 7 of 13 Introduction to Pattern Recognition Applet Signal Tracking o Algorithms track a time- based signal from left to right o A signal’s next state is predicted given the previous states o Regular interval sampling by interpolation o Algorithms are recursive in nature o Noise is simulated

Page 8 of 13 Introduction to Pattern Recognition Applet Signal Tracking – Kalman Filter o Observation equation relates observations and states o The state equation predicts the next state o Algorithm runs two steps repeatedly  State prediction stage uses state equation and state gain factor to predict next state  Update state stage compares previous state and observation with noises to make final prediction o Upon completion mean square error is given

Page 9 of 13 Introduction to Pattern Recognition Applet Signal Tracking – Unscented Kalman Filter o Algorithm has same basic operation as conventional Kalman Filter o Sigma points are used (alpha, beta, and kappa) o Each sigma point has a weight that ends up effecting the overall mean of the filtered signal o Modification generally reduces the mean square error

Page 10 of 13 Introduction to Pattern Recognition Applet Signal Tracking – Particle Filtering o Based on sequential Monte Carlo techniques o Has state and observation equations like KF o Particles are used for prediction o They form a probability distribution of an observation at each step o Algorithm functions best when applied with non-linear signals

Page 11 of 13 Introduction to Pattern Recognition Applet Important points o Pattern Classification  Multiple classes  Not time-based data  Performance based on percentage of correctly classified points o Signal Tracking  Single class of points  Time-based and interpolated data  Performance based on mean square error o Is there a need for separate applets?

Page 12 of 13 Introduction to Pattern Recognition Applet Tutorials o Detailed operation of each algorithm is given o More algorithm detail is given in the tutorial section Go to tutorials

Page 13 of 13 Introduction to Pattern Recognition Applet References S. Haykin and E. Moulines, "From Kalman to Particle Filters," IEEE International Conference on Acoustics, Speech, and Signal Processing, Philadelphia, Pennsylvania, USA, March M.W. Andrews, "Learning And Inference In Nonlinear State-Space Models," Gatsby Unit for Computational Neuroscience, University College, London, U.K., December P.M. Djuric, J.H. Kotecha, J. Zhang, Y. Huang, T. Ghirmai, M. Bugallo, and J. Miguez, "Particle Filtering," IEEE Magazine on Signal Processing, vol 20, no 5, pp , September N. Arulampalam, S. Maskell, N. Gordan, and T. Clapp, "Tutorial On Particle Filters For Online Nonlinear/ Non- Gaussian Bayesian Tracking," IEEE Transactions on Signal Processing, vol. 50, no. 2, pp , February R. van der Merve, N. de Freitas, A. Doucet, and E. Wan, "The Unscented Particle Filter," Technical Report CUED/F- INFENG/TR 380, Cambridge University Engineering Department, Cambridge University, U.K., August S. Gannot, and M. Moonen, "On The Application Of The Unscented Kalman Filter To Speech Processing," International Workshop on Acoustic Echo and Noise, Kyoto, Japan, pp 27-30, September J.P. Norton, and G.V. Veres, "Improvement Of The Particle Filter By Better Choice Of The Predicted Sample Set," 15th IFAC Triennial World Congress, Barcelona, Spain, July J. Vermaak, C. Andrieu, A. Doucet, and S.J. Godsill, "Particle Methods For Bayesian Modeling And Enhancement Of Speech Signals," IEEE Transaction on Speech and Audio Processing, vol 10, no. 3, pp , March M. Gabrea, “Robust Adaptive Kalman Filtering-based Speech Enhancement Algorithm,” ICASSP 2004, vol 1, pp. I- 301-I-304, May K. Paliwal, :Estiamtion og noise variance from the noisy AR signal and its application in speech enhancement,” IEEE transaction on Acoustics, Speech, and Signal Processing, vol 36, no 2, pp , Feb 1988.