ISOMAP TRACKING WITH PARTICLE FILTER Presented by Nikhil Rane

Dimensionality Reduction Let x i be H-dimensional and y i be L- dimensional then dimensionality reduction solves the problem x i = f (y i ) where H>L

Dimensionality Reduction Techniques Linear PCA Transforms data into a new coordinate system so that largest variance in on the 1 st dimension, 2 nd largest along 2 nd dimension … Classical MDS Preserves Euclidean distances between points Nonlinear Isomap Preserves geodesic distances between points LLE Preserves local configurations in data

Face Database

Principal Components Analysis (PCA) 1) Make the mean of the data zero 2) Compute covariance matrix C 3) Compute eigenvalues and eigenvectors of C 4) Choose the principal components 5) Generate low-dimensional points using principal components

Performance of PCA on Face- data

Classical Multidimensional Scaling (MDS) Compute Distance Matrix S Compute inner product matrix B = -0.5JSJ where J = I N – (1/N)11 T Decompose B into eigenvectors and eigenvalues Use top d eigenvectors and eigenvalues to form the d dimensional embedding.

Performance of MDS on face- data

Locally Linear Embedding (LLE) Find neighbors of each data point Compute weights that best reconstruct each data point from its neighbors Compute low-dimensional vectors best reconstructed by the weights

Performance of LLE on Face- data

Geodesic Distance Geodesic distance – the length of the shortest curve between two points taken along the surface of a manifold

Isometric Feature Mapping (Isomap) Construct neighborhood graph Compute shortest paths between points Apply classical MDS

Performance of Isomap on face- data

Tracking vs. Detection Detection - locating an object independent of the past information When motion is unpredictable For reacquisition of a lost target Tracking - locating an object based on past information Saves computation time

Recursive Bayesian Framework Estimate the pdf of state at time t given the pdf of state at time t - 1 and measurement at time t Predict Predict state of the system at time t using a system- model and pdf from time t – 1 Update Update the predicted state using measurement at time t by Bayes’ rule

Kalman Filtering vs. Particle Filtering Kalman filter assumes the pdf of the state to be Gaussian at all times and requires the measurement and process noise to be Gaussian Particle filter makes no such assumption and in fact estimates the pdf at every time-step

Resampling

Condensation algorithm Algorithm – 1) Resample 2) Predict 3) Measure

Condensation algorithm

Isomap Tracking with Particle Filtering Create training set of a person’s face (off-line) Use Isomap to reduce dimensionality of the training set (off-line) Run particle filter on test sequence to track the person

Training Data

Isomap of Training Data

Isomap Discrepancy Isomap gave dimensionality of 2 when head poses moving up were removed. Thus, the dimensionality of 3 recovered by training data can be attributed to the non-symmetry of the face about the horizontal axis.

Weighting Particles by SSD

Weighting Particles by Chamfer distance

State evolution without resampling

State evolution with resampling

Experimental Results

Videos

Videos Continued

Conclusion and Future work Isomap provides good frame-work for pose estimation Algorithm can track and estimate a person’s pose at the same time Use of particle filter allows parallel implementation Goal is to be able to build an Isomap on-line so that the particle filter tracker can learn as it tracks

Thank You!