1. ISOMAP TRACKING WITH PARTICLE FILTER Presented by Nikhil Rane

2. Dimensionality Reduction Let x i be H-dimensional and y i be L- dimensional then dimensionality reduction solves the problem x i = f (y i ) where H>L

3. Dimensionality Reduction Techniques Linear PCA Transforms data into a new coordinate system so that largest variance in on the 1 st dimension, 2 nd largest along 2 nd dimension … Classical MDS Preserves Euclidean distances between points Nonlinear Isomap Preserves geodesic distances between points LLE Preserves local configurations in data

4. Face Database

5. Principal Components Analysis (PCA) 1) Make the mean of the data zero 2) Compute covariance matrix C 3) Compute eigenvalues and eigenvectors of C 4) Choose the principal components 5) Generate low-dimensional points using principal components

6. Performance of PCA on Face- data

7. Classical Multidimensional Scaling (MDS) Compute Distance Matrix S Compute inner product matrix B = -0.5JSJ where J = I N – (1/N)11 T Decompose B into eigenvectors and eigenvalues Use top d eigenvectors and eigenvalues to form the d dimensional embedding.

8. Performance of MDS on face- data

9. Locally Linear Embedding (LLE) Find neighbors of each data point Compute weights that best reconstruct each data point from its neighbors Compute low-dimensional vectors best reconstructed by the weights

10. Performance of LLE on Face- data

11. Geodesic Distance Geodesic distance – the length of the shortest curve between two points taken along the surface of a manifold

12. Isometric Feature Mapping (Isomap) Construct neighborhood graph Compute shortest paths between points Apply classical MDS

13. Performance of Isomap on face- data

14. Tracking vs. Detection Detection - locating an object independent of the past information When motion is unpredictable For reacquisition of a lost target Tracking - locating an object based on past information Saves computation time

15. Recursive Bayesian Framework Estimate the pdf of state at time t given the pdf of state at time t - 1 and measurement at time t Predict Predict state of the system at time t using a system- model and pdf from time t – 1 Update Update the predicted state using measurement at time t by Bayes’ rule

16. Kalman Filtering vs. Particle Filtering Kalman filter assumes the pdf of the state to be Gaussian at all times and requires the measurement and process noise to be Gaussian Particle filter makes no such assumption and in fact estimates the pdf at every time-step

17. Resampling

18. Condensation algorithm Algorithm – 1) Resample 2) Predict 3) Measure

19. Condensation algorithm

20. Isomap Tracking with Particle Filtering Create training set of a person’s face (off-line) Use Isomap to reduce dimensionality of the training set (off-line) Run particle filter on test sequence to track the person

21. Training Data

22. Isomap of Training Data

23. Isomap Discrepancy Isomap gave dimensionality of 2 when head poses moving up were removed. Thus, the dimensionality of 3 recovered by training data can be attributed to the non-symmetry of the face about the horizontal axis.

24. Weighting Particles by SSD

25. Weighting Particles by Chamfer distance

26. State evolution without resampling

27. State evolution with resampling

28. Experimental Results

29. Videos

30. Videos Continued

31. Conclusion and Future work Isomap provides good frame-work for pose estimation Algorithm can track and estimate a person’s pose at the same time Use of particle filter allows parallel implementation Goal is to be able to build an Isomap on-line so that the particle filter tracker can learn as it tracks

32. Thank You!