São Paulo Advanced School of Computing (SP-ASC’10). São Paulo, Brazil, July 12-17, 2010 Looking at People Using Partial Least Squares William Robson Schwartz

2 Introduction Partial Least Squares Proposed Method Experimental Results Conclusions Introduction How to improve results: don’t use pixel, use feature combination! Consequence: high dimensional data (>170,000 dimensions!) Solution: Partial Least Squares (PLS)

3 Introduction Partial Least Squares Proposed Method Experimental Results Conclusions Introduction Works:  Face recognition (ECCV 2010)  Pedestrian detection (ICCV 2009)  Human Detection under Partial Occlusion (ICB 2009)  Appearance-based modeling (SIBGRAPI 2009)  Data-driven detection optimization (under submission) Pedestrian detection (ICCV 2009)

4 Introduction Partial Least Squares Proposed Method Experimental Results Conclusions Introduction Characteristics of humans in standing positions:  Strong vertical edges along the boundaries of the body;  Clothing is generally uniform. Clothing textures are different from natural textures;  Discriminatory color information is found in the face/head regions;

5 Introduction Partial Least Squares Proposed Method Experimental Results Conclusions Introduction Therefore, edges, colors and textures capture important cues for discriminating humans from the background. Features used:  Histogram of Oriented Gradient (HOG) descriptors;  Color frequency;  Texture features computed from co-occurrence matrices;

6 Introduction Partial Least Squares Proposed Method Experimental Results Conclusions Introduction Consequences of feature augmentation:  High dimensional feature spaces (> 170,000);  The number of samples for training is much smaller than features;  Sampling with overlapping regions increases the multicollinearity of the feature set; An ideal setting for a statistical technique known as Partial Least Squares (PLS).

7 Introduction Partial Least Squares Proposed Method Experimental Results Conclusions Partial Least Squares PLS is a class of methods for modeling relations between sets of variables using latent spaces. Regression and class aware dimensionality reduction. Feature vectors are projected onto projection vectors estimated by PLS, then a classifier is used in the resulting low dimensional sub-space.

8 Introduction Partial Least Squares Proposed Method Experimental Results Conclusions Partial Least Squares PCA: PLS: X: data matrix (n x p) y: response variable (n x 1) W: projection matrix (p x m) PLS constructs orthogonal latent components

9 Introduction Partial Least Squares Proposed Method Experimental Results Conclusions Framework Dimensionality reduction Feature extraction Classification PLS model Detection windows Probability mapFinal Result Input Image Detection window split into overlapping blocks (blocks of multiple sizes)

10 Introduction Partial Least Squares Proposed Method Experimental Results Conclusions Feature Combination Features extract different information. Combination leads to an improvement over single feature- based approach.

11 Introduction Partial Least Squares Proposed Method Experimental Results Conclusions Experimental Results (170,820 features)(33,312 features) INRIA Pedestrian DatasetDaimlerChrysler Dataset

12 Introduction Partial Least Squares Proposed Method Experimental Results Conclusions Experimental Results

13 Introduction Partial Least Squares Proposed method Experimental Results Conclusions 13 Conclusions Combination of different features leads to more accurate results. This richer set of features analyzed by PLS provides improvements in human detection.