Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui Graduate school of Systems and Information Engineering University of Tsukuba (Japan)

2 Abstract  We proposed a statistical method for object classification based on multi-view.  The proposed method is an extension of MSM.  Problems of previous works (MSM, KMSM) has been solved.  We evaluated the classification performance of the proposed method and previous works.

3 Table of Contents  Backgrounds  Multi-view classification and problems of existing methods for nonlinear distribution  The proposed method  Approximation by local subspaces and ensemble learning  Experimental results  Performance comparison using multi-view images of objects  Summary

4 Multi-view Based Object Classification  Multiple view is more beneficial for object classification than single.  View based approach does not need 3D model for classification.  Subspace related methods has been proposed for multi-view frame work.

5 Existing view based approaches  Mutual Subspace Method (MSM) O.Yamaguchi, K.Fukui, K.Maeda: Face recognition using temporal image sequence. Proc.IEEE 3rd International Conference on Automatic Face and Gesture Recognition, pp ,1998.  Pro Low computation cost  Con Cannot handle nonlinear distribution

6 Multi-view Based Object Classification  Feature vectors from multi-view input are likely to have nonlinear distribution.  Subspace approximation therefore is not accurate.

7 Existing view based approaches  Kernel Mutual Subspace Method (KMSM) H.Sakano, N.Mukawa: Kernel mutual subspace method for robust facial image recognition. Proc. 4th International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies, Vol.1, pp ,  Project feature vectors to a high dimensional space to reduce their nonlinearity.  Pros:  Can handle nonlinear distribution.  Cons:  Consume more computation time, and this increases at order N^2 as learning data increases.  Some critical parameters need to be optimized

8 Motivation of the Proposed Method  Approximate nonlinear distribution.  Divide the original distribution, and approximate each subset.  Achieve comparable classification performance with KMSM, using less computation.  Perform classification without kernel framework.

9 Table of Contents  Backgrounds  Multi-view classification and problems of existing methods for nonlinear distribution  The proposed method  Approximation by local subspaces and ensemble learning  Experimental results  Performance comparison using multi-view images of objects  Summary

10 The Proposed Method  Divide the nonlinear distribution into several subsets based on Euclidean distance.  Nonlinearity of each subset is weaker.  We approximate each subset with local subspace.

11 Ensemble Classification  Division number and dimension of each local subspace are parameters affect classification performance.  We construct local subspaces under multiple combination.  We assume each case as weak classifier and apply ensemble learning.

12 The Proposed Method with Weight  Each local subspace carries a weight coefficient based on its classification performance.  This coefficient weights the canonical angle.  The coefficients are simply the accuracy rate of each local subspace in preliminary experiments.

13 The Proposed Method

14 Table of Contents  Backgrounds  Multi-view classification and problems of existing methods for nonlinear distribution  The proposed method  Approximation by local subspaces and ensemble learning  Experimental results  Performance comparison using multi-view images of objects  Summary

15 Classification Experiment  Compare the classification performance and computation cost of MSM, KMSM and the proposed method.  Use multi-view image data set ‘ The ETH-80 Image Set ’ B.Leibe, B.Schiele: Analyzing appearance and contour based methods for object categorization. Proc. CVPR'03, Vol.2, pp , 2003

16 Classification Experiment  The dataset contains 8 classes, 10 objects for each class.  41 points of view for each object  Feature vector is the resized image (16x16)

17 Classification Experiment  Use 164 images to learn for each class. (generate dictionary)  Evaluation input is images from an unknown object. (10 images/points of view)  By exchanging learning data and evaluation data (leave-one-out), we repeated experiment for 1640 times.

18 Classification Experiment  Classification performance of each method MethodAccuracy Rate(%) SeparabilityEER(%) MSM KMSM Proposed method Proposed method with weight

19 Classification Experiment  The proposed method improved classification performance from MSM  The proposed method achieved comparable performance with KMSM  Classification performance of the proposed method was improved by introducing weight to each local subspace

20 Classification Experiment  Computation cost of each method MethodCalculation Time (seconds per input) Order MSM0.1O(n) Proposed method 0.4O(n) Proposed method with weight 0.4O(n) KMSM3.1O(n^2)

21 Classification Experiment  The proposed method consumes less computation time compared with KMSM  The computation time of the proposed method increases in order N as learning data increases

22 Classification Experiment  Classification performance of weak classifiers Proposed method

23 Table of Contents  Backgrounds  Multi-view classification and problems of existing methods for nonlinear distribution  The proposed method  Approximation by local subspaces and ensemble learning  Experimental results  Performance comparison using multi-view images of objects  Summary

24 Summary  We proposed a method to achieve comparable performance with KMSM by less calculation.  Classification performance is further improved by introducing weight to each local subspace  The advantages of the proposed method is shown with classification experiment of objects.

Thank You Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces (ThBT4.3) Jianing Wu, Kazuhiro Fukui