1. WSCG2008, Plzen, 04-07, Febrary 2008 Comparative Evaluation of Random Forest and Fern classifiers for Real-Time Feature Matching I. Barandiaran 1, C.Cottez 1, C.Paloc 1, M.Graña 2 1 Departamento de Aplicaciones Biomédicas Asociación VICOMTech, San Sebastián, {ibarandiaran,ccottez,cpaloc}@vicomtech.org 2 University of Basque Country Computer Science School, Pº. Manuel de Lardizabal, 1 20009, San Sebastián, Spain ccpgrrom@si.ehu.es VISUAL INTERACTION AND COMMUNICATIONS TECHNOLOGIES

2. 2 Summary 1.Introduction. 2.Random Forest, FERNS 1.Mixed/Augmented Reality Application. 2.Conclusions/Questions.

3. 3 Introduction Motivation and objectives Motivated by the work of Vincent LePetit (CVLab). Real-Time Augmented Reality. Camera Pose Estimation. Markerless tracking. Model-based tracking. Tracking by detection. Test and compare different parameters. Scale. Size of the Training Set. Number of Classes. Training Time.

4. 4 Augmented Reality Features: Mix Virtual and Real Objects.. Real-Time. Portable Devices ( Head Mounted Display, Tablet PC, PDA Device, Movil Phone.. ) Introduction

5. 5 Problems: Rendering. Real-Time(Delay). Registration/Pose Estimation. Introduction

6. 6 Non model-based Tracking No a priori knowledge of the object to be tracked. Updates/Propagates an estimation over time. Partial object occlusions. Tend to tracking reinitialization. Model Based Tracking Some a priori knowledge is available. May not depend on the past. Frame by Frame estimation. Robust against partial object occlusion. Automatic tracking initialization. Introduction

7. 7 1.Introduction 2.Random Forest, FERNS 3.Mixed/Augmented Reality 4.Conclusions/Questions Summary

8. 8 Random Forest, FERNS Tracking of Planar Surfaces. The Classifiers are applied for interest point (feature) matching. Matched Points are used during camera pose estimation Process.

9. 9 Random Forest, FERNS Building the training set. Frontal view of the object to be detected. Feature Point extraction FAST (Rosten06) and YAPE (CvLab). Sub-images (patches) are generated for each class. Classes to Be recognized by the Classifier

10. 10 Random Forest, FERNS Building the training set. Generate Random Affine transformations. Generate new examples of each Class. ….. Random Affine transformations Training Set (examples)

11. 11 Random Forest Multiclassifier based on Randomized Trees. Firstly introduced in 1997 handwritten recognition (Amit, Y.,German, D.) Developed by Leo Breiman (Medical Data Analisys). Recently Applied to tracking by detection (LePetit06). Main Features Fast Training Step, and execution. Good Precision. Random selection of the independent variables (features). Random selection of Examples. Easy to Implement and paralelizable.

12. 12 Random Forest Classifier Training. N Binary-Trees are Grown. Pixel intensity tests are executed in any non-terminal node. Pixels can be selected at Random. Posterior Distributions P(Y=c |T=Tk,n) are stored in leave nodes.

13. 13 Random Forest Example Classification. Every example is dropped down the trees. The Example traverse the tree towards the leaf nodes. Pixels to be tested

14. T1T1T1T1 T2T2T2T2 TnTnTnTn Random Forest 14 Random Forest Combine Results The example labeling is obtained as a combination of partial results obtained by every tree in the forest.

15. 15 FERNS Introduced in 2007 (Mustafa Özuysal). Multiclassifier. Applied to 3D keypoint recognition. Successfully applied to image recognition/retrieval (Zisserman07). Main Features Non hierarchical structure. Semi Naive-Bayes Combination Strategy. Random selection of the independent variables (features). Random selection of Examples. Easy to Implement and paralelizable.

16. 16 FERNS Semi-Naive Bayes Combination.

17. 17 FERNS Classifier Training xxx Possible Outputs 0 7...... Posterior Distributions (Look-up Tables)

18. 18 FERNS Classifier Training 110 000 010 6 0 011 2 3 Class 1 Class 2 Class 1 Class 2.......... Fern 1 Fern 2 Fern n Posterior Distributions (Look-up Tables)....

19. 19 FERNS Example Classification. 261 Posterior Distributions (Look-up Tables) Fern 1 Fern 2 Fern 3

20. 20 Random Forest vs FERNS Rotation Range 20 Trees, 15 Depth. 225 Different Clases. 400 Images per class.

21. 21 Random Forest vs FERNS Scale Range 20 Trees, 15 Depth. 225 Different Clases. 400 Images per class.

22. 22 Random Forest vs FERNS Size of the training Set 20 Trees, 15 Depth. 225 Different Classes. [0.5-1.5] Scale Range.

23. 23 Random Forest vs FERNS Number of different Classes. 20 Trees, 15 Depth. [0.8-1.2] Scale Range. 1500 Training images per class.

24. 24 Random Forest vs FERNS Training time. 20 Trees, 15 Depth. 225 Different Classes. [0.5-1.5] Scale Range.

25. 25 Pose Estimation Homography Estimation Robust Estimation (RANSAC). Non-Linear Minimization (Levenberg-Marquardt).

26. 26 Summary 1.Introduction. 2.Random Forest, FERNS. 3.Mixed/Augmented Reality Application. 4.Conclusions/Questions.

27. 27 European Project IMPROVE (Improving Display and Rendering Technology for Virtual Environments) Develop of new interaction metaphors. Develop of new Displays. Photo Realistic Rendering. Development of Markerless Tracking Techniques. Augmented Reality Application

28. 28 Architectural Scenario Automotive Scenario

29. 29 Marker-Less tracking (InDoor Scenario) Feature Points Tracking Textured plane Image Augmentation Augmented Reality Application

30. 30 Marker-Less tracking (OutDoor Scenario) Image Acquisition Feature points Tracking Image Augmentation Augmented Reality Application

31. 31 Performance 20 Trees. Full Rotation Range and [0.8-1.2] Scale Range. 1000 images per Class. 250 Different Classes. Augmented Reality Application

32. 32 Summary 1.Introduction. 2.Random Forest, FERNS. 3.Mixed/Augmented Reality Application. 4.Conclusions/Questions.

33. 33 Conclusions Both Approaches are very Similar. The classifier is more sensitive to variations in scale. The classifier is robust against variations in object orientation. When the classifier converges, increase the number of trees does not improve accuracy. The node test can be selected at random. FERNS Requires more Memmory Than Random Forest. Training and classification Time is Higher in FERNS than in Random Forest. Random Forest are Faster than FERNS (without heuristics) FERNS Supports more classes than Random Forest. The Output of both classifiers must be filtered. The higher the classification accuracy, the better the performance of the tracking.

34. Thanks For Listening Iñigo Barandiaran Martirena (ibarandiaran@vicomtech.org) Researcher, VICOMTech Paseo Mikeletegi 57 20009 San Sebastián Tfno: +34 943 30 92 30Fax : +34 943 30 93 93