1. Training and Evaluating of Object Bank Models Presenter ： Changyu Liu Advisor ： Prof. Alex Interest ： Multimedia Analysis May 16 th, 2013

2. CMU - Language Technologies Institute 2 Contents Dataset Setting Model Training Model Evaluation in Deformable Part Model Evaluation in Object Bank Conclusion and Plan

3. CMU - Language Technologies Institute 3 Dataset Setting --- Object Lists OB IDObject NameWNID 10477knifen02973904 1253balloonn02782093 12498snailn01944390 11515candlen02948072 1176soccer balln04254680 1190laptopn03642806 1232airplanen02690373 12982carn02701002 1329boatn03329663 1103cown01887787 In this experiment, we firstly choose 10 objects, as: Table 1 Selected 10 Objects

4. CMU - Language Technologies Institute 4 Dataset Setting --- Sample Configuration 1.Then, choose 961 total image(about 100 for each object) for training, 958 total image for evaluation, and 1331 total image for testing. 2.All these images are divided by 1:4 for positive and negative samples and are all from Image Net (http://www.image-net.org/) with most of them having a bounding box annotation.http://www.image-net.org/

5. CMU - Language Technologies Institute 5 Dataset Setting --- Sample Configuration 3. We use these images to substitute VOC 2008 dataset and have generated as well as evaluated four deformable part models (other six models are on the way).

6. CMU - Language Technologies Institute 6 Contents Dataset Setting Model Training Model Evaluation in Deformable Part Model Evaluation in Object Bank Conclusion and Plan

7. CMU - Language Technologies Institute 7 Model Training ---Overview In order to use Object Bank features, object models should be trained firstly. Here we introduced a Deformable Part Model(Felzenszwalb, CVPR 2008) for such training. The current adopted version was voc-release 3.l.

8. CMU - Language Technologies Institute 8 Fig. 1 Deformable Part Model Model Training --- Deformable Part The deformable model include both a coarse global template and higher resolution part templates. The templates represent histogram of gradient features (b1) coarse template (b2)part templates(b3) spatial model (a) person detection Example

9. CMU - Language Technologies Institute 9 Model Training --- Results On average, it generated 1.5 models each day on the CQ-serials desktop. After training, we got 9.mat model file, as: balloon_final.mat snail_final.mat candle_final.mat soccer ball_final.mat laptop_final.mat airplane_final.mat car_final.mat boat_final.mat cow_final.mat

10. CMU - Language Technologies Institute 10 Contents Dataset Setting Model Training Model Evaluation in Deformable Part Model Evaluation in Object Bank Conclusion and Plan

11. CMU - Language Technologies Institute 11 Model Evaluation --- Deformable Part Then, we had a evaluation of each object on the selected 958 images, and got the Average Precision distribution map, as:

12. CMU - Language Technologies Institute 12 Model Evaluation --- Deformable Part Fig. 2 AP of Airplane In which AP is average precision, Bbox 1 is bounding box from root placements, and Bbox 2 is bounding box from using predictor function.

13. CMU - Language Technologies Institute 13 Model Evaluation --- Deformable Part Fig. 3 AP of Balloon

14. CMU - Language Technologies Institute 14 Model Evaluation --- Deformable Part Last, we got 9 objects average precision, as: ObjectAP of Bbox1AP of Bbox2 balloon0.4280.439 snail0.1840.201 candle0.2030.196 soccer ball0.376 laptop0.4720.479 airplane0.6440.652 car0.5180.526 boat0.4950.488 cow0.4160.405 Table 2 Average precision of nine objects Then, got 9 google images(1 image for each object for a bounding box test.

15. CMU - Language Technologies Institute 15 Model Evaluation --- Deformable Part Fig. 4 Balloon

16. CMU - Language Technologies Institute 16 Model Evaluation --- Deformable Part Fig. 5 Candle

17. CMU - Language Technologies Institute 17 Model Evaluation --- Deformable Part Fig. 6 Cow

18. CMU - Language Technologies Institute 18 Model Evaluation --- Deformable Part Fig. 7 Laptop

19. CMU - Language Technologies Institute 19 Model Evaluation --- Deformable Part Fig. 8 Soccer ball

20. CMU - Language Technologies Institute 20 Contents Dataset Setting Model Training Model Evaluation in Deformable Part Model Evaluation in Object Bank Conclusion and Plan

21. CMU - Language Technologies Institute 21 Model Evaluation --- Object Bank ObjectCorrelation Coefficient balloon0.71806 snail0.86498 candle0.85893 soccer ball0.84165 laptop0.73821 airplane0.79783 car0.48926 boat0.75255 cow0.71712 Table 3 Correlation Coefficient The second evaluation was tested on Object Bank.

22. CMU - Language Technologies Institute 22 Model Evaluation --- Object Bank

23. CMU - Language Technologies Institute 23 Contents Dataset Setting Model Training Model Evaluation in Deformable Part Model Evaluation in Object Bank Conclusion and Plan

24. CMU - Language Technologies Institute 24 Conclusion Conclusion, 1)The width or height of selected image must >= 4 HOG bin(4*8 pixels). 2)It is feasible to use v3.1(not v5) code to generate object models for getting Object Bank features, and it took 1/1.5 day to get one model. The plan for next steps is, 1) Move these codes to PSC for a further test in order to improve the process speed. 2) Find what the needed 1000 objects names are. 3) Choose and Make the dataset from Image Net.

25. CMU - Language Technologies Institute 25 Reference [1] P. Felzenszwalb, D. McAllester, D. Ramanan. A Discriminatively Trained, Multiscale, Deformable Part Model. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2008 [2] P. Felzenszwalb, R. Girshick, D. McAllester, D. Ramanan. Object Detection with Discriminatively Trained Part Based Models. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 32, No. 9, Sep. 2010. [3] Level Image Representation for Scene Classification and Semantic Feature Sparsification. Proceedings of the Neural Information Processing Systems (NIPS), 2010.

26. CMU - Language Technologies Institute 26 Thank you!