1. Learning Collections of Parts for Object Recognition and Transfer Learning University of Illinois at Urbana- Champaign

2. Flexible Part Based Model Why? – Interested in learning a large number of object categories – Avoid learning new category from scratch when useful information can be borrowed from other categories

3. Flexible Part Based Model: Objectives Simple to train – Minimal manual initialization effort – Train each part independently – Simple spatial model 3 How Can We Adapt Existing Part Models to New Categories?

4. Boosted Collections of Parts 4 Simple to train – Minimal manual initialization effort – Train each part independently – Simple spatial model ECCV 2010 How Can We Adapt Existing Part Models to New Categories?

5. Part Refinement Retrain with new examples Train Part Detector Collect Consistent Positives Initialize with Single Exemplar 5

6. Compute expected part position from exemplar: Transfer to other examples: Encouraging Spatial Consistency 6 How Can We Adapt Existing Part Models to New Categories?

7. Only allow candidates with sufficient overlap with expected position Encouraging Consistency 7 GOOD BAD How Can We Adapt Existing Part Models to New Categories?

8. Learned Part Models 8 How Can We Adapt Existing Part Models to New Categories?

9. Learned Part Models 9 How Can We Adapt Existing Part Models to New Categories?

10. Part Evaluation: Discrimination 1.How discriminative are our parts? (mean AP) PlaneBikeBoatCatDogSofa Exemplar15.217.43.523.618.16.6 Refined: All-in36.539.74.042.325.88.0 Selective: Appearance38.139.95.746.529.58.3 Selective: App.+Spatial37.337.24.639.524.48.7

11. Part Evaluation: Spatial Consistency 1.How discriminative are our parts? (mean AP) 2.How well can we localize Poselet Keypoint annotations? (mean best AP per keypoint type) PlaneBikeBoatCatDogSofa Exemplar14.134.612.412.88.99.1 Refined: All-in21.341.39.622.012.97.2 Selective: Appearance23.941.613.922.514.711.1 Selective: App.+Spatial27.342.414.822.213.310.8

12. Pooling Part Detections Propose 500 candidate object regions per image (Endres and Hoiem 2010)

13. Pooling Part Detections Collect highest scoring response for each part:

14. Pooling Part Detections Collect highest scoring response for each part:

15. Scoring Object Candidates Classify vector of scores using boosted classifier

16. Post-Processing Without an explicit spatial model, overlapping regions tend to receive similar category scores Re-predict bounding box by having each part detector cast a weighted vote for a refined bounding box

17. Relocalization Loose spatial model: Good parts can be assigned to bad regions

18. Relocalization Loose spatial model: Good parts can be assigned to bad regions Solution 1: Region shape features to down-weight bad regions HOG of segmentation mask

19. Relocalization Loose spatial model: Good parts can be assigned to bad regions Solution 2: Use parts to repredict bounding box Each part votes for box Weighted average based on appearance score and learned reliability

20. Results: Beating state of the art Our Model Felzenszwalb et al. Aeroplane 44.3 -> 48.4 AP Cat 24.1 -> 36.9 AP Dog 8.5 -> 20.9 AP

21. Results: Competitive with state of the art Bicycle 49.6 -> 43.0 AP Boat 6.7 -> 5.0 AP Sofa 17.2 -> 14.1 AP Our Model Felzenszwalb et al.

22. Adapting Models: Finding analogous parts

23. Transfer

24. Adapting Parts: Refining Model Adapt each part’s appearance using object model transfer from e.g. Aytar and Zisserman, 2012 24

25. Conclusion Goal: Recognition systems that can give as much detail about any object they encounter Consider supervised tasks that generalize across categories Capture shared similarities across categories and differences within categories 25