Learning Collections of Parts for Object Recognition and Transfer Learning University of Illinois at Urbana- Champaign

Flexible Part Based Model Why? – Interested in learning a large number of object categories – Avoid learning new category from scratch when useful information can be borrowed from other categories

Flexible Part Based Model: Objectives Simple to train – Minimal manual initialization effort – Train each part independently – Simple spatial model 3 How Can We Adapt Existing Part Models to New Categories?

Boosted Collections of Parts 4 Simple to train – Minimal manual initialization effort – Train each part independently – Simple spatial model ECCV 2010 How Can We Adapt Existing Part Models to New Categories?

Part Refinement Retrain with new examples Train Part Detector Collect Consistent Positives Initialize with Single Exemplar 5

Compute expected part position from exemplar: Transfer to other examples: Encouraging Spatial Consistency 6 How Can We Adapt Existing Part Models to New Categories?

Only allow candidates with sufficient overlap with expected position Encouraging Consistency 7 GOOD BAD How Can We Adapt Existing Part Models to New Categories?

Learned Part Models 8 How Can We Adapt Existing Part Models to New Categories?

Learned Part Models 9 How Can We Adapt Existing Part Models to New Categories?

Part Evaluation: Discrimination 1.How discriminative are our parts? (mean AP) PlaneBikeBoatCatDogSofa Exemplar Refined: All-in Selective: Appearance Selective: App.+Spatial

Part Evaluation: Spatial Consistency 1.How discriminative are our parts? (mean AP) 2.How well can we localize Poselet Keypoint annotations? (mean best AP per keypoint type) PlaneBikeBoatCatDogSofa Exemplar Refined: All-in Selective: Appearance Selective: App.+Spatial

Pooling Part Detections Propose 500 candidate object regions per image (Endres and Hoiem 2010)

Pooling Part Detections Collect highest scoring response for each part:

Pooling Part Detections Collect highest scoring response for each part:

Scoring Object Candidates Classify vector of scores using boosted classifier

Post-Processing Without an explicit spatial model, overlapping regions tend to receive similar category scores Re-predict bounding box by having each part detector cast a weighted vote for a refined bounding box

Relocalization Loose spatial model: Good parts can be assigned to bad regions

Relocalization Loose spatial model: Good parts can be assigned to bad regions Solution 1: Region shape features to down-weight bad regions HOG of segmentation mask

Relocalization Loose spatial model: Good parts can be assigned to bad regions Solution 2: Use parts to repredict bounding box Each part votes for box Weighted average based on appearance score and learned reliability

Results: Beating state of the art Our Model Felzenszwalb et al. Aeroplane > 48.4 AP Cat > 36.9 AP Dog 8.5 -> 20.9 AP

Results: Competitive with state of the art Bicycle > 43.0 AP Boat 6.7 -> 5.0 AP Sofa > 14.1 AP Our Model Felzenszwalb et al.

Adapting Models: Finding analogous parts

Transfer

Adapting Parts: Refining Model Adapt each part’s appearance using object model transfer from e.g. Aytar and Zisserman,

Conclusion Goal: Recognition systems that can give as much detail about any object they encounter Consider supervised tasks that generalize across categories Capture shared similarities across categories and differences within categories 25