1. Category Independent Region Proposals Ian Endres and Derek Hoiem University of Illinois at Urbana-Champaign

2. Finding Objects

3. Scanning Window Horse Dog Cat Car Train … 10,000+ windows

4. Category Independent Search ~100 regions

5. Finding Unfamiliar Objects

6. Finding Objects Objectives: 1.Minimize number of proposed regions 2.Maintain high recall of all objects 3.Provide detailed spatial support (i.e. segmentation)

7. Challenges Objects extremely diverse – Variety of shapes, sizes – Many different appearances Within object variation – Multiple materials and textures – Strong interior boundaries Many objects in an image

8. Overview 1 2 3 4... Generate Proposals: Maximize recall Rank Proposals: Small diverse set of object regions

9. Generating Proposals 1. Select Seed2. Compute affinities for seed 3. Construct binary CRF + Unary term: Affinities Pairwise term: Occlusion Boundaries 4. Compute proposal 5. Change parameters Repeat

10. Generating Seeds Compute occlusion boundaries (Hoiem et al. ICCV ‘07) Generate hierarchal segmentation – Incrementally merge regions of oversegmentation Use regions with sufficient size and boundary strength – Avoids redundant or uninformative seeds

11. Region Affinity Learned from pairs of regions belonging to an object – Computed between the seed and each region of the hierarchy – Features: color and texture similarity, boundary crossings, layout agreement

12. Color/Texture Similarity Color, texture histograms for each region Compute histogram intersection distance between two regions

13. Boundary Crossing Draw line between region centers of mass Compute strength of occlusion boundaries crossed

14. Layout Agreement Predict object extent from each region Compute strength of agreement between two regions

15. CRF Segmentation Binary segmentation Graph composition: – Nodes: Superpixels – Edges: Adjacent superpixels +

16. CRF Segmentation Graph Potentials – Unary Potential: affinity values for each superpixel – Edge Potential: occlusion boundary strength Parameters (25 combinations) – Node/Edge weight tradeoff – Node bias + Unary potential: Affinities Edge potential: Occlusion Boundaries

17. Ranking Proposals w T X 1 w T X 3 Appearance scores w T X 4 1. 2. 3. 4. w T X 2 Sort scores Generated Ranking

18. Lacks Diversity But in an image with many objects, one object may dominate 1 2 3 4 … 20 150 100 … 50 … …

19. Encouraging Diversity Suppress regions with high overlap with previous proposals … 1 2 3 10 4 … 20 50 100 … …

20. Ranking as Structured Prediction Find the max scoring ordering of proposals Greedily add proposals with best overall score Appearance score Overlap penalty Gives higher weight to higher ranked proposals Overall score

21. Learning to Rank (Max-margin Structured Learning) Score of ground truth ordering (R (n) ) should be greater than all other orderings (R): Loss ( ) encourages good orderings: – Higher quality proposals should have higher rank – Each object should have a highly ranked proposal

22. Experimental Setup Train on 200 BSDS images Test 1: 100 BSDS images Test 2: 512 Images from Pascal 2008 Seg. Val.

23. Evaluation Region overlap Recall at 50% region overlap – Typically more strict that 50% bounding box overlap – Measures detection quality and segment quality A i A j

24. Qualitative Results Pascal BSDS (Rank, % overlap)

25. Vs. Standard Segmentation Standard: 53% 3000 proposals Ours: 53% 18 proposals Standard: 80% 70,000 proposals (merge 2 adjacent regions) Ours: 80% 180 proposals

26. Recalling Pascal Categories

27. Future work Object Discovery Incorporate into detection systems – Label regions directly – Voting from proposed regions Refine proposals with domain knowledge – i.e. wheel or head models