1. From Interactive to Semantic Image Segmentation Varun Gulshan Supervisors: Prof. Andrew Blake Prof. Andrew Zisserman 20 Jan 2012

2. Two segmentation tasks car building person tree sky road bench object background Interactive segmentationSemantic segmentation

3. Interactive segmentation Semantic segmentation Thesis Flow Chapter 3: Texture Features Low level cues Chapter 4: Star convexity Mid level cues Chapter 5: Segmenting humans Bounding box interaction + Top down cues Chapter 6: Superpixel based classification Fully automatic segmentation

4. Chapter 3: Features for interactive segmentation Low level texture features for improving interactive segmentation methods.

5. Texture features Pure Texture Feature (L-shape): Texture + Gray Feature (L-shape):

6. Texture features Pure Texture Feature (Plus-shape): Texture + Gray Feature (Plus-shape):

7. Camouflage Image Dataset Introduced a dataset of 50 Camouflage images, to demonstrate the power of texture features.

8. Quantitative evaluation +14% +21% +7%+4% Gray RGB Huge gain in accuracy obtained using texture features on top of gray scale images. Significant improvement on top of RGB images.

9. Chapter 4: Star Convexity and Extensions Mid level shape constraints for reducing user effort in interactive segmentation systems.

10. Chapter 4: Star convexity Single Star Multiple Stars Geodesic Star

11. Robot user evaluation Updated segmentation Biggest connected component Initial brush strokes Segmentation output with current interaction Error segmentation False positive False negative New Brush Stroke Centre of connected component New Brush Stroke New brush stroke placed Process is repeated upto 20 strokes Segmentation after 20 strokes

12. Robot user evaluation MethodSP-IGSP-SIGSP-LIGBJRWGSCseq Effort17.7815.7715.1412.3512.319.63 Our method takes least effort

13. Chapter 5: Learning to segment humans Using top down cues to segment specific object categories.

14. Segmenting humans Bounding box (given/detected) Top down HOG prediction Bottom up refinement

15. Kinect Data Acquisition RGB imageKinect scene labelsCleaned up Ground truth Dataset of roughly 3500 images acquired using the Kinect

16. Top down learning Local ImageLocal HOGLocal mask Classifier trained to predict segmentation masks for local windows based on their HOG descriptor.

17. Bottom up refinement Top down segmentation Local Color model window Local color model unaries Final segmentation …..

18. Chapter 6: Semantic segmentation Fully automatic segmentation based upon learning from multiple superpixelisations.

19. Combing multiple superpixelisations Various methods to learn from multiple superpixelisations explored: 1. Avg-Indep 2. Avg-Union 3. LPβ-Indep 4. IofR-Joint GlobalPbVekslerQuickShift

20. Quantitative evaluation +7%+5%+6%+3% Single superpixelisation Multiple superpixelisations Combining multiple superpixelisations improves performance.

21. Novel pairwise features.... CRF trained jointly for appearance and novel pairwise features.

22. Over to you!