Presentation is loading. Please wait.

Presentation is loading. Please wait.

Unsupervised Learning of Categorical Segments in Image Collections *California Institute of Technology **Technion Marco Andreetto*, Lihi Zelnik-Manor**,

Published by Modified over 9 years ago

Similar presentations

Presentation on theme: "Unsupervised Learning of Categorical Segments in Image Collections *California Institute of Technology **Technion Marco Andreetto*, Lihi Zelnik-Manor**,"— Presentation transcript:

1 Unsupervised Learning of Categorical Segments in Image Collections *California Institute of Technology **Technion Marco Andreetto*, Lihi Zelnik-Manor**, Pietro Perona* The Sixth IEEE Computer Society Workshop on Perceptual Organization in Computer Vision (POCV 2008)

2 Outline Motivation and related work A probabilistic model for single image segmentation Unsupervised learning of categorical segments Experimental results Conclusions and future works

3 Outline Motivation and related work A probabilistic model for single image segmentation Unsupervised learning of categorical segments Experimental results Conclusions and future works

4 Motivation

5 Normalized cuts: Shi and Malik PAMI 2000

6 Motivation

7

8 Categorical segments: from human segmentation

9 Motivation

10 Related works Russell et al. CVPR 2006 Cao and Fei-Fei ICCV 2007 Wang and Grimson NIPS 2007 Andreetto et al. ICCV 2007

11 Outline Motivation and related work A probabilistic model for single image segmentation Unsupervised learning of categorical segments Experimental results Conclusions and future works

12 An image as a set of segments

13 K = 2 N

14 An image as a set of segments K = 2  Segment probability N

15 An image as a set of segments K = 2  Segment probability fkfk K Segment density N

16 Image formation K = 2  c Segment probability Label fkfk K x Segment density N

17 What we’re looking forObserved Probabilistic model for clustering  c fkfk KN x Likelihood of x to be in cluster k

18 Non-parametric densities Sum of local kernels

19 Outline Motivation and related work A probabilistic model for single image segmentation Unsupervised learning of categorical segments Experimental results Conclusions and future works

20 N Learning categorical segments  c fkfk K x w gkgk K Segment appearance Joint for all images Segment shape/color Specific per image M

21 Visual words Filter Bank VQ w1w1 w2w2 w3w3 wNwN Filter bank: 17 outputs 256 visual words Winn et al. ICCV 2005 … …

22 Inference N  c fkfk K x w gkgk K M

23 Gibbs sampling

24 Prior term: Number of pixels in image m assigned to segment k

25 Gibbs sampling Prior term: Visual words term: Number of pixels in image m assigned to segment k Number of visual word h assigned to segments k

26 Gibbs sampling Prior term: Visual words term: Segment term: Number of pixels in image m assigned to segment k Number of visual word h assigned to segments k Non-parametric density Estimate for segment k Affinity between observations i and j

27 Outline Motivation and related work A probabilistic model for single image segmentation Unsupervised learning of categorical segments Experimental results Conclusions and future works

28 Experimental results (MSRC)

29 Classification results (MSRC) Class NameWang and GrimsonOur model DetectionFalse Al.DetectionFalse Al. Cow0.56620.03340.48890.0823 GrassN/A 0.63890.0337 Cars0.68380.24370.33130.1732 SkyN/A 0.99540.0096 FoliageN/A 0.47350.1122 SeaN/A 0.61990.0174 Bikes0.56610.37140.54360.0646 Faces0.69730.42170.61610.0429 Running time: 18.75 sec. per image

30 Experimental results (Labelme)

31 Categorical segments (Labelme) Segment 1: FoliageSegment 2: Buildings Segment 1: SkySegment 3: Street pavement

32 Categorical segments (scenes)

33 Outline Motivation and related work A probabilistic model for single image segmentation Unsupervised learning of categorical segments Experimental results Conclusions and future works

34 Conclusions We presented a model for unsupervised learning of categorical segments We describe an inference method based on Gibbs sampling We show some experimental results on a standard dataset MSRC v1.

35 Future work Faster inference method (variational approximation) Automatic inference of the number of segments Learning geometric relationships between segments

36 Thank You

37

Download ppt "Unsupervised Learning of Categorical Segments in Image Collections *California Institute of Technology **Technion Marco Andreetto*, Lihi Zelnik-Manor**,"

Similar presentations

Putting Objects in Perspective Derek Hoiem Alexei A. Efros Martial Hebert Carnegie Mellon University Robotics Institute.

Putting Objects in Perspective Derek Hoiem Alexei A. Efros Martial Hebert Carnegie Mellon University Robotics Institute.
Weakly supervised learning of MRF models for image region labeling Jakob Verbeek LEAR team, INRIA Rhône-Alpes.

Weakly supervised learning of MRF models for image region labeling Jakob Verbeek LEAR team, INRIA Rhône-Alpes.
Unsupervised Learning Clustering K-Means. Recall: Key Components of Intelligent Agents Representation Language: Graph, Bayes Nets, Linear functions Inference.

Unsupervised Learning Clustering K-Means. Recall: Key Components of Intelligent Agents Representation Language: Graph, Bayes Nets, Linear functions Inference.
Foreground Focus: Finding Meaningful Features in Unlabeled Images Yong Jae Lee and Kristen Grauman University of Texas at Austin.

Foreground Focus: Finding Meaningful Features in Unlabeled Images Yong Jae Lee and Kristen Grauman University of Texas at Austin.
Spectral graph reduction for image and streaming video segmentation Fabio Galasso 1 Margret Keuper 2 Thomas Brox 2 Bernt Schiele 1 1 Max Planck Institute.

Spectral graph reduction for image and streaming video segmentation Fabio Galasso 1 Margret Keuper 2 Thomas Brox 2 Bernt Schiele 1 1 Max Planck Institute.
A generic model to compose vision modules for holistic scene understanding Adarsh Kowdle *, Congcong Li *, Ashutosh Saxena, and Tsuhan Chen Cornell University,

A generic model to compose vision modules for holistic scene understanding Adarsh Kowdle *, Congcong Li *, Ashutosh Saxena, and Tsuhan Chen Cornell University,
Agenda Introduction Bag-of-words models Visual words with spatial location Part-based models Discriminative methods Segmentation and recognition Recognition-based.

Agenda Introduction Bag-of-words models Visual words with spatial location Part-based models Discriminative methods Segmentation and recognition Recognition-based.
Learning to Combine Bottom-Up and Top-Down Segmentation Anat Levin and Yair Weiss School of CS&Eng, The Hebrew University of Jerusalem, Israel.

Learning to Combine Bottom-Up and Top-Down Segmentation Anat Levin and Yair Weiss School of CS&Eng, The Hebrew University of Jerusalem, Israel.
Scene Labeling Using Beam Search Under Mutex Constraints ID: O-2B-6 Anirban Roy and Sinisa Todorovic Oregon State University 1.

Scene Labeling Using Beam Search Under Mutex Constraints ID: O-2B-6 Anirban Roy and Sinisa Todorovic Oregon State University 1.
Simultaneous Image Classification and Annotation Chong Wang, David Blei, Li Fei-Fei Computer Science Department Princeton University Published in CVPR.

Simultaneous Image Classification and Annotation Chong Wang, David Blei, Li Fei-Fei Computer Science Department Princeton University Published in CVPR.
Object class recognition using unsupervised scale-invariant learning Rob Fergus Pietro Perona Andrew Zisserman Oxford University California Institute of.

Object class recognition using unsupervised scale-invariant learning Rob Fergus Pietro Perona Andrew Zisserman Oxford University California Institute of.
LARGE-SCALE IMAGE PARSING Joseph Tighe and Svetlana Lazebnik University of North Carolina at Chapel Hill road building car sky.

LARGE-SCALE IMAGE PARSING Joseph Tighe and Svetlana Lazebnik University of North Carolina at Chapel Hill road building car sky.
Patch to the Future: Unsupervised Visual Prediction

Patch to the Future: Unsupervised Visual Prediction
Structural Human Action Recognition from Still Images Moin Nabi Computer Vision Lab. ©IPM - Oct. 2010.

Structural Human Action Recognition from Still Images Moin Nabi Computer Vision Lab. ©IPM - Oct
LOCUS (Learning Object Classes with Unsupervised Segmentation) A variational approach to learning model- based segmentation. John Winn Microsoft Research.

LOCUS (Learning Object Classes with Unsupervised Segmentation) A variational approach to learning model- based segmentation. John Winn Microsoft Research.
Ziming Zhang *, Ze-Nian Li, Mark Drew School of Computing Science, Simon Fraser University, Vancouver, B.C., Canada {zza27, li, Learning.

Ziming Zhang *, Ze-Nian Li, Mark Drew School of Computing Science, Simon Fraser University, Vancouver, B.C., Canada {zza27, li, Learning.
Optimization & Learning for Registration of Moving Dynamic Textures Junzhou Huang 1, Xiaolei Huang 2, Dimitris Metaxas 1 Rutgers University 1, Lehigh University.

Optimization & Learning for Registration of Moving Dynamic Textures Junzhou Huang 1, Xiaolei Huang 2, Dimitris Metaxas 1 Rutgers University 1, Lehigh University.
Biased Normalized Cuts 1 Subhransu Maji and Jithndra Malik University of California, Berkeley IEEE Conference on Computer Vision and Pattern Recognition.

Biased Normalized Cuts 1 Subhransu Maji and Jithndra Malik University of California, Berkeley IEEE Conference on Computer Vision and Pattern Recognition.
Beyond bags of features: Part-based models Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba.

Beyond bags of features: Part-based models Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba.
Unsupervised Learning of Visual Taxonomies IEEE conference on CVPR 2008 Evgeniy Bart – Caltech Ian Porteous – UC Irvine Pietro Perona – Caltech Max Welling.

Unsupervised Learning of Visual Taxonomies IEEE conference on CVPR 2008 Evgeniy Bart – Caltech Ian Porteous – UC Irvine Pietro Perona – Caltech Max Welling.

Similar presentations

Ads by Google