Unsupervised Learning of Categorical Segments in Image Collections *California Institute of Technology **Technion Marco Andreetto*, Lihi Zelnik-Manor**,

Slides:

Advertisements

Similar presentations

Putting Objects in Perspective Derek Hoiem Alexei A. Efros Martial Hebert Carnegie Mellon University Robotics Institute.

Advertisements

Weakly supervised learning of MRF models for image region labeling Jakob Verbeek LEAR team, INRIA Rhône-Alpes.

Unsupervised Learning Clustering K-Means. Recall: Key Components of Intelligent Agents Representation Language: Graph, Bayes Nets, Linear functions Inference.

Foreground Focus: Finding Meaningful Features in Unlabeled Images Yong Jae Lee and Kristen Grauman University of Texas at Austin.

Spectral graph reduction for image and streaming video segmentation Fabio Galasso 1 Margret Keuper 2 Thomas Brox 2 Bernt Schiele 1 1 Max Planck Institute.

A generic model to compose vision modules for holistic scene understanding Adarsh Kowdle *, Congcong Li *, Ashutosh Saxena, and Tsuhan Chen Cornell University,

Agenda Introduction Bag-of-words models Visual words with spatial location Part-based models Discriminative methods Segmentation and recognition Recognition-based.

Learning to Combine Bottom-Up and Top-Down Segmentation Anat Levin and Yair Weiss School of CS&Eng, The Hebrew University of Jerusalem, Israel.

Scene Labeling Using Beam Search Under Mutex Constraints ID: O-2B-6 Anirban Roy and Sinisa Todorovic Oregon State University 1.

Simultaneous Image Classification and Annotation Chong Wang, David Blei, Li Fei-Fei Computer Science Department Princeton University Published in CVPR.

Object class recognition using unsupervised scale-invariant learning Rob Fergus Pietro Perona Andrew Zisserman Oxford University California Institute of.

LARGE-SCALE IMAGE PARSING Joseph Tighe and Svetlana Lazebnik University of North Carolina at Chapel Hill road building car sky.

Patch to the Future: Unsupervised Visual Prediction

Structural Human Action Recognition from Still Images Moin Nabi Computer Vision Lab. ©IPM - Oct

LOCUS (Learning Object Classes with Unsupervised Segmentation) A variational approach to learning model- based segmentation. John Winn Microsoft Research.

Ziming Zhang *, Ze-Nian Li, Mark Drew School of Computing Science, Simon Fraser University, Vancouver, B.C., Canada {zza27, li, Learning.

Optimization & Learning for Registration of Moving Dynamic Textures Junzhou Huang 1, Xiaolei Huang 2, Dimitris Metaxas 1 Rutgers University 1, Lehigh University.

Biased Normalized Cuts 1 Subhransu Maji and Jithndra Malik University of California, Berkeley IEEE Conference on Computer Vision and Pattern Recognition.

Beyond bags of features: Part-based models Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba.

Unsupervised Learning of Visual Taxonomies IEEE conference on CVPR 2008 Evgeniy Bart – Caltech Ian Porteous – UC Irvine Pietro Perona – Caltech Max Welling.

Beyond bags of features: Adding spatial information Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba.

Robust Higher Order Potentials For Enforcing Label Consistency

Statistical Recognition Slides adapted from Fei-Fei Li, Rob Fergus, Antonio Torralba, and Kristen Grauman.

LARGE-SCALE NONPARAMETRIC IMAGE PARSING Joseph Tighe and Svetlana Lazebnik University of North Carolina at Chapel Hill CVPR 2011Workshop on Large-Scale.

1 Unsupervised Modeling and Recognition of Object Categories with Combination of Visual Contents and Geometric Similarity Links Gunhee Kim Christos Faloutsos.

Beyond bags of features: Adding spatial information Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba.

CVR05 University of California Berkeley 1 Familiar Configuration Enables Figure/Ground Assignment in Natural Scenes Xiaofeng Ren, Charless Fowlkes, Jitendra.

Oxford Brookes Seminar Thursday 3 rd September, 2009 University College London1 Representing Object-level Knowledge for Segmentation and Image Parsing:

CSE 221: Probabilistic Analysis of Computer Systems Topics covered: Statistical inference (Sec. )

Unsupervised discovery of visual object class hierarchies Josef Sivic (INRIA / ENS), Bryan Russell (MIT), Andrew Zisserman (Oxford), Alyosha Efros (CMU)

Object Recognition: Conceptual Issues Slides adapted from Fei-Fei Li, Rob Fergus, Antonio Torralba, and K. Grauman.

Computational Vision Jitendra Malik University of California at Berkeley Jitendra Malik University of California at Berkeley.

Object Recognition: Conceptual Issues Slides adapted from Fei-Fei Li, Rob Fergus, Antonio Torralba, and K. Grauman.

Visual Object Recognition Rob Fergus Courant Institute, New York University

1 Outline Overview Integrating Vision Models CCM: Cascaded Classification Models Learning Spatial Context TAS: Things and Stuff Descriptive Querying of.

Exercise Session 10 – Image Categorization

Unsupervised Learning of Categories from Sets of Partially Matching Image Features Kristen Grauman and Trevor Darrel CVPR 2006 Presented By Sovan Biswas.

A General Framework for Tracking Multiple People from a Moving Camera

(Infinitely) Deep Learning in Vision Max Welling (UCI) collaborators: Ian Porteous (UCI) Evgeniy Bart UCI/Caltech) Pietro Perona (Caltech)

City University of Hong Kong 18 th Intl. Conf. Pattern Recognition Self-Validated and Spatially Coherent Clustering with NS-MRF and Graph Cuts Wei Feng.

Multi-task Low-rank Affinity Pursuit for Image Segmentation Bin Cheng, Guangcan Liu, Jingdong Wang, Zhongyang Huang, Shuicheng Yan (ICCV’ 2011) Presented.

Tag Ranking Present by Jie Xiao Dept. of Computer Science Univ. of Texas at San Antonio.

Bag-of-features models. Origin 1: Texture recognition Texture is characterized by the repetition of basic elements or textons For stochastic textures,

A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The.

Category Discovery from the Web slide credit Fei-Fei et. al.

Yao, B., and Fei-fei, L. IEEE Transactions on PAMI(2012)

Scale-less Dense Correspondences Tal Hassner The Open University of Israel ICCV’13 Tutorial on Dense Image Correspondences for Computer Vision.

80 million tiny images: a large dataset for non-parametric object and scene recognition CS 4763 Multimedia Systems Spring 2008.

BING: Binarized Normed Gradients for Objectness Estimation at 300fps

Eric Xing © Eric CMU, Machine Learning Latent Aspect Models Eric Xing Lecture 14, August 15, 2010 Reading: see class homepage.

MSRI workshop, January 2005 Object Recognition Collected databases of objects on uniform background (no occlusions, no clutter) Mostly focus on viewpoint.

Bayesian Parameter Estimation Liad Serruya. Agenda Introduction Bayesian decision theory Scale-Invariant Learning Bayesian “One-Shot” Learning.

Topic Models Presented by Iulian Pruteanu Friday, July 28 th, 2006.

Real-Time Tracking with Mean Shift Presented by: Qiuhua Liu May 6, 2005.

Image Classification over Visual Tree Jianping Fan Dept of Computer Science UNC-Charlotte, NC

Final Review Course web page: vision.cis.udel.edu/~cv May 21, 2003  Lecture 37.

Learning Jigsaws for clustering appearance and shape John Winn, Anitha Kannan and Carsten Rother NIPS 2006.

Jigsaws: joint appearance and shape clustering John Winn with Anitha Kannan and Carsten Rother Microsoft Research, Cambridge.

Towards Total Scene Understanding: Classiﬁcation, Annotation and Segmentation in an Automatic Framework N 工科所錢雅馨 2011/01/16 Li-Jia Li, Richard.

IEEE 2015 Conference on Computer Vision and Pattern Recognition Active Learning for Structured Probabilistic Models with Histogram Approximation Qing SunAnkit.

Hybrid Deep Learning for Reflectance Confocal Microscopy Skin Images

Nonparametric Semantic Segmentation

Finding Clusters within a Class to Improve Classification Accuracy

Learning to Combine Bottom-Up and Top-Down Segmentation

Anomaly Detection in Crowded Scenes

Unsupervised learning of models for recognition

Liyuan Li, Jerry Kah Eng Hoe, Xinguo Yu, Li Dong, and Xinqi Chu

Nonparametric Bayesian Texture Learning and Synthesis

Adarsh Kowdle*, Congcong Li*, Ashutosh Saxena, and Tsuhan Chen

Clustering appearance and shape by Jigsaw, and comparing it with Epitome. Papers (1) Clustering appearance and shape by learning jigsaws (2006 NIPS) (2)

Presentation transcript:

Unsupervised Learning of Categorical Segments in Image Collections *California Institute of Technology **Technion Marco Andreetto*, Lihi Zelnik-Manor**, Pietro Perona* The Sixth IEEE Computer Society Workshop on Perceptual Organization in Computer Vision (POCV 2008)

Outline Motivation and related work A probabilistic model for single image segmentation Unsupervised learning of categorical segments Experimental results Conclusions and future works

Outline Motivation and related work A probabilistic model for single image segmentation Unsupervised learning of categorical segments Experimental results Conclusions and future works

Motivation

Normalized cuts: Shi and Malik PAMI 2000

Motivation

Categorical segments: from human segmentation

Motivation

Related works Russell et al. CVPR 2006 Cao and Fei-Fei ICCV 2007 Wang and Grimson NIPS 2007 Andreetto et al. ICCV 2007

Outline Motivation and related work A probabilistic model for single image segmentation Unsupervised learning of categorical segments Experimental results Conclusions and future works

An image as a set of segments

K = 2 N

An image as a set of segments K = 2  Segment probability N

An image as a set of segments K = 2  Segment probability fkfk K Segment density N

Image formation K = 2  c Segment probability Label fkfk K x Segment density N

What we’re looking forObserved Probabilistic model for clustering  c fkfk KN x Likelihood of x to be in cluster k

Non-parametric densities Sum of local kernels

Outline Motivation and related work A probabilistic model for single image segmentation Unsupervised learning of categorical segments Experimental results Conclusions and future works

N Learning categorical segments  c fkfk K x w gkgk K Segment appearance Joint for all images Segment shape/color Specific per image M

Visual words Filter Bank VQ w1w1 w2w2 w3w3 wNwN Filter bank: 17 outputs 256 visual words Winn et al. ICCV 2005 … …

Inference N  c fkfk K x w gkgk K M

Gibbs sampling

Prior term: Number of pixels in image m assigned to segment k

Gibbs sampling Prior term: Visual words term: Number of pixels in image m assigned to segment k Number of visual word h assigned to segments k

Gibbs sampling Prior term: Visual words term: Segment term: Number of pixels in image m assigned to segment k Number of visual word h assigned to segments k Non-parametric density Estimate for segment k Affinity between observations i and j

Outline Motivation and related work A probabilistic model for single image segmentation Unsupervised learning of categorical segments Experimental results Conclusions and future works

Experimental results (MSRC)

Classification results (MSRC) Class NameWang and GrimsonOur model DetectionFalse Al.DetectionFalse Al. Cow GrassN/A Cars SkyN/A FoliageN/A SeaN/A Bikes Faces Running time: sec. per image

Experimental results (Labelme)

Categorical segments (Labelme) Segment 1: FoliageSegment 2: Buildings Segment 1: SkySegment 3: Street pavement

Categorical segments (scenes)

Outline Motivation and related work A probabilistic model for single image segmentation Unsupervised learning of categorical segments Experimental results Conclusions and future works

Conclusions We presented a model for unsupervised learning of categorical segments We describe an inference method based on Gibbs sampling We show some experimental results on a standard dataset MSRC v1.

Future work Faster inference method (variational approximation) Automatic inference of the number of segments Learning geometric relationships between segments

Thank You