Part 4: combined segmentation and recognition Li Fei-Fei.

Slides:

Advertisements

Similar presentations

POSE–CUT Simultaneous Segmentation and 3D Pose Estimation of Humans using Dynamic Graph Cuts Mathieu Bray Pushmeet Kohli Philip H.S. Torr Department of.

Advertisements

OBJ CUT & Pose Cut CVPR 05 ECCV 06

O BJ C UT M. Pawan Kumar Philip Torr Andrew Zisserman UNIVERSITY OF OXFORD.

The Layout Consistent Random Field for detecting and segmenting occluded objects CVPR, June 2006 John Winn Jamie Shotton.

Agenda Introduction Bag-of-words models Visual words with spatial location Part-based models Discriminative methods Segmentation and recognition Recognition-based.

Part 4: Combined segmentation and recognition by Rob Fergus (MIT)

Learning to Combine Bottom-Up and Top-Down Segmentation Anat Levin and Yair Weiss School of CS&Eng, The Hebrew University of Jerusalem, Israel.

Fitting: The Hough transform. Voting schemes Let each feature vote for all the models that are compatible with it Hopefully the noise features will not.

Object class recognition using unsupervised scale-invariant learning Rob Fergus Pietro Perona Andrew Zisserman Oxford University California Institute of.

1 Part 1: Classical Image Classification Methods Kai Yu Dept. of Media Analytics NEC Laboratories America Andrew Ng Computer Science Dept. Stanford University.

Generative Models of Images of Objects S. M. Ali Eslami Joint work with Chris Williams Nicolas Heess John Winn June 2012 UoC TTI.

Qualifying Exam: Contour Grouping Vida Movahedi Supervisor: James Elder Supervisory Committee: Minas Spetsakis, Jeff Edmonds York University Summer 2009.

LOCUS (Learning Object Classes with Unsupervised Segmentation) A variational approach to learning model- based segmentation. John Winn Microsoft Research.

Contour Based Approaches for Visual Object Recognition Jamie Shotton University of Cambridge Joint work with Roberto Cipolla, Andrew Blake.

Pedestrian Detection in Crowded Scenes Dhruv Batra ECE CMU.

Simultaneous Segmentation and 3D Pose Estimation of Humans or Detection + Segmentation = Tracking? Philip H.S. Torr Pawan Kumar, Pushmeet Kohli, Matt Bray.

Image Parsing: Unifying Segmentation and Detection Z. Tu, X. Chen, A.L. Yuille and S-C. Hz ICCV 2003 (Marr Prize) & IJCV 2005 Sanketh Shetty.

In Search of Objects: 50 years of wondering : Learning-Based Methods in Vision A. Efros, CMU, Spring 2009.

Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,

Beyond bags of features: Part-based models Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba.

Fitting: The Hough transform

Beyond bags of features: Adding spatial information Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba.

Statistical Recognition Slides adapted from Fei-Fei Li, Rob Fergus, Antonio Torralba, and Kristen Grauman.

Object Recognition Szeliski Chapter 14.

1 Image Recognition - I. Global appearance patterns Slides by K. Grauman, B. Leibe.

Beyond bags of features: Adding spatial information Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba.

Oxford Brookes Seminar Thursday 3 rd September, 2009 University College London1 Representing Object-level Knowledge for Segmentation and Image Parsing:

Object Recognition: Conceptual Issues Slides adapted from Fei-Fei Li, Rob Fergus, Antonio Torralba, and K. Grauman.

The Layout Consistent Random Field for Recognizing and Segmenting Partially Occluded Objects By John Winn & Jamie Shotton CVPR 2006 presented by Tomasz.

Object Recognition: Conceptual Issues Slides adapted from Fei-Fei Li, Rob Fergus, Antonio Torralba, and K. Grauman.

Fitting: The Hough transform

LOCUS Demo Stefan Zickler. Two “different” classes Class “Car Side Views” Class “Car Rears”

Object Recognition by Parts Object recognition started with line segments. - Roberts recognized objects from line segments and junctions. - This led to.

Perceptual and Sensory Augmented Computing Integrating Recognitoin and Reconstruction Integrating Recognition and Reconstruction for Cognitive Scene Interpretation.

1. Introduction Motion Segmentation The Affine Motion Model Contour Extraction & Shape Estimation Recursive Shape Estimation & Motion Estimation Occlusion.

Fitting: The Hough transform. Voting schemes Let each feature vote for all the models that are compatible with it Hopefully the noise features will not.

Bag-of-features models. Origin 1: Texture recognition Texture is characterized by the repetition of basic elements or textons For stochastic textures,

Leo Zhu CSAIL MIT Joint work with Chen, Yuille, Freeman and Torralba 1.

Object Detection 01 – Advance Hough Transformation JJCAO.

Supervised Learning of Edges and Object Boundaries Piotr Dollár Zhuowen Tu Serge Belongie.

Perceptual and Sensory Augmented Computing Discussion Session: Sliding Windows Sliding Windows – Silver Bullet or Evolutionary Deadend? Alyosha Efros,

MSRI workshop, January 2005 Object Recognition Collected databases of objects on uniform background (no occlusions, no clutter) Mostly focus on viewpoint.

Class-Specific Hough Forests for Object Detection Zhen Yuan Hsu Advisor：S.J.Wang Gall, J., Lempitsky, V.: Class-specic hough forests for object detection.

Efficient Matching of Pictorial Structures By Pedro Felzenszwalb and Daniel Huttenlocher Presented by John Winn.

Fitting: The Hough transform

Layered Object Detection for Multi-Class Image Segmentation UC Irvine Yi Yang Sam Hallman Deva Ramanan Charless Fowlkes.

O BJ C UT M. Pawan Kumar Philip Torr Andrew Zisserman UNIVERSITY OF OXFORD.

Discussion of Pictorial Structures Pedro Felzenszwalb Daniel Huttenlocher Sicily Workshop September, 2006.

CS 1699: Intro to Computer Vision Detection II: Deformable Part Models Prof. Adriana Kovashka University of Pittsburgh November 12, 2015.

Inference in generative models of images and video John Winn MSR Cambridge May 2004.

Category Independent Region Proposals Ian Endres and Derek Hoiem University of Illinois at Urbana-Champaign.

Learning Jigsaws for clustering appearance and shape John Winn, Anitha Kannan and Carsten Rother NIPS 2006.

Jigsaws: joint appearance and shape clustering John Winn with Anitha Kannan and Carsten Rother Microsoft Research, Cambridge.

Object Recognition by Integrating Multiple Image Segmentations Caroline Pantofaru, Cordelia Schmid, Martial Hebert ECCV 2008 E.

Real-Time Hierarchical Scene Segmentation and Classification Andre Uckermann, Christof Elbrechter, Robert Haschke and Helge Ritter John Grossmann.

Fast Human Detection in Crowded Scenes by Contour Integration and Local Shape Estimation Csaba Beleznai, Horst Bischof Computer Vision and Pattern Recognition,

Object Recognition by Parts

Summary of “Efficient Deep Learning for Stereo Matching”

LOCUS: Learning Object Classes with Unsupervised Segmentation

Object Recognition by Parts

Adversarially Tuned Scene Generation

Object Recognition by Parts

Learning to Combine Bottom-Up and Top-Down Segmentation

Learning Layered Motion Segmentations of Video

Object Recognition by Parts

Brief Review of Recognition + Context

Object Recognition by Parts

“Traditional” image segmentation

Object Recognition with Interest Operators

Learning complex visual concepts

Presentation transcript:

Part 4: combined segmentation and recognition Li Fei-Fei

Aim Given an image and object category, to segment the object Segmentation should (ideally) be shaped like the object e.g. cow-like obtained efficiently in an unsupervised manner able to handle self-occlusion Segmentation Object Category Model Cow Image Segmented Cow

In this section: brief paper reviews Jigsaw approach: Borenstein & Ullman, 2001, 2002 Concurrent recognition and segmentation: Yu and Shi, 2002 Image parsing: Tu et al Interleaved segmentation: Liebe & Schiele, 2004, 2005 OBJCUT: Kumar et al LOCUS: Winn and Jojic, 2005

Jigsaw approach: Borenstein and Ullman, 2001, 2002

Jigsaw approach Each patch has foreground/background mask

Object-Specific Figure-Ground Segregation Stella X. Yu and Jianbo Shi, 2002

Object-Specific Figure-Ground Segregation Some segmentation/detection results Yu and Shi, 2002

Image parsing: Tu, Zhu and Yuille 2003

Perceptual and Sensory Augmented Computing Interleaved Object Categorization and Segmentation Implicit Shape Model - Recognition Backprojected Hypotheses Interest Points Matched Codebook Entries Probabilistic Voting Voting Space (continuous) Backprojection of Maxima Segmentation Refined Hypotheses (uniform sampling) Liebe and Schiele, 2003, 2005

Perceptual and Sensory Augmented Computing Interleaved Object Categorization and Segmentation Interpretation of p(figure) map  per-pixel confidence in object hypothesis  Use for hypothesis verification p(figure) p(ground) Segmentation p(figure) p(ground) Original image Liebe and Schiele, 2003, 2005

Perceptual and Sensory Augmented Computing Interleaved Object Categorization and Segmentation Cows: Results Segmentations from interest points Single-frame recognition - No temporal continuity used! Liebe and Schiele, 2003, 2005

OBJCUT: shape prior -- Layered Pictorial Structures (LPS) Generative model Composition of parts + spatial layout Layer 2 Layer 1 Parts in Layer 2 can occlude parts in Layer 1 Spatial Layout (Pairwise Configuration) Kumar, et al. 2004, 2005

OBJCUT Probability of labelling in addition has Unary potential which depend on distance from Θ (shape parameter) D (pixels) m (labels) Θ (shape parameter) Image Plane Object Category Specific MRF x y mxmx mymy Unary Potential Φ x (m x |Θ) Kumar, et al. 2004, 2005

In the absence of a clear boundary between object and background SegmentationImage OBJCUT: Results Using LPS Model for Cow

LOCUS model Deformation field D Position & size T Class shape π Class edge sprite μ o,σ o Edge image e Image Object appearance λ 1 Background appearance λ 0 Mask m Shared between images Different for each image Winn and Jojic, 2005

Summary Strength –Explains every pixel of the image –Useful for image editing, layering, etc. Issues –Invariance issues (especially) scale, view-point variations