Presentation is loading. Please wait.

Presentation is loading. Please wait.

O BJ C UT M. Pawan Kumar Philip Torr Andrew Zisserman UNIVERSITY OF OXFORD.

Published bySibyl Singleton Modified over 9 years ago

Similar presentations

Presentation on theme: "O BJ C UT M. Pawan Kumar Philip Torr Andrew Zisserman UNIVERSITY OF OXFORD."— Presentation transcript:

1 O BJ C UT M. Pawan Kumar Philip Torr Andrew Zisserman UNIVERSITY OF OXFORD

2 Aim Given an image, to segment the object Segmentation should (ideally) be shaped like the object e.g. cow-like obtained efficiently in an unsupervised manner able to handle self-occlusion Segmentation Object Category Model Cow Image Segmented Cow

3 Challenges Self Occlusion Intra-Class Shape Variability Intra-Class Appearance Variability

4 Motivation Magic Wand Current methods require user intervention Object and background seed pixels (Boykov and Jolly, ICCV 01) Bounding Box of object (Rother et al. SIGGRAPH 04) Cow Image Object Seed Pixels

5 Motivation Magic Wand Current methods require user intervention Object and background seed pixels (Boykov and Jolly, ICCV 01) Bounding Box of object (Rother et al. SIGGRAPH 04) Cow Image Object Seed Pixels Background Seed Pixels

6 Motivation Magic Wand Current methods require user intervention Object and background seed pixels (Boykov and Jolly, ICCV 01) Bounding Box of object (Rother et al. SIGGRAPH 04) Segmented Image

7 Motivation Magic Wand Current methods require user intervention Object and background seed pixels (Boykov and Jolly, ICCV 01) Bounding Box of object (Rother et al. SIGGRAPH 04) Cow Image Object Seed Pixels Background Seed Pixels

8 Motivation Magic Wand Current methods require user intervention Object and background seed pixels (Boykov and Jolly, ICCV 01) Bounding Box of object (Rother et al. SIGGRAPH 04) Segmented Image

9 Problem Manually intensive Segmentation is not guaranteed to be ‘object-like’ Non Object-like Segmentation Motivation

10 Our Method Combine object detection with segmentation –Borenstein and Ullman, ECCV ’02 –Leibe and Schiele, BMVC ’03 Incorporate global shape priors in MRF Detection provides – Object Localization – Global shape priors Automatically segments the object –Note our method completely generic –Applicable to any object category model

11 Outline Problem Formulation Form of Shape Prior Optimization Results

12 Problem Labelling m over the set of pixels D Shape prior provided by parameter Θ Energy E (m,Θ) = ∑Φ x (D|m x )+Φ x (m x |Θ) + ∑ Ψ xy (m x,m y )+ Φ(D|m x,m y ) Unary terms –Likelihood based on colour –Unary potential based on distance from Θ Pairwise terms –Prior –Contrast term Find best labelling m* = arg min ∑ w i E (m,Θ i ) –w i is the weight for sample Θ i Unary terms Pairwise terms

13 MRF Probability for a labelling consists of Likelihood Unary potential based on colour of pixel Prior which favours same labels for neighbours (pairwise potentials) Prior Ψ xy (m x,m y ) Unary Potential Φ x (D|m x ) D (pixels) m (labels) Image Plane x y mxmx mymy

14 Example Cow Image Object Seed Pixels Background Seed Pixels Prior x … y … … … x … y … … … Φ x (D|obj) Φ x (D|bkg) Ψ xy (m x,m y ) Likelihood Ratio (Colour)

15 Example Cow Image Object Seed Pixels Background Seed Pixels Prior Likelihood Ratio (Colour)

16 Contrast-Dependent MRF Probability of labelling in addition has Contrast term which favours boundaries to lie on image edges D (pixels) m (labels) Image Plane Contrast Term Φ(D|m x,m y ) x y mxmx mymy

17 Example Cow Image Object Seed Pixels Background Seed Pixels Prior + Contrast x … y … … … x … y … … … Likelihood Ratio (Colour) Ψ xy (m x,m y ) + Φ(D|m x,m y ) Φ x (D|obj) Φ x (D|bkg)

18 Example Cow Image Object Seed Pixels Background Seed Pixels Prior + Contrast Likelihood Ratio (Colour)

19 Our Model Probability of labelling in addition has Unary potential which depend on distance from Θ (shape parameter) D (pixels) m (labels) Θ (shape parameter) Image Plane Object Category Specific MRF x y mxmx mymy Unary Potential Φ x (m x |Θ)

20 Example Cow Image Object Seed Pixels Background Seed Pixels Prior + Contrast Distance from Θ Shape Prior Θ

21 Example Cow Image Object Seed Pixels Background Seed Pixels Prior + Contrast Likelihood + Distance from Θ Shape Prior Θ

22 Example Cow Image Object Seed Pixels Background Seed Pixels Prior + Contrast Likelihood + Distance from Θ Shape Prior Θ

23 Outline Problem Formulation –E (m,Θ) = ∑Φ x (D|m x )+Φ x (m x |Θ) + ∑ Ψ xy (m x,m y )+ Φ(D|m x,m y ) Form of Shape Prior Optimization Results

24 Layered Pictorial Structures (LPS) Generative model Composition of parts + spatial layout Layer 2 Layer 1 Parts in Layer 2 can occlude parts in Layer 1 Spatial Layout (Pairwise Configuration)

25 Layer 2 Layer 1 Transformations Θ 1 P(Θ 1 ) = 0.9 Cow Instance Layered Pictorial Structures (LPS)

26 Layer 2 Layer 1 Transformations Θ 2 P(Θ 2 ) = 0.8 Cow Instance Layered Pictorial Structures (LPS)

27 Layer 2 Layer 1 Transformations Θ 3 P(Θ 3 ) = 0.01 Unlikely Instance Layered Pictorial Structures (LPS)

28 LPS for Detection Learning – Learnt automatically using a set of examples Detection –Matches LPS to image using Loopy Belief Propagation –Localizes object parts

29 Outline Problem Formulation Form of Shape Prior Optimization Results

30 Optimization Given image D, find best labelling as m* = arg max p(m|D) Treat LPS parameter Θ as a latent (hidden) variable EM framework –E : sample the distribution over Θ –M : obtain the labelling m

31 E-Step Given initial labelling m ’, determine p(Θ|m ’,D) Problem Efficiently sampling from p(Θ|m ’,D) Solution We develop efficient sum-product Loopy Belief Propagation (LBP) for matching LPS. Similar to efficient max-product LBP for MAP estimate –Felzenszwalb and Huttenlocher, CVPR ‘04

32 Results Different samples localize different parts well. We cannot use only the MAP estimate of the LPS.

33 M-Step Given samples from p(Θ|m ’,D), get new labelling m new Sample Θ i provides –Object localization to learn RGB distributions of object and background –Shape prior for segmentation Problem –Maximize expected log likelihood using all samples –To efficiently obtain the new labelling

34 M-Step Cow Image Shape Θ 1 w 1 = P(Θ 1 |m’,D) RGB Histogram for Object RGB Histogram for Background

35 Cow Image Shape Θ 1 M-Step w 1 = P(Θ 1 |m’,D) Θ1Θ1 Image Plane D (pixels) m (labels) Best labelling found efficiently using a Single Graph Cut

36 Segmentation using Graph Cuts x … y ……… z …… Obj Bkg Cut Φ x (D|bkg) + Φ x (bkg|Θ) m Φ z (D|obj) + Φ z (obj|Θ) Ψ xy (m x,m y )+ Φ(D|m x,m y )

37 Segmentation using Graph Cuts x … y ……… z …… Obj Bkg m

38 M-Step Cow Image Shape Θ 2 w 2 = P(Θ 2 |m’,D) RGB Histogram for Background RGB Histogram for Object

39 M-Step Cow Image Shape Θ 2 w 2 = P(Θ 2 |m’,D) Θ2Θ2 Image Plane D (pixels) m (labels) Best labelling found efficiently using a Single Graph Cut

40 M-Step Θ2Θ2 Image Plane Θ1Θ1 w1w1 + w 2 + …. Best labelling found efficiently using a Single Graph Cut m* = arg min ∑ w i E (m,Θ i )

41 Outline Problem Formulation Form of Shape Prior Optimization Results

42 SegmentationImage Results Using LPS Model for Cow

43 In the absence of a clear boundary between object and background SegmentationImage Results Using LPS Model for Cow

44 SegmentationImage Results Using LPS Model for Cow

45 SegmentationImage Results Using LPS Model for Cow

46 SegmentationImage Results Using LPS Model for Horse

47 SegmentationImage Results Using LPS Model for Horse

48 Our MethodLeibe and SchieleImage Results

49 Appearance ShapeShape+Appearance Results Without Φ x (D|m x ) Without Φ x (m x |Θ)

50 Conclusions –New model for introducing global shape prior in MRF –Method of combining detection and segmentation –Efficient LBP for detecting articulated objects Future Work –Other shape parameters need to be explored –Method needs to be extended to handle multiple visual aspects

Download ppt "O BJ C UT M. Pawan Kumar Philip Torr Andrew Zisserman UNIVERSITY OF OXFORD."

Similar presentations

Using Strong Shape Priors for Multiview Reconstruction Yunda SunPushmeet Kohli Mathieu BrayPhilip HS Torr Department of Computing Oxford Brookes University.

Using Strong Shape Priors for Multiview Reconstruction Yunda SunPushmeet Kohli Mathieu BrayPhilip HS Torr Department of Computing Oxford Brookes University.
POSE–CUT Simultaneous Segmentation and 3D Pose Estimation of Humans using Dynamic Graph Cuts Mathieu Bray Pushmeet Kohli Philip H.S. Torr Department of.

POSE–CUT Simultaneous Segmentation and 3D Pose Estimation of Humans using Dynamic Graph Cuts Mathieu Bray Pushmeet Kohli Philip H.S. Torr Department of.
OBJ CUT & Pose Cut CVPR 05 ECCV 06

OBJ CUT & Pose Cut CVPR 05 ECCV 06
O BJ C UT M. Pawan Kumar Philip Torr Andrew Zisserman UNIVERSITY OF OXFORD.

O BJ C UT M. Pawan Kumar Philip Torr Andrew Zisserman UNIVERSITY OF OXFORD.
Combinatorial Optimization and Computer Vision Philip Torr.

Combinatorial Optimization and Computer Vision Philip Torr.
Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.

Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.
The Layout Consistent Random Field for detecting and segmenting occluded objects CVPR, June 2006 John Winn Jamie Shotton.

The Layout Consistent Random Field for detecting and segmenting occluded objects CVPR, June 2006 John Winn Jamie Shotton.
Pose Estimation and Segmentation of People in 3D Movies Karteek Alahari, Guillaume Seguin, Josef Sivic, Ivan Laptev Inria, Ecole Normale Superieure ICCV.

Pose Estimation and Segmentation of People in 3D Movies Karteek Alahari, Guillaume Seguin, Josef Sivic, Ivan Laptev Inria, Ecole Normale Superieure ICCV.
Joint Optimisation for Object Class Segmentation and Dense Stereo Reconstruction Ľubor Ladický, Paul Sturgess, Christopher Russell, Sunando Sengupta, Yalin.

Joint Optimisation for Object Class Segmentation and Dense Stereo Reconstruction Ľubor Ladický, Paul Sturgess, Christopher Russell, Sunando Sengupta, Yalin.
Agenda Introduction Bag-of-words models Visual words with spatial location Part-based models Discriminative methods Segmentation and recognition Recognition-based.

Agenda Introduction Bag-of-words models Visual words with spatial location Part-based models Discriminative methods Segmentation and recognition Recognition-based.
Part 4: Combined segmentation and recognition by Rob Fergus (MIT)

Part 4: Combined segmentation and recognition by Rob Fergus (MIT)
Learning to Combine Bottom-Up and Top-Down Segmentation Anat Levin and Yair Weiss School of CS&Eng, The Hebrew University of Jerusalem, Israel.

Learning to Combine Bottom-Up and Top-Down Segmentation Anat Levin and Yair Weiss School of CS&Eng, The Hebrew University of Jerusalem, Israel.
Interactive Segmentation with Super-Labels Andrew Delong Western Yuri BoykovOlga VekslerLena GorelickFrank Schmidt TexPoint fonts used in EMF. Read the.

Interactive Segmentation with Super-Labels Andrew Delong Western Yuri BoykovOlga VekslerLena GorelickFrank Schmidt TexPoint fonts used in EMF. Read the.
ICCV 2007 tutorial Part III Message-passing algorithms for energy minimization Vladimir Kolmogorov University College London.

ICCV 2007 tutorial Part III Message-passing algorithms for energy minimization Vladimir Kolmogorov University College London.
I Images as graphs Fully-connected graph – node for every pixel – link between every pair of pixels, p,q – similarity w ij for each link j w ij c Source:

I Images as graphs Fully-connected graph – node for every pixel – link between every pair of pixels, p,q – similarity w ij for each link j w ij c Source:
Mixture of trees model: Face Detection, Pose Estimation and Landmark Localization Presenter: Zhang Li.

Mixture of trees model: Face Detection, Pose Estimation and Landmark Localization Presenter: Zhang Li.
Outline  Facial Attributes Analysis  Animated Pose Templates(APT) for Modeling and Detecting Human Actions  Unsupervised Structure Learning of Stochastic.

Outline  Facial Attributes Analysis  Animated Pose Templates(APT) for Modeling and Detecting Human Actions  Unsupervised Structure Learning of Stochastic.
3D Shape Representation Tianqiang 04/01/2014. Image/video understanding Content creation Why do we need 3D shapes?

3D Shape Representation Tianqiang 04/01/2014. Image/video understanding Content creation Why do we need 3D shapes?
LOCUS (Learning Object Classes with Unsupervised Segmentation) A variational approach to learning model- based segmentation. John Winn Microsoft Research.

LOCUS (Learning Object Classes with Unsupervised Segmentation) A variational approach to learning model- based segmentation. John Winn Microsoft Research.
GrabCut Interactive Image (and Stereo) Segmentation Carsten Rother Vladimir Kolmogorov Andrew Blake Antonio Criminisi Geoffrey Cross [based on Siggraph.

GrabCut Interactive Image (and Stereo) Segmentation Carsten Rother Vladimir Kolmogorov Andrew Blake Antonio Criminisi Geoffrey Cross [based on Siggraph.

Similar presentations

Ads by Google