Presentation is loading. Please wait.

Presentation is loading. Please wait.

Learning to Detect A Salient Object Reporter: 鄭綱 (3/2)

Published by Modified over 9 years ago

Similar presentations

Presentation on theme: "Learning to Detect A Salient Object Reporter: 鄭綱 (3/2)"— Presentation transcript:

1 Learning to Detect A Salient Object Reporter: 鄭綱 (3/2)

2 Outline  Introduction  CRF  Formulation of Static Salient Object  Strong Contrast  Center-Surround Histogram  Color Spatial-Distribution  Learning & Inference for the Model  Formulation of Dynamic Salient Object  Results  Conclusion  References

3 Introduction  Image Labeling Problem Sky Building Lawn Plane Tree

4 Introduction  Which kinds of information can be used for labeling? Features from individual sites Intensity, color, texture, … Interactions with neighboring sites Contextual information Vegetation Sky or Building?

5 Introduction  Contextual information: 2 types of interactions Interaction with neighboring labels (Spatial smoothness of labels) neighboring sites tend to have similar labels(except at the discontinuities) Interactions with neighboring observed data Building Sky

6 Introduction  Let be the label of the node of the image set S, and N i be the neighboring nodes of node i.  Three kinds of information for image labeling  Features from local node  Interaction with neighboring labels  Interaction with neighboring observed data node iS-{i}NiNi

7 Introduction  General formulation: where and are called association potential and interaction potential.

8 Introduction Labels in Spatial Data are NOT independent! – spatially adjacent labels are often the same (Markov Random Fields and Conditional Random Fields) – spatially adjacent elements that have similar features often receive the same label (Conditional Random Fields) – spatially adjacent elements that have different features may not have correlated labels (Conditional Random Fields)

9 Salient Object? Salience Map?

10 Formulation of Static Salient Object Detection  CRF model (static image): Salient object featurePairwise feature Strong contrast Center- surround histogram Color spatial- distribution Maximize!! Minimize!!

11 Strong Contrast  Generate contrast map for each level of 6- level Gaussian pyramid. Then do linear combination. Input imageLevel 1 Level 4

12 Center-Surround Histogram  Salient object usually has a “huge” difference from local area. ……. More different between 2 rectangles !

13 Center-Surround Histogram where …... ??

14 Color Spatial-Distribution  The wider a color is distributed in the image, the less possible a salient object contains this color.  Each pixel is assigned to a color component with the probability:

15 Color Spatial-Distribution  Then the feature can be defined as a weighted sum:

16 Formulation of Static Salient Object Detection  CRF model (static image): Salient object featurePairwise feature Strong contrast Center- surround histogram Color spatial- distribution Maximize!! Minimize!!

17 Learning & Inference for the Model  The goal of CRF learning is to estimate the linear weights. Gradient descent Training images

18 Learning & Inference for the Model A training image for example: where An training image Label inferred this iterate Possible label Label per pixel

19 Learning & Inference for the Model Gradient descent: How to use ground-truth information? Where is the labeled ground-truth. t: iteration

20 Learning & Inference for the Model  Situations: Ground-truth mistake!!!

21 Ground-Truth Mistake  Solution: apply Gaussian function to weight every pixel in the rectangle.

22 Inference  We should find the most probable labeling to maximize in training & detection.  BP – Max-product Belief Propagation [Pearl ‘86] + Can be applied to any energy function – In vision results are usually worse than that of graph cuts – Does not always converge  TRW - Max-product Tree-reweighted Message Passing [Wainwright, Jaakkola, Willsky ‘02], [Kolmogorov ‘05] + Can be applied to any energy function + Convergence guarantees for the algorithm in [Kolmogorov ’05]

23 Formulation of Dynamic Salient Object Detection  Similar as static salient object detection! Static Salient feature Maximize!! Contrast of motion Center- surround histogram Spatial- distribution of motion Penalty term of motion

24 Results From left to right: input image, multi-scale contrast, center- surround histogram, color spatial distribution, and binary mask by CRF.

25 Results From left to right: Fuzzy growing based method Salience map Their approach Ground-truth

26 Results 1. multi-scale contrast 2. center-surround histogram 3. color spatial distribution 4. combination all Image set AImage set B 1. FG 2. SM 3. their approach

27 Conclusion  They model the salient object detection by CRF, where a group of salient features are combined through CRF learning.  It’s a set of novel local, regional & global salient features to define a generic salient object.  Multi-object & no object cases are left as future work.

28 References (paper & book)  “Learning to Detect A Salient Object”, CVPR 2007, PAMI 2010.  http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.91.4387&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.91.4387&rep=rep1&type=pdf  “A Model of Saliency-Based Visual Attention for Rapid Scene Analysis”, PAMI, 1998.  http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.65.2199&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.65.2199&rep=rep1&type=pdf  “Pattern Recognition and Machine Learning”, C. M. Bishop.  http://www.library.wisc.edu/selectedtocs/bg0137.pdf http://www.library.wisc.edu/selectedtocs/bg0137.pdf

29 References (about CRF)  “Discriminative Random Fields”, IJCV, 2006.  http://citeseerx.ist.psu.edu/viewdoc/download?do i=10.1.1.116.4695&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?do i=10.1.1.116.4695&rep=rep1&type=pdf  “Conditional Random Fields: An Introduction”, H. M. Wallach.  http://citeseerx.ist.psu.edu/viewdoc/download?do i=10.1.1.64.436&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?do i=10.1.1.64.436&rep=rep1&type=pdf  “Log-linear models & conditional random fields”, C. Elkan, 2008.  http://cseweb.ucsd.edu/~elkan/250B/cikmtutoria l.pdf http://cseweb.ucsd.edu/~elkan/250B/cikmtutoria l.pdf

30 References (about TRW)  “MAP estimation via agreement on (hyper) trees: Message-passing and linear programming approaches”, IEEE transaction on Information Theory, 2005.  http://arxiv.org/pdf/cs/0508070 http://arxiv.org/pdf/cs/0508070  “Convergent Tree-Reweighted Message Passing for Energy Minimization”, PAMI, 2006.  http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1. 100.2409&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1. 100.2409&rep=rep1&type=pdf  “A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors”, PAMI, 2008.  http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1. 142.4997&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1. 142.4997&rep=rep1&type=pdf

Download ppt "Learning to Detect A Salient Object Reporter: 鄭綱 (3/2)"

Similar presentations

Mean-Field Theory and Its Applications In Computer Vision1 1.

Mean-Field Theory and Its Applications In Computer Vision1 1.
Algorithms for MAP estimation in Markov Random Fields Vladimir Kolmogorov University College London Tutorial at GDR (Optimisation Discrète, Graph Cuts.

Algorithms for MAP estimation in Markov Random Fields Vladimir Kolmogorov University College London Tutorial at GDR (Optimisation Discrète, Graph Cuts.
Bayesian Belief Propagation

Bayesian Belief Propagation
HOPS: Efficient Region Labeling using Higher Order Proxy Neighborhoods Albert Y. C. Chen 1, Jason J. Corso 1, and Le Wang 2 1 Dept. of Computer Science.

HOPS: Efficient Region Labeling using Higher Order Proxy Neighborhoods Albert Y. C. Chen 1, Jason J. Corso 1, and Le Wang 2 1 Dept. of Computer Science.
Weakly supervised learning of MRF models for image region labeling Jakob Verbeek LEAR team, INRIA Rhône-Alpes.

Weakly supervised learning of MRF models for image region labeling Jakob Verbeek LEAR team, INRIA Rhône-Alpes.
Change Detection C. Stauffer and W.E.L. Grimson, “Learning patterns of activity using real time tracking,” IEEE Trans. On PAMI, 22(8):747-757, Aug 2000.

Change Detection C. Stauffer and W.E.L. Grimson, “Learning patterns of activity using real time tracking,” IEEE Trans. On PAMI, 22(8): , Aug 2000.
SOFT SCISSORS: AN INTERACTIVE TOOL FOR REALTIME HIGH QUALITY MATTING International Conference on Computer Graphics and Interactive Techniques ACM SIGGRAPH.

SOFT SCISSORS: AN INTERACTIVE TOOL FOR REALTIME HIGH QUALITY MATTING International Conference on Computer Graphics and Interactive Techniques ACM SIGGRAPH.
Graph cut Chien-chi Chen.

Graph cut Chien-chi Chen.
ICCV 2007 tutorial Part III Message-passing algorithms for energy minimization Vladimir Kolmogorov University College London.

ICCV 2007 tutorial Part III Message-passing algorithms for energy minimization Vladimir Kolmogorov University College London.
Lecture 07 Segmentation Lecture 07 Segmentation Mata kuliah: T0283 - Computer Vision Tahun: 2010.

Lecture 07 Segmentation Lecture 07 Segmentation Mata kuliah: T Computer Vision Tahun: 2010.
電腦視覺 Computer and Robot Vision I Chapter2: Binary Machine Vision: Thresholding and Segmentation Instructor: Shih-Shinh Huang 1.

電腦視覺 Computer and Robot Vision I Chapter2: Binary Machine Vision: Thresholding and Segmentation Instructor: Shih-Shinh Huang 1.
Foreground Modeling The Shape of Things that Came Nathan Jacobs Advisor: Robert Pless Computer Science Washington University in St. Louis.

Foreground Modeling The Shape of Things that Came Nathan Jacobs Advisor: Robert Pless Computer Science Washington University in St. Louis.
Markov Networks.

Markov Networks.
IEEE TCSVT 2011 Wonjun Kim Chanho Jung Changick Kim

IEEE TCSVT 2011 Wonjun Kim Chanho Jung Changick Kim
Yilin Wang 11/5/2009. Background Labeling Problem Labeling: Observed data set (X) Label set (L) Inferring the labels of the data points Most vision problems.

Yilin Wang 11/5/2009. Background Labeling Problem Labeling: Observed data set (X) Label set (L) Inferring the labels of the data points Most vision problems.
1 Image Completion using Global Optimization Presented by Tingfan Wu.

1 Image Completion using Global Optimization Presented by Tingfan Wu.
1 On the Statistical Analysis of Dirty Pictures Julian Besag.

1 On the Statistical Analysis of Dirty Pictures Julian Besag.
Abstract We present a model of curvilinear grouping using piecewise linear representations of contours and a conditional random field to capture continuity.

Abstract We present a model of curvilinear grouping using piecewise linear representations of contours and a conditional random field to capture continuity.
1 Markov random field: A brief introduction Tzu-Cheng Jen Institute of Electronics, NCTU 2007-03-28.

1 Markov random field: A brief introduction Tzu-Cheng Jen Institute of Electronics, NCTU
Abstract Extracting a matte by previous approaches require the input image to be pre-segmented into three regions (trimap). This pre-segmentation based.

Abstract Extracting a matte by previous approaches require the input image to be pre-segmented into three regions (trimap). This pre-segmentation based.

Similar presentations

Ads by Google