Download presentation
Presentation is loading. Please wait.
Published byThomasina Henry Modified over 9 years ago
1
Pseudo-Bound Optimization for Binary Energies Presenter: Meng Tang Joint work with Ismail Ben AyedYuri Boykov 1 / 27
2
Labeling Problems in Computer Vision foreground selection Geometric model fitting StereoSemantic segmentationDenoising inpainting Binary label Multi-label 2 / 27
3
Energy Minimization for Labeling Problem foreground selection Semantic segmentation S * = arg S min E(S) s p = 1 (FG) or 0 (BG) s p = ‘sky’ or ‘road’ or ‘bike’ etc. 3 / 27
4
Basic Pairwise Energies Common in computer vision Unary termPairwise term Example: interactive segmentation S * = arg S min E(S) e.g. Boros & Hammer. 2002 Boykov & Jolly. 2001 4 / 27 Submodular case: fast global solver (Graph Cuts ) Unary termPairwise term
5
More difficult energies High-order energies Entropy minimization for image segments Matching target distribution Volume constraints Convex shape prior Pairwise nonsubmodular energies Curvature regularization Segmentation with repulsion Binary image deconvolution e.t.c. Roof duality [Boros & Hammer. 2002] QPBO-mincut [Kolmogorov, Rother et al. 2007] TRWS, SRMP [Kolmogorov et al. 2006, 2014] Parallel ICM [Leordeanu et al. 2009] ……………. Region Competition[Zhu, Lee & Yuille. 1995] GrabCut [Rother et al. 2004] [Vicente et al. 2009] [Gould et al. 2011, 2012][Kohli et al. 2007, 2009] [Ayed et al. 2010, 2013][Gorelick et al. 2013, 2014] …………….. 5 / 27
6
Our framework (Pseudo-Bound Opt.) High-order energies Entropy minimization for image segments Matching target distribution Volume constraints Convex shape prior Pairwise nonsubmodular energies Curvature regularization Segmentation with repulsion Binary image deconvolution e.t.c. Roof duality [Boros & Hammer. 2002] QPBO-mincut [Kolmogorov, Rother et al. 2007] TRWS, SRMP [Kolmogorov et al. 2006, 2014] Parallel ICM [Leordeanu et al. 2009] ……………. Region Competition[Zhu, Lee & Yuille. 1995] GrabCut [Rother et al. 2004] [Vicente et al. 2009] [Gould et al. 2011, 2012][Kohli et al. 2007, 2009] [Ayed et al. 2010, 2013][Gorelick et al. 2013, 2014] …………….. 6 / 27
7
Example of high-order energy With known appearance models θ 0,θ 1. Appearance models can be optimized Boykov & Jolly. 2001 GrabCut [Rother et al. 2004] 7 / 27 fixed variables
8
From model fitting to entropy optimization: entropy of intensities in entropy of intensities in Note: H(P|Q) H(P) for any two distributions cross-entropyentropy [Delong et al, IJCV 2012] [Tang et al. ICCV 2013] common energy for categorical clustering, e.g. [Li et al. ICML’04] Decision Forest Classification, e.g. [Criminisi & Shotton. 2013] binary optimization mixed optimization high-order energy 8 / 27
9
Energy example: color entropy low entropyhigh entropy S S S S S S S S S S S S 9 / 27
10
Pseudo-bound optimization example: minimize our entropy-based energy E(S) 10 / 27
11
BCD could be seen as bound optimization for entropy 11 / 27 one standard approach: Block-Coordinate Descent (BCD) GrabCut [Rother et al. 2004] ≥ our entropy energy mixed var. energy
12
one standard approach: Block-Coordinate Descent (BCD) GrabCut [Rother et al. 2004] BCD could be seen as bound optimization for entropy ≥ our entropy energy mixed var. energy E(S)E(S) E ( S|θ 0,θ 1 ) t StSt S t+1 E ( S|θ 0,θ 1 ) t+1 12 / 27 E(St)E(St) E(S t+1 )
13
Converges to a local minimum E(S)E(S) StSt S t+1 E(St)E(St) E(S t+1 ) Bound optimization, in general A t+1 (S) At(S)At(S) 13 / 27 (Majorize-Minimize, Auxiliary Function, Surrogate Function)
14
Local minima examples (for GrabCut) E=2.37×10 6 E=2.41×10 6 E=1.39×10 6 E=1.410×10 6 14 / 27
15
This work: Pseudo-Bound Optimization solution space S StSt S t+1 At(S)At(S) E(S)E(S) E(St)E(St) (a) (b) (c) Make larger move! General framework for energy-minimization 15 / 27 E(S t+1 )
16
General form of Pseudo-Bounds Ƒ t (S, λ) = + λ R t (S) Pairwise Submodular Unary Bounding relaxation S λ = min s Ƒ t (S, λ) S t+1 = argmin S λ E(S λ ) Parametric Maxflow Update Iterate BoundsParametricPseudo-Cuts A t (S) (pPBC) n-links s t a cut t-link edge capacities depend linearly on λ. Parametric Max-flow Gallo et al. 1989 Hochbaum et al. 2010 Kolmogorov et al. 2007 Bound 16 / 27
17
Specific pseudo-bounds Example 1 How to choose auxiliary and bound relaxation functions? Pairwise Submodular Unary Ƒ t (S, λ) = + λ R t (S) A t (S) Ƒ t (S,λ) + λ(|S|-|S t |) = E ( S|θ 0,θ 1 ) t 17 / 27
18
Specific pseudo-bounds Example 2 Pairwise Submodular Unary |S||S| |St||St| f(|S|) |V0||V0| High-order example: Soft volume constriants Ƒ t (S, λ) = + λ R t (S) A t (S) How to choose auxiliary and bound relaxation functions? 18 / 27
19
Specific pseudo-bounds Example 3 Ƒ t (S, λ) = + λ R t (S) A t (S) Pairwise Submodular Unary Pairwise example: Nonsubmodluar pairwise m pq s p s q, for m pq >0 How to choose auxiliary and bound relaxation functions? 19 / 27 Gorelick et al. in CVPR 2014
20
EXPERIMENTAL RESULTS 20 / 27
21
Experiment results (high-order) Interactive segmentation (entropy minimization) 21 / 27
22
Experiment results (high-order) Interactive segmentation (GrabCut database) 22 / 27 Rother et al. 2004 Tang et al. 2013 Vicente et al. 2009
23
Unsupervised binary segmentation – without prior (bounding box, appearance etc.) Experiment results (high-order) 23 / 27
24
Matching appearance distribution Input imageGround truthOur method 24 / 27
25
Experiment results (high-order) Matching color distribution 25 / 27 Ayed et al. 2013 Gorelick et al. 2013
26
Experiment results (pairwise) Segmentation with curvature regularization 26 / 27 *Submodularization for Binary Pairwise Energies, Gorelick et al. in CVPR 2014 * * * *
27
Conclusion Achieve state-of-the-art for many binary energy minimization problems Several options of pseudo-bounds BCD as in GrabCut is a bound optimization Optimize pseudo-bounds efficiently with parametric maxflow General optimization framework for high- order and pairwise binary energy minimization Code available: www.csd.uwo.ca/~mtang73 www.csd.uwo.ca/~mtang73 27 / 27
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.