Auxiliary Cuts for General Classes of Higher-Order Functionals 1 Ismail Ben Ayed, Lena Gorelick and Yuri Boykov

Standard Segmentation Functionals 2 S

Historic Data Linear terms are not enough 3 Standard model Learned distributions

Linear terms are not enough 3 Segmentation with log likelihoods Learned distributions

Linear terms are not enough 3 Standard model Target distributions Segmentation with log likelihoods

Linear terms are not enough 3 Standard model Target distributions

Segmentation with log likelihoods Linear terms are not enough 3 Standard model Target distributions

Segmentation with log likelihoods Linear terms are not enough 3 Standard model Learned distributions Obtained distributions

From log-likelihoods to higher-order terms 4 Rother et al. 06, Ben Ayed CVPR 10, Gorelick et al. ECCV 12, Jiang et al. CVPR 12

Standard vs. High-order 5 Input High-order Likelihoods High-order Input

Regional Functional Examples Volume Constraint 6

Bin Count Constraint Regional Functional Examples Volume Constraint 6

7 Contribution: Bound Optimization of General Higher-Order Terms Non-Linear Combination of Linear Terms

Optimization Higher-order Pairwise Sub-modular 8

9 Prior Art: General-Purpose Techniques Based on Functional Derivatives

9 -- Level Sets: Ben Ayed et al. CVPR Line search: Gorelick et al. ECCV 2012 Can be slow

9 -- Level Sets: Ben Ayed et al. CVPR Line search: Gorelick et al. ECCV 2012 F differentiable Prior Art: General-Purpose Techniques Based on Functional Derivatives

9 -- Level Sets: Ben Ayed et al. CVPR Line search: Gorelick et al. ECCV 2012 Parameters? Prior Art: General-Purpose Techniques Based on Functional Derivatives

Prior Art: Specialized Techniques 9  Volume constraint: Werner, CVPR 2008  Norms between bin counts: Mukherjee et al. CVPR 2009, Jiang et al. CVPR 2012  Bhattacharyya : Ben Ayed et al. CVPR 2010, Punithakumar et al. SIAM 2012 Only particular cases

Auxiliary Function Optimization 10

Auxiliary Function Optimization 10

Auxiliary Function Optimization 10

Standard Tricks for Deriving Auxiliary Functions 12 Cauchy-Schwarz inequality Quadratic bound principle First-order expansion Jensen’s inequality  E.g.: EM is based on this approach

Jensen’s Inequality bound 11

Unary Terms Jensen’s Inequality bound

11 Jensen’s Inequality bound

Auxiliary Function Derivation 13

Auxiliary Function Derivation 13

Auxiliary Function Derivation 13

Auxiliary Function Derivation 13 Constant

Auxiliary Function Derivation 13 Sum to 1

Auxiliary Function Derivation 13 Jensen’sLinear auxiliary function

Difference with other methods: the volume constraint case 14

Difference with other methods: the volume constraint case 14 Gradient Descent

Difference with other methods: the volume constraint case 14 Trust Region: Gorelick et al. CVPR 13

Difference with other methods: the volume constraint case 14 Auxiliary Cuts

General Form of the Functionals 15 Higher-order Sub-modular

General Form of the Functionals 15 Linear bound Sub-modular Higher-order Sub-modular

General Form of the Functionals 15 Graph Cut Higher-order Sub-modular Linear bound Sub-modular

Experimental examples

L2 Bin Count (Aux. Cuts vs. Level Sets) Level-Set, dt=1 Level-Set, dt=50 Level-Set, dt=1000 Init Aux. Cuts 16

User input Result User input Iter 2 User input ResultB-J Initial segment Iter. 3Iter L1 Bin Count

18 inputs Input L2 Volume Constraint User input B-J B-J and Volume

Conclusions 19 Advantages: Derivative-free No optimization parameters, e.g., step size Easy to implement Never worsen the energy at each iteration

Conclusions 19 Limitations: The form of F should verify some conditions Limited to nested evolutions of segments

Conclusions 19 Extensions: More general forms of F Arbitrary evolutions of segments

19 inputs Input Thanks