1. Lena Gorelick joint work with O. Veksler I. Ben Ayed A. Delong Y. Boykov

2. 2 Potts Model

3. 3 Submodular Energy global optimum with graphcut (Boros & Hammer, 2002) Submodular Energy global optimum with graphcut (Boros & Hammer, 2002) Potts Model

4. Non-Submodular Energy NP-hard Non-Submodular Energy NP-hard 4 Middlebury Image credit: Carlos Hernandes

5. General energy - NP-hard  Approximate methods:  Global Linearization: QPBO, TRWS, SRMP (Kolmogorov et al. 2006, 2014)  Local Linearization: parallel ICM, IPFP (Leordeanu, 2009)  Message Passing: BP (Pearl 1989) 5

6. 6  QPBO, TRWS, SRMP (Kolmogorov et al. 2006, 2014) Linearize introducing large number of variables and constraints Linearize introducing large number of variables and constraints Solve relaxed LP or its dual  Integrality Gap Rounding

7. 7  parallel ICM (Leordeanu, 2009)  large steps  weak min  IPFP (Leordeanu, 2009)  controls step size by relaxation   Integrality Gap Et(x)Et(x) ~ Bounded domain of discrete configurations

8.  Local Submodular Approximation model  Non-linear  Two ways to control step size 8 Et(x)Et(x) ~ Bounded domain of discrete configurations

9.  Trust Region  Local submodular approximation  Auxiliary Functions = Surrogate Functions = Upper Bounds = Majorize-Minimize  Local submodular upper bound  Never leave the discrete domain 9 LSA-AUX LSA-TR

10.  Trust Region:  Discrete High Order Energies  Relaxed Quadratic Binary Energies  Levenberg Marquardt  Auxiliary Functions=Surrogate Functions =Upper Bounds = Majorize-Minimize  Discrete High Order Energies 10 Gorelick et al. 2012,2013 Ben Ayed et al. 2013 Olsson et al. 2008 Narasimhan & Bilmes 2005 Rother et al. 2006 Hartley & Zisserman 2004

11. 11 + + - -

12. 12

13. Approximate around 13 Et(x)Et(x) ~

14. Submodular function LSA Submodular function LSA Approximate around 14 Linear Approximation Et(x)Et(x) ~

15. 15

16. 16

17. 17

18. 18 0 1 1 1,0 0,0 1,1 0,1

19. 19 0 1 1 1,0

20. 1 0 1 0,0 1,1 20 Linear (Unary) approximation

21. 0 1 1 21

22. 22

23. 23 Et(x)Et(x) ~ Newton Step

24. 24 Et(x)Et(x) ~ Trust Region Trust Region Sub-Problem 24 NP-hard! Constrained Submodular Optimization

25. fixed in each iteration inversely related to trust region size adjusted based on quality of approximation 25 Unary Terms Boykov et al. 2006 Unary Terms Boykov et al. 2006 Gorelick et al. 2013 Submodular

26. 26

27.  Binary De-convolution  All pairwise terms supermodular 27 Original ImgConvolved Convolved+Noise ? ?

28. 28 Noise: N(0,0.05 ) LSA-TR (0.3 sec.) E=21.13 LSA-TR (0.3 sec.) E=21.13 LSA-AUX (0.04 sec) E=34.70 LSA-AUX (0.04 sec) E=34.70 TRWS: 5000 iter. E=65.07 TRWS: 5000 iter. E=65.07 LBP 5000 iter. E=40.15 LBP 5000 iter. E=40.15 QPBO (0.1 sec.) QPBO-I (0.2 sec.) E=66.44 QPBO-I (0.2 sec.) E=66.44 IPFP (0.4 sec.) E=32.90 IPFP (0.4 sec.) E=32.90 SRMP: 5000 iter. E=39.06 SRMP: 5000 iter. E=39.06

29. 29 QPBOQPBO-I E= -77.08 LBP E= -84.54 IPFP E= 163.25 Image Potts, v<0 (submodular) with edge repulsion, v>0 (non-submodular) TRWS E= -67.21 LSA-TR E= -175.05 LSA-TR E= -175.05 LSA-AUX E= -120.03 SRMP E= -101.61 Repulsion = Reward different labels across high contrast edges

30.  dtf-chinesechar database 30 LSA-TR Input Img Ground Truth Kappes et al., 2013

31. 31

32. 32  Efficient Squared Curvature model – (Nieuwenhuis et al. 2014, poster on Friday) Potts ModelElastica 90-degree curvature Heber et al. 2012 El-Zehiry&Grady, 2010 Our curvature Using LSA-TR

33.  Two novel discrete optimization methods  Simple, efficient, state-of-art results  The code is available online - http://vision.csd.uwo.ca/code/  Extensions:  Find new applications ▪ Convexity Shape Prior (in ECCV14)  Alternative optimization framework with LSA ▪ Pseudo-Bounds (in ECCV14)  Please come by our poster 33