1. Image Segmentation by Branch-and-Mincut Victor Lempitsky Andrew Blake Carsten Rother

2. V. Lempitsky, A. Blake, C. Rother “Image Segmentation by Branch-And-Mincut” An example 1 0 F p =0, B p =1 F p =1, B p =0 P pq image from UIUC car dataset Standard “graph cut” segmentation energy [Boykov, Jolly 01]:... pq s t [Freedman, Zhang 05], [Ali, Farag, El-Baz 07],... 1 0 [Hammer 64] [Greig, Porteous, Seheult 89]

3. V. Lempitsky, A. Blake, C. Rother “Image Segmentation by Branch-And-Mincut” 1 0 A harder example image from UIUC car dataset Optimal ω P pq Optimal x min x, ω

4. V. Lempitsky, A. Blake, C. Rother “Image Segmentation by Branch-And-Mincut” Energy optimization Optimization options: Choose reasonable ω, solve for x [Freedman, Zhang 05], [Pawan Kumar, Torr, Zisserman’ObjCut 05], [Ali, Farag, El-Baz 07].... Alternate between x and ω (EM) [Rother, Kolmogorov, Blake’ GrabCut 04], [Bray, Kohli, Torr’PoseCut 06], [Kim, Zabih 03].... Optimize continuously [Chan, Vese 01], [Leventon, Grimson, Faugeras 00], [Cremers, Osher, Soatto 06], [Wang, Staib 98]... Exhaustive search constantunary potentials (costs) pairwise potentials (costs) ≥ 0 ω – global parameter ( ω ∊ Ω 0 )....shape priors, color distribution/intensity priors (Chan-Vese, GrabCut)...

5. V. Lempitsky, A. Blake, C. Rother “Image Segmentation by Branch-And-Mincut” Our approach [Gavrila, Philomin 99], [Lampert, Blaschko, Hofman 08], [Cremers, Schmidt, Barthel 08] along x dimension along ω dimension Extremely large, structured domain Specific “graph cut” function Low-dimensional (discretized) domain Function of the general form Branch-and-bound Mincut Branch-and-Mincut

6. V. Lempitsky, A. Blake, C. Rother “Image Segmentation by Branch-And-Mincut” Search tree Ω0Ω0

7. V. Lempitsky, A. Blake, C. Rother “Image Segmentation by Branch-And-Mincut” Bounding the energy: an example F p =1, B p =0 F p =0, B p =1 ω1ω1 ω2ω2 F p =0, B p =0 min

8. V. Lempitsky, A. Blake, C. Rother “Image Segmentation by Branch-And-Mincut” The lower bound pq s t Computable with mincut!

9. V. Lempitsky, A. Blake, C. Rother “Image Segmentation by Branch-And-Mincut” Lower bound Monotonic increase towards leaves Tightness at leaf nodes:

10. V. Lempitsky, A. Blake, C. Rother “Image Segmentation by Branch-And-Mincut” Lower bound: example precomputedcomputed at runtime

11. V. Lempitsky, A. Blake, C. Rother “Image Segmentation by Branch-And-Mincut” Branch-and-Bound Standard best-first branch-and-bound search: additional speed-up from “reusing” maxflow computations [Kohli,Torr 05] Small fraction of nodes is visited lowest lower bound A B C

12. V. Lempitsky, A. Blake, C. Rother “Image Segmentation by Branch-And-Mincut” Results: shape prior 30,000,000 shapes Exhaustive search: 30,000,000 mincuts Branch-and-Mincut: 12,000 mincuts Speed-up: 2500 times (30 seconds per 312x272 image)

13. V. Lempitsky, A. Blake, C. Rother “Image Segmentation by Branch-And-Mincut” Results: shape prior Left ventricle epicardium tracking (work in progress) Our segmentation Shape prior from other sequences 5,200,000 templates ≈20 seconds per frame Speed-up 1150 Data courtesy: Dr Harald Becher, Department of Cardiovascular Medicine, University of Oxford Original sequenceNo shape prior

14. V. Lempitsky, A. Blake, C. Rother “Image Segmentation by Branch-And-Mincut” Result: shape prior Can add feature- based detector here UIUC car dataset

15. V. Lempitsky, A. Blake, C. Rother “Image Segmentation by Branch-And-Mincut” Results: Discrete Chan-Vese functional Global minima of the discrete Chan-Vese functional: Chan-Vese functional [Chan, Vese 01]: ∊ [0;255]x[0;255]: quad-tree clustering Speed-up 28-58 times c_p cfcf cbcb

16. V. Lempitsky, A. Blake, C. Rother “Image Segmentation by Branch-And-Mincut” Performance Sample Chan-Vese problem:

17. V. Lempitsky, A. Blake, C. Rother “Image Segmentation by Branch-And-Mincut” Results: GrabCut E = -618E = -624 (speed-up 481)E = -628 E = -593E = -584 (speed-up 141)E = -607 ω corresponds to color mixtures [Rother, Kolmogorov, Blake’ GrabCut 04] uses EM-like search Branch-and-Mincut searches over 65,536 starting points

18. V. Lempitsky, A. Blake, C. Rother “Image Segmentation by Branch-And-Mincut” Conclusion ̶ good energy to integrate low-level and high-level knowledge in segmentation. Branch-and-Mincut framework can find its global optimum efficiently in many cases Ongoing work: Branch-and-X algorithms C++ code at http://research.microsoft.com/~victlem/ Branch-and-bound Mincut Dynamic Programming