IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 26, NO. 9, SEPTEMBER 2004 Yuri Boykov, Member, IEEE Vladimir Kolmogorov, Member, IEEE Guan-Yu Liu 1

Outline  Introduction  Overview  Method A : growth B : augmentation C : adoption  Experimental Results Part1 - introduce algorithm Part2 - results  Conclusion  Reference  Q & A 2

Introduction (1/3)  GREIG et al. [15] were the first to discover that powerful min-cut/max-flow algorithms from combinatorial optimization can be used to minimize certain important energy functions in vision. 123 where L = is a labeling of image, is a data penalty function, is an interaction potential, and is a set of all pairs of neighboring pixels. 3 [15] D. Greig, B. Porteous, and A. Seheult, “Exact Maximum A Posteriori Estimation for Binary Images,” J. Royal Statistical Soc., Series B, vol. 51, no. 2, pp , 1989.

Introduction (2/3)  Algorithm Dinic’s algorithm [14] Push-Relabel algorithm with the queue-based selection rule [12] Push-Relabel algorithm with the highest level selection rule [12] 4 [12] A.V. Goldberg and R.E. Tarjan, “A New Approach to the Maximum-Flow Problem,” J. ACM, vol. 35, no. 4, pp , Oct [14] E.A. Dinic, “Algorithm for Solution of a Problem of Maximum Flow in Networks with Power Estimation,” Soviet Math. Dokl., vol. 11, pp , 1970.

Introduction (3/3)  This paper use Min-Cut/Max-Flow to find minimum energy. Image Restoration ○ Potts model ○ Linear interaction Stereo ○ Pixel-Labeling stereo with the Potts model ○ Stereo with occlusion ○ Multi-camera scene reconstruction Segmentation 5

Overview (1/9) 6 A B C D

Overview (2/9) 7 A B C D S T Source Sink

Overview (3/9) 8 A B C D S T 2 2

Overview (4/9) 9 A B C D 2/9 2/2 S T

10 Overview (5/9)  There are two basic way to search max-flow Breadth-first search(BFS) Push  A typical augmenting path algorithm stores information about the distribution of the current s→t flow f among the edges of G using a residual graph G f. The topology of G f is identical to G, but the capacity of an edge in G f reflects the residual capacity of the same edge in G given the amount of flow already in the edge.

Overview (6/9) 11 A B C D 9 9 2/2 S T

Overview (7/9) 12 A B C D 9 9 2/2 S T 2 2 2

Overview (8/9) 13 A B C D 2/9 2/2 S T 2 2

14 Overview (9/9)  There are three steps on the proposed algorithm. A : growth B : augmentation C : adoption

Method  A : growth search trees S and T grow until they touch giving an s→t path  B : augmentation the found path is augmented, search tree(s) break into forest(s)  C : adoption trees S and T are restored. 15

Method.A (1/3)  At the growth stage, the search trees(s and t) expand. The active nodes explore adjacent nonsaturated edges and acquire new children from a set of free nodes. Active node Passive node Free node 16

Method.A (2/3)  The growth stage terminates if an active node encounters a neighboring node that belongs to the opposite tree. 17

Method.A (3/3) 18

Method  A : growth search trees S and T grow until they touch giving an s→t path  B : augmentation the found path is augmented, search tree(s) break into forest(s)  C : adoption trees S and T are restored. 19

Method.B (1/3)  The augmentation stage augments the path found at the growth stage. Since we push through the largest flow possible, some edge(s) in the path become saturated. Orphan node 20

Method.B (2/3) 21

Method.B (3/3)  The nodes that become orphan nodes keep their original own parents. Change another parent Become free nodes 22

Method  A : growth search trees S and T grow until they touch giving an s→t path  B : augmentation the found path is augmented, search tree(s) break into forest(s)  C : adoption trees S and T are restored. 23

Method.C (1/3)  The goal of the adoption stage is to restore the single- tree structure of sets S and T with roots in the source and the sink. Try to find a new valid parent for each orphan. A new parent should belong to the same set, S or T, as the orphan. If there is no qualifying parent, we remove the orphan from S or T and make it a free node. Also declare all its former children orphans. 24

Method.C (2/3) 25

Method.C (3/3)  The stage terminates when no orphans are left and, thus, the search tree structures of S and T are restored. Since some orphan nodes in S and T may become free, the adoption stage results in contraction of these sets. 26

Method.conclusion  After the adoption stage is completed, the algorithm returns to the growth stage. The algorithm terminates when the search trees S and T cannot grow (no active nodes) and the trees are separated by saturated edges. This implies that a maximum flow is achieved. 27

Experimental Results.part1  A : Dinic’s algorithm  B : Push-Relabel algorithm with the queue-based  C : Push-Relabel algorithm with the highest level 28

29

Experimental Results.part1  A : Dinic’s algorithm  B : Push-Relabel algorithm with the queue-based  C : Push-Relabel algorithm with the highest level 30

Experimental Results.B (1/1)  To push water from source to sink. Every node can store water temporally. Terminate when bottleneck. 31

Experimental Results.part1  A : Dinic’s algorithm  B : Push-Relabel algorithm with the queue-based  C : Push-Relabel algorithm with the highest level 32

Experimental Results.C (1/1)  Improved queue-based Set height levels to avoid water flow back 33

Experimental Results.part2  A : Image Restoration Potts model Linear interaction energy  B : Stereo Pixel-Labeling stereo with the potts model Stereo with occlusion Multi-camera scene reconstruction  C : Segmentation 34

Experimental Results.A (1/4)  Potts model 123  Linear interaction 1  Reference [2], [4], [8], [15], [18], [21], [32], [36], [38] 35

Experimental Results.A (2/4) 36

Experimental Results.A (3/4) 37

Experimental Results.A (4/4) 38

Experimental Results.part2  A : Image Restoration Potts model Linear interaction energy  B : Stereo Pixel-Labeling stereo with the Potts model Stereo with occlusion Multi-camera scene reconstruction  C : Segmentation 39

Experimental Results.B (1/6)  Potts model 1  Stereo with occlusion 1  Reference [4], [7], [23], [24] 40

Experimental Results.B (2/6) 41

Experimental Results.B (3/6) 42

Experimental Results.B (4/6)  Multi-camera scene reconstruction 1 ○ 1 1 ○ The smoothness term is the sum of Potts energy terms over all cameras. ○ The visibility term is infinity if a configuration f violate the visibility constraint and zero otherwise. 43

Experimental Results.B (5/6) 44

Experimental Results.B (6/6) 45

Experimental Results.part2  A : Image Restoration Potts model Linear interaction energy  B : Stereo Pixel-Labeling stereo with the potts model Stereo with occlusion Multi-camera scene reconstruction  C : Segmentation 46

Experimental Results.C (1/4)  Potts model 123  Reference [3], [5], [28], [29], [33], [35] 47

Experimental Results.C (2/4) 48

Experimental Results.C (3/4) 49

Experimental Results.C (4/4) 50

Conclusions  In most examples, our new min-cut/max-flow algorithm worked 2-5 times faster than any of the other methods, including the push-relabel and the Dinic algorithms (which are known to outperform other min-cut/max- flow techniques).  However, our algorithm is not a clear outperformer when the complexity of underlying graphs is increased. 51

Reference (1/4)  [2] Y. Boykov and D. Huttenlocher, “A New Bayesian Framework for Object Recognition,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. II, pp ,  [3] Y. Boykov and V. Kolmogorov, “Computing Geodesics and Minimal Surfaces via Graph Cuts,” Proc. Int’l Conf. Computer Vision, vol. I, pp ,  [4] Y. Boykov, O. Veksler, and R. Zabih, “Markov Random Fields with Efficient Approximations,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp ,  [5] Y. Boykov and G. Funka-Lea, “Optimal Object Extraction via Constrained Graph-Cuts,” Int’l J. Computer Vision, to appear.  [7] Y. Boykov and V. Kolmogorov, “An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision,” Proc. Int’l Workshop Energy Minimization Methods in Computer Vision and Pattern Recognition, pp , Sept

Reference (2/4)  [8] Y. Boykov, O. Veksler, and R. Zabih, “Fast Approximate Energy Minimization via Graph Cuts,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp , Nov  [15] D. Greig, B. Porteous, and A. Seheult, “Exact Maximum A Posteriori Estimation for Binary Images,” J. Royal Statistical Soc., Series B, vol. 51, no. 2, pp ,  [18] H. Ishikawa and D. Geiger, “Segmentation by Grouping Junctions,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp ,  [21] J. Kim, J.W. Fisher III, A. Tsai, C. Wible, A.S. Willsky, and W.M. Wells III, “Incorporating Spatial Priors into an Information Theoretic Approach for f MRI Data Analysis,” Medical Image Computing and Computer-Assisted Intervention, pp ,

Reference (3/4)  [23] V. Kolmogorov and R. Zabih, “Computing Visual Correspondence with Occlusions via Graph Cuts,” Proc. Int’l Conf. Computer Vision, July  [24] V. Kolmogorov and R. Zabih, “Multi-Camera Scene Reconstruction via Graph Cuts,” Proc. Seventh European Conf. Computer Vision, vol. III, pp , May  [28] S. Osher and N. Paragios, Geometric Level Set Methods in Imaging, Vision, and Graphics. Springer Verlag  [29] S. J. Osher and R.P. Fedkiw, Level Set Methods and Dynamic Implicit Surfaces. Springer Verlag,  [32] S. Roy and I. Cox, “A Maximum-Flow Formulation of the N-camera Stereo Correspondence Problem,” IEEE Proc. Int’l Conf. Computer Vision, pp ,

Reference (4/4)  [33] G. Sapiro, Geometric Partial Differential Equations and Image Analysis. Cambridge Univ. Press,  [35] J.A. Sethian, Level Set Methods and Fast Marching Methods. Cambridge Univ. Press,  [36] D. Snow, P. Viola, and R. Zabih, “Exact Voxel Occupancy with Graph Cuts,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp ,  [38] B. Thirion, B. Bascle, V. Ramesh, and N. Navab, “Fusion of Color, Shading and Boundary Information for Factory Pipe Segmentation,” IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp ,

Q & A 56