1. 1 Markov Random Fields with Efficient Approximations Yuri Boykov, Olga Veksler, Ramin Zabih Computer Science Department CORNELL UNIVERSITY

2. 2 Introduction MAP-MRF approach (Maximum Aposteriori Probability estimation of MRF) Bayesian framework suitable for problems in Computer Vision (Geman and Geman, 1984) Problem: High computational cost. Standard methods (simulated annealing) are very slow.

3. 3 Outline of the talk n Models where MAP-MRF estimation is equivalent to min-cut problem on a graph generalized Potts model linear clique potential model n Efficient methods for solving the corresponding graph problems n Experimental results stereo, image restoration

4. 4 MRF framework in the context of stereo MRF defining property: Hammersley-Clifford Theorem: neighborhood relationships ( n-links ) image pixels ( vertices ) - disparity at pixel p - configuration

5. 5 MAP estimation of MRF configuration Observed data Likelihood function (sensor noise) Prior (MRF model) Bayes rule

6. 6 Energy minimization Find that minimizes the Posterior Energy Function : Data term (sensor noise) Smoothness term (MRF prior)

7. 7 Generalized Potts model Clique potential Penalty for discontinuity at (p,q) Energy function

8. 8 Static clues - selecting Stereo Image : White Rectangle in front of the black background Disparity configurations minimizing energy E( f ):

9. 9 Minimization of E(f) via graph cuts p-vertices (pixels) Cost of n-link Cost of t-link Terminals (possible disparity labels)

10. 10 Multiway cut vertices V = pixels + terminals edges E = n-links + t-links A multiway cut C yields some disparity configuration Remove a subset of edges C C is a multiway cut if terminals are separated in G(C) Graph G = Graph G(C) =

11. 11 Main Result (generalized Potts model) n Under some technical conditions on the multiway min-cut C on G gives___ that minimizes E( f ) - the posterior energy function for the generalized Potts model. Multiway cut Problem: find minimum cost multiway cut C graph G

12. 12 Solving multiway cut problem n Case of two terminals: max-flow algorithm (Ford, Fulkerson 1964) polinomial time (almost linear in practice). n NP-complete if the number of labels >2 (Dahlhaus et al., 1992) n Efficient approximation algorithms that are optimal within a factor of 2

13. 13 Our algorithm Initialize at arbitrary multiway cut C 1. Choose a pair of terminals 2. Consider connected pixels

14. 14 Our algorithm Initialize at arbitrary multiway cut C 1. Choose a pair of terminals 2. Consider connected pixels 3. Reallocate pixels between two terminals by running max-flow algorithm

15. 15 Our algorithm Initialize at arbitrary multiway cut C 1. Choose a pair of terminals 2. Consider connected pixels 3. Reallocate pixels between two terminals by running max-flow algorithm 4. New multiway cut C’ is obtained Iterate until no pair of terminals improves the cost of the cut

16. 16 Experimental results (generalized Potts model) n Extensive benchmarking on synthetic images and on real imagery with dense ground truth From University of Tsukuba Comparisons with other algorithms

17. 17 Synthetic example Image CorrelationMultiway cut

18. 18 Real imagery with ground truth Ground truth Our results

19. 19 Comparison with ground truth

20. 20 Gross errors (> 1 disparity)

21. 21 Comparative results: normalized correlation DataGross errors

22. 22 Statistics

23. 23 Related work (generalized Potts model) n Greig et al., 1986 is a special case of our method (two labels) n Two solutions with sensor noise (function g) highly restricted Ferrari et al., 1995, 1997

24. 24 Linear clique potential model Clique potential Penalty for discontinuity at (p,q) Energy function

25. 25 Minimization of via graph cuts Cost of n-link Cost of t-link {p,q} part of graph a cut C yields some configuration cut C

26. 26 Main Result (linear clique potential model) n Under some technical conditions on the min-cut C on gives that minimizes - the posterior energy function for the linear clique potential model.

27. 27 Related work (linear clique potential model) n Ishikawa and Geiger, 1998 earlier independently obtained a very similar result on a directed graph n Roy and Cox, 1998 undirected graph with the same structure no optimality properties since edge weights are not theoretically justified

28. 28 Experimental results (linear clique potential model) n Benchmarking on real imagery with dense ground truth From University of Tsukuba n Image restoration of synthetic data

29. 29 Ground truth stereo image ground truth Generalized Potts model Linear clique potential model

30. 30 Image restoration Noisy diamond image Generalized Potts model Linear clique potential model