1. Interactive Image Segmentation using Graph Cuts Mayuresh Kulkarni and Fred Nicolls Digital Image Processing Group University of Cape Town PRASA 2009

2. Outline Image Segmentation Problem Our Approach Graph cuts and Gaussian Mixture Models Results and Discussion Future Research

3. What is foreground?

4. Image Segmentation

5. Our Approach Graph Cuts Segmentation Cost Function : E(A) = λ R(A) + B(A) Region informationBoundary informationPixel connectivity 8 – pixel neighbourhood Difference between adjacent pixels Image properties eg. colour, texture

6. Graph Cuts Source (foreground) Sink (background) Cost Function : E(A) = λ R(A) + B(A) Pixel connectivity (boundaries) Inter-pixel weights (boundaries) Source and Sink weights (regions)

7. Gaussian Mixture Models Background GMMForeground GMM

8. Gaussian Mixture Models Foreground GMM Background GMM Log Likelihood Ratio = log(K *pf/pb) pf pb

9. GMM components Greyscale images – Intensity values – Intensity values and MR8 filters Colour images – RGB values – G, (G-R), (G-B) values – Luv values – MR8 filters

10. Boundary information Inter-pixel weights – Edge detection – Difference between adjacent pixels – Gradient Pixel connectivity

11. Results Κ = 0.01Κ = 0.1Κ = 1

12. Results Original Image RGB, Luv and MR8 (Fscore = 0.916) Luv and MR8 (Fscore = 0.921) Luv (Fscore = 0.934)

13. Results Original ImageRGB, Luv and MR8 (Fscore = 0.906) RGB (Fscore = 0.951) Luv (Fscore = 0.945)

14. Analysis of Results Accurate segmentation achieved Components in the GMM depend on image Segmentation can be controlled using K and λ

15. Future Research Different grid (non-pixel grid) Ratio cuts Exploring other statistical models ObjCut – segmenting particular objects

16. References Y. Boykov and M. P. Jolly. Interactive graph cuts for optimal boundary and region segmentation of objects in N-D images. In ICCV, volume 1, pages 105–112, July 2001. Yuri Boykov and Vladimir Kolmogorov. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell., 26(9):1124–1137, 2004. Pushmeet Kohli, Jonathan Rihan, Matthieu Bray, and Philip H. S. Torr. Simultaneous segmentation and pose estimation of humans using dynamic graph cuts. International Journal of Computer Vision, 79(3):285–298, 2008. H. Permuter, J. Francos, and I. Jermyn. Gaussian mixture models of texture and colour for image database. In ICASSP, pages 25–88, 2003. D. Martin, C. Fowlkes, D. Tal, and J. Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proc. 8th Int’l Conf. Computer Vision, volume 2, pages 416–423, July 2001. Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. “GrabCut”: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph., 23(3):309–314, August 2004.