Secrets of GrabCut and Kernel K-means

Slides:

Advertisements

Similar presentations

O BJ C UT M. Pawan Kumar Philip Torr Andrew Zisserman UNIVERSITY OF OXFORD.

Advertisements

CSCE643: Computer Vision Mean-Shift Object Tracking Jinxiang Chai Many slides from Yaron Ukrainitz & Bernard Sarel & Robert Collins.

SOFT SCISSORS: AN INTERACTIVE TOOL FOR REALTIME HIGH QUALITY MATTING International Conference on Computer Graphics and Interactive Techniques ACM SIGGRAPH.

Graph cut Chien-chi Chen.

Pattern Recognition and Machine Learning

Interactive Segmentation with Super-Labels Andrew Delong Western Yuri BoykovOlga VekslerLena GorelickFrank Schmidt TexPoint fonts used in EMF. Read the.

1School of CS&Eng The Hebrew University

GrabCut Interactive Image (and Stereo) Segmentation Carsten Rother Vladimir Kolmogorov Andrew Blake Antonio Criminisi Geoffrey Cross [based on Siggraph.

GrabCut Interactive Foreground Extraction using Iterated Graph Cuts Carsten Rother Vladimir Kolmogorov Andrew Blake Microsoft Research Cambridge-UK.

GrabCut Interactive Foreground Extraction using Iterated Graph Cuts Carsten Rother Vladimir Kolmogorov Andrew Blake Microsoft Research Cambridge-UK.

Interactive Image Segmentation using Graph Cuts Mayuresh Kulkarni and Fred Nicolls Digital Image Processing Group University of Cape Town PRASA 2009.

Stephen J. Guy 1. Photomontage Photomontage GrabCut – Interactive Foreground Extraction 1.

1 s-t Graph Cuts for Binary Energy Minimization  Now that we have an energy function, the big question is how do we minimize it? n Exhaustive search is.

Graph-based image segmentation Václav Hlaváč Czech Technical University in Prague Faculty of Electrical Engineering Department of Cybernetics Prague, Czech.

GrabCut Interactive Image (and Stereo) Segmentation Joon Jae Lee Keimyung University Welcome. I will present Grabcut – an Interactive tool for foreground.

Pseudo-Bound Optimization for Binary Energies Presenter: Meng Tang Joint work with Ismail Ben AyedYuri Boykov 1 / 27.

Modeling Pixel Process with Scale Invariant Local Patterns for Background Subtraction in Complex Scenes (CVPR’10) Shengcai Liao, Guoying Zhao, Vili Kellokumpu,

Robust Higher Order Potentials For Enforcing Label Consistency

CS 376b Introduction to Computer Vision 04 / 08 / 2008 Instructor: Michael Eckmann.

Region Segmentation. Find sets of pixels, such that All pixels in region i satisfy some constraint of similarity.

Learning to Detect Natural Image Boundaries Using Local Brightness, Color, and Texture Cues David R. Martin Charless C. Fowlkes Jitendra Malik.

Schedule Introduction Models: small cliques and special potentials Tea break Inference: Relaxation techniques:

Segmentation CSE P 576 Larry Zitnick Many slides courtesy of Steve Seitz.

Problem Sets Problem Set 3 –Distributed Tuesday, 3/18. –Due Thursday, 4/3 Problem Set 4 –Distributed Tuesday, 4/1 –Due Tuesday, 4/15. Probably a total.

Image Segmentation Some slides: courtesy of O. Capms, Penn State, J.Ponce and D. Fortsyth, Computer Vision Book.

An Iterative Optimization Approach for Unified Image Segmentation and Matting Hello everyone, my name is Jue Wang, I’m glad to be here to present our paper.

Image Segmentation Today’s Readings Forsyth & Ponce, Chapter 14

Statistical analysis and modeling of neural data Lecture 4 Bijan Pesaran 17 Sept, 2007.

Measuring Uncertainty in Graph Cut Solutions Pushmeet Kohli Philip H.S. Torr Department of Computing Oxford Brookes University.

Perceptual Organization: Segmentation and Optical Flow.

Dorin Comaniciu Visvanathan Ramesh (Imaging & Visualization Dept., Siemens Corp. Res. Inc.) Peter Meer (Rutgers University) Real-Time Tracking of Non-Rigid.

Graph-based Segmentation

Image Segmentation Image segmentation is the operation of partitioning an image into a collection of connected sets of pixels. 1. into regions, which usually.

Jinhui Tang †, Shuicheng Yan †, Richang Hong †, Guo-Jun Qi ‡, Tat-Seng Chua † † National University of Singapore ‡ University of Illinois at Urbana-Champaign.

Image Segmentation Rob Atlas Nick Bridle Evan Radkoff.

Computer Vision James Hays, Brown

MRFs and Segmentation with Graph Cuts Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 03/31/15.

Mean-shift and its application for object tracking

1 Mean shift and feature selection ECE 738 course project Zhaozheng Yin Spring 2005 Note: Figures and ideas are copyrighted by original authors.

Graph Cut & Energy Minimization

A General Framework for Tracking Multiple People from a Moving Camera

Region Segmentation Readings: Chapter 10: 10.1 Additional Materials Provided K-means Clustering (text) EM Clustering (paper) Graph Partitioning (text)

MRFs and Segmentation with Graph Cuts Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/24/10.

City University of Hong Kong 18 th Intl. Conf. Pattern Recognition Self-Validated and Spatially Coherent Clustering with NS-MRF and Graph Cuts Wei Feng.

Object Stereo- Joint Stereo Matching and Object Segmentation Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on Michael Bleyer Vienna.

CSE 185 Introduction to Computer Vision Pattern Recognition 2.

INTRODUCTION TO Machine Learning 3rd Edition

O BJ C UT M. Pawan Kumar Philip Torr Andrew Zisserman UNIVERSITY OF OXFORD.

Boosted Particle Filter: Multitarget Detection and Tracking Fayin Li.

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

A Dynamic Conditional Random Field Model for Object Segmentation in Image Sequences Duke University Machine Learning Group Presented by Qiuhua Liu March.

Pseudo-Bound Optimization for Binary Energies Meng Tang 1 Ismail Ben Ayed 2 Yuri Boykov 1 1 University of Western Ontario, Canada 2 GE Healthcare Canada.

Journal of Visual Communication and Image Representation

CS654: Digital Image Analysis Lecture 28: Advanced topics in Image Segmentation Image courtesy: IEEE, IJCV.

Tree and Forest Classification and Regression Tree Bagging of trees Boosting trees Random Forest.

Image segmentation.

MRFs and Segmentation with Graph Cuts Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 03/27/12.

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 1: INTRODUCTION.

GrabCut Interactive Foreground Extraction using Iterated Graph Cuts Carsten Rother Vladimir Kolmogorov Andrew Blake Microsoft Research Cambridge-UK.

Nonparametric Semantic Segmentation

A segmentation and tracking algorithm

Unsupervised-learning Methods for Image Clustering

Statistical Learning Dong Liu Dept. EEIS, USTC.

Normalized Cut meets MRF

PRAKASH CHOCKALINGAM, NALIN PRADEEP, AND STAN BIRCHFIELD

“grabcut”- Interactive Foreground Extraction using Iterated Graph Cuts

Seam Carving Project 1a due at midnight tonight.

Segmentation (continued)

Model generalization Brief summary of methods

Presentation transcript:

Secrets of GrabCut and Kernel K-means Meng Tang1 Ismail Ben Ayed2 Dmitrii Marin1 Yuri Boykov1 1University of Western Ontario, Canada 2University of Quebec, Canada GrabCut as Color Clustering Probabilistic K-means vs Kernel K-means Our Energy Formulation & Optimization Initialization Basic K-means (e.g. Chan-Vese) Variance Criterion Kernel K-means plus smoothness regularization: Pairwise clustering term MRF regular. GMMs fitting more complex data representation probability models solution space S St St+1 At(S) E(S) E(St) E(St+1) bound optimization for E(S) unary bound for kernel k-means[4] minimize bound with graph cuts Probabilistic K-means (e.g. Zhu&Yuille, GrabCut[1]) kernel K-means[4] (ours) 3D color space descriptive GMMs over-fit the data sensitive to local minima log-likelihoods with parametric probability Model parameters θ often jointly optimized Experiments of GrabCut and our KKM Kernel give implicit feature Model-fitting & Kernel K-means Low dimension: RGB only GrabCut KKM aKKM Box and GT example θ : descriptive models (histogram or GMM) example k : normalized Gaussian (fixed small width) GrabCut KKM A B C Smoothness map Entropy Criterion Gini Criterion H(S) - entropy for intensities in S G(S) – Gini impurity for intensities in S (a) initialization (b) K-means (c) Elliptic K-means intensity histogram Solution A Solution B Secrets of Kernel K-means: the bias of optimal Gini clustering (see Breiman [3], for histograms) Figure 1: Robustness to regularization weight Figure 2: sample results on GrabCut dataset (d) Histogram fitting (e) GMM: local mini. (f) GMM: from G.T. bias to small & dense cluster observed for average association in [2]. We extend Breiman’s analysis [3] to continuous case common adaptive kernels (e.g. KNN) eliminate such bias (see paper) High dimension: RGBD (depth) depth map GrabCut Our aKKM solution A and B are equally good when using histograms local minima issue using GMMs (g) K-mode (mean-shift) (h) Our Kernel K-means model-fitting: local min., binning kernel K-means: non-linear separa. Figure 4: Scalability to higher dimension on RGB+wD Figure 3: Average error on GrabCut dataset High dimension feature : RGBXYM (location XY, motion M) Proposed Kernel K-means “World Map” of Segmentation Fixed Gaussian Kernel pKM Probabilistic K-means (ML model fitting) kKM kernel K-means (average distortion AD or association AA) basic K-means p.s.d. kernel weak kernel clustering (unary) Hilbertian distortion Normalized Cuts K-modes (mean-shift) GMM fitting histogram fitting geometric fitting gamma fitting Gaussian kernel K-means Spectral clustering distortion Gibbs fitting Average Cut Gini Bias Adaptive Kernel good clustering (a)Video frames (b)Optical flow (c)Our aKKM(RGBXY) (d)Our aKKM(RGBXYM) [1] C. Rother, V. Kolmogorov and A. Blake, GrabCut: Interactive Foreground Extraction using Iterated Graph Cuts, in SIGGRAPH 2004. [2] J. Shi, and J. Malik. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 22:888-905, 2000. [3] L. Breiman. Technical note: Some properties of splitting criteria. Machine Learning, 24(1):41-47, 1996. [4] I. Dhillon, Y. Guan, and B. Kulis. Kernel K-means, spectral clustering and normalized cuts. In KDD, 2004.