1. Normalized Cut meets MRF
Meng Tang1 Dmitrii Marin1 Ismail Ben Ayed2 Yuri Boykov1 1University of Western Ontario, Canada École de Technologie Supérieure, Canada
Energy Formulation
Optimization: Kernel Cut and Spectral Cut
Normalized Cut (NC)
MRF
General optimization approach: bound optimization
E(S)
At (S) St
St+1
At+1 (S)
balanced
regularization
unary kernel or spectral bound
clustering
constraints
to optimize bound At(S) we use move-making graph cuts
[Boykov, Veksler, Zabih, 2001]
Why MRF for NC
semi-supervision is challenging
How to incorporate group priors?
weak edge alignment
How many clusters?
Kernel bound for NC (exact)
8 clusters
Normalized Cut is concave:
NC approach:
cannot-link
3 clusters
Lemma (concavity) Function e : R|Ω| → R is concave over region Sk > 0
given p.s.d. affinity matrix A := [Apq].
must-link
[Chew et al. 2015] [Eriksson et al. 2010] [Yu & Shi 2004]
[Arbelaez et al., 2010]
1st order Taylor expansion
is an upper bound for NC:
at(Sk) St
St+1
at+1(Sk)
e(Sk)
our approach:
(sparsity prior)
NC + label cost
NC+Potts
NC+Potts+seeds
NC + Pn Potts
MRF
MRF
MRF
MRF
Why NC for MRF
(equivalently kernel k-means [Dhillon t al., 2004])
Spectral bound for NC (approx.)
ML term for 𝜃 𝑘
Step 1. low-rank (m) approximation
Step 2. low-dim. isomeric Euclidean embedding
for SVDs
Probabilistic K-means [Kearns et al., UAI’97]
bad clustering
(overfitting)
Model Fitting (e.g. GMM)
(GrabCut without smoothness)
Normalized Cut
good clustering
and
low-rank m
implying isometry
MDS [Cox & Cox., 2000]
Kernel PCA [Schölkopf et al., 1998] NC
K-means
unary spectral bound
Summary of contribution
new unary kernel and spectral bounds for NC
can combine NC with any MRF constraints
can combine MRF with balanced clustering
MRF with features of any dimension (RGBD, RGBM, RGBXYM, deep,…)
We show NC ≈ K-means via low-rank (m) approximation
In contrast, K-means is a discretization heuristic in NC spectral relaxation
[Shi & Malik 2000, Yu & Shi 2003]
Spectral bound accuracy
(depending on m)
m = 3
K-means
m = 10 NC
Experiments: NC helps MRF
Experiments: MRF helps NC
poor scalability to high-dim
GrabCut
Kernel Cut
Potts model improves edge alignment
RGBD segmentation
BSDS500 measure
Covering
PRI
VOI
Spectral Clustering
0.34
0.76
2.76
Our Kernel Cut
0.41
0.78
2.44
Our Spectral Cut
0.42
2.34
Spectral Clustering
(no Potts)
Our Kernel Cut
(NC with Potts)
Our Spectral Cut
(NC with Potts)
High dimension feature : RGBXYM (location XY, motion M)
Label cost
Robust Pn Potts
using image tags (e.g. beach, car) to help clustering
Image clustering with
tags consistency prior
Our Kernel Cut with increasing label cost
(a)Video frames
(b)Optical flow
(c)Kernel Cut(RGBXY)
(d)Kernel Cut(RGBXYM)