PCA Extension By Jonash

Outline Robust PCA Generalized PCA Clustering points on a line Clustering lines on a plane Clustering hyperplanes in a space

Robust PCA Rrbust Principal Component Analysis for Computer Vision Fernando De la Torre Mochael J. Black CS, Brown University

PCA is Least-Square Fit

PCA is Least-Square Fit

Robust Statistics Recover the best fit for the majority of the data Detect and reject outliers

Robust PCA

Robust PCA

Robust PCA Training images

Robust PCA Naïve PCA Simply reject Robust PCA

RPCA BBTdi di B BTdi = ci In traditional PCA, we minimize Σni = 0 (di – B BT di)2 = Σni = 0 (di - Bci)2 EM PCA Limσ -> 0(D = BC + σ2I) E-step C = (BTB)-1BTD M-step B = DCT(CCT)-1 BBTdi di B BTdi = ci

RPCA Xu and Yuille [1995] tries to minimize Σni = 1 [ V i(di – B ci)2 + n(1-Vi) ] Hard to solve (continuous + discrete)

RPCA Gabriel and Zamir [1979] tries to minimize Σni = 1 Σdp = 1 [ wpi(dpi – Bci)2] Impratical for high dimension “Low rank approximation of matrices by least squares with any choice of weights” 1979

RPCA Idea is to use a robust function ρ Geman-McClureρ (x,σ) = x2/(x2 + σ2) Σni = 1 Σdp = 1 ρ[ (dpi–μp –Σkj = 1 bpjcji), σp] Approximated by local quadratic function Use gradient descent The rest is nothing but heuristics

RPCA

Robust PCA - Experiment 256 training images (120x160) Obtain 20 RPCA basis 3 hrs on 900MHz Pentinum III in Matlab

Outline Robust PCA Generalized PCA Clustering points on a line Clustering lines on a plane Clustering hyperplanes in a space

Generalized PCA Generalized Principal Component Analysis Rene Vidal Yi Ma Shankar Sastry UC Berkley and UIUC

GPCA

GPCA Example 1

GPCA Example 2

GPCA Example 3

GPCA Goals # of subspaces and their dimension Basis for subspace Segmentation of data

GPCA Ideas Union of subspaces = certain polynomials Noise free case

Outline Robust PCA Generalized PCA Clustering points on a line Clustering lines on a plane Clustering hyperplanes in a space

GPCA 1D Case

GPCA 1D Case Cont’d

To have a unique solution, GPCA 1D Case Cont’d MN = n+1 unknowns To have a unique solution, rank(Vn) = n = Mn- 1

GPCA 1D Example n = 2 groups pn(x) = ( x – μ1) ( x – μ2) No polynomial of degree 1 Infinite polynomial of degree 3 pn(x) = x2 + c1x + c2 => Polynomial factor

Outline Robust PCA Generalized PCA Clustering points on a line Clustering lines on a plane Clustering hyperplanes in a space

GPCA 2D Case L j = { X = [x, y]T: bj1x + bj2y = 0 } (b11x + b12y = 0) or (b21x + b22y = 0)…

GPCA 2D Case Cont’d (b11x + b12y = 0) or (b21x + b22y = 0)… Pn(x) = (b11x + b12y)…(bn1x + bn2y) = 0 = Σck xn-k yk

GPCA 2D Case Cont’d Take n = 2 for example… p2(x) = (b11x + b12y)(b21x + b22y) ▽p2(x) = (b21x + b22y)b1 + (b11x + b12y)b2 , bj = [bj1, bj2] T if x ~ L1, then ▽p2(x) ~ b1, otherwise ~ b2

GPCA 2D Case Cont’d Given that {yj ε Lj}, the normal vector of Lj is bj ~ ▽pn(yj) 3 things… Determine “ n ” as min{ j: rank(Vj) = j } Solve cn for Vncn = 0 Find normal vector bj

Outline Robust PCA Generalized PCA Clustering points on a line Clustering lines on a plane Clustering hyperplanes in a space

GPCA Hyperplanes Still assume d1 = … = dn = d = D – 1 Sj = { bjTx = bj1x1 + bj2x2 + … + bjDxD = 0}

GPCA Hyperplanes MN = C(D+n-1, D)

GPCA Hyperplanes

GPCA Hyperplanes Since we know n, we can solve for ck ck => bk by ▽pn(x) If we know yj on each Sj, finding bj will be easy

GPCA Hyperplanes One point yj on each hyperplane Sj Consider a random line L = t * v + x0 Obtain yj by intersecting L and Sj yj = tj * v + x0 Find roots tj by … Pn(t v + xo)

GPCA Hyperplanes Summarize We want to find n to solve for c To get b (normal) for each S, find ▽pn(x) To get label j, solve pn(yj = tj * v + x0) = 0

One More Thing

One More Thing Previously we assume d1 = … =dn= D – 1 Actually we cannot assume that… Please read section 4.2 & 4.3 … by yourself Discuss how to recursively reduce dimension