1. Generalized Principal Component Analysis Tutorial @ CVPR 2008
Yi Ma
ECE Department
University of Illinois
Urbana Champaign
René Vidal
Center for Imaging Science
Institute for Computational Medicine
Johns Hopkins University
Title: Segmentation of Dynamic Scenes and Textures
Abstract: Dynamic scenes are video sequences containing multiple objects moving in front of dynamic backgrounds, e.g. a bird floating on water. One can model such scenes as the output of a collection of dynamical models exhibiting discontinuous behavior both in space, due to the presence of multiple moving objects, and in time, due to the appearance and disappearance of objects. Segmentation of dynamic scenes is then equivalent to the identification of this mixture of dynamical models from the image data. Unfortunately, although the identification of a single dynamical model is a well understood problem, the identification of multiple hybrid dynamical models is not. Even in the case of static data, e.g. a set of points living in multiple subspaces, data segmentation is usually thought of as a "chicken-and-egg" problem. This is because in order to estimate a mixture of models one needs to first segment the data and in order to segment the data one needs to know the model parameters. Therefore, static data segmentation is usually solved by alternating data clustering and model fitting using, e.g., the Expectation Maximization (EM) algorithm.
Our recent work on Generalized Principal Component Analysis (GPCA) has shown that the "chicken-and-egg" dilemma can be tackled using algebraic geometric techniques. In the case of data living in a collection of (static) subspaces, one can segment the data by fitting a set of polynomials to all data points (without first clustering the data) and then differentiating these polynomials to obtain the model parameters for each group. In this talk, we will present ongoing work addressing the extension of GPCA to time-series data living in a collection of multiple moving subspaces. The approach combines classical GPCA with newly developed recursive hybrid system identification algorithms. We will also present applications of DGPCA in image/video segmentation, 3-D motion segmentation, dynamic texture segmentation, and heart motion analysis.

2. Data segmentation and clustering
Given a set of points, separate them into multiple groups
Discriminative methods: learn boundary
Generative methods: learn mixture model, using, e.g. Expectation Maximization

3. Dimensionality reduction and clustering
In many problems data is high-dimensional: can reduce dimensionality using, e.g. Principal Component Analysis
Image compression
Recognition
Faces (Eigenfaces)
Image segmentation
Intensity (black-white)
Texture

4. Segmentation problems in dynamic vision
Segmentation of video and dynamic textures
Segmentation of rigid-body motions

5. Segmentation problems in dynamic vision
Segmentation of rigid-body motions from dynamic textures

6. Clustering data on non Euclidean spaces
Mixtures of linear spaces
Mixtures of algebraic varieties
Mixtures of Lie groups
“Chicken-and-egg” problems
Given segmentation, estimate models
Given models, segment the data
Initialization?
Need to combine
Algebra/geometry, dynamics and statistics

7. Outline of the tutorial
Introduction ( )
Part I: Theory ( )
Basic GPCA theory and algorithms ( )
Advanced statistical methods for GPCA ( )
Questions ( )
Break ( )
Part II: Applications ( )
Applications to motion and video segmentation ( )
Applications to image representation & segmentation ( )
Questions ( )

8. Part I: Theory Introduction to GPCA (8.00-8.15)
Basic GPCA theory and algorithms ( )
Review of PCA and extensions
Introductory cases: line, plane and hyperplane segmentation
Segmentation of a known number of subspaces
Segmentation of an unknown number of subspaces
Advanced statistical and methods for GPCA ( )
Lossy coding of samples from a subspace
Minimum coding length principle for data segmentation
Agglomerative lossy coding for subspace clustering
PART I = 38 slices = 1 hour
Motivation: 7 slides
GPCA: 31 slides
Part II: 31 slides = 1 hour
Computer vision: 18 slides
Biomedical imaging: 6 slides
Control: 13 slides

9. Part II: Applications in computer vision
Applications to motion & video segmentation ( )
2-D and 3-D motion segmentation
Temporal video segmentation
Dynamic texture segmentation
Applications to image representation and segmentation ( )
Multi-scale hybrid linear models for sparse image representation
Hybrid linear models for image segmentation
PART I = 38 slices = 1 hour
Motivation: 7 slides
GPCA: 31 slides
Part II: 31 slides = 1 hour
Computer vision: 18 slides
Biomedical imaging: 6 slides
Control: 13 slides

10. References: Springer-Verlag 2008

11. Slides, MATLAB code, papers