Generalized Principal Component Analysis Tutorial @ CVPR 2008 Yi Ma ECE Department University of Illinois Urbana Champaign René Vidal Center for Imaging Science Institute for Computational Medicine Johns Hopkins University Title: Segmentation of Dynamic Scenes and Textures Abstract: Dynamic scenes are video sequences containing multiple objects moving in front of dynamic backgrounds, e.g. a bird floating on water. One can model such scenes as the output of a collection of dynamical models exhibiting discontinuous behavior both in space, due to the presence of multiple moving objects, and in time, due to the appearance and disappearance of objects. Segmentation of dynamic scenes is then equivalent to the identification of this mixture of dynamical models from the image data. Unfortunately, although the identification of a single dynamical model is a well understood problem, the identification of multiple hybrid dynamical models is not. Even in the case of static data, e.g. a set of points living in multiple subspaces, data segmentation is usually thought of as a "chicken-and-egg" problem. This is because in order to estimate a mixture of models one needs to first segment the data and in order to segment the data one needs to know the model parameters. Therefore, static data segmentation is usually solved by alternating data clustering and model fitting using, e.g., the Expectation Maximization (EM) algorithm. Our recent work on Generalized Principal Component Analysis (GPCA) has shown that the "chicken-and-egg" dilemma can be tackled using algebraic geometric techniques. In the case of data living in a collection of (static) subspaces, one can segment the data by fitting a set of polynomials to all data points (without first clustering the data) and then differentiating these polynomials to obtain the model parameters for each group. In this talk, we will present ongoing work addressing the extension of GPCA to time-series data living in a collection of multiple moving subspaces. The approach combines classical GPCA with newly developed recursive hybrid system identification algorithms. We will also present applications of DGPCA in image/video segmentation, 3-D motion segmentation, dynamic texture segmentation, and heart motion analysis.

Data segmentation and clustering Given a set of points, separate them into multiple groups Discriminative methods: learn boundary Generative methods: learn mixture model, using, e.g. Expectation Maximization

Dimensionality reduction and clustering In many problems data is high-dimensional: can reduce dimensionality using, e.g. Principal Component Analysis Image compression Recognition Faces (Eigenfaces) Image segmentation Intensity (black-white) Texture

Segmentation problems in dynamic vision Segmentation of video and dynamic textures Segmentation of rigid-body motions

Segmentation problems in dynamic vision Segmentation of rigid-body motions from dynamic textures

Clustering data on non Euclidean spaces Mixtures of linear spaces Mixtures of algebraic varieties Mixtures of Lie groups “Chicken-and-egg” problems Given segmentation, estimate models Given models, segment the data Initialization? Need to combine Algebra/geometry, dynamics and statistics

Outline of the tutorial Introduction (8.00-8.15) Part I: Theory (8.15-9.45) Basic GPCA theory and algorithms (8.15-9.00) Advanced statistical methods for GPCA (9.00-9.45) Questions (9.45-10.00) Break (10.00-10.30) Part II: Applications (10.30-12.00) Applications to motion and video segmentation (10.30-11.15) Applications to image representation & segmentation (11.15-12.00) Questions (12.00-12.15)

Part I: Theory Introduction to GPCA (8.00-8.15) Basic GPCA theory and algorithms (8.15-9.00) Review of PCA and extensions Introductory cases: line, plane and hyperplane segmentation Segmentation of a known number of subspaces Segmentation of an unknown number of subspaces Advanced statistical and methods for GPCA (9.00-9.45) Lossy coding of samples from a subspace Minimum coding length principle for data segmentation Agglomerative lossy coding for subspace clustering PART I = 38 slices = 1 hour Motivation: 7 slides GPCA: 31 slides Part II: 31 slides = 1 hour Computer vision: 18 slides Biomedical imaging: 6 slides Control: 13 slides

Part II: Applications in computer vision Applications to motion & video segmentation (10.30-11.15) 2-D and 3-D motion segmentation Temporal video segmentation Dynamic texture segmentation Applications to image representation and segmentation (11.15-12.00) Multi-scale hybrid linear models for sparse image representation Hybrid linear models for image segmentation PART I = 38 slices = 1 hour Motivation: 7 slides GPCA: 31 slides Part II: 31 slides = 1 hour Computer vision: 18 slides Biomedical imaging: 6 slides Control: 13 slides

References: Springer-Verlag 2008

Slides, MATLAB code, papers