Lecture 8:Eigenfaces and Shared Features CAP 5415: Computer Vision Fall 2006

What I did on Monday

What I did on Monday

Questions on PS2?

Task: Face Recognition ? From “Bayesian Face Recognition” by Moghaddam, Jebara, and Pentland

Easiest way to recognize Compute the squared difference between the test image and each of the examples Nearest-Neighbor Classifier

Easiest way to recognize Problems Slow Distracted by things not related to faces

Easiest way to recognize How can we find a way to represent a face with a few numbers?

Linear Algebra to the Rescue Raster-Scan the image into a vector

Stack Examples into a Matrix If we have M example faces in the database we get a matrix: One face

Introducing the SVD Stands for Singular Value Decomposition The matrix M can be factored as m x n n x n m x n m x m

Special Properties of these Matrices U – unitary, for a real-valued matrix, that means UTU = I Σ – diagonal. Only non-zero along the diagonal. These non-zero entries are called the singular values V – also unitary

Another way to think of this ΣV is a set of weights that tells us how to add together the columns of U to produce the data What if there are too many observations in each column?

Approximating M Modify Σ by setting all but a few of the singular values to zero. Effectively, only using some columns of U to reconstruct M Same as using some small number of parameters to measure a N-dimensional signal

Approximating M Question: If I were going to approximate M using a few columns of U, which few should I use? Answer: Find the rows in Σ with the largest singular values and use the corresponding columns of M

Simple Example

Calculate SVD [u,s,v] = svd([x' y']') >> u u = -0.6715 -0.7410 -0.6715 -0.7410 -0.7410 0.6715 >> s(:,1:2) ans = 5.5373 0 0 0.7852 >>

U defines a set of axes [u,s,v] = svd([x' y']') >> u u = -0.6715 -0.7410 -0.7410 0.6715 >> s(:,1:2) ans = 5.5373 0 0 0.7852 >>

Now, let's reduce these points to one dimension >> sp=s; >> sp(2,2)=0; >> nm = u*sp*v'; >>

What have we done? We have used one dimension to describe the points instead of two We used the SVD to find the best set of axes Using the SVD minimizes the Frobenius norm of difference between M and ~M In other words, minimizes squared error

How does this relate to faces? Let's assume that we have images of faces with N pixels Don't N pixels to represent the face Assume that the space of face images can be represented by relatively few number of axes Called eigenfaces From “Probabilistic Visual Learning” by Moghaddam and Pentland

Why “eigenfaces”? If you subtract the mean, MMT is the covariance matrix of the data The matrix U also contains the eigenvectors of MMT These eigenvectors are the vectors that maximize the variance of the reconstruction Same result, different motivation than SVD Called Principle Components Analysis or PCA Take Pattern Recognition!

Empirical proof that faces lie in relatively low-dimensional subspace From “Probabilistic Visual Learning” by Moghaddam and Pentland

From “Probabilistic Visual Learning” by Moghaddam and Pentland

Empirical proof that faces lie in relatively low-dimensional subspace From “Probabilistic Visual Learning” by Moghaddam and Pentland

Basic Eigenface Recognition Find the face Normalize the image Project it into onto the eigenfaces Do nearest -neighbor classification Lots of variations that I won't get into

What makes it harder or easier? From “How Features of the Human Face Affect Recognition: a Statistical Comparison of Three Face Recognition Algorithms” by Givens et al.

Does it work in the real world?

Guessing Gender from Faces From Learning Gender with Support Faces by Moghaddam and Yeang