Outline Multilinear Analysis M. A. O. Vasilescu and D. Terzopoulos, “Multilinear Independent Components Analysis,” CVPR 2005
Motivations Natural images are generated by the interaction of multiple factors related to scene structure, illumination, and imaging November 10, 2018 Computer Vision
Motivations November 10, 2018 Computer Vision
Separating Styles from Content November 10, 2018 Computer Vision
Separating Styles from Content November 10, 2018 Computer Vision
Separating Styles from Content November 10, 2018 Computer Vision
Bilinear Model Suppose that we want to represent both style s and content c with vectors of parameters Let ysc denote a K-dimensional observation vector in style s and content class c November 10, 2018 Computer Vision
How to Learn the Bases and Coefficients Learning is done by minimizing total squared error over the entire training set Again the solution is given by a SVD decomposition November 10, 2018 Computer Vision
How to Learn the Bases and Coefficients For symmetric models, we also minimize the total squared error over the training set This is minimized using an iterative procedure November 10, 2018 Computer Vision
Classification Example November 10, 2018 Computer Vision
Classification Example When the model is trained on 10 faces and tested on the remaining one 1-NN achieves an accuracy of 53.9%±4.3% The proposed method achieves 73.9%±6.7% when parameters are determined automatically It achieves 80.6%±7.5% when optimal parameter values are used November 10, 2018 Computer Vision
Extrapolation November 10, 2018 Computer Vision
Extrapolation November 10, 2018 Computer Vision
Translation November 10, 2018 Computer Vision
Multilinear Analysis Tensors are multilinear mappings over a set of vector spaces An order N tensor is given by Mode-n vectors are given by They result in a mode-n flattening November 10, 2018 Computer Vision
Multilinear Analysis November 10, 2018 Computer Vision
Multilinear Analysis November 10, 2018 Computer Vision
Multilinear Analysis Mode-n product of a tensor and matrix is given by A tensor of n factors November 10, 2018 Computer Vision
Mode-n SVD November 10, 2018 Computer Vision
TensorFaces Weizmann face image database 28 male subjects photographed in 5 viewpoints, 4 illuminations, and 3 expressions Images are aligned to a reference face using a global rigid optical flow and then downsized by a factor of 3 and cropped, yielding a total of 7943 pixels per image within the elliptical cropping window November 10, 2018 Computer Vision
Weizmann Face Dataset November 10, 2018 Computer Vision
TensorFaces The dataset is then represented by a tensor of order 5 November 10, 2018 Computer Vision
TensorFaces November 10, 2018 Computer Vision
TensorFaces November 10, 2018 Computer Vision
TensorFaces November 10, 2018 Computer Vision
TensorFaces The bases for TensorFaces are given by November 10, 2018 Computer Vision
Face Recognition First experiment TensorFaces are trained on an ensemble comprising images of 23 people, captured from 3 viewpoints (0,±34 degrees), with 4 illumination conditions (center, left, right, left + right) It is tested on other images in this 23 person dataset acquired from 2 different viewpoints (±17 degrees) under the same 4 illumination conditions In this test scenario, the PCA method recognized the person correctly 61% of the time while TensorFaces recognized the person correctly 80% of the time. November 10, 2018 Computer Vision
Face Recognition The second experiment In a second experiment, TensorFaces is trained on images of 23 people 5 viewpoints (0,±17,±34 degrees), 3 illuminations (center light, left light, right light) Tested on the 4th illumination (left + right) PCA yielded a poor recognition rate of 27% while Tensorfaces achieved a recognition rate of 88% November 10, 2018 Computer Vision
Dimension Reduction November 10, 2018 Computer Vision
N-Mode Orthogonal Iteration Algorithm November 10, 2018 Computer Vision
Multilinear ICA Similar to MPCA, MICA is done using an n-mode ICA algorithm November 10, 2018 Computer Vision
Different ICA Architectures Architecture I ICA computes independent components of DT Architecture II ICA computes independent components of D November 10, 2018 Computer Vision
MICA November 10, 2018 Computer Vision
MICA November 10, 2018 Computer Vision
MICA November 10, 2018 Computer Vision
Recognition Experiment November 10, 2018 Computer Vision
TensorTextures Similar idea can be used for image synthesis November 10, 2018 Computer Vision
TensorTextures November 10, 2018 Computer Vision
TensorTextures November 10, 2018 Computer Vision
TensorTextures November 10, 2018 Computer Vision
TensorTextures November 10, 2018 Computer Vision
TensorTextures November 10, 2018 Computer Vision
TensorTextures November 10, 2018 Computer Vision
Comparison November 10, 2018 Computer Vision