Affine Registration in R m 5. The matching function allows to define tentative correspondences and a RANSAC-like algorithm can be used to estimate the full correpsondences. - Affine Iterative Closest Point (Affine ICP) 1. Once the correspondences are given, the optimal affine transform A can be computed quickly by minimizing the error function above. 2. A can be solved via a linear system of equations. 3. Iteratively estimate the correspondences and the affine transform A. Experiments  Experiment with Synthetic Data Points - Experiments with point sets in different dimensions. - Added noises range from 0% to 10%. - Exact affine transformation is recovered in noiseless setting.  2D point matching. Small view differences can be accounted for by affine transformation of the images. 1. No image features (i.e., intensities) are required. 2. Matching results have small RMS error.  Image set matching. The algorithm correctly estimates all the correspondences. Convergence is observed in this experiment while a direct application of the affine ICP does not converge. Abstract Affine Registration has a long and venerable history in computer vision literature, and extensive work have been done for affine registration in R2 and R3. In this paper, we study affine registrations in R m for m > 3. To justify breaking this dimension barrier, we show two interesting types of matching problems that can be formulated and solved as affine registration problems in dimensions higher than three: stereo correspondence under motion and image set matching. More specifically, for an object undergoing non-rigid motion that can be linearly modeled using a small number of shape basis vectors, the stereo correspondence problem can be solved by affine registering points in R 3n. Given two collections of images related by an unknown linear transformation of the image space, the correspondences between images in the two collections can be recovered by solving an affine registration problem in R m, where m is the dimension of a PCA subspace. The algorithm proposed in this paper estimates the affine transformation between two points in Rm. It does not require continuous optimization, and our analysis shows that in the absence of data noise, the algorithm will recover the exact affine transformation for almost all point sets. We validate the proposed algorithm on a variety of synthetic point sets in different dimensions with varying degrees of deformation and noise, and we also show experimentally that the two types of matching problems can indeed be solved satisfactorily using the proposed affine registration algorithm. Overview  Affine registration in dimension higher than three.  Why do we need to study it?  Stereo Correspondences under Motion - Non-rigid motion modeled with linear shape basis. - Establishing correspondences across different views.  Image Set Matching - Two collections of images, related by some affine transformation of the image space. - A global approach for matching that does not require matching individual images. Affine Registration and Vision Applications  Stereo Correspondences under Motion Assumption: Non-rigid motion that can be modeled using linear shape bases. Setup: 1. Two (affine) cameras observing an object undergoing non-rigid motion. 2. Feature points on object are tracked consistently in each camera. 3. Problem: Compute the correspondences between the observed feature points in two cameras. The affine connection comes from the factorization: where X i t is the 2D tracked points in images, B is the matrix of shape basis. The tensor product is between the shape coefficients and camera matrices. In the paper, we show that the points X 1 t X 2 t … X k t belong to a subspace S of dimension 3m, where m is the number of basis elements. The correspondences, between {X 1 t, X 2 t … X k t } and {Y 1 t, Y 2 t, … Y k t }, are given by an affine transformation A on S.  Image Set Matching Assumption: Two collections of images are related by some affine transformation of the image space. Example: 1. Images related by a 2D orthogonal transform. 2. Images from two different cameras. Method: 1. { X 1 t, X 2 t, …, X k t } and { Y 1 t, Y 2 t, …, Y k t } are two collections of images. 2. Compute PCA subspaces for the image collections and projects. 3. Estimate the affine transformation between the projected points. Affine Registration in R m Let P ={ p 1, p 2, …, p k } and Q = { q 1, q 2, …, q k } be two point sets in R m related by where A is a nonsingular matrix in GL(m) and gives the correspondences. Affine Registration in R m  Algorithm - Reduce the problem to orthogonal case. Minimizing the error function Coordinate transforms by the squared-root of the covariance matrix: The transformed points are related by an unknown orthogonal transformation. - Spectral Approach 1. Pair-wise distances are preserved under orthogonal transform. 2. Use the spectral information of a symmetric matrix to estimate the correspondences. One example for the function and this gives the (un-normalized) discrete Laplacian. 3. The eigenvector of the symmetric matrix L can be considered as functions on the point set. If two point-sets are related by an orthogonal transform, there will be correspondences between the eigenvalues of their matrices. 4. Using eigenvectors with corresponding eigenvalues as the features for computing correspondences. Let be the two sets of corresponding eigenvectors. A matching function Higher Dimensional Affine Registration and Applications Jason ChiNejhum ShahedJeffrey HoMing-Hsuan Yang CISE Department, University of Florida, Gainesville, FL CISE Department, University of Florida, Gainesville, FL CISE Department, University of Florida, Gainesville, FL EECS Department, University of California, Merced, CA