Motion Analysis Slides are from RPI Registration Class.

2 Registration Images are described as discrete sets of point locations associated with a geometric measurement Locations may have additional properties such as intensities and orientations Locations may have additional properties such as intensities and orientations Registration problem involves two parts: Finding correspondences between features Finding correspondences between features Estimating the transformation parameters based on these correspondences Estimating the transformation parameters based on these correspondences

3 Notation Set of moving image features Set of fixed image features Each feature must include a point location in the coordinate system of its image. Set of correspondences:

4 Error objective function depends on unknown transformation parameters and unknown feature correspondences Each may depend on the other! Each may depend on the other! Transformation may include mapping of more than just locations Distance function, D, could be as simple as the Euclidean distance between location vectors. Mathematical Formulation

5 Correspondence Problem Determine correspondences before estimating transformation parameters Based on rich description of features Based on rich description of features Error prone Error prone OR Other way around? OR Other way around? Determine correspondences at the same time as estimation of parameters “Chicken-and-egg” problem “Chicken-and-egg” problem For the next few minutes we will assume a set of correspondences is given and proceed to the estimation of parameters We will return to the correspondence problem later We will return to the correspondence problem later

6 Example: Estimating Parameters 2d point locations: Similarity transformation: Euclidean distance:

7 Putting This Together

8 What Do We Have? Least-squares objective function Quadratic function of each parameter We can Take the derivative with respect to each parameter Take the derivative with respect to each parameter Set the resulting gradient to 0 (vector) Set the resulting gradient to 0 (vector) Solve for the parameters through matrix inversion Solve for the parameters through matrix inversion We’ll do this in two forms: component and matrix/vector

9 Component Derivative (a)

10 Component Derivative (b) At this point, we’ve dropped the leading factor of 2. It will be eliminated when this is set to 0.

11 Component Derivatives t x and t y

12 Gathering Setting each of these equal to 0 we obtain a set of 4 linear equations in 4 unknowns. Gathering into a matrix we have:

13 Solving This is a simple equation of the form Provided the 4x4 matrix X is full-rank (evaluate SVD) we easily solve as

14 Matrix Version We can do this in a less painful way by rewriting the following intermediate expression in terms of vectors and matrices:

15 Matrix Version (continued) Equal to zero. Take transpose on either side..

16 Matrix Version (continued) Taking the derivative of this wrt the transformation parameters (we didn’t cover vector derivatives, but this is fairly straightforward): Setting this equal to 0 and solving yields:

17 Comparing the Two Versions Final equations are identical (if you expand the symbols) Matrix version is easier (once you have practice) and less error prone Sometimes efficiency requires hand- calculation and coding of individual terms

18 Registration (rigid+scale) Given two point sets Our goal is to find the similarity transformation that best aligns the moments of the two point sets. In particular we want to find The rotation matrix R, t The rotation matrix R, t The scale factor s, and The scale factor s, and The translation vector t The translation vector t

19 The first moment is the center of mass

20 Second Moments Scatter matrix Eigenvalue / eigenvector decomposition (S p ) where i are the eigenvalues and the columns of V p hold the corresponding eigenvectors:

21 Rotation, Translation and Scale Rotated, uniformly scaled points: Center of the rotated data is the same as the rotated, scaled and translated center before rotation:

22 Scatter Matrix Rotated, scaled scatter matrix:

23 Eigenvectors and Eigenvalues An eigenvector is a favorite direction for a matrix. Any square matrix M has at least one nonzero vector v which is mapped in a particularly simple way M v = v v is eigenvector of A and is corresponding eigenvalue If > 0, M v is parallel to v. If 0, M v is parallel to v. If < 0, they are antiparallel. If zero, v is in the nullspace. In all cases, M v is just a multiple of v.

24 Eigenvalues and Eigenvectors Given an eigenvector v j of S p : So, v j ’ = Rv j : Eigenvectors are rotated eigenvectors v j ’ = Rv j : Eigenvectors are rotated eigenvectors Eigenvalues are multiplied by the square of the scale Eigenvalues are multiplied by the square of the scaleSo, v j ’ = Rv j : Eigenvectors are rotated eigenvectors v j ’ = Rv j : Eigenvectors are rotated eigenvectors Eigenvalues are multiplied by the square of the scale Eigenvalues are multiplied by the square of the scale Multiply both sides by s 2 R, and manipulate: R is rotation matrix j

25 The process compute similarity transformation that best aligns the first and second moments Assume the 2nd moments (eigenvalues) are distinct Procedure: Center the data in each coordinate system Center the data in each coordinate system Compute the scatter matrices and their eigenvalues and eigenvectors Compute the scatter matrices and their eigenvalues and eigenvectors Using this, compute Scaling and rotation Scaling and rotation Then, translation Then, translation

26 Rotation The rotation matrix should align the eigenvectors in order: Manipulating:Manipulating: As a result:

27 Scale Ratios of corresponding eigenvalues should determine the scale: Taking the derivative with respect to s 2, which we are treating at Setting the result to 0 and solving yields

28 Translation Once we have the rotation and scale we can compute the translation based on the centers of mass:

29 Summary Calculations are straightforward, non-iterative and do not require correspondences: Moments, first Moments, first Rotation and scale separately Rotation and scale separately Translation Translation Assumes the viewpoints of the two data sets coincide approximately Can fail miserably for significantly differing viewpoints Can fail miserably for significantly differing viewpoints