Principal Warps: Thin-Plate Splines and the Decomposition of Deformations 김진욱 ( 이동통신망연구실 ; 박천현 (3D 모델링 및 처리연구실 ; 이승재 ( 멀티미디어 이동통신연구실 ; 최종윤 (3D 모델링 및 처리연구실 ;

Thin-Plate Splines Warping Contents Introduction Interpolation and special function U(r) Algebraic properties of TPS Mapping function f(x, y) Spectral analysis of principal warps Examples Conclusion

Thin-Plate Splines Warping Related Works Thin-Plate Spline Approximation for Image Registration [2]  Uses approximation approach (this paper uses interpolation) Landmark-Based Elastic Registration Using Approximating This-Plate Splines [3]  Refinement of [2] Consistent Landmark and Intensity-Based Image Registration [4]  Uses Intensity based information additionally

Thin-Plate Splines Warping Thin-Plate Spline The name “thin plate spline” refers to a physical analogy involving the bending of a thin sheet of metal In physical setting, the deflection is in the direction orthogonal to the plane In coordinate transformation, one interprets the lifting of the plates as a displacement of the coordinate within the plane

Thin-Plate Splines Warping Thin-plate spline as an interpolant TPS is 2D analog of the cubic spline of 1D Purpose : describe the deformation specified by  Finitely many point-correspondence  With irregular spacing In this paper, we use special function U(r)   Define U(0) = 0

Thin-Plate Splines Warping Special function U(r) Graph of –U(r) X : (0, 0, 0) Circle  Zeros of function  Radius : 1/√e

Thin-Plate Splines Warping Special function U(r) : example D k s are the corners (1, 0), (0, 1), (-1, 0), (0, -1) of the square z(x, y) approximates

Thin-Plate Splines Warping Special function U(r) Given a set of data points weighted combination of U(r) centered at each data points,  Gives interpolation function that passes through the points exactly  And linear combination of U(r)s minimize the “bending energy”, defined by

Thin-Plate Splines Warping Algebra of the Thin-Plate Spline for Arbitrary Sets of Landmarks Define matrices (K,P,L)  P 1 =(x 1,y 1 ), …, P n =(x n,y n ) are n landmark points in the ordinary Euclidean plane  r ij = |P i - P j | : the distance between points i and j

Thin-Plate Splines Warping Algebra of the Thin-Plate Spline for Arbitrary Sets of Landmarks Define the vector W = (w 1, …, w n ) and the coefficients a 1, a x, a y by the equation  V = (v 1, …, v n ) is any n-vector  Y = (V | ) T

Thin-Plate Splines Warping Algebra of the Thin-Plate Spline for Arbitrary Sets of Landmarks From the previous equation,  (3) means interpolation property of TPS  (4) means boundary condition of TPS

Thin-Plate Splines Warping The resulting function f(x,y) Define the function f(x,y)  First 3 terms : describes global affine transform  Rest terms : describes (nonglobal) non-linear transformation f(x i,y i ) = v i for all i (interpolation property) minimizes the bending energy, since U(r) is the fundamental solution of biharmonic equation

Thin-Plate Splines Warping The resulting function f(x,y) In the application we take the points (x i,y i ) to be landmarks and V to be the n X 2 matrix  Each (x i ’, y i ’) is the landmark homologus to (x i, y i ) in another copy of R 2 Maps each point (x i, y i ) to its homolog(x i ’,y i ’) Least bent of all such mapping function

Thin-Plate Splines Warping Principal Warps The value of I f (bending energy) is  Proportional to quadratic form Note that notation used represents vector as (v0, v1, v2, … ) L n is the upper left n by n submatrix of L inverse  In non-degenerate cases, U’s can be represented as one dimensional displacement of any single landmark holding the others in fixed position

Thin-Plate Splines Warping Spectrum of bending energy matrix The vector is,  (W | A) T Vector A describes the affine part of transformation  Rotation, Translation, Scaling Vector W describes the non-linearity of transformation Matrix L n -1 KL n -1 means  Bending energy as a function of changes in the coordinates of the landmarks  0-valued eigenvalues are corresponding to affine transform

Thin-Plate Splines Warping Spectrum of bending energy matrix Non-zero eigenvalues  If there are N landmarks, We have N – 3 non-zero eigenvalues (and eigenvectors)  Corresponding eigenvectors There are N components in eigenvectors, and they are coefficients for the N functions U based at N landmarks Called “principal warps” of the configuration of landmarks The transforms are affine-free (not-global transform) Higher eigenvalue means higher bending energy, and smaller physical scale of deformation

Thin-Plate Splines Warping Spectrum of bending energy matrix When the number of landmarks, `N’ is,  N = 3 All transformations are described by affine transform No principal warps  N = 4 Only one (trivial) principal warp  N > 4 Successive eigenvectors describe the more smaller scale of deformation

Thin-Plate Splines Warping Examples Warped image Original image

Thin-Plate Splines Warping Examples The matrix L, vector V  Positions of landmarks : V  Matrix L

Thin-Plate Splines Warping Examples Global affine transform  are vectors  From the a of the vector (w|a)  We get the affine part of the mapping function 

Thin-Plate Splines Warping Examples SVD of linear parts in affine transform yields  Rotation matrix (44.89º)  Scaling matrix in x-coordinate in y-coordinate  Rotation matrix (-53.34º)

Thin-Plate Splines Warping Examples Remaining terms   The matrix has 5 eigenvalues 3 0-valued eigenvalues : for affine transform 2 non-zero eigenvalues which have their own eigenvectors  f =(0.2152, , , , ) for  f =( , , , , ) for

Thin-Plate Splines Warping Examples Net changes in x, y coordinate  From the above net-change in each coordinate, we can calculate the weights of principal warps

Thin-Plate Splines Warping Examples Expand the function f x, f y of the mapping function in terms of principal warps  Principal warps are used as basis of deformation description

Thin-Plate Splines Warping Real world examples APERT syndrome (trace of x – ray images)

Thin-Plate Splines Warping Real world examples Landmarks assignment

Thin-Plate Splines Warping Discussion Image Discrimination  If landmarks can be chosen consistently on the left-hand form, the space of the decompositions explored in the paper is a natural context for interpretation of all multivariate findings

Thin-Plate Splines Warping Discussion Landmark Identification  Some points biologically homologous between images are not clearly identifiable by local processing. When landmarks can be localized to edges, likely-hood ratio which can be computed by principal warp analysis can be used

Thin-Plate Splines Warping Discussion Description of Actual Deformation  Can provide actual measure of deformation description, not an unwarp method

Thin-Plate Splines Warping Discussion Instantiation of Primitives  In biomedical imaging, details are ordinaily strongly intercorrelated, and cannot observe all the details all at once.  Regression analysis upon principal warps can help this problem

Thin-Plate Splines Warping Discussion 3 dimensions extension  We used the 2-dimensional form r 2 log r 2  We can use U(r) = |r| in 3-dimension Called thin-`hyperplane’ spline

Thin-Plate Splines Warping Conclusion Method proposed in the paper,  Decompose the deformation  Introduces feature space for deformation Feature space is finite-dimensional space Easy to analyze  Principal warps can represent the multivariate distribution of configuration of landmarks easily Landmark assignment problem  For biologists, not for computer scientists