1. Principal Warps: Thin-Plate Splines and the Decomposition of Deformations 김진욱 ( 이동통신망연구실 ; rein@mobilenet.snu.ac.kr) 박천현 (3D 모델링 및 처리연구실 ; amihk@3map.snu.ac.kr) 이승재 ( 멀티미디어 이동통신연구실 ; sjlee@mmlab.snu.ac.kr).snu.ac.kr 최종윤 (3D 모델링 및 처리연구실 ; joesmith@3map.snu.ac.kr)

2. Thin-Plate Splines Warping2 2005-12-01 Contents Introduction Interpolation and special function U(r) Algebraic properties of TPS Mapping function f(x, y) Spectral analysis of principal warps Examples Conclusion

3. Thin-Plate Splines Warping3 2005-12-01 Related Works Thin-Plate Spline Approximation for Image Registration [2]  Uses approximation approach (this paper uses interpolation) Landmark-Based Elastic Registration Using Approximating This-Plate Splines [3]  Refinement of [2] Consistent Landmark and Intensity-Based Image Registration [4]  Uses Intensity based information additionally

4. Thin-Plate Splines Warping4 2005-12-01 Thin-Plate Spline The name “thin plate spline” refers to a physical analogy involving the bending of a thin sheet of metal In physical setting, the deflection is in the direction orthogonal to the plane In coordinate transformation, one interprets the lifting of the plates as a displacement of the coordinate within the plane

5. Thin-Plate Splines Warping5 2005-12-01 Thin-plate spline as an interpolant TPS is 2D analog of the cubic spline of 1D Purpose : describe the deformation specified by  Finitely many point-correspondence  With irregular spacing In this paper, we use special function U(r)   Define U(0) = 0

6. Thin-Plate Splines Warping6 2005-12-01 Special function U(r) Graph of –U(r) X : (0, 0, 0) Circle  Zeros of function  Radius : 1/√e

7. Thin-Plate Splines Warping7 2005-12-01 Special function U(r) : example D k s are the corners (1, 0), (0, 1), (-1, 0), (0, -1) of the square z(x, y) approximates

8. Thin-Plate Splines Warping8 2005-12-01 Special function U(r) Given a set of data points weighted combination of U(r) centered at each data points,  Gives interpolation function that passes through the points exactly  And linear combination of U(r)s minimize the “bending energy”, defined by

9. Thin-Plate Splines Warping9 2005-12-01 Algebra of the Thin-Plate Spline for Arbitrary Sets of Landmarks Define matrices (K,P,L)  P 1 =(x 1,y 1 ), …, P n =(x n,y n ) are n landmark points in the ordinary Euclidean plane  r ij = |P i - P j | : the distance between points i and j

10. Thin-Plate Splines Warping10 2005-12-01 Algebra of the Thin-Plate Spline for Arbitrary Sets of Landmarks Define the vector W = (w 1, …, w n ) and the coefficients a 1, a x, a y by the equation  V = (v 1, …, v n ) is any n-vector  Y = (V | 0 0 0 ) T

11. Thin-Plate Splines Warping11 2005-12-01 Algebra of the Thin-Plate Spline for Arbitrary Sets of Landmarks From the previous equation,  (3) means interpolation property of TPS  (4) means boundary condition of TPS

12. Thin-Plate Splines Warping12 2005-12-01 The resulting function f(x,y) Define the function f(x,y)  First 3 terms : describes global affine transform  Rest terms : describes (nonglobal) non-linear transformation f(x i,y i ) = v i for all i (interpolation property) minimizes the bending energy, since U(r) is the fundamental solution of biharmonic equation

13. Thin-Plate Splines Warping13 2005-12-01 The resulting function f(x,y) In the application we take the points (x i,y i ) to be landmarks and V to be the n X 2 matrix  Each (x i ’, y i ’) is the landmark homologus to (x i, y i ) in another copy of R 2 Maps each point (x i, y i ) to its homolog(x i ’,y i ’) Least bent of all such mapping function

14. Thin-Plate Splines Warping14 2005-12-01 Principal Warps The value of I f (bending energy) is  Proportional to quadratic form Note that notation used represents vector as (v0, v1, v2, … ) L n is the upper left n by n submatrix of L inverse  In non-degenerate cases, U’s can be represented as one dimensional displacement of any single landmark holding the others in fixed position

15. Thin-Plate Splines Warping15 2005-12-01 Spectrum of bending energy matrix The vector is,  (W | A) T Vector A describes the affine part of transformation  Rotation, Translation, Scaling Vector W describes the non-linearity of transformation Matrix L n -1 KL n -1 means  Bending energy as a function of changes in the coordinates of the landmarks  0-valued eigenvalues are corresponding to affine transform

16. Thin-Plate Splines Warping16 2005-12-01 Spectrum of bending energy matrix Non-zero eigenvalues  If there are N landmarks, We have N – 3 non-zero eigenvalues (and eigenvectors)  Corresponding eigenvectors There are N components in eigenvectors, and they are coefficients for the N functions U based at N landmarks Called “principal warps” of the configuration of landmarks The transforms are affine-free (not-global transform) Higher eigenvalue means higher bending energy, and smaller physical scale of deformation

17. Thin-Plate Splines Warping17 2005-12-01 Spectrum of bending energy matrix When the number of landmarks, `N’ is,  N = 3 All transformations are described by affine transform No principal warps  N = 4 Only one (trivial) principal warp  N > 4 Successive eigenvectors describe the more smaller scale of deformation

18. Thin-Plate Splines Warping18 2005-12-01 Examples Warped image Original image

19. Thin-Plate Splines Warping19 2005-12-01 Examples The matrix L, vector V  Positions of landmarks : V  Matrix L

20. Thin-Plate Splines Warping20 2005-12-01 Examples Global affine transform  are vectors  From the a of the vector (w|a)  We get the affine part of the mapping function 

21. Thin-Plate Splines Warping21 2005-12-01 Examples SVD of linear parts in affine transform yields  Rotation matrix (44.89º)  Scaling matrix 1.0749 in x-coordinate 0.7441 in y-coordinate  Rotation matrix (-53.34º)

22. Thin-Plate Splines Warping22 2005-12-01 Examples Remaining terms   The matrix has 5 eigenvalues 3 0-valued eigenvalues : for affine transform 2 non-zero eigenvalues which have their own eigenvectors  f 0.2837 =(0.2152, -0.3265, 0.1346, -0.6554, 0.6321) for 0.2837  f 0.1480 =(-0.4941, -0.2415, -0.3370, 0.4700, 0.6026) for 0.1480

23. Thin-Plate Splines Warping23 2005-12-01 Examples Net changes in x, y coordinate  From the above net-change in each coordinate, we can calculate the weights of principal warps

24. Thin-Plate Splines Warping24 2005-12-01 Examples Expand the function f x, f y of the mapping function in terms of principal warps  Principal warps are used as basis of deformation description

25. Thin-Plate Splines Warping25 2005-12-01 Real world examples APERT syndrome (trace of x – ray images)

26. Thin-Plate Splines Warping26 2005-12-01 Real world examples Landmarks assignment

27. Thin-Plate Splines Warping27 2005-12-01 Discussion Image Discrimination  If landmarks can be chosen consistently on the left-hand form, the space of the decompositions explored in the paper is a natural context for interpretation of all multivariate findings

28. Thin-Plate Splines Warping28 2005-12-01 Discussion Landmark Identification  Some points biologically homologous between images are not clearly identifiable by local processing. When landmarks can be localized to edges, likely-hood ratio which can be computed by principal warp analysis can be used

29. Thin-Plate Splines Warping29 2005-12-01 Discussion Description of Actual Deformation  Can provide actual measure of deformation description, not an unwarp method

30. Thin-Plate Splines Warping30 2005-12-01 Discussion Instantiation of Primitives  In biomedical imaging, details are ordinaily strongly intercorrelated, and cannot observe all the details all at once.  Regression analysis upon principal warps can help this problem

31. Thin-Plate Splines Warping31 2005-12-01 Discussion 3 dimensions extension  We used the 2-dimensional form r 2 log r 2  We can use U(r) = |r| in 3-dimension Called thin-`hyperplane’ spline

32. Thin-Plate Splines Warping32 2005-12-01 Conclusion Method proposed in the paper,  Decompose the deformation  Introduces feature space for deformation Feature space is finite-dimensional space Easy to analyze  Principal warps can represent the multivariate distribution of configuration of landmarks easily Landmark assignment problem  For biologists, not for computer scientists