1. CSE554Extrinsic DeformationsSlide 1 CSE 554 Lecture 10: Extrinsic Deformations Fall 2014

2. CSE554Extrinsic DeformationsSlide 2 Review Non-rigid deformation – Intrinsic methods: deforming the boundary points – An optimization problem Minimize shape distortion Maximize fit – Example: Laplacian-based deformation Source Target Before After

3. CSE554Extrinsic DeformationsSlide 3 Extrinsic Deformation Computing deformation of each point in the plane or volume – Not just points on the boundary curve or surface Credits: Adams and Nistri, BMC Evolutionary Biology (2010)

4. CSE554Extrinsic DeformationsSlide 4 Extrinsic Deformation Applications – Registering contents between images and volumes – Interactive animation

5. CSE554Extrinsic DeformationsSlide 5 Techniques Thin-plate spline deformation Free form deformation Cage-based deformation

6. CSE554Extrinsic DeformationsSlide 6 Thin-Plate Spline Given corresponding source and target points Computes a spatial deformation function for every point in the 2D plane or 3D volume Credits: Sprengel et al, EMBS (1996)

7. CSE554Extrinsic DeformationsSlide 7 Thin-Plate Spline A minimization problem – Minimizing distances between source and target points – Minimizing distortion of the space There is a closed-form solution – Solving a linear system of equations

8. CSE554Extrinsic DeformationsSlide 8 Thin-Plate Spline Input – Source points: p 1,…,p n – Target points: q 1,…,q n Output – A deformation function f[p] for any point p pipi qiqi p f[p]

9. CSE554Extrinsic DeformationsSlide 9 Thin-Plate Spline Minimization formulation – E f : fitting term Measures how close is the deformed source to the target – E d : distortion term Measures how much the space is warped – : weight Controls how much distortion is allowed

10. CSE554Extrinsic DeformationsSlide 10 Thin-Plate Spline Fitting term – Minimizing sum of squared distances between deformed source points and target points

11. CSE554Extrinsic DeformationsSlide 11 Thin-Plate Spline Distortion term – Penalizes non-linear deformation: – The energy is zero when the deformation is a linear transformation Translation, rotation, scaling, shearing

12. CSE554Extrinsic DeformationsSlide 12 Thin-Plate Spline Finding the minimizer for – Uniquely exists, and has a closed form: M: a constant transformation matrix v i : translation vectors (one per source point) Both M and v i are determined by p i,q i, where

13. CSE554Extrinsic DeformationsSlide 13 Thin-Plate Spline Result – At higher, the deformation is closer to an affine transformation Credits: Sprengel et al, EMBS (1996)

14. CSE554Extrinsic DeformationsSlide 14 Thin-Plate Spline Application: landmark-based image registration – Manual or automatic detection of landmarks and correspondences SourceTargetDeformed source Credits: Rohr et al, TMI (2001)

15. CSE554Extrinsic DeformationsSlide 15 Thin-Plate Spline Application: landmark-based image registration – Manual or automatic detection of landmarks and correspondences SourceTargetSource Credits: Rohr et al, TMI (2001)

16. CSE554Extrinsic DeformationsSlide 16 Thin-Plate Spline Pros – Closed-form solution makes it efficient to compute – Only requires scattered point correspondences Cons – Difficult to control deformation away from the point handles

17. CSE554Extrinsic DeformationsSlide 17 Free Form Deformation Uses a control lattice that embeds the shape Deforming the lattice points warps the embedded shape Credits: Sederberg and Parry, SIGGRAPH (1986)

18. CSE554Extrinsic DeformationsSlide 18 Free Form Deformation “Blending” the deformation at the control points – Each deformed point is a weighted sum of deformed lattice points

19. CSE554Extrinsic DeformationsSlide 19 Free Form Deformation Input – Source control points: p 1,…,p n – Target control points: q 1,…,q n Output – A deformation function f[p] for any point p within the space of the grid. w i [p]: pre-computed “influence” of p i on p pipi qiqi p f[p]

20. CSE554Extrinsic DeformationsSlide 20 Free Form Deformation Desirable properties of the weights w i [p] – Greater when p is closer to p i So that the influence of each control point is local – Smoothly varies with location of p So that the deformation is smooth – So that f[p] =  w i [p] q i is an affine combination of q i – So that f[p]=p if the lattice stays unchanged pipi qiqi p f[p]

21. CSE554Extrinsic DeformationsSlide 21 Free Form Deformation Finding weights (2D) – Let the lattice points be p i,j for i=0,…,k and j=0,…,l – Compute p’s relative location in the grid (s,t) Let (x min,x max ), (y min,y max ) be the range of grid p 0,0 p 0,1 p 0,2 p 1,0 p 1,1 p 1,2 p 2,0 p 2,1 p 2,2 p 3,0 p 3,1 p 3,2 p s t

22. CSE554Extrinsic DeformationsSlide 22 Free Form Deformation Finding weights (2D) – Let the lattice points be p i,j for i=0,…,k and j=0,…,l – Compute p’s relative location in the grid (s,t) – The weight w i,j for lattice point p i,j is: i,j : importance of p i,j B: Bernstein basis function: p 0,0 p 0,1 p 0,2 p 1,0 p 1,1 p 1,2 p 2,0 p 2,1 p 2,2 p 3,0 p 3,1 p 3,2 p s t

23. CSE554Extrinsic DeformationsSlide 23 Free Form Deformation Finding weights (2D) – Weight distribution for one control point (max at that control point): p 0,0 p 0,1 p 0,2 p 1,0 p 1,1 p 2,0 p 3,0 p 3,1 p 3,2

24. CSE554Extrinsic DeformationsSlide 24 Free Form Deformation A deformation example

25. CSE554Extrinsic DeformationsSlide 25 Free Form Deformation Image registration – Embed the source in a lattice – Compute new lattice positions over the target Manually, or solve it as an optimization problem (maximally matching images contents while minimizing distortion) – Deform each source pixel using FFD www.slicer.org

26. CSE554Extrinsic DeformationsSlide 26 Free Form Deformation Pros – Even simpler then thin-plate spline – Well controlled deformation over the entire grid Cons – Specifying target locations of all grid points is a lot of work Especially for interactive animation – Cartesian grid is not a good fit for complex shapes

27. CSE554Extrinsic DeformationsSlide 27 Cage-based Deformation Use a control mesh (“cage”) to embed the shape Deforming the cage vertices warps the embedded shape Credits: Ju, Schaefer, and Warren, SIGGRAPH (2005)

28. CSE554Extrinsic DeformationsSlide 28 Cage-based Deformation “Blending” the deformation at the cage vertices – w i [p]: pre-computed “influence” of p i on p p f[p] pipi qiqi

29. CSE554Extrinsic DeformationsSlide 29 Cage-based Deformation Finding weights (2D) – Problem: given a closed polygon (cage) with vertices p i and an interior point p, find smooth weights w i [p] such that: 1) 2) pipi p

30. CSE554Extrinsic DeformationsSlide 30 Cage-based Deformation Finding weights (2D) – A simple case: the cage is a triangle – The weights are unique (3 eqs, 3 vars) – Known as the barycentric coordinates of p p1p1 p2p2 p3p3 p

31. CSE554Extrinsic DeformationsSlide 31 Finding weights (2D) – The harder case: the cage is an arbitrary (possibly concave) polygon – The weights are not unique A good choice: Mean Value Coordinates (MVC) Can be extended to 3D Cage-based Deformation pipi p αiαi α i+1 p i-1 p i+1

32. CSE554Extrinsic DeformationsSlide 32 Cage-based Deformation Finding weights (2D) – Weight distribution of one cage vertex in MVC: pipi

33. CSE554Extrinsic DeformationsSlide 33 Cage-based Deformation Application: character animation

34. CSE554Extrinsic DeformationsSlide 34 Cage-based Deformation Registration – Embed source in a cage – Compute new locations of cage vertices over the target Minimizing some fitting and energy objectives – Deform source pixels using MVC Not seen in literature yet… future work!

35. CSE554Extrinsic DeformationsSlide 35 Cage-based Deformation Pros – Simple to compute – Well-behaved deformation within the cage – Fewer control points and better fitting to shape than FFD Cons – Constructing the cage is mostly a manual process

36. CSE554Extrinsic DeformationsSlide 36 Further Readings Thin-plate spline deformation – “Principal warps: thin-plate splines and the decomposition of deformations”, by Bookstein (1989) – “Landmark-Based Elastic Registration Using Approximating Thin-Plate Splines”, by Rohr et al. (2001) Free form deformation – “Free-Form Deformation of Solid Geometric Models”, by Sederberg and Parry (1986) – “Extended Free-Form Deformation: A sculpturing Tool for 3D Geometric Modeling”, by Coquillart (1990) Cage-based deformation – “Mean value coordinates for closed triangular meshes”, by Ju et al. (2005) – “Harmonic coordinates for character animation”, by Joshi et al. (2007) – “Green coordinates”, by Lipman et al. (2008)