1. CSE554Extrinsic DeformationsSlide 1 CSE 554 Lecture 9: Extrinsic Deformations Fall 2012

2. CSE554Extrinsic DeformationsSlide 2 Review Non-rigid deformation – Intrinsic methods: deforming the boundary points – An optimization problem Minimize shape distortion Maximize fit – Example: Laplacian-based deformation Source Target Before After

3. CSE554Extrinsic DeformationsSlide 3 Extrinsic Deformation Computing deformation of each point in the plane or volume – Not just points on the boundary curve or surface Credits: Adams and Nistri, BMC Evolutionary Biology (2010)

4. CSE554Extrinsic DeformationsSlide 4 Extrinsic Deformation Applications – Registering contents between images and volumes – Interactive spatial deformation

5. CSE554Extrinsic DeformationsSlide 5 Techniques Thin-plate spline deformation Free form deformation Cage-based deformation

6. CSE554Extrinsic DeformationsSlide 6 Thin-Plate Spline Given corresponding source and target points Computes a spatial deformation function for every point in the 2D plane or 3D volume Credits: Sprengel et al, EMBS (1996)

7. CSE554Extrinsic DeformationsSlide 7 Thin-Plate Spline A minimization problem – Minimizing distances between source and target points – Minimizing distortion of the space (as if bending a thin sheet of metal) There is a closed-form solution – Solving a linear system of equations

8. CSE554Extrinsic DeformationsSlide 8 Thin-Plate Spline Input – Source points: p 1,…,p n – Target points: q 1,…,q n Output – A deformation function f[p] for any point p pipi qiqi p f[p]

9. CSE554Extrinsic DeformationsSlide 9 Thin-Plate Spline Minimization formulation – E f : fitting term Measures how close is the deformed source to the target – E d : distortion term Measures how much the space is warped – : weight Controls how much non-rigid warping is allowed

10. CSE554Extrinsic DeformationsSlide 10 Thin-Plate Spline Fitting term – Minimizing sum of squared distances between deformed source points and target points

11. CSE554Extrinsic DeformationsSlide 11 Thin-Plate Spline Distortion term – Minimizing a physical bending energy on a metal sheet (2D): – The energy is zero when the deformation is affine Translation, rotation, scaling, shearing

12. CSE554Extrinsic DeformationsSlide 12 Thin-Plate Spline Finding the minimizer for – Uniquely exists, and has a closed form: M: an affine transformation matrix v i : translation vectors (one per source point) Both M and v i are determined by p i,q i, where

13. CSE554Extrinsic DeformationsSlide 13 Thin-Plate Spline Result – At higher, the deformation is closer to an affine transformation Credits: Sprengel et al, EMBS (1996)

14. CSE554Extrinsic DeformationsSlide 14 Thin-Plate Spline Application: image registration – Manual or automatic feature pair detection SourceTargetDeformed source Credits: Rohr et al, TMI (2001)

15. CSE554Extrinsic DeformationsSlide 15 Thin-Plate Spline Advantages – Smooth deformations, with physical analogy – Closed-form solution – Few free parameters (no tuning is required) Disadvantages – Solving the equations still takes time (hence cannot perform “Interactive” deformation)

16. CSE554Extrinsic DeformationsSlide 16 Free Form Deformation Uses a control lattice that embeds the shape Deforming the lattice points warps the embedded shape Credits: Sederberg and Parry, SIGGRAPH (1986)

17. CSE554Extrinsic DeformationsSlide 17 Free Form Deformation Warping the space by “blending” the deformation at the control points – Each deformed point is a weighted sum of deformed lattice points

18. CSE554Extrinsic DeformationsSlide 18 Free Form Deformation Input – Source lattice points: p 1,…,p n – Target lattice points: q 1,…,q n Output – A deformation function f[p] for any point p in the lattice grid: w i [p]: determined by relative position of p with respect to p i pipi qiqi p f[p]

19. CSE554Extrinsic DeformationsSlide 19 Free Form Deformation Desirable properties of the weights w i [p] – Greater when p is closer to p i So that the influence of each control point is local – Smoothly varies with location of p So that the deformation is smooth – So that f[p] =  w i [p] q i is an affine combination of q i – So that f[p]=p if the lattice stays unchanged pipi qiqi p f[p]

20. CSE554Extrinsic DeformationsSlide 20 Free Form Deformation Finding weights (2D) – Let the lattice points be p i,j for i=0,…,k and j=0,…,l – Compute p’s relative location in the grid (s,t) Let (x min,x max ), (y min,y max ) be the range of grid p 0,0 p 0,1 p 0,2 p 1,0 p 1,1 p 1,2 p 2,0 p 2,1 p 2,2 p 3,0 p 3,1 p 3,2 p s t

21. CSE554Extrinsic DeformationsSlide 21 Free Form Deformation Finding weights (2D) – Let the lattice points be p i,j for i=0,…,k and j=0,…,l – Compute p’s relative location in the grid (s,t) – The weight w i,j for lattice point p i,j is: i,j : importance of p i,j B: Bernstein basis function: p 0,0 p 0,1 p 0,2 p 1,0 p 1,1 p 1,2 p 2,0 p 2,1 p 2,2 p 3,0 p 3,1 p 3,2 p s t

22. CSE554Extrinsic DeformationsSlide 22 Free Form Deformation Finding weights (2D) – Weight distribution for one control point (max at that control point): p 0,0 p 0,1 p 0,2 p 1,0 p 1,1 p 2,0 p 3,0 p 3,1 p 3,2

23. CSE554Extrinsic DeformationsSlide 23 Free Form Deformation A deformation example

24. CSE554Extrinsic DeformationsSlide 24 Free Form Deformation Registration – Embed the source in a lattice – Compute new lattice positions over the target – Fitting term Distance between deformed source shape and the target shape Agreement of the deformed image content and the target image – Distortion term Thin-plate spline energy Non-folding constraint Local rigidity constraint (to prevent stretching)

25. CSE554Extrinsic DeformationsSlide 25 Free Form Deformation Registration example Credits: Loeckx et al, MICCAI (2004) Source Target Deformed w/o rigidity Deformed with rigidity

26. CSE554Extrinsic DeformationsSlide 26 Free Form Deformation Advantages – Smooth deformations – Easy to implement (no equation solving) – Efficient and localized controls for interactive editing – Can be coupled with different fitting or energy objectives Disadvantages – Too many lattice points in 3D

27. CSE554Extrinsic DeformationsSlide 27 Cage-based Deformation Use a control mesh (“cage”) to embed the shape Deforming the cage vertices warps the embedded shape Credits: Ju, Schaefer, and Warren, SIGGRAPH (2005)

28. CSE554Extrinsic DeformationsSlide 28 Cage-based Deformation Warping the space by “blending” the deformation at the cage vertices – w i [p]: determined by relative position of p with respect to p i p f[p] pipi qiqi

29. CSE554Extrinsic DeformationsSlide 29 Cage-based Deformation Finding weights (2D) – Problem: given a closed polygon (cage) with vertices p i and an interior point p, find weights w i [p] such that: 1) 2) pipi p

30. CSE554Extrinsic DeformationsSlide 30 Cage-based Deformation Finding weights (2D) – A simple case: the cage is a triangle – The weights are unique, and are the barycentric coordinates of p p1p1 p2p2 p3p3 p

31. CSE554Extrinsic DeformationsSlide 31 Finding weights (2D) – The harder case: the cage is an arbitrary (possibly concave) polygon – The weights are not unique A good choice: Mean Value Coordinates (MVC) Can be extended to 3D Cage-based Deformation pipi p αiαi α i+1 p i-1 p i+1

32. CSE554Extrinsic DeformationsSlide 32 Cage-based Deformation Finding weights (2D) – Weight distribution of one cage vertex in MVC: pipi

33. CSE554Extrinsic DeformationsSlide 33 Cage-based Deformation Application: character animation – Using improved weights: Harmonic Coordinates

34. CSE554Extrinsic DeformationsSlide 34 Cage-based Deformation Registration – Embed source in a cage – Compute new locations of cage vertices over the target Minimizing some fitting and energy objectives Not seen in literature yet… future work!

35. CSE554Extrinsic DeformationsSlide 35 Cage-based Deformation Advantages over free form deformations – Much smaller number of control points – Flexible structure suitable for organic shapes Disadvantages – Setting up the cage can be time-consuming – The cage needs to be a closed shape Not as flexible as point handles

36. CSE554Extrinsic DeformationsSlide 36 Further Readings Thin-plate spline deformation – “Principal warps: thin-plate splines and the decomposition of deformations”, by Bookstein (1989) – “Landmark-Based Elastic Registration Using Approximating Thin-Plate Splines”, by Rohr et al. (2001) Free form deformation – “Free-Form Deformation of Solid Geometric Models”, by Sederberg and Parry (1986) – “Extended Free-Form Deformation: A sculpturing Tool for 3D Geometric Modeling”, by Coquillart (1990) Cage-based deformation – “Mean value coordinates for closed triangular meshes”, by Ju et al. (2005) – “Harmonic coordinates for character animation”, by Joshi et al. (2007) – “Green coordinates”, by Lipman et al. (2008)