CSE 554 Lecture 10: Extrinsic Deformations Fall 2016

Review Non-rigid deformation Intrinsic methods: deforming the boundary points An optimization problem Minimize shape distortion Maximize fit Example: Laplacian-based deformation Source Target Before After

Extrinsic Deformation Given source and target point pairs (handles) Compute deformation of any point in the plane or volume Not just points on the boundary curve or surface

Extrinsic Deformation Applications Registering contents between images and volumes Interactive animation

Techniques Thin-plate spline deformation Free form deformation Cage-based deformation

Thin-Plate Spline A minimization problem Minimizing distances between source and target points Minimizing distortion of the space There is a closed-form solution Solving a linear system of equations

Thin-Plate Spline Input Output Source points: p1,…,pn Corresponding target points: q1,…,qn Output A deformation function f[p] (p is any point) pi qi p f[p]

Thin-Plate Spline Minimization formulation Ef: fitting term Measures how close is the deformed source to the target Ed: distortion term Measures how much the space is warped : weight Controls how much distortion is allowed

Thin-Plate Spline Fitting term Minimizing sum of squared distances between deformed source points and target points

Thin-Plate Spline Distortion term Penalizes non-linear deformation: The energy is zero when the deformation is a linear transformation Translation, rotation, scaling, shearing

Thin-Plate Spline Finding the minimizer for Uniquely exists, and has a closed form: M: a global transformation vi: translation vectors (one per source point) Both M and vi are determined by pi,qi, By solving a linear equation system where

Thin-Plate Spline Result At higher , the deformation is closer to an affine transformation Credits: Sprengel et al, EMBS (1996)

Thin-Plate Spline Application: landmark-based image registration Manual or automatic detection of landmarks and correspondences Source Target Deformed source Credits: Rohr et al, TMI (2001)

Thin-Plate Spline Application: landmark-based image registration Manual or automatic detection of landmarks and correspondences Source Target Source Credits: Rohr et al, TMI (2001)

Thin-Plate Spline Pros Cons Only requires scattered point correspondences Closed-form solution makes it efficient to compute the deformation Once the global transformation (M) and local vectors (vi) are solved Cons Solving M, vi could be time-consuming for large number of handles Not for interactive deformation (e.g., user moving the handles in real-time)

Free Form Deformation Uses a control lattice that embeds the shape Deforming the lattice points (control points) warps the embedded shape Credits: Sederberg and Parry, SIGGRAPH (1986)

Free Form Deformation Blending the deformation at the control points (handles) Each deformed point is a weighted sum of deformed lattice points

Free Form Deformation Input Output Source control points: p1,…,pn Target control points: q1,…,qn Output A deformation function f[p] for any point p within the space of the grid. wi[p]: “influence” of pi on p (regardless of qi, so it can be pre- computed) p f[p] pi qi

Free Form Deformation Desirable properties of the weights wi[p] Decreases with distance from p to pi So that the influence of each control point is local Smoothly varies with location of p So that the deformation is smooth So that f[p] =  wi[p] qi is an affine combination of qi So that f[p]=p if the lattice stays unchanged pi qi p f[p]

Free Form Deformation Finding weights (2D) Let the lattice points be pi,j for i=0,…,k and j=0,…,l Compute p’s relative location in the grid (s,t) Let (xmin,xmax), (ymin,ymax) be the range of grid p0,2 p1,2 p2,2 p3,2 p s t p0,1 p1,1 p2,1 p3,1 p0,0 p1,0 p2,0 p3,0

Free Form Deformation Finding weights (2D) Let the lattice points be pi,j for i=0,…,k and j=0,…,l Compute p’s relative location in the grid (s,t) The weight wi,j for lattice point pi,j is: i,j: importance of pi,j B: Bernstein basis function: p0,2 p1,2 p2,2 p3,2 p t p0,1 p1,1 p2,1 p3,1 p0,0 p1,0 p2,0 p3,0 s

Free Form Deformation Finding weights (2D) Weight distribution for one control point (max at that control point): p3,2 p3,1 p0,2 p1,1 p3,0 p0,1 p2,0 p1,0 p0,0

Free Form Deformation A deformation example

Free Form Deformation Image registration Embed the source in a lattice Compute new lattice positions over the target Manually or automatically Deform each source pixel using FFD www.slicer.org

Free Form Deformation Pros Cons Fast updates when control points are moved Cons Too many control points (a lot of them are interior to the shape) Cartesian grid is too inflexible for complex shapes

Cage-based Deformation Use a control mesh (“cage”) to embed the shape Deforming the cage vertices warps the embedded shape Credits: Ju, Schaefer, and Warren, SIGGRAPH (2005)

Cage-based Deformation “Blending” the deformation at the cage vertices wi[p]: pre-computed “influence” of pi on p More challenging to compute than FFD: not regular lattice structure pi p qi f[p]

Cage-based Deformation Finding weights (2D) Problem: given a closed polygon (cage) with vertices pi and an interior point p, find smooth weights wi[p] such that: 1) 2) pi p

Cage-based Deformation Finding weights (2D) A simple case: the cage is a triangle The weights are unique (3 eqs, 3 vars) wi are known as the barycentric coordinates of p p1 p p3 p2

Cage-based Deformation Finding weights (2D) The harder case: the cage is an arbitrary (possibly concave) polygon The weights are not unique (“generalized barycentric coordinates”) A good choice: Mean Value Coordinates (MVC) [Floater, 2003] pi-1 pi αi αi+1 p pi+1

Cage-based Deformation Finding weights (2D) Weight distribution of one cage vertex in MVC: pi

Cage-based Deformation MVC be extended to 3D [Ju, 2005; Floater, 2005] Other types of coordinates: Harmonic coordinates; Green coordinates; etc. Application: character animation

Cage-based Deformation Registration Embed source in a cage Compute new locations of cage vertices over the target Deform source pixels using MVC or other generalized barycentric coordinates

Cage-based Deformation Pros Fast update when control points are moving Fewer control points and better fitting to shape than FFD Cons Constructing the cage is mostly a manual process Although there is some recent progress in automation Sacht et al., Siggraph Asia 2015

Further Readings Thin-plate spline deformation Free form deformation “Principal warps: thin-plate splines and the decomposition of deformations”, by Bookstein (1989) “Landmark-Based Elastic Registration Using Approximating Thin-Plate Splines”, by Rohr et al. (2001) Free form deformation “Free-Form Deformation of Solid Geometric Models”, by Sederberg and Parry (1986) “Extended Free-Form Deformation: A sculpturing Tool for 3D Geometric Modeling”, by Coquillart (1990) Cage-based deformation “Mean value coordinates for closed triangular meshes”, by Ju et al. (2005) “Harmonic coordinates for character animation”, by Joshi et al. (2007) “Green coordinates”, by Lipman et al. (2008)