Image Deformation Using Moving Least Squares Scott Schaefer, Travis McPhail, Joe Warren SIGGRAPH 2006 Presented by Nirup Reddy

InputAffineSimilarityRigid

Outline Introduction Previous work MLS with points MLS with line segments Conclusions

Introduction Image deformation Animation, morphing, medical imaging … Controlled by handles: points, lines … Function f Interpolation: f(p i )=q i (handles) Smoothness ~ smooth deformations Identity: q i =p i  f(x)=x Similar to scattered data interpolation

Previous Work Grid/Polygon handles Free-form deformations [Sederberg and Parry 1986; Lee et al. 1995] Require aligning grid lines with image features Line handles [Beier and Neely 1992] Undesirable folding Point handles RBF Thin-plate splines [Bookstein 1989] Local non-uniform scaling and shearing As-rigid-as-possible deformations [Igarashi et al. 2005] Solve on the whole triangulation Discontinuities

Previous Work

As-Rigid-As-Possible Shape Manipulation

Thin-plate spline As-rigid-as-possible

Previous Work Grid/Polygon handles Free-form deformations [Sederberg and Parry 1986; Lee et al. 1995] Require aligning grid lines with image features Line handles [Beier and Neely 1992] Undesirable folding Point handles RBF Thin-plate splines [Bookstein 1989] Local non-uniform scaling and shearing As-rigid-as-possible deformations [Igarashi et al. 2005] Solve on the whole triangulation Discontinuities

Moving Least Squares Deformation Build image deformations based on handles (p i,q i ) Interpolation Smoothness Identity Given a point v, solve for the best l v (x): Deformation function f(v)=l v (v) Locally defined --- MOVING Least Squares Satisfy the three properties

Affine Transform Affine transform: l v (x)=xM+T Translation can be removed: T=q * -p * M So, l v (x)=(x-p * )M+q * The new cost function: M could be different class of transformations

Affine Deformations Solution The deformation function A j can be precomputed Contains non-uniform scaling and shear

Similarity Deformations Only include translation, rotation, and uniform scaling Remove shear from deformation

Similarity Deformations Only include translation, rotation, and uniform scaling Remove shear from deformation

Similarity Deformations (Cont.) A special subset of affine transforms Translation Rotation Constraints: Uniform-scaling Requirement: M T M=λ 2 I Define, where M 2 =M 1 ┴ Cost function (Least squares problem) still quadratic in M 1 where (x,y) ┴ =(-y,x)

Similarity Deformations (Cont.) Solution for matrix M where Solution for deformation function where Property Preserves angles better than affine deformations Local scaling can hurt realism

Rigid Deformations Deformations should not include scaling and shear [Alexa 2000; Igarashi et al. 2005] Best rigid transformation can be found from best similarity transformation Remove local uniform scaling M T M=I

Rigid Deformations Deformations should not include scaling and shear [Alexa 2000; Igarashi et al. 2005] Best rigid transformation can be found from best similarity transformation Remove local uniform scaling M T M=I

Rigid Deformations (Cont.) Solution for matrix M Solution for deformation function where, and A i is as in similarity deformations. Limited precomputation

Example: Actual Image

Deformation ’ s Affine Similarity Rigid

Comparison: AffineSimilarityRigid Non-uniform scaling and shear Local scaling can often lead to undesirable Deformations But Preserves angles Deformation is quite realistic Computation time very low abt 1.5ms Highest was computed as 3.4ms Highest was computed 3.8ms

Some Rigid Deformation Results

Deformation with Curves Handles are control curves instead of control points Cost function T still can be removed

Limitations: May be difficult to evaluate for arbitrary functions We restrict these functions to be line segments and derive closed-form solutions for the deformations in terms of the end-points of these segments

Affine Lines Represent line segments as matrix products Cost function Minimizer

Affine Lines (Cont.) Deformation

Similarity Lines Cost function Minimizer

Similarity Lines (Cont.) Deformation

Rigid Lines Derive from similarity lines Deformation

Implementation Pixel by pixel --- expensive Deformation on a downsampled grid Fill quads using bilinear interpolation

Timing Grid size: 100x100

Limitation: Foldbacks

Conclusions Moving Least Squares 2x2 linear system at each grid point Real-time Similarity and rigid transformations More realistic results Extension: line segments Closed-form expressions

Applications Of MLS: Very powerful tool in computer graphics Animation Medical Imaging Image Warping Image Morphing

Conclusions (Cont.) Future Work Adding topological information Generalizing to 3D to deform surfaces Handles can be any curves