1. Image Deformation Using Moving Least Squares Scott Schaefer, Travis McPhail, Joe Warren SIGGRAPH 2006 Presented by Nirup Reddy

4. InputAffineSimilarityRigid

5. Outline Introduction Previous work MLS with points MLS with line segments Conclusions

6. Introduction Image deformation Animation, morphing, medical imaging … Controlled by handles: points, lines … Function f Interpolation: f(p i )=q i (handles) Smoothness ~ smooth deformations Identity: q i =p i  f(x)=x Similar to scattered data interpolation

7. Previous Work Grid/Polygon handles Free-form deformations [Sederberg and Parry 1986; Lee et al. 1995] Require aligning grid lines with image features Line handles [Beier and Neely 1992] Undesirable folding Point handles RBF Thin-plate splines [Bookstein 1989] Local non-uniform scaling and shearing As-rigid-as-possible deformations [Igarashi et al. 2005] Solve on the whole triangulation Discontinuities

8. Previous Work

9. As-Rigid-As-Possible Shape Manipulation

10. Thin-plate spline As-rigid-as-possible

11. Previous Work Grid/Polygon handles Free-form deformations [Sederberg and Parry 1986; Lee et al. 1995] Require aligning grid lines with image features Line handles [Beier and Neely 1992] Undesirable folding Point handles RBF Thin-plate splines [Bookstein 1989] Local non-uniform scaling and shearing As-rigid-as-possible deformations [Igarashi et al. 2005] Solve on the whole triangulation Discontinuities

19. Moving Least Squares Deformation Build image deformations based on handles (p i,q i ) Interpolation Smoothness Identity Given a point v, solve for the best l v (x): Deformation function f(v)=l v (v) Locally defined --- MOVING Least Squares Satisfy the three properties

20. Affine Transform Affine transform: l v (x)=xM+T Translation can be removed: T=q * -p * M So, l v (x)=(x-p * )M+q * The new cost function: M could be different class of transformations

21. Affine Deformations Solution The deformation function A j can be precomputed Contains non-uniform scaling and shear

22. Similarity Deformations Only include translation, rotation, and uniform scaling Remove shear from deformation

23. Similarity Deformations Only include translation, rotation, and uniform scaling Remove shear from deformation

24. Similarity Deformations (Cont.) A special subset of affine transforms Translation Rotation Constraints: Uniform-scaling Requirement: M T M=λ 2 I Define, where M 2 =M 1 ┴ Cost function (Least squares problem) still quadratic in M 1 where (x,y) ┴ =(-y,x)

25. Similarity Deformations (Cont.) Solution for matrix M where Solution for deformation function where Property Preserves angles better than affine deformations Local scaling can hurt realism

26. Rigid Deformations Deformations should not include scaling and shear [Alexa 2000; Igarashi et al. 2005] Best rigid transformation can be found from best similarity transformation Remove local uniform scaling M T M=I

27. Rigid Deformations Deformations should not include scaling and shear [Alexa 2000; Igarashi et al. 2005] Best rigid transformation can be found from best similarity transformation Remove local uniform scaling M T M=I

28. Rigid Deformations (Cont.) Solution for matrix M Solution for deformation function where, and A i is as in similarity deformations. Limited precomputation

30. Example: Actual Image

31. Deformation ’ s Affine Similarity Rigid

32. Comparison: AffineSimilarityRigid Non-uniform scaling and shear Local scaling can often lead to undesirable Deformations But Preserves angles Deformation is quite realistic Computation time very low abt 1.5ms Highest was computed as 3.4ms Highest was computed 3.8ms

37. Some Rigid Deformation Results

41. Deformation with Curves Handles are control curves instead of control points Cost function T still can be removed

42. Limitations: May be difficult to evaluate for arbitrary functions We restrict these functions to be line segments and derive closed-form solutions for the deformations in terms of the end-points of these segments

43. Affine Lines Represent line segments as matrix products Cost function Minimizer

44. Affine Lines (Cont.) Deformation

45. Similarity Lines Cost function Minimizer

46. Similarity Lines (Cont.) Deformation

47. Rigid Lines Derive from similarity lines Deformation

51. Implementation Pixel by pixel --- expensive Deformation on a downsampled grid Fill quads using bilinear interpolation

52. Timing Grid size: 100x100

53. Limitation: Foldbacks

54. Conclusions Moving Least Squares 2x2 linear system at each grid point Real-time Similarity and rigid transformations More realistic results Extension: line segments Closed-form expressions

55. Applications Of MLS: Very powerful tool in computer graphics Animation Medical Imaging Image Warping Image Morphing

56. Conclusions (Cont.) Future Work Adding topological information Generalizing to 3D to deform surfaces Handles can be any curves