1. Projective Transformations for Image Transition Animations
Planar surfaces are related
by projective transformation
TzuYen Wong, Peter Kovesi & Amitava Datta
School of Computer Science & Software Engineering
The University of Western Australia

2. Objective 1 Inputs – 2D images + quadrilateral pairs
Output – smooth transition
Novelty – perspective correctness
Approach – decomposing homography
Issue – Normalisation

3. Objective 2 Inputs – 2D shapes + multiple quadrilateral pairs
Output – smooth transition
Novelty – good shape interpolation with few primitives
Approach – force equilibrium scheme for alignment

4. Motivation
Image transition animations usually use affine transformations on triangle patches. (6 DOF)
Eg. image morphing, shape interpolation
Igarashi, T., Moscovich, T., and Hughes, J. F. As-rigid-as-possible shape manipulation. In ACM Transactions on Graphics, SIGGRAPH 2005, ACM Press, 2005.
Transformation of image patches is a common requirement for 2D transition animations such as shape interpolation and image morphing. It is usually done by applying affine transformations to triangular patches.
However, the affine transformation does not model the perspective transformation frequently found in images.
Hence, such techniques can only produce approximate results and usually use an excessively large number of triangles to compensate for this shortcoming. This paper proposes the application of projective transformations on quadrilateral image patches as a solution to this problem.

5. Motivation
Image transition animations usually use affine transformations on triangle patches. (6 DOF)
Eg. image morphing, shape interpolation
Perspective effect is not modelled (requires 8 DOF)
Approximated results
large number of triangles
Transformation of image patches is a common requirement for 2D transition animations such as shape interpolation and image morphing. It is usually done by applying affine transformations to triangular patches.
However, the affine transformation does not model the perspective transformation frequently found in images.
Hence, such techniques can only produce approximate results and usually use an excessively large number of triangles to compensate for this shortcoming. This paper proposes the application of projective transformations on quadrilateral image patches as a solution to this problem.

6. Motivation
Image transition animations usually use affine transformations on triangle patches. (6 DOF)
Eg. image morphing, shape interpolation
Perspective effect is not modelled (requires 8 DOF)
Approximated results
large number of triangles
Propose: projective transformations (8 DOF) on quadrilateral image patches as a solution
Transformation of image patches is a common requirement for 2D transition animations such as shape interpolation and image morphing. It is usually done by applying affine transformations to triangular patches.
However, the affine transformation does not model the perspective transformation frequently found in images.
Hence, such techniques can only produce approximate results and usually use an excessively large number of triangles to compensate for this shortcoming. This paper proposes the application of projective transformations on quadrilateral image patches as a solution to this problem.

7. Previous work – Affine Transformation
Matrix animation and polar decomposition (Shoemake 1992)
Shortcomings of direct interpolations of vertices or matrix elements – shrinking, unpredictable
Polar decomposition
Affine matrix = rotation + scaling + translation
Interpolate the angle of rotation produces good result
Close form solution for least distortion affine interpolation of multiple triangles
quality of the transition strongly depends on avoiding having irregular triangles in the mesh through complex isomorphic dissections.
K. Shoemake and T. Duff. Matrix animation and polar decomposition. In Proceedings of the conference on Graphics interface ’92, pages 258–264, San Francisco, CA, USA, Morgan Kaufmann Publishers Inc.

8. Previous work – Affine Transformation
Matrix animation and polar decomposition (Shoemake 1992)
Interpolation using
Vertex positions
Matrix elements
rotation angle
+ scaling + translation
Close form solution for least distortion affine interpolation of multiple triangles
quality of the transition strongly depends on avoiding having irregular triangles in the mesh through complex isomorphic dissections.

9. Previous work – Affine Transformation
As-rigid-as-possible shape interpolation (Alexa 2000)
Close form solution for least distortion affine interpolation of multiple triangles
Triangles disconnect from each other in rigid interpolation
Find the vertices of connected triangles which have transformation matrices that are closest to the transformation matrices of the vertices of disconnected rigid interpolation
Close form solution for least distortion affine interpolation of multiple triangles
quality of the transition strongly depends on avoiding having irregular triangles in the mesh through complex isomorphic dissections.
M. Alexa, D. Cohen-Or, and D. Levin. As-rigid-as-possible shape interpolation. In K. Akeley, editor, Siggraph 2000, Computer Graphics Proceedings, pages 157–164. ACM Press/Addison-Wesley Publishing Co., 2000.

10. Previous work – Affine Transformation
As-rigid-as-possible shape interpolation (Alexa 2000)
Close form solution for least distortion affine interpolation of multiple triangles
Quality of the transition strongly depends on avoiding having irregular triangles through complex isomorphic dissections.
Close form solution for least distortion affine interpolation of multiple triangles
quality of the transition strongly depends on avoiding having irregular triangles in the mesh through complex isomorphic dissections.
M. Alexa, D. Cohen-Or, and D. Levin. As-rigid-as-possible shape interpolation. In K. Akeley, editor, Siggraph 2000, Computer Graphics Proceedings, pages 157–164. ACM Press/Addison-Wesley Publishing Co., 2000.

11. Previous work – Projective Transformation
Other projectively correct interpolation
2D View morphing (Seitz 1996, Xiao 2004) I1 ^ I2
Prewarping Ca Ia ^
Morphing Ia
Postwarping X C1 C2 I1 I2

12. Previous work – Projective Transformation
Other projectively correct interpolation
2D View morphing (Seitz 1996, Xiao 2004)
Prewarp for parallel view interpolation
Postwarp is manual or with affine approximation X C1 C2 I1 I2 ^ Ca Ia

13. Previous work – Projective Transformation
Other projectively correct interpolation
2D View morphing (Seitz 1996, Xiao 2004)
Prewarp for parallel view interpolation
Postwarp is manual or with affine approximation
3D graphics
Interpolate planar surfaces in 3D
Required 3D information

14. Method overview Source quadrilateral Target quadrilateral
Decomposition of projective matrix, H (homography)
perspective + rotation + scaling + translation
Last row of homography – vanishing line vector

15. Projective matrix decomposition
Normalised Homography decomposed into
Perspective
Axis alignment
Scaling
Rotation
Translation

16. Projective matrix decomposition
Normalised Homography decomposed into
Perspective
Axis alignment
Scaling
Rotation
Translation

17. Projective matrix decomposition
Normalised Homography decomposed into
Perspective
Axis alignment
Scaling
Rotation
Translation
Singular Value Decomposition (SVD)

18. Projective matrix Interpolation
Interpolation with Identity matrix
Perspective - Interpolation of vanishing line vector
Find the axis of rotation and interpolate angle

19. Projective matrix Interpolation
Interpolation with Identity matrix
Perspective - Interpolation of vanishing line vector
Find the axis of rotation and interpolate angle
Rotation - Interpolation of angle (q)
Scaling & Translation - linear interpolation
Sa = (1-a)I + aS
Axis alignment - no interpolation

20. Normalisation issues Vanishing line normalisation
Arbitrary scaled homography hH
Set , unit magnitude vanishing line
Avoid having two scaling interpolations as the combination is non-linear and creates visual artefact

21. Normalisation issues Vanishing line normalisation
Arbitrary scaled homography hH
Set , unit magnitude vanishing line
Avoid having two scaling interpolations as the combination is non-linear and creates visual artefact
Example:
Scaling of P : 
Scaling of S : 
Combination: 

22. Normalisation issues Vanishing line normalisation
Arbitrary scaled homography hH
Set , unit magnitude vanishing line
Avoid having two scaling interpolations as the combination is non-linear and creates visual artefact
Example:
Scaling of P : 1.0      2.0
Scaling of S : 1.0      0.5
Combination: 

23. Normalisation issues Vanishing line normalisation
Arbitrary scaled homography hH
Set , unit magnitude vanishing line
Avoid having two scaling interpolations as the combination is non-linear and creates visual artefact
Example:
Scaling of P : 1.0      2.0
Scaling of S : 1.0      0.5
Combination: 1.0  1.08  1.12  1.12  1.08  1.0
>

24. Normalisation issues
Translation and scale of source quadrilateral affect the vanishing line vector
Translation normalisation of source quadrilateral
The source points are translated to be centered at the origin before computing the homography
Scale normalisation of source quadrilateral
Only excessive scaling creates undesirable results

25. Normalisation issues Examples (a) Perspectively correct transition
(b) Source translated away from origin (75% of object size)
(c) Source scaled down (coordinate magnitudes ~ 0.1)
(d) Vanishing line vector with magnitude = 10.

26. Shape interpolation with multiple quadrilaterals
Replace affine interpolation of triangles
physical correctness of projective transformation
smaller number of primitives
Disconnected boundaries among vertices of different intermediate quadrilaterals
Force-equilibrium scheme to enforce boundary connectivity
The projective interpolation of quadrilaterals can be used as a primitive where triangles are traditionally used, such as in image transition applications. It has the advantages of preserving the physical correctness of projective transformation and requiring a smaller number of primitives.
Similar to triangle interpolation, the interpolation of adjacent quadrilaterals suffer from disconnected boundaries when they have different transformations. We propose a closed form force-equilibrium post processing scheme to line up the disconnected intermediate adjacent quadrilaterals and to minimise distortions.

27. Shape interpolation with multiple quadrilaterals
Force on each quad
vertex position
Example
i = 1
translation
The projective interpolation of quadrilaterals can be used as a primitive where triangles are traditionally used, such as in image transition applications. It has the advantages of preserving the physical correctness of projective transformation and requiring a smaller number of primitives.
Similar to triangle interpolation, the interpolation of adjacent quadrilaterals suffer from disconnected boundaries when they have different transformations. We propose a closed form force-equilibrium post processing scheme to line up the disconnected intermediate adjacent quadrilaterals and to minimise distortions.

28. Shape interpolation with multiple quadrilaterals
Force on each quad
At force equilibrium,
all Fi = 0
With Nq equations,
C Teq = Fp (close form solution)
The projective interpolation of quadrilaterals can be used as a primitive where triangles are traditionally used, such as in image transition applications. It has the advantages of preserving the physical correctness of projective transformation and requiring a smaller number of primitives.
Similar to triangle interpolation, the interpolation of adjacent quadrilaterals suffer from disconnected boundaries when they have different transformations. We propose a closed form force-equilibrium post processing scheme to line up the disconnected intermediate adjacent quadrilaterals and to minimise distortions.

29. Results

30. Results Non-rigid object shape interpolation
a) Projective interpolation with quadrilaterals
b) Alexa’s as-rigid-as-possible affine interpolation with triangles.

31. Results Non-rigid object shape interpolation
a) Projective interpolation with quadrilaterals
b) Alexa’s as-rigid-as-possible affine interpolation with triangles.

32. Results Rigid object shape interpolation
a) Projective interpolation with quadrilaterals
b) Alexa’s as-rigid-as-possible affine interpolation with triangles.

33. Results Projective interpolation applied on a real image
5 pairs of separate homographies for 5 planar surfaces H1 H2 H3 H4 H5

34. Results Projective interpolation applied on a real image
5 pairs of separate homographies for 5 planar surfaces

35. Results Projective shape interpolation –
da Vinci’s Vitruvian Man to Coyote

36. Results Projective shape interpolation –
da Vinci’s Vitruvian Man to Coyote

37. Conclusions
Enable perspectively correct interpolation of image patches, without 3D information.
Improve interpolation results by normalisation of homography and source points position
Enable quadrilaterals to be used as primitives for shape interpolation and morphing applications.
In this paper, we successfully extend matrix decomposition and interpolation techniques so that, rather than just
being applicable to affine transformation, they can be also applied to general projective transformation. This enables perspectively correct interpolation of image patches to be generated without 3D information.
We suggest normalisation of the projective matrix before decomposition and position normalisation of the source
points for better interpolation results. We present an efficient force-equilibrium method to handle disconnected intermediate quadrilaterals and enable quadrilaterals to be used as primitives for more complex shape interpolation and morphing applications.
Future research directions include the optimisation of the source position for optimal interpolation and exploration of
applications of this technique for shape interpolation and image morphing.

38. Future Work
Optimisation of the source position for optimal interpolation
Exploration of applications of this technique besides shape interpolation and image morphing.
In this paper, we successfully extend matrix decomposition and interpolation techniques so that, rather than just
being applicable to affine transformation, they can be also applied to general projective transformation. This enables perspectively correct interpolation of image patches to be generated without 3D information.
We suggest normalisation of the projective matrix before decomposition and position normalisation of the source
points for better interpolation results. We present an efficient force-equilibrium method to handle disconnected intermediate quadrilaterals and enable quadrilaterals to be used as primitives for more complex shape interpolation and morphing applications.
Future research directions include the optimisation of the source position for optimal interpolation and exploration of
applications of this technique for shape interpolation and image morphing.

39. Projective Transformations for Image Transition Animations
Enable perspectively correct interpolation of image patches, without 3D information
Improve interpolation results by normalisation of homography and source points position
Enable quadrilaterals to be used as primitives for shape interpolation and morphing applications
TzuYen Wong, Peter Kovesi
& Amitava Datta