1. Affine Structure from Motion
EECS 274 Computer Vision
Affine Structure from Motion

2. Affine structure from motion
Structure from motion (SFM)
Elements of affine geometry
Affine SFM from two views
Geometric approach
Affine epipolar geometry
Affine SFM from multiple views
From affine to Euclidean images
Reading: FP Chapter 12

3. Affine structure from motion
Given a sequence of images
Find out feature points in 2D images
Find out corresponding features
Find out their 3D positions
Find out their affine motion

4. Affine Structure from Motion
Reprinted with permission from “Affine Structure from Motion,” by J.J. (Koenderink and A.J.Van Doorn, Journal of the Optical Society of America A,
8: (1990).  Optical Society of America.
Given m pictures of n points, can we recover
the three-dimensional configuration of these points?
the camera configurations (projection matrices)?
(structure)
(motion)

5. Scene relief
When the scene relief is small (compared with the overall distance separating it from the observing camera), affine projection models can be used to approximate the imaging process

6. Orthographic Projection
R is a scene reference point
Parallel Projection
consider the points off optical axis
viewing rays
are parallel

7. Affine projection models
Weak-Perspective Projection (generalizes orthographic projection)
R is a scene reference point
Paraperspective Projection (generalizes parallel projection)
consider the
distortions for points off the optical axis

8. Affine projection equations
Consider weak perspective projection and let zr denote the depth of a reference point R, then P  P’  p
p is the non-homogenous
coordinate
R2 is a 2 × 3 matrix of the first 2 rows of R and t2 is a vector formed by first 2 elements of t

9. Weak perspective projection
k and s denote the aspect ratio and skew of the camera
M is a 2 × 4 matrix defined by
2 intrinsic parameters
5 extrinsic parameters
1 scene-dependent structure parameter zr
See Chapter 2.3 of FP

10. The Affine Structure-from-Motion Problem
Given m images of n matched points Pj we can write
Here pij is 2 × 1 non-homogenous coordinate, and Mi = (Ai bi)
Problem: estimate the m 2 × 4 affine projection matrices Mi and
the n positions Pj from the mn correspondences pij
2mn equations in 8m+3n unknowns
Overconstrained problem, that can be solved
using (non-linear) least squares!

11. The Affine Ambiguity of Affine SFM
When the intrinsic and extrinsic parameters are unknown
If M and P are solutions, i j
So are M’ and P’ where i j
and
Q is an affine
transformation.
C is a 3 × 3 non-singular matrix and d is in R3

12. Affine Structure from Motion
Any solution of the affine structure from motion (sfm) can only be defined up to an affine transformation ambiguity
Taking into account the 12 parameters define general affine transformation, for 2 views (m=2), we need at least 4 point correspondences to determine the projection matrices and 3D points
2mn ≥ 8m + 3n - 12

13. With known intrinsic parameters
Exploit constraints of Mi = (Ai bi) (See Chapter 2.3) to eliminate ambiguity
First find affine shape
Use additional views and constraints to determine Euclidean structure

14. 2D planar transformations
Preserve parallelism and ratio of distance between colinear points

15. Affine Spaces: (Semi-Formal) Definition

16. 2
Example: R as an Affine Space

17. In General
The notation
is justified by the fact that choosing some origin O in X
allows us to identify the point P with the vector OP, i.e.
u=OP , Φu(O)=P
Warning: P+u and Q-P are defined independently of O!!

18. Barycentric Combinations
Can “add” a vector to a point and “subtract” two points
Can we add points? R=P+Q
NO!
But, when
we can define
Note by introducing an arbitrary origin O:

19. Can be defined purely in terms of points
Affine Subspaces
defined by y a point O and a vector subspace U
Can be defined purely in terms of points
m+1 points define a m-dimensional subspace

20. Coordinate system for U:
Affine Coordinates
Coordinate system for U:
Coordinate system for Y=O+U:
Affine coordinates:
Coordinate system for Y:
Barycentric
coordinates:
Affine coordinates of P in the basis formed by points Ai

21. Affine Transformations
Bijections from X to Y that:
map m-dimensional subspaces of X onto m-dimensional
subspaces of Y;
map parallel subspaces onto parallel subspaces; and
preserve affine (or barycentric) coordinates.
Bijections from X to Y that:
map lines of X onto lines of Y; and
preserve the ratios of signed lengths of
line segments.
The affine coordinates of D in the basis of A,B,C are the same as those of D’ in the
basis of A’,B’, and C’ – namely 2/3 and ½.
In E3 they are combinations of rigid transformations, non-uniform scalings and shears

22. Affine Transformations II
Given two affine spaces X and Y of dimension m, and two
coordinate frames (A) and (B) for these spaces, there exists
a unique affine transformation mapping (A) onto (B).
Given an affine transformation from X to Y, one can always
write:
When coordinate frames have been chosen for X and Y,
this translates into:

23. Affine projections induce affine transformations from planes
onto their images.
Preserve ratio of distance between colinear points, parallelism,
and affine coordinates
Weak- and paraperspective projections are affine transformations

24. Affine Shape
Two point sets S and S’ in some affine space X are
affinely equivalent when there exists an affine
transformation y: X X such that X’ = y ( X ).
Affine structure from motion = affine shape recovery.
= recovery of the corresponding motion equivalence classes.

25. Geometric affine scene reconstruction from two images
(Koenderink and Van Doorn, 1991).
4 points define 2 affine views
Affine projection of a plane onto
another plane is an affine transformation
Affine coordinates in π can be measured
by other two images
affine coordinates of P in
the basis (A,B,C,D)

26. Affine Structure from Motion
Reprinted with permission from “Affine Structure from Motion,” by J.J. (Koenderink and A.J.Van Doorn, Journal of the Optical Society of America A,
8: (1990).  Optical Society of America.
(Koenderink and Van Doorn, 1991)
Given 2 affine views of 4 non-coplanar ponits, the affine shape of
the scene is uniquely determined

27. Algebraic motion estimation using affine epipolar constraint
A, A’, b, b’ are known
α,β, α’,β’ are constants of A, A’, b, b’
Note: the epipolar lines are parallel.

28. Affine Epipolar Geometry
Given point p=(u,v)T, the matching point p’=(u’,’v)T lies on α’u’+β’v’+(αu+βv+δ)=0

29. The Affine Fundamental Matrix
where

30. Algebraic Scene Reconstruction Method

31. An Affine Trick..
Algebraic Scene Reconstruction Method

32. The Affine Structure of Affine Images
Suppose we observe a scene with m fixed cameras..
The set of all images of a fixed scene is a 3D affine space!

33. has rank 4!

34. From Affine to Vectorial Structure
Idea: pick one of the points (or their center of mass)
as the origin.

35. What if we could factorize D? (Tomasi and Kanade, 1992)
Affine SFM is solved!
Singular Value Decomposition
We can take

36. From uncalibrated to calibrated cameras
Weak-perspective camera:
Calibrated camera:
Problem: what is Q ?
Note: Absolute scale cannot be recovered. The Euclidean shape
(defined up to an arbitrary similitude) is recovered.

37. Reconstruction Results (Tomasi and Kanade, 1992)
Reprinted from “Factoring Image Sequences into Shape and Motion,” by C. Tomasi and
T. Kanade, Proc. IEEE Workshop on Visual Motion (1991).  1991 IEEE.

38. Photo tourism/photosynth