Projective cameras Motivation Elements of Projective Geometry Projective structure from motion Planches : – –

Weak-Perspective Projection Model r (p and P are in homogeneous coordinates) p = A P + b (neither p nor P is in hom. coordinates) p = M P (P is in homogeneous coordinates)

Affine Spaces: (Semi-Formal) Definition

Affine projections induce affine transformations from planes onto their images.

Affine Structure from Motion Reprinted with permission from “Affine Structure from Motion,” by J.J. (Koenderink and A.J.Van Doorn, Journal of the Optical Society of America A, 8: (1990).  1990 Optical Society of America. Given m pictures of n points, can we recover the three-dimensional configuration of these points? the camera configurations? (structure) (motion)

Geometric affine scene reconstruction from two images (Koenderink and Van Doorn, 1991).

The Projective Structure-from-Motion Problem Given m perspective images of n fixed points P we can write Problem: estimate the m 3x4 matrices M and the n positions P from the mn correspondences p. i j ij 2mn equations in 11m+3n unknowns Overconstrained problem, that can be solved using (non-linear) least squares! j

The Projective Ambiguity of Projective SFM If M and P are solutions, i j So are M’ and P’ where i j and Q is an arbitrary non-singular 4x4 matrix. When the intrinsic and extrinsic parameters are unknown Q is a projective transformation.

Projective Spaces: (Semi-Formal) Definition

A Model of P( R ) 3

Projective Subspaces and Projective Coordinates

Projective coordinates P

Projective Subspaces Given a choice of coordinate frame Line:Plane:

When do m+1 points define a p-dimensional subspace Y of an n-dimensional projective space X equipped with some coordinate frame? Writing that all minors of size (p+1)x(p+1) of D are equal to zero gives the equations of Y. Rank ( D ) = p+1, where

Hyperplanes and duality This can be rewritten as u 0 x 0 +u 1 x 1 +…+u n x n = 0, or  T P = 0, where  = (u 0,u 1,…,u n ) T. Consider n+1 points P 0, …, P n-1, P in a projective space X of dimension n. They lie in the same hyperplane when Det(D)=0. Hyperplanes form a dual projective space X * of X, and any theorem that holds for points in X holds for hyperplanes in X *. What is the dual of a straight line?

Affine and Projective Spaces

Affine and Projective Coordinates

Cross-Ratios Collinear points Pencil of coplanar lines Pencil of planes {A,B;C,D}= sin(  +  )sin(  +  ) sin(  +  +  )sin 

Cross-Ratios and Projective Coordinates Along a line equipped with the basis In a plane equipped with the basis In 3-space equipped with the basis *

Projective Transformations Bijective linear map: Projective transformation: ( = homography ) Projective transformations map projective subspaces onto projective subspaces and preserve projective coordinates. Projective transformations map lines onto lines and preserve cross-ratios.

Perspective Projections induce projective transformations between planes.

Projective Shape Two point sets S and S’ in some projective space X are projectively equivalent when there exists a projective transformation  : X X such that S’ =  ( S ). Projective structure from motion = projective shape recovery. = recovery of the corresponding motion equivalence classes.

Epipolar Geometry Epipolar Plane Epipoles Epipolar Lines Baseline

Geometric Scene Reconstruction Idea: use (A,B,C,D,F) as a projective basis and reconstruct O’ and O’’, assuming that the epipoles are known. A B C D F G H I J K E O’ O’’

Geometric Scene Reconstruction II Idea: use (A,O”,O’,B,C) as a projective basis, assuming again that the epipoles are known.

Multi-View Geometry Epipolar Geometry The Essential Matrix The Fundamental Matrix

Epipolar Geometry Epipolar Plane Epipoles Epipolar Lines Baseline

Epipolar Constraint Potential matches for p have to lie on the corresponding epipolar line l’. Potential matches for p’ have to lie on the corresponding epipolar line l.

Epipolar Constraint: Calibrated Case Essential Matrix (Longuet-Higgins, 1981)

Properties of the Essential Matrix E p’ is the epipolar line associated with p’. E p is the epipolar line associated with p. E e’=0 and E e=0. E is singular. E has two equal non-zero singular values (Huang and Faugeras, 1989). T T

Epipolar Constraint: Small Motions To First-Order: Pure translation: Focus of Expansion

Epipolar Constraint: Uncalibrated Case Fundamental Matrix (Faugeras and Luong, 1992)

Properties of the Fundamental Matrix F p’ is the epipolar line associated with p’. F p is the epipolar line associated with p. F e’=0 and F e=0. F is singular. T T

The Eight-Point Algorithm (Longuet-Higgins, 1981) | F | =1. Minimize: under the constraint 2

Non-Linear Least-Squares Approach (Luong et al., 1993) Minimize with respect to the coefficients of F, using an appropriate rank-2 parameterization.

The Normalized Eight-Point Algorithm (Hartley, 1995) Center the image data at the origin, and scale it so the mean squared distance between the origin and the data points is 2 pixels: q = T p, q’ = T’ p’. Use the eight-point algorithm to compute F from the points q and q’. Enforce the rank-2 constraint. Output T F T’. T iiii ii

Data courtesy of R. Mohr and B. Boufama.

Without normalization With normalization Mean errors: 10.0pixel 9.1pixel Mean errors: 1.0pixel 0.9pixel

Trinocular Epipolar Constraints These constraints are not independent!

Trinocular Epipolar Constraints: Transfer Given p and p, p can be computed as the solution of linear equations. 123

Trifocal Constraints

All 3x3 minors must be zero! Calibrated Case Trifocal Tensor

Trifocal Constraints Uncalibrated Case Trifocal Tensor

Trifocal Constraints: 3 Points Pick any two lines l and l through p and p. Do it again T( p, p, p )=0 12 3

Properties of the Trifocal Tensor Estimating the Trifocal Tensor Ignore the non-linear constraints and use linear least-squares a posteriori. Impose the constraints a posteriori. For any matching epipolar lines, l G l = 0. The matrices G are singular. They satisfy 8 independent constraints in the uncalibrated case (Faugeras and Mourrain, 1995). 213 Ti 1 i

For any matching epipolar lines, l G l = Ti The backprojections of the two lines do not define a line!

Multiple Views (Faugeras and Mourrain, 1995)

Two Views Epipolar Constraint

Three Views Trifocal Constraint

Four Views Quadrifocal Constraint (Triggs, 1995)

Geometrically, the four rays must intersect in P..

Quadrifocal Tensor and Lines

Scale-Restraint Condition from Photogrammetry