1. Computer Graphics
Recitation 12

2. The plan today Projective geometry basics Epipolar geometry
Reminder of homogeneous coordinates
Algebraic properties
Epipolar geometry
Pinhole camera model
Fundamental matrix of two views
Reprojection using the fundamental matrix

3. Motivation Applications in computer vision
Object recognition in images
Reconstruction of 3D models from images

4. Motivation – image based rendering
We are given several images of a 3D world.
Want to generate images of novel views without reconstructing the 3D model and rendering it
Reconstructing 3D model is a hard task
Rendering complex 3D models under complex lighting environment is hard

5. Motivation – image based rendering
Input: two images of a statue

6. Motivation – image based rendering
Output: images of novel views, without reconstruction of 3D!!

7. Holes may appear…
Due to disocclusion

8. Holes may appear…
Due to discretization

9. Projective geometry
Images of 3D world are some projective transformations 3D  2D
Expressed in matrix (linear) form when homogeneous coordinates are used

10. Relation between different images
The different images are related by Fundamental Matrices

11. Projective transformations
A general projective transformation
Preserves collinearity (three points on a line will stay on the line)
Doesn’t preserve parallelism, lengths, angles

12. Projective transformations
Projective transformation is not linear
Points at infinity may be mapped to finite points in the image and vice versa

13. Homogeneous coordinates
Add one dimension to the representation
Any linear transformation in homogeneous coordinates is in fact projective transformation

14. Homogeneous coordinates
The representation is unique up to scale

15. Homogeneous coordinates
Points at infinity:

16. Representation of 2D lines
ax + by +c = 0
Homogeneous vector (a, b, c)T
(a, b, c)T,   0 represents the same line
The vector (0, 0, 1) represents the line at infinity

17. Incidence relation
Point p = (x, y, 1)T is on line l = (a, b, c)T  lTp = pTl = 0

18. Therefore, both p1 and p2 lie on l
Incidence relation
A line through two points p1 and p2 is l = p1  p2
p1T l = <p1, p1  p2> = 0
p2T l = <p2, p1  p2> = 0
Therefore, both p1 and p2 lie on l

19. Therefore, p lies on both l1 and l2
Lines intersection
Two lines in the plane always intersect (maybe at infinite point)
The intersection point of l1 and l2 is p = l1  l2
l1Tp = <l1, l1  l2> = 0
l2Tp = <l2, l1  l2> = 0
Therefore, p lies on both l1 and l2

20. Projective transformation
Linear transformation of homogeneous coordinates
From 3D model to 2D image: 34 matrix

21. Simple pinhole camera model
The camera center C is in the origin of R3
The image plane is z = f Y Z f X

22. Simple pinhole camera model
Local coordinates in the image plane
principal point Y y Z x X

23. Simple pinhole camera model
The simple camera matrix: y
Iy / f = y / z
Iy = fy/z Iy f z

24. Simple pinhole camera model
The simple camera matrix:

25. General pinhole camera model
1) The image plane origin may be translated: Y Z
(px, py) X

26. General pinhole camera model
K is called calibration matrix (internal camera parameters):

27. General pinhole camera model
2) The world coordinates may be rotated and translated
camera center (31)
rotation 33 matrix
from world to camera Y
(px, py) Z X

28. Geometry of two views P p p’ C’ C

29. The epipoles
e is the image of C’ (second camera origin) under the first camera P p p’ e’ e C’ C

30. The epipoles
Clearly, C, C’, e, e’, P, p, p’ all lie in the same plane!
A plane that contains the baseline CC’ is called epipolar plane P p p’ e’ e C’ C

31. Epipolar lines
Epipolar plane intersects the image plane at epipolar line
The epipolar line will always pass through the epipole P p p’ e’ e C’ C

32. Epipolar lines
Epipolar line l’ is the image of the ray CP in the second camera P Q p’ p e’ e C’ C

33. Much nicer figure C C’

34. Another nice figure C C’

35. Relation between corresponding points
p is the image of P in the first camera
p’ is the corresponding point in the second camera
p’ must lie on the intersection of the image plane with the epipolar plane through C, p, e! P l’ p p’ e’ e C’ C

36. The basic incidence equation
p’ must lie on the epipolar line in the second image plane:
p’T l’ = 0
It can be shown that the epipolar line l’ is:
l’ = Fp
where F is a 33 matrix that depends on the camera matrices only

37. The fundamental matrix
F is a 33 matrix
Expresses the incidence relation:
p’T F p = 0
Fp is the epipolar line,
so we get back p’T l’ = 0

38. Properties of the fundamental matrix
Rank = 2 (because Fe = 0)
Thus, 7 degrees of freedom out of 9 coefficients
one constraint is det F = 0
scaling – multiplying F by a constant doesn’t change anything because of homogeneity

39. Finding the fundamental matrix
Given 2 images with unknown camera matrices
We can find F by 7 point correspondences (gives us 7 equations for the coefficients of F)
Practical algorithms find many more correspondences and find a least-squares solution – more robust

40. Generating novel viewsC3
e23
e21
e23
e12
e31
e13 C2 C2 C1
We have three cameras C1, C2, C3
eij denotes intersection of CiCj with the image plane of Ci

41. Generating novel viewsC3
p’’
e23
e21 P
e23
e12 p’ p
e31
e13 C2 C2 C1
The image of P in the 3rd camera is
the intersection of the epipolar lines from C1, C2

42. The algorithm p’ p p’’ Given:
Fundamental matrices
Point correspondences between the two given images
This is found using “optical flow” algorithm
For each correspondence p  p’ find the position of the corresponding point p’’ in the third view
Copy the intensity to p’’ p’ p
p’’

43. Generating novel views
Denote the fundamental matrix from Cj to Ci by Fij (i, j = 1, 2, 3)
The intersection of the epipolar lines is given by:
p’’ = (F31 p)(F32 p’)

44. Degeneracies
Can’t reproject points on the plane of C1, C2 , C3 (no proper intersection of the epipolar lines)
Numerical problems for all points close to that plane
Epipolar geometry is not defined when Ci = Ci (only rotational movement of the camera)
Can’t reconstruct third view if C1, C2 , C3 are on one line

45. Good luck!