1. Overview Pin-hole model From 3D to 2D Camera projection
Homogeneous coordinates
Camera calibration
Vanishing points & lines
Perspective cues
Distortions

2. Perspective cues

3. Perspective cues

4. Perspective cues

5. Comparing heights
Vanishing point
Vanishing point

6. Comparing heights
Vanishing point
Vanishing point

7. Measuring heights 50 40 33 30 Camera height: 27 25 20 10
Vanishing point 40
Vanishing point
Horizon line 33 30
Camera height: 27 25 20 10

8. The cross ratio
What if we have no ruler in the scene for the measuring?
Vanishing points are not enough for this task.

9. The cross ratio
The cross-ratio of 4 collinear points is the fundamental projection invariant of projective geometry.
The points ordering can be permuted. Therefore there are 4! = 24 different orders (but only 6 distinct values!)

10. The cross ratio

11. The cross ratio
Scene cross ratio
Image cross ratio

12. The cross ratio
Vanishing point
Horizon line
Vanishing point

13. The cross ratio
Vanishing point
Horizon line
Vanishing point

14. Overview Pin-hole model From 3D to 2D Camera projection
Homogeneous coordinates
Camera calibration
Vanishing points & lines
Perspective cues
Distortions

15. Fronto-parallel planes
What happens to the projection of a pattern on a plane parallel to the image plane?
All points on that plane are at a fixed depth z
The pattern gets scaled by a factor of f/z, but angles and ratios of lengths/areas are preserved

16. Fronto-parallel planes
images from 1X.com

17. Perspective distortions

18. Architectural photography

19. Architectural photography
When we want to take a photo of a tall building we may face a difficulty for capturing all the building in the wanted perspective.
(a) Keeping the camera level, with an ordinary lens, captures only the bottom portion of the building.

20. Architectural photography
When we want to take a photo of a tall building we may face a difficulty for capturing all the building in the wanted perspective.
(b) Tilting the camera upwards results in converging verticals.

21. Architectural photography
When we want to take a photo of a tall building we may face a difficulty for capturing all the building in the wanted perspective.
(c) solution: “Shifting” the lens upwards results in a picture of the entire subject.

22. Architectural photography
When we want to take a photo of a tall building we may face a difficulty for capturing all the building in the wanted perspective.
(c) solution: “Shifting” the lens upwards results in a picture of the entire subject.
So how can we actually “shift” the lens of the camera for solving the problem?

23. Architectural photography
The solutions:
View camera – used by photographers for controlling focus and convergence of parallel lines. Image control is done by moving the front and/or rear standards.

24. Architectural photography
The solutions:
Tilt-Shift lens – a modern cameras solving this problem. Disadvantages: Tilting around the vertical axis resulted in a very small region in which objects appear sharp.

25. Tilt-Shift photography

26. Tilt-Shift photography

27. Tilt-Shift photography

28. Architectural photography
The solutions:
Homographic projections …

29. Points on the plane

30. Points on the plane
In many cases, such as the perspective distortions problem, we can consider some points in the image as on the same plane in the 3D world.
Using an appropriate coordinates system for the scene, we can say all these points on the plane satisfy the fact that: Z = 0 .

31. Points on the plane

32. Points on the plane

33. Points on the plane

34. Points on the plane

35. Planar homography
We got a 3 × 3 matrix, appropriate for the cases of points in the scene that lies on the same plane. We call this matrix Homography Matrix.
Once again the scale factor is arbitrary and ignorable.
Only 8 unique numbers left to be determined in the homography matrix.
Can be estimated from (at least) 4 world points and their corresponding image points.

36. Planar homography
Pseudo Inverse

37. Planar homography
Pseudo Inverse

38. Planar homography

39. Planar homography

40. Planar homography

41. Planar homography

42. Planar homography

43. Demo time!

44. The effect of virtual camera
Using the Homography matrix for dealing perspective distortions and other problematic cameras’ positions may be seen as projection of the handheld camera in the reality into a virtual camera which roles as a camera in which the result image of the homography is taken.
This virtual camera’s position is suitable for the actual perspective that we want to see the objects in the image.

45. The effect of virtual camera
Handled Camera
Virtual Camera

46. Planar homography - Some problems…
In most interesting images and cases, not all the points in the image are on the same plane. By wrapping the image according to the homography matrix, created for a specific plane, some distortions may occur regarding the these points, and especially regarding edges in the image which connecting two planes in the world.

47. Planar homography - more uses

48. Planar homography - more uses

49. More perspective distortions
The problem pointed initially by Da Vinci:
When we project some equal flat vertical objects, standing on a line parallel to the image plane, all these objects looks with the same length also in the projected 2D image.
But if we use equal vertical cylindrical objects instead of the flat ones, under the same conditions, we notice that the exterior objects appear bigger.
Important to note that this distortion has nothing to do with lens flaws.

50. More perspective distortions

51. Lens distortions
Sometimes distortions in the images may stem from imperfect camera’s lens.
So far all the imaging models assume that cameras obeys a linear projection model, where straight lines in the world result in straight lines in the image. Unfortunately, many wide-angle lenses have noticeable radial distortion.
Such derivations from the anticipated output images are most noticeable for rays that pass through the edge of the lens.

52. Lens distortions

53. Pin-cushion distortion
Lens distortions
The coordinates in the observed images are displayed away from (Barrel) or towards (Pin-cushion) the image center by an amount proportional to their radial distance.
Barrel distortion
Pin-cushion distortion

54. Lens distortions

55. Lens distortions
Let be the pixel coordinates obtained after perspective division (from the homogeneous coordinates , but before scaling by focal length 𝑓 and shifting the optical centre ).

56. 𝜅1 and 𝜅2 are called the radial distortion parameters.
Lens distortions
Apply radial distortion according to the quartic model.
𝜅1 and 𝜅2 are called the radial distortion parameters.

57. Lens distortions
Compute the final pixel coordinates by applying focal length and translate image center.
A variety of techniques can be used to estimate the radial distortion parameters for a given lens:
The simplest and most useful is to take an image of a scene with a lot of straight lines aligned with and near edges of the image. Then the parameters can be adjusted until all the lines in the image becomes straight.
Another approach is to use several overlapping images and to combine the estimation of the parameters with the image alignment process which involves a quadratic radial distortion correction term.

58. Lens distortions - some exceptions
Fish-eye lens – produce strong visual distortion intended to create a wide panoramic or hemispherical image.

59. Lens distortions - some exceptions
Anamorphic lenses – widely used in feature file production in order a wider range of aspect ratios could fit within a standard image sensor. These lens do not follow the radial distortion model we suggest. Instead, they can be thought of as inducing different vertical and horizontal scaling, i.e. non-square pixels.
images from

60. References Szeliski: Ch. 2.1.3-2.1.6, 6.2, 6.3
cs.cornell.edu:
csail.mit.edu:
cs.haifa.ac.il/hagit: