1. Lecturer: Dr. A.H. Abdul Hafez
Image Processing and Analysis Computer Vision Lecture 2: Image Formation Geometry
Lecturer: Dr. A.H. Abdul Hafez
Hasan Kalyoncu University
Faculty of Computer Engineering
Fall
30 November 2018
HKU, AH Abdul Hafez

2. Physical parameters of image formation
Geometric
Type of projection
Camera pose
Optical
Sensor’s lens type
focal length, field of view, aperture
Photometric
Type, direction, intensity of light reaching sensor
Surfaces’ reflectance properties
Sensor
sampling, etc.

3. Physical parameters of image formation
Geometric
Type of projection
Camera pose
Optical
Sensor’s lens type
focal length, field of view, aperture
Photometric
Type, direction, intensity of light reaching sensor
Surfaces’ reflectance properties
Sensor
sampling, etc.

4. Perspective and art
Use of correct perspective projection indicated in 1st century B.C. frescoes
Skill resurfaces in Renaissance: artists develop systematic methods to determine perspective projection (around )
Raphael
Durer, 1525
K. Grauman

5. Perspective projection equations
3d world mapped to 2d projection in image plane
Image plane
Focal length
Optical axis
Camera frame ‘ ’
Scene / world points
Scene point
Image coordinates ‘’
Forsyth and Ponce

6. Homogeneous coordinates
Is this a linear transformation?
no—division by z is nonlinear
Trick: add one more coordinate:
homogeneous image
coordinates
homogeneous scene
coordinates
Converting from homogeneous coordinates
Slide by Steve Seitz

7. Perspective Projection Matrix
Projection is a matrix multiplication using homogeneous coordinates:
divide by the third coordinate to convert back to non-homogeneous coordinates
Complete mapping from world points to image pixel positions?
Slide by Steve Seitz

8. Perspective projection & calibration
Perspective equations so far in terms of camera’s reference frame….
Camera’s intrinsic and extrinsic parameters needed to calibrate geometry.
Camera frame
K. Grauman 8

9. Perspective projection & calibration
World frame
Extrinsic:
Camera frame World frame
Intrinsic:
Image coordinates relative to camera  Pixel coordinates
Camera frame
World to camera coord. trans. matrix (4x4)
Perspective projection matrix (3x4)
Camera to pixel coord. trans. matrix (3x3) = 2D
point (3x1)
3D point (4x1)
K. Grauman 9

10. Intrinsic parameters: from idealized world coordinates to pixel values
Forsyth&Ponce
Perspective projection
W. Freeman

11. Intrinsic parameters But “pixels” are in some arbitrary spatial units
Maybe pixels are not square
W. Freeman

12. Intrinsic parameters
We don’t know the origin of our camera pixel coordinates
W. Freeman

13. Intrinsic parameters
May be skew between camera pixel axes
W. Freeman

14. Intrinsic parameters, homogeneous coordinates
Using homogenous coordinates,
we can write this as:
or:
In pixels
In camera-based coords
W. Freeman

15. Extrinsic parameters: translation and rotation of camera frame
Non-homogeneous coordinates
Homogeneous coordinates
W. Freeman

16. Combining extrinsic and intrinsic calibration parameters, in homogeneous coordinates
pixels
Intrinsic
Camera coordinates
World coordinates
Extrinsic
Forsyth&Ponce
W. Freeman

17. Other ways to write the same equation
pixel coordinates
world coordinates
Conversion back from homogeneous coordinates leads to:
W. Freeman

18. Calibration target
Find the position, ui and vi, in pixels, of each calibration object feature point.

19. Camera calibration
From before, we had these equations relating image positions,
u,v, to points at 3-d positions P (in homogeneous coordinates):
So for each feature point, i, we have:
W. Freeman

20. Camera calibration Stack all these measurements of i=1…n points
into a big matrix:
W. Freeman

21. Camera calibration In vector form: Showing all the elements:
W. Freeman

22. Camera calibration Q m = 0
We want to solve for the unit vector m (the stacked one)
that minimizes
The minimum eigenvector of the matrix QTQ gives us that
(see Forsyth&Ponce, 3.1), because it is the unit vector x that minimizes xT QTQ x.
W. Freeman

23. Camera calibration
Once you have the M matrix, can recover the intrinsic and extrinsic parameters as in Forsyth&Ponce, sect
W. Freeman

24. Recall, perspective effects…
Far away objects appear smaller
Forsyth and Ponce

25. Perspective effects

26. Perspective effects

27. Perspective effects Parallel lines in the scene intersect in the image
Converge in image on horizon line
Image plane
(virtual)
pinhole
Scene

28. Projection Types/properties
Many-to-one: any points along same ray map to same point in image
Points  ?
points
Lines  ?
lines (collinearity preserved)
Distances and angles are / are not ? preserved
are not
Degenerate cases:
– Line through focal point projects to a point.
– Plane through focal point projects to line
– Plane perpendicular to image plane projects to part of the image.

29. Projection Types/Weak perspective
Approximation: treat magnification as constant
Assumes scene depth << average distance to camera
Image plane
World points:

30. Projection Types/Orthographic projection
Given camera at constant distance from scene
World points projected along rays parallel to optical access

31. 2D rigid Transformations

32. 2D Rigid Transformations

33. 3D Rigid Transformations

34. Slide Credits Bill Freeman Steve Seitz Kristen Grauman
Forsyth and Ponce
Rick Szeliski
Trevol Darrell
and others, as marked…

35. END

37. Other types of projection
Lots of intriguing variants…
(I’ll just mention a few fun ones)
S. Seitz

38. 360 degree field of view… Basic approach
Take a photo of a parabolic mirror with an orthographic lens (Nayar)
Or buy one a lens from a variety of omnicam manufacturers…
See
S. Seitz

39. Titlt-shift images from Olivo Barbieri and Photoshop imitations
Tilt-shift
Titlt-shift images from Olivo Barbieri and Photoshop imitations
S. Seitz

40. tilt, shift

41. Tilt-shift perspective correction

42. normal lens tilt-shift lens

43. Rotating sensor (or object)
Rollout Photographs © Justin Kerr
Also known as “cyclographs”, “peripheral images”
S. Seitz

44. Photofinish
S. Seitz