1. Computer Vision : CISC4/689
Computer Graphics
Output
Image
Model
Synthetic
Camera
(slides courtesy of Michael Cohen)
Computer Vision : CISC4/689

2. Computer Vision : CISC4/689
Output
Model
Real Scene
Real Cameras
(slides courtesy of Michael Cohen)
Computer Vision : CISC4/689

3. Computer Vision : CISC4/689
Combined
Output
Image
Model
Real Scene
Synthetic
Camera
Real Cameras
(slides courtesy of Michael Cohen)
Computer Vision : CISC4/689

4. Computer Vision : CISC4/689
Pinhole cameras
Abstract camera model - box with a small hole in it
Pinhole cameras work in practice
The point to make here is that each point on the image plane sees light from only
one direction, the one that passes through the pinhole.
Computer Vision : CISC4/689

5. Distant objects are smaller
Computer Vision : CISC4/689

6. Consequences: Parallel lines meet
There exist vanishing points
Marc Pollefeys
Computer Vision : CISC4/689

7. The Effect of Perspective
Computer Vision : CISC4/689

8. Computer Vision : CISC4/689
Vanishing points H
VPL
VPR
VP1
VP2
Different directions correspond
to different vanishing points
VP3
Marc Pollefeys
Computer Vision : CISC4/689

9. Computer Vision : CISC4/689
Vanishing points
each set of parallel lines (=direction) meets at a different point
The vanishing point for this direction
Sets of parallel lines on the same plane lead to collinear vanishing points.
The line is called the horizon for that plane
Good ways to spot faked images
scale and perspective don’t work
vanishing points behave badly
supermarket tabloids are a great source.
Computer Vision : CISC4/689

10. Computer Vision : CISC4/689
Slide credit: David Jacobs

11. Properties of Projection
Points project to points
Lines project to lines
Vanishing points for parallel lines
Parallel lines parallel to image plane donot converge
Closer objects appear bigger
Angles are not preserved
Degenerate cases
Line through focal point projects to a point.
Plane through focal point projects to line
Computer Vision : CISC4/689

12. Pinhole Camera Terminology
Image plane
Optical axis
Principal point/
image center
Focal length
Camera center/ pinhole
Camera point
Image point
Computer Vision : CISC4/689

13. The equation of projection
Computer Vision : CISC4/689

14. The equation of projection
Cartesian coordinates:
We have, by similar triangles, that (x, y, z) -> (f x/z, f y/z, -f)
Ignore the third coordinate, and get
Computer Vision : CISC4/689

15. Computer Vision : CISC4/689
The camera matrix
Turn previous expression into HC’s
HC’s for 3D point are (X,Y,Z,T)
HC’s for point in image are (U,V,W)
Computer Vision : CISC4/689

16. Computer Vision : CISC4/689
Weak perspective
Issue
perspective effects, but not over the scale of individual objects
collect points into a group at about the same depth, then divide each point by the depth of its group
Adv: easy
Disadv: wrong
Computer Vision : CISC4/689

17. Weak Perspective ProjectionZ O -x Z Z f
Computer Vision : CISC4/689

18. The Equation of Weak Perspective (scaled Orthographic)
s is constant for all points.
Parallel lines no longer converge, they remain parallel.
Computer Vision : CISC4/689
Slide credit: David Jacobs

19. Computer Vision : CISC4/689
Generalization of Orthographic Projection
When the camera is at a
(roughly constant) distance
from the scene, take m=1.
Computer Vision : CISC4/689
Marc Pollefeys

20. Computer Vision : CISC4/689
Pictorial Comparison
Weak perspective
Perspective
Computer Vision : CISC4/689 
Marc Pollefeys

21. Summary: Perspective Laws
Weak perspective
Orthographic
Computer Vision : CISC4/689

22. Pros and Cons of These Models
Weak perspective has simpler math.
Accurate when object is small and distant.
Most useful for recognition.
Pinhole perspective much more accurate for scenes.
Used in structure from motion.
When accuracy really matters, we must model the real camera
Use perspective projection with other calibration parameters (e.g., radial lens distortion)
Computer Vision : CISC4/689
Slide credit: David Jacobs

23. The projection matrix for orthographic projection
Computer Vision : CISC4/689

24. Computer Vision : CISC4/689
Affine cameras
Computer Vision : CISC4/689

25. The Image Formation Pipeline
Computer Vision : CISC4/689

26. Computer Vision : CISC4/689
Camera parameters
Issue
camera may not be at the origin, looking down the z-axis
extrinsic parameters
one unit in camera coordinates may not be the same as one unit in world coordinates
intrinsic parameters - focal length, principal point, aspect ratio, angle between axes, etc.
Note the matrix dimensions
Computer Vision : CISC4/689

27. Computer Vision : CISC4/689
Camera calibration
Issues:
what are intrinsic parameters of the camera?
what is the camera matrix? (intrinsic+extrinsic)
General strategy:
view calibration object
identify image points
obtain camera matrix by minimizing error
obtain intrinsic parameters from camera matrix
Error minimization:
Linear least squares
easy problem numerically
solution can be rather bad
Minimize image distance
more difficult numerical problem
solution usually rather good,
start with linear least squares
Numerical scaling is an issue
Computer Vision : CISC4/689

28. Computer Vision : CISC4/689
Outline
Vector, matrix basics
2-D point transformations
Translation, scaling, rotation, shear
Homogeneous coordinates and transformations
Homography, affine transformation
Computer Vision : CISC4/689

29. Computer Vision : CISC4/689
Notes on Notation
Vectors, points: x, v (assume column vectors)
Matrices: R, T
Scalars: x, a
Axes, objects: X, Y, O
Coordinate systems: W, C
Number systems: R, Z
Specials
Transpose operator: xT (as opposed to x0)
Identity matrix: Id
Matrices/vectors of zeroes, ones: 0, 1
Computer Vision : CISC4/689

30. Block Notation for Matrices
Often convenient to write matrices in terms of parts
Smaller matrices for blocks
Row, column vectors for ranges of entries on rows, columns, respectively
E.g.: If A is 3 x 3 and :
Computer Vision : CISC4/689

31. Computer Vision : CISC4/689
2-D Transformations
Types
Scaling
Rotation
Shear
Translation
Mathematical representation
Computer Vision : CISC4/689

32. Computer Vision : CISC4/689
2-D Scaling
Computer Vision : CISC4/689

33. Computer Vision : CISC4/689
2-D Scaling
Computer Vision : CISC4/689

34. Computer Vision : CISC4/689
2-D Scaling sx 1
Horizontal shift proportional to horizontal position
Computer Vision : CISC4/689

35. Computer Vision : CISC4/689
2-D Scaling sy 1
Vertical shift proportional to vertical position
Computer Vision : CISC4/689

36. Computer Vision : CISC4/689
2-D Scaling
Computer Vision : CISC4/689

37. Matrix form of 2-D Scaling
Computer Vision : CISC4/689

38. Computer Vision : CISC4/689
2-D Scaling
Computer Vision : CISC4/689

39. Computer Vision : CISC4/689
2-D Rotation
Computer Vision : CISC4/689

40. Computer Vision : CISC4/689
2-D Rotation
Computer Vision : CISC4/689

41. Computer Vision : CISC4/689
2-D Rotation
Computer Vision : CISC4/689

42. Matrix form of 2-D Rotation
(this is a counterclockwise rotation; reverse signs of sines to get a clockwise one)
Computer Vision : CISC4/689

43. Matrix form of 2-D Rotation
Computer Vision : CISC4/689

44. Computer Vision : CISC4/689
2-D Shear (Horizontal)
Computer Vision : CISC4/689

45. Computer Vision : CISC4/689
2-D Shear (Horizontal)
Horizontal displacement proportional to vertical position
Computer Vision : CISC4/689

46. 2-D Shear (Horizontal)
(Shear factor h is positive for the figure above)
Computer Vision : CISC4/689

47. Computer Vision : CISC4/689
2-D Shear (Horizontal)
Computer Vision : CISC4/689

48. Computer Vision : CISC4/689
2-D Shear (Vertical)
(v is negative for the figure above)
Computer Vision : CISC4/689

49. Computer Vision : CISC4/689
2-D Translation
Computer Vision : CISC4/689

50. Computer Vision : CISC4/689
2-D Translation
Computer Vision : CISC4/689

51. Computer Vision : CISC4/689
2-D Translation
Computer Vision : CISC4/689

52. Computer Vision : CISC4/689
2-D Translation
Computer Vision : CISC4/689

53. Computer Vision : CISC4/689
2-D Translation
Computer Vision : CISC4/689

54. Computer Vision : CISC4/689
2-D Translation
Computer Vision : CISC4/689

55. Representing Transformations
Note that we’ve defined translation as a vector addition but rotation, scaling, etc. as matrix multiplications
It’s inconvenient to have two different operations (addition and multiplication) for different forms of transformation
It would be desirable for all transformations to be expressed in a common, linear form (since matrix multiplications are linear transformations)
Computer Vision : CISC4/689

56. Example: “Trick” of additional coordinate makes this possible
Old way:
New way:
Computer Vision : CISC4/689

57. Computer Vision : CISC4/689
Translation Matrix
We can write the formula using this “expanded” transformation matrix as:
From now on, assume points are in this “expanded” form unless otherwise noted:
Computer Vision : CISC4/689

58. Homogeneous Coordinates
“Expanded” form is called homogeneous coordinates or projective space
Change to projective space by adding a scale factor (usually but not always 1):
Computer Vision : CISC4/689

59. Homogeneous Coordinates: Projective Space
Equivalence is defined up to scale ¸ (non-zero for finite points)
Think of projective points in P2 as rays in R3, where z coordinate is scale factor
All Euclidean points along ray are “same” in this sense
Computer Vision : CISC4/689

60. Leaving Projective Space
Can go back to non-homogeneous representation by dividing by scale factor and dropping extra coordinate:
This is the same as saying “Where does the ray intersect the plane defined by z = 1”?
Analogy to perspective projection, where f=1 (image plane) and lambda is z of any point in the ray. For different lambda’s along the line, projected point is the same, thus Equivalence class
Computer Vision : CISC4/689

61. Homogeneous Coordinates: Rotations, etc.
A 2-D rotation, scaling, shear or other transformation normally expressed by a 2 x 2 matrix R is written in homogeneous coordinates with the following 3 x 3 matrix:
The non-commutativity of matrix multiplication explains why different transformation orders give different results—i.e., RT  TR
In homogeneous
form
In homogeneous form
Computer Vision : CISC4/689

62. Example: Transformations Don’t Commute
Computer Vision : CISC4/689

63. Example: Transformations Don’t Commute
Computer Vision : CISC4/689

64. Example: Transformations Don’t Commute
Computer Vision : CISC4/689

65. Example: Transformations Don’t Commute
Computer Vision : CISC4/689

66. Computer Vision : CISC4/689
2-D Transformations
Full-generality 3 x 3 homogeneous transformation is called a homography
Computer Vision : CISC4/689

67. 2-D Transformations
Full-generality 3 x 3 homogeneous transformation is called a homography
Translation components
Computer Vision : CISC4/689

68. 2-D Transformations Full-generality 3 x 3 homogeneous
transformation is called a homography
Scale/rotation components
Computer Vision : CISC4/689

69. 2-D Transformations
Full-generality 3 x 3 homogeneous transformation is called a homography
Shear/rotation components
Shear/rotation off-diagonal (more about relationship between shearing and rotation at
Computer Vision : CISC4/689

70. 2-D Transformations
Full-generality 3 x 3 homogeneous transformation is called a homography
Homogeneous scaling factor
Computer Vision : CISC4/689

71. Computer Vision : CISC4/689
2-D Transformations
Full-generality 3 x 3 homogeneous transformation is called a homography
When these are zero (as they have been so far), H is an affine transformation
Computer Vision : CISC4/689