1. EEM 561 Machine Vision Week 10 :Image Formation and Cameras
Spring 2015
Instructor: Hatice Çınar Akakın, Ph.D.
Anadolu University

2. Image formation 3D world 2D image
Images are projections of the 3-D world onto a 2-D plane…
Figures © Stephen E. Palmer, 2002
Slide source: A.Torralba

3. Image formation Let’s design a camera
Idea 1: put a piece of film in front of an object
Do we get a reasonable image?
Slide source: Seitz

4. The barrier blocks off most of the rays
Pinhole camera
The barrier blocks off most of the rays
It gets inverted
Add a barrier to block off most of the rays
This reduces blurring
The opening known as the aperture
How does this transform the image?
It gets inverted!!
Slide source: Seitz

5. Light rays from many different parts of the scene strike the same point on the paper.
Each point on the image plane sees light from only one direction, the one that passes through the pinhole.
The point to make here is that each point on the image plane sees light from only
one direction, the one that passes through the pinhole.
Forsyth & Ponce

6. Pinhole camera is a simple model to approximate imaging process, perspective projectionf c
f = focal length
c = center of the camera
If we treat pinhole as a point, only one ray from any given point can enter the camera.
Figure from Forsyth

7. Pinhole camera Photograph by Abelardo Morell, 1991
Slide source: A.Torralba

8. Pinhole camera Photograph by Abelardo Morell, 1991
Slide source: A.Torralba

9. Pinhole camera Photograph by Abelardo Morell, 1991
Slide source: A.Torralba

10. Pinhole camera Photograph by Abelardo Morell, 1991
Slide source: A.Torralba

11. Effect of pinhole size
Wandell, Foundations of Vision, Sinauer, 1995

12. Wandell, Foundations of Vision, Sinauer, 1995

13. Shrinking the aperture
Why not make the aperture as small as possible?
Less light gets through
Diffraction effects...
Slide source:N.Snavely

14. Shrinking the aperture
Slide source:N.Snavely

15. Camera obscura: The pre-camera
In Latin, means ‘dark room’
"Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January 24, 1544, and later he used this illustration of the event in his book De Radio Astronomica et Geometrica, It is thought to be the first published illustration of a camera obscura..."
Hammond, John H., The Camera Obscura, A Chronicle
CS 376 Lecture 15

16. Camera Obscura

17. Freestanding camera obscura at UNC Chapel Hill
Illustration of Camera Obscura
Freestanding camera obscura at UNC Chapel Hill
Photo by Seth Ilys

18. Camera obscura at home
Sketch from
CS 376 Lecture 15

19. Accidental pinhole camera
Outside scene *
Aperture
See Zomet, A.; Nayar, S.K. CVPR 2006 for a detailed analysis.
Slide source: A.Torralba

20. Measuring distance Object size decreases with distance to the pinhole
There, given a single projection, if we know the size of the object we can know how far it is.
But for objects of unknown size, the 3D information seems to be lost.

21. Adding a lens A lens focuses light onto the film
“circle of
confusion”
A lens focuses light onto the film
There is a specific distance at which objects are “in focus”
other points project to a “circle of confusion” in the image
Changing the shape of the lens changes this distance
Slide source:N.Snavely

22. (Center Of Projection)
Cameras with lenses F
focal point
optical center
(Center Of Projection)
A lens focuses parallel rays onto a single focal point
Gather more light, while keeping focus; make pinhole perspective projection practical
Slide source:K.Grauman
CS 376 Lecture 15

23. Thin lens equation
𝑦 ′ 𝑦 = 𝑣 𝑢 𝑦
𝑦 ′ 𝑦 = (𝑣−𝑓) 𝑓 𝑦′
Any object point satisfying this equation is in focus
Slide source:K.Grauman
CS 376 Lecture 15

24. Combining Lenses

25. The eye The human eye is a camera
Note that the retina is curved
The human eye is a camera
Iris - colored annulus with radial muscles
Pupil - the hole (aperture) whose size is controlled by the iris
What’s the “film”?
photoreceptor cells (rods and cones) in the retina

26. Perspective projection
camera
world f y z y’
Cartesian coordinates:
We have, by similar triangles, that (x, y, z) -> (f x/z, f y/z, -f)
Ignore the third coordinate, and get
f: focal length
O: camera center
Slide source: A.Torralba

27. Geometric properties of projection
Points go to points
Lines go to lines
Planes go to whole image or half-planes.
Polygons go to polygons
Degenerate cases
line through focal point to point
plane through focal point to line
Slide source: A.Torralba

28. Modeling projection Is this a linear transformation?
no—division by z is nonlinear
Homogeneous coordinates to the rescue!
homogeneous image
coordinates
homogeneous scene
coordinates
Converting from homogeneous coordinates
Slide by Steve Seitz

29. Perspective Projection Matrix
Projection is a matrix multiplication using homogeneous coordinates:
divide by the third coordinate to convert back to non-homogeneous coordinates
Slide by Steve Seitz
CS 376 Lecture 15

30. Perspective Projection -- Ideal Case

31. Perspective Projection -- Ideal Case

32. Orthographic projection
Given camera at constant distance from scene
World points projected along rays parallel to optical access
CS 376 Lecture 15

33. Projection properties
Parallel lines converge at a vanishing point
Each direction in space has its own vanishing point
But parallels parallel to the image plane remain parallel
Slide source:N.Snavely

34. Vanishing points and lines
Vertical vanishing
point
(at infinity)
Vanishing
line
Vanishing
point
Vanishing
point
source:J.Hays
Slide from Efros, Photo from Criminisi

35. Homogeneous coordinates
2D Points:
2D Lines: d
(nx, ny)
Slide source: A.Torralba

36. Homogeneous coordinates
Intersection between two lines:
Slide source: A.Torralba

37. Homogeneous coordinates
Line joining two points:
Slide source: A.Torralba

38. 2D Transformations Example: translation = . + = . =tx ty 1 tx ty 1 = + . 1 tx ty = . =
Now we can chain transformations
Slide source: A.Torralba

39. Recall:Summary of Affine Transformations

40. More Realistic Perspective Projection

41. Perspective projection
(converts from 3D rays in camera coordinate system to pixel coordinates)
(intrinsics)
in general,
(upper triangular matrix)
: aspect ratio (1 unless pixels are not square)
: skew (0 unless pixels are shaped like rhombi/parallelograms)
: principal point ((0,0) unless optical axis doesn’t intersect projection plane at origin)
Slide source:N.Snavely

42. Projection matrix jw kw Ow iw R,T x: Image Coordinates: (u,v,1)
K: Intrinsic Matrix (3x3)
R: Rotation (3x3)
t: Translation (3x1)
X: World Coordinates: (X,Y,Z,1)
We can use homogeneous coordinates to write camera matrix in linear form.
Slide Credit: Saverese

43. Projection matrix Intrinsic Assumptions Unit aspect ratio
Optical center at (0,0)
No skew
Extrinsic Assumptions
No rotation
Camera at (0,0,0) K
Work through equations for u and v on board
Slide Credit: Saverese

44. Remove assumption: known optical center
Intrinsic Assumptions
Unit aspect ratio
No skew
Extrinsic Assumptions
No rotation
Camera at (0,0,0)

45. Remove assumption: square pixels
Intrinsic Assumptions
No skew
Extrinsic Assumptions
No rotation
Camera at (0,0,0)

46. Remove assumption: non-skewed pixels
Intrinsic Assumptions
Extrinsic Assumptions
No rotation
Camera at (0,0,0)
Note: different books use different notation for parameters
Slide Credit: J. Hays

47. Oriented and Translated Camerajw t kw Ow iw
Slide Credit: J. Hays

48. Allow camera translation
Intrinsic Assumptions
Extrinsic Assumptions
No rotation
Slide Credit: J. Hays

49. 3D Rotation of Points
Rotation around the coordinate axes, counter-clockwise: p p’ g y z
Slide Credit: Saverese

50. Allow camera rotation
Slide source:J.Hays

51. Degrees of freedom How many known points are needed to estimate this?5 6
How many known points are needed to estimate this?
Slide source:J.Hays

52. Camera calibration Use the camera to tell you things about the world:
Relationship between coordinates in the world and coordinates in the image: geometric camera calibration, see Szeliski, section 5.2, 5.3 for references
(Relationship between intensities in the world and intensities in the image: photometric image formation, see Szeliski, sect. 2.2.)
Slide source: A.Torralba

53. Things to remember Vanishing points and vanishing lines
Vertical vanishing
(at infinity)
Vanishing points and vanishing lines
Pinhole camera model and camera projection matrix
Homogeneous coordinates