EEM 561 Machine Vision Week 10 :Image Formation and Cameras Spring 2015 Instructor: Hatice Çınar Akakın, Ph.D. haticecinarakakin@anadolu.edu.tr Anadolu University

Image formation 3D world 2D image Images are projections of the 3-D world onto a 2-D plane… Figures © Stephen E. Palmer, 2002 Slide source: A.Torralba

Image formation Let’s design a camera Idea 1: put a piece of film in front of an object Do we get a reasonable image? Slide source: Seitz

The barrier blocks off most of the rays Pinhole camera The barrier blocks off most of the rays It gets inverted Add a barrier to block off most of the rays This reduces blurring The opening known as the aperture How does this transform the image? It gets inverted!! Slide source: Seitz

Light rays from many different parts of the scene strike the same point on the paper. Each point on the image plane sees light from only one direction, the one that passes through the pinhole. The point to make here is that each point on the image plane sees light from only one direction, the one that passes through the pinhole. Forsyth & Ponce

Pinhole camera is a simple model to approximate imaging process, perspective projection f c f = focal length c = center of the camera If we treat pinhole as a point, only one ray from any given point can enter the camera. Figure from Forsyth

Pinhole camera Photograph by Abelardo Morell, 1991 Slide source: A.Torralba

Pinhole camera Photograph by Abelardo Morell, 1991 Slide source: A.Torralba

Pinhole camera Photograph by Abelardo Morell, 1991 Slide source: A.Torralba

Pinhole camera Photograph by Abelardo Morell, 1991 Slide source: A.Torralba

Effect of pinhole size Wandell, Foundations of Vision, Sinauer, 1995

Wandell, Foundations of Vision, Sinauer, 1995

Shrinking the aperture Why not make the aperture as small as possible? Less light gets through Diffraction effects... Slide source:N.Snavely

Shrinking the aperture Slide source:N.Snavely

Camera obscura: The pre-camera In Latin, means ‘dark room’ "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January 24, 1544, and later he used this illustration of the event in his book De Radio Astronomica et Geometrica, 1545. It is thought to be the first published illustration of a camera obscura..." Hammond, John H., The Camera Obscura, A Chronicle http://www.acmi.net.au/AIC/CAMERA_OBSCURA.html CS 376 Lecture 15

Camera Obscura

Freestanding camera obscura at UNC Chapel Hill Illustration of Camera Obscura Freestanding camera obscura at UNC Chapel Hill Photo by Seth Ilys

Camera obscura at home http://blog.makezine.com/archive/2006/02/how_to_room_sized_camera_obscu.html Sketch from http://www.funsci.com/fun3_en/sky/sky.htm CS 376 Lecture 15

Accidental pinhole camera Outside scene * Aperture See Zomet, A.; Nayar, S.K. CVPR 2006 for a detailed analysis. Slide source: A.Torralba

Measuring distance Object size decreases with distance to the pinhole There, given a single projection, if we know the size of the object we can know how far it is. But for objects of unknown size, the 3D information seems to be lost.

Adding a lens A lens focuses light onto the film “circle of confusion” A lens focuses light onto the film There is a specific distance at which objects are “in focus” other points project to a “circle of confusion” in the image Changing the shape of the lens changes this distance Slide source:N.Snavely

(Center Of Projection) Cameras with lenses F focal point optical center (Center Of Projection) A lens focuses parallel rays onto a single focal point Gather more light, while keeping focus; make pinhole perspective projection practical Slide source:K.Grauman CS 376 Lecture 15

Thin lens equation 𝑦 ′ 𝑦 = 𝑣 𝑢 𝑦 𝑦 ′ 𝑦 = (𝑣−𝑓) 𝑓 𝑦′ Any object point satisfying this equation is in focus Slide source:K.Grauman CS 376 Lecture 15

Combining Lenses

The eye The human eye is a camera Note that the retina is curved The human eye is a camera Iris - colored annulus with radial muscles Pupil - the hole (aperture) whose size is controlled by the iris What’s the “film”? photoreceptor cells (rods and cones) in the retina

Perspective projection camera world f y z y’ Cartesian coordinates: We have, by similar triangles, that (x, y, z) -> (f x/z, f y/z, -f) Ignore the third coordinate, and get f: focal length O: camera center Slide source: A.Torralba

Geometric properties of projection Points go to points Lines go to lines Planes go to whole image or half-planes. Polygons go to polygons Degenerate cases line through focal point to point plane through focal point to line Slide source: A.Torralba

Modeling projection Is this a linear transformation? no—division by z is nonlinear Homogeneous coordinates to the rescue! homogeneous image coordinates homogeneous scene coordinates Converting from homogeneous coordinates Slide by Steve Seitz

Perspective Projection Matrix Projection is a matrix multiplication using homogeneous coordinates: divide by the third coordinate to convert back to non-homogeneous coordinates Slide by Steve Seitz CS 376 Lecture 15

Perspective Projection -- Ideal Case

Perspective Projection -- Ideal Case

Orthographic projection Given camera at constant distance from scene World points projected along rays parallel to optical access CS 376 Lecture 15

Projection properties Parallel lines converge at a vanishing point Each direction in space has its own vanishing point But parallels parallel to the image plane remain parallel Slide source:N.Snavely

Vanishing points and lines Vertical vanishing point (at infinity) Vanishing line Vanishing point Vanishing point source:J.Hays Slide from Efros, Photo from Criminisi

Homogeneous coordinates 2D Points: 2D Lines: d (nx, ny) Slide source: A.Torralba

Homogeneous coordinates Intersection between two lines: Slide source: A.Torralba

Homogeneous coordinates Line joining two points: Slide source: A.Torralba

2D Transformations Example: translation = . + = . = tx ty 1 tx ty 1 = + . 1 tx ty = . = Now we can chain transformations Slide source: A.Torralba

Recall:Summary of Affine Transformations

More Realistic Perspective Projection

Perspective projection (converts from 3D rays in camera coordinate system to pixel coordinates) (intrinsics) in general, (upper triangular matrix) : aspect ratio (1 unless pixels are not square) : skew (0 unless pixels are shaped like rhombi/parallelograms) : principal point ((0,0) unless optical axis doesn’t intersect projection plane at origin) Slide source:N.Snavely

Projection matrix jw kw Ow iw R,T x: Image Coordinates: (u,v,1) K: Intrinsic Matrix (3x3) R: Rotation (3x3) t: Translation (3x1) X: World Coordinates: (X,Y,Z,1) We can use homogeneous coordinates to write camera matrix in linear form. Slide Credit: Saverese

Projection matrix Intrinsic Assumptions Unit aspect ratio Optical center at (0,0) No skew Extrinsic Assumptions No rotation Camera at (0,0,0) K Work through equations for u and v on board Slide Credit: Saverese

Remove assumption: known optical center Intrinsic Assumptions Unit aspect ratio No skew Extrinsic Assumptions No rotation Camera at (0,0,0)

Remove assumption: square pixels Intrinsic Assumptions No skew Extrinsic Assumptions No rotation Camera at (0,0,0)

Remove assumption: non-skewed pixels Intrinsic Assumptions Extrinsic Assumptions No rotation Camera at (0,0,0) Note: different books use different notation for parameters Slide Credit: J. Hays

Oriented and Translated Camera jw t kw Ow iw Slide Credit: J. Hays

Allow camera translation Intrinsic Assumptions Extrinsic Assumptions No rotation Slide Credit: J. Hays

3D Rotation of Points Rotation around the coordinate axes, counter-clockwise: p p’ g y z Slide Credit: Saverese

Allow camera rotation Slide source:J.Hays

Degrees of freedom How many known points are needed to estimate this? 5 6 How many known points are needed to estimate this? Slide source:J.Hays

Camera calibration Use the camera to tell you things about the world: Relationship between coordinates in the world and coordinates in the image: geometric camera calibration, see Szeliski, section 5.2, 5.3 for references (Relationship between intensities in the world and intensities in the image: photometric image formation, see Szeliski, sect. 2.2.) Slide source: A.Torralba

Things to remember Vanishing points and vanishing lines Vertical vanishing (at infinity) Vanishing points and vanishing lines Pinhole camera model and camera projection matrix Homogeneous coordinates