Digital Camera and Computer Vision Laboratory Department of Computer Science and Information Engineering National Taiwan University, Taipei, Taiwan, R.O.C. Computer and Robot Vision I Chapter 13 Perspective Projection Geometry Presented by: 傅楸善 & 張博思 指導教授 : 傅楸善 博士

DC & CV Lab. CSIE NTU 13.1 Introduction Computer vision problems often involve interpreting the information on a two- dimensional (2D) image of a three-dimensional (3D) world in order to determine the placement of the 3D objects portrayed in the image. To do this requires understanding the perspective transformation governing the geometric way 3D information is projected onto the 2D image.

DC & CV Lab. CSIE NTU 13.1 Introduction image formation on the retina, according to Descartes scrape ox eye, observe from darkened room inverted image of scene

DC & CV Lab. CSIE NTU Nalwa, Scrape ox eye, observe from darkened room inverted image of scene

DC & CV Lab. CSIE NTU 13.2 One-Dimensional Perspective Projection f: focal length of lens u: distance between object and lens center v: distance between image and lens center thin-lens equation: lens law: 1/f=1/u+1/v light passing lens center dose not deflect light parallel to optical axis will pass focus

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU pinhole camera: infinitesimally small aperture pinhole camera: approximated by lens with aperture adjusted to the smallest pinhole camera: simplest device to form image of 3D scene on 2D surface

DC & CV Lab. CSIE NTU 13.2 One-Dimensional Perspective Projection aperture size decreased: image become sharper diameter of aperture is 0.06 inch, inch, inch aperture below certain size: diffraction: bending of light rays around edge

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU JOKE

DC & CV Lab. CSIE NTU 13.2 One-Dimensional Perspective Projection

DC & CV Lab. CSIE NTU 13.2 One-Dimensional Perspective Projection f: camera constant (different from above equation) (r, s, 1): homogeneous coordinate system for point (r, s) first linear transformation: translates (r, s, 1) by distance of f

DC & CV Lab. CSIE NTU 13.2 One-Dimensional Perspective Projection second linear transformation: takes perspective transformation to image line 1D image line coordinate:

DC & CV Lab. CSIE NTU 13.2 One-Dimensional Perspective Projection (Xp,Yp) (X, Y) X Y

DC & CV Lab. CSIE NTU 13.2 One-Dimensional Perspective Projection (Xp,Yp) (X, Y) X Y

DC & CV Lab. CSIE NTU 13.2 One-Dimensional Perspective Projection lens: at origin and looks down - axis image line: distance f in front of lens and parallel to -axis : the x - y axes rotated anticlockwise by angle

DC & CV Lab. CSIE NTU 13.2 One-Dimensional Perspective Projection

DC & CV Lab. CSIE NTU 13.2 One-Dimensional Perspective Projection rewriting the relationship in terms of homogeneous coordinate system

DC & CV Lab. CSIE NTU 13.3 The Perspective Projection in 3D camera lens: along line parallel to z-axis position of lens: center of perspectivity: (u, v): coordinates of perspective projection of (x, y, z) on image plane

DC & CV Lab. CSIE NTU JOKE

DC & CV Lab. CSIE NTU Smaller Appearance of Farther Objects without loss of generality: take center of perspectivity to be origin perspective projection: objects appear smaller the farther they are

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU foreshortening: line segments in plane parallel to image has maximum size

DC & CV Lab. CSIE NTU Lines to Lines lines in 3D world transform to lines in the image plane parallel lines in 3D with nonzero z slope: meet in a vanishing point

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU Perspective Projection of Convex Polyhedra are Convex Proofs in textbook, simple but tedious, study as exercise by yourself

DC & CV Lab. CSIE NTU Vanishing Point Perspective projections of parallel 3D lines having nonzero slope along the optic z-axis meet in a vanishing point on the image projection plane.

DC & CV Lab. CSIE NTU Vanishing Line

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU D Lines-2D perspective Projection Lines There is a relationship between the parameters of a 3D line and the parameters of the perspective projection of the line.

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU JOKE

DC & CV Lab. CSIE NTU D to 3D Inference Using Perspective Projection perspective projection on unknown 3D line: provides four of six constraints additional constraints: 3D-world-model information about points, lines

DC & CV Lab. CSIE NTU Inverse Perspective Projection : perspective projection of a point f: image plane distance from camera lens thus : 3D coordinate of the point in image plane camera lens: at the origin line L: inverse perspective projection of the point

DC & CV Lab. CSIE NTU Line Segment with Known Direction Cosines and Known Length known: : line segment length : line segment direction cosine, : perspective projections of endpoints unknown:, : 3D coordinates of endpoints

DC & CV Lab. CSIE NTU Line Segment with Known Direction Cosines and Known Length

DC & CV Lab. CSIE NTU Line Segment with Known Direction Cosines and Known Length

DC & CV Lab. CSIE NTU Collinear Points with Known Interpoint Distances known: : perspective projection of nth collinear points, n = 0, …, N - 1 distance between (n+1)th point and first point unknown: : direction cosine of line, : 3D coordinates of points

DC & CV Lab. CSIE NTU Collinear Points with known Interpoint Distances

DC & CV Lab. CSIE NTU N Parallel Lines known: : perspective projection of nth parallel line unknown: : direction cosine of line

DC & CV Lab. CSIE NTU N Parallel Lines

DC & CV Lab. CSIE NTU N Lines Intersecting at a Point with Known Angles known: : perspective projection of intersecting point : perspective projection of nth intersecting line : known angle between and unknown: : 3D nth intersecting line

DC & CV Lab. CSIE NTU N Lines Intersecting in a Known Plane known: : perspective projection of intersecting point : perspective projection of nth intersecting line : plane equation unknown: : 3D nth intersecting line

DC & CV Lab. CSIE NTU N Lines Intersecting in a Known Plane

DC & CV Lab. CSIE NTU Three Lines in a Plane with One Perpendicular to the Other Two known: : perspective projection of line unknown: three lines in same plane, perpendicular to, : perspective projection of line : since is perpendicular to,

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU Point with Given Distance to a Known Point known: : perspective projection of unknown point : known 3D points : distance between the two points unknown: : direction cosine between two points Inverse perspective projection: u v f

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU Point in a Known Plane known: : perspective projection of unknown point : known plane equation where point lies unknown: : 3D coordinate of the point:

DC & CV Lab. CSIE NTU Point in a Known Plane

DC & CV Lab. CSIE NTU Line in a Known Plane known: : known plane equation where line lies : perspective projection of line unknown: : 3D line A*i + B*j + C*k = 0,

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU JOKE

DC & CV Lab. CSIE NTU Angle known: : perspective projection of the unknown line : direction cosine for the known line : angle between the 3D lines unknown: : direction cosine for the unknown line =.

DC & CV Lab. CSIE NTU Angle

DC & CV Lab. CSIE NTU Parallelogram known: perspective projection of four corner points of a parallelogram unknown: : normal to the plane on which the parallelogram lies, : direction cosines of two sides of parallelogram,

DC & CV Lab. CSIE NTU Triangle with One Vertex Known known:,, : perspective projection of three vertices : one known 3D vertex of the three vertices, : known lengths of the triangle in 3D unknown:, : two unknown 3D vertices of the three vertices

DC & CV Lab. CSIE NTU Triangle with One Vertex Known

DC & CV Lab. CSIE NTU Triangle with Orientation of One Leg Known known:,, : perspective projection of three vertices : known direction cosines between the first two vertices, : known lengths of the triangle in 3D unknown:,, : three unknown 3D vertices

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU Triangle: three-point spatial resection problem in photogrammetry known:,, : perspective projection of three vertices, : known lengths of the triangle in 3D unknown:,, : three unknown 3D vertices four solutions =, =, =

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU Determining the Principal Point by Using Parallel Lines principle point: point through which the optic axis passes principle point: so far assumes origin of image reference frame known:, n = 1, …, N: perspective projection of nth parallel line unknown: : coordinate of the principal point

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU Circles known: perspective projection of a circle having known radius unknown: plane on which the circle lies the 3D center of the circle:

DC & CV Lab. CSIE NTU JOKE

DC & CV Lab. CSIE NTU 13.6 Range from Structured Light structured light: active visual sensing technique upon perspective geometry structured light: controlled light source with regular pattern onto scene regular pattern: stripes, grid, …

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU intensity and range images

DC & CV Lab. CSIE NTU 13.6 Range from Structured Light Two light sources with cylindrical lenses produce sheets of light that intersect in a line lying on the surface of a conveyor belt. A camera above the belt is aimed so that this line is imaged on a linear array of photo sensors. When there is no object present, all the sensor cells are brightly illuminated. When part of an object interrupts the incident light, the corresponding region on the linear array is darkened. The motion of the belt scans the object past the sensor, generating the second image dimension.

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU 13.7 Cross-Ratio cross-ratio: of perspective projection of 4 collinear points, takes same value

DC & CV Lab. CSIE NTU Cross-Ratio Definitions and Invariance four collinear points: q, r: centers of perspectivity for two projection images

DC & CV Lab. CSIE NTU Cross-Ratio Definitions and Invariance Let,. by perspective projection equations, cross-ratio: cross-ratio: independent of reference frame, point p, direction cosine b cross-ratio: depends only on directed of collinear points

DC & CV Lab. CSIE NTU Only One Cross-Ratio each of 4! Cross-ratios is a function of cross-ratio

DC & CV Lab. CSIE NTU Cross–Ratio in Three Dimensions The cross-ratio derived from one-dimensional perspective projections in a two-dimensional world can be generalized to two-dimensional perspective projection in a three-dimensional world. five co-planar points : cross-ratio for the line segment and

DC & CV Lab. CSIE NTU Cross–Ratio in Three Dimensions

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU Using Cross-Ratios cross-ratio: to aid in establishing correspondences

DC & CV Lab. CSIE NTU END

DC & CV Lab. CSIE NTU JOKE

DC & CV Lab. CSIE NTU Using Cross-Ratios cross-ratio: to aid in establishing correspondences Term Project JPEG 2000

DC & CV Lab. CSIE NTU 1. Neural Network

DC & CV Lab. CSIE NTU 3. Image Compression JPEG, MPEG, H. 264

DC & CV Lab. CSIE NTU 5. segmentation based on texture

DC & CV Lab. CSIE NTU 6. optical character reading

DC & CV Lab. CSIE NTU 7. stereo vision

DC & CV Lab. CSIE NTU 7. stereo vision

DC & CV Lab. CSIE NTU 8. Handwriting recognition

DC & CV Lab. CSIE NTU 8. Handwriting recognition

DC & CV Lab. CSIE NTU 9. histogram specification

DC & CV Lab. CSIE NTU Term Project

DC & CV Lab. CSIE NTU 10. homomorphic filtering

DC & CV Lab. CSIE NTU 10. homomorphic filtering

DC & CV Lab. CSIE NTU 12. calculating the sizes of stones, cells, cell nucleus.

DC & CV Lab. CSIE NTU 13. trademark resemblance, semi-automatic similarity classification

DC & CV Lab. CSIE NTU 15. structured light 3-D reconstruction

DC & CV Lab. CSIE NTU 15. structured light 3-D reconstruction

DC & CV Lab. CSIE NTU 16. object classification with moments invariant to rotation, scaling, translation

DC & CV Lab. CSIE NTU 17. photometric stereo

DC & CV Lab. CSIE NTU 18. shape from focus

DC & CV Lab. CSIE NTU 19. shape from polarization Sec. 12.5, p. 22

DC & CV Lab. CSIE NTU 20. shape from shading

DC & CV Lab. CSIE NTU 20. shape from shading

DC & CV Lab. CSIE NTU 21. shape from texture

DC & CV Lab. CSIE NTU 21. shape from texture

DC & CV Lab. CSIE NTU 22. solving correspondence problem or optic flow field

DC & CV Lab. CSIE NTU 23. motion and shape parameter recovery

DC & CV Lab. CSIE NTU 24. segmentation of newspaper, documents into title, figure, caption, …

DC & CV Lab. CSIE NTU 25. optical distortion correction

DC & CV Lab. CSIE NTU JOKE

DC & CV Lab. CSIE NTU Term Project

DC & CV Lab. CSIE NTU 26. line labeling of 2D line drawing of 3D objects

DC & CV Lab. CSIE NTU 27. Computer Tomography

DC & CV Lab. CSIE NTU 27. Computer Tomography

DC & CV Lab. CSIE NTU 29. X Ray diagnostic

DC & CV Lab. CSIE NTU 30. finger-print validation

DC & CV Lab. CSIE NTU 31. face recognition (intensity image, range image)

DC & CV Lab. CSIE NTU 33. digital morphing

DC & CV Lab. CSIE NTU 33. digital morphing

DC & CV Lab. CSIE NTU Term Project

DC & CV Lab. CSIE NTU 39. Printed music sheet recognition and translation into MIDI (Musical Instrument Digital Interface) format file

DC & CV Lab. CSIE NTU 40. wafer defect inspection

DC & CV Lab. CSIE NTU 41. wafer critical dimension measurement

DC & CV Lab. CSIE NTU 42. IC pin inspection

DC & CV Lab. CSIE NTU 43. IC mark printing inspection

DC & CV Lab. CSIE NTU Term Project

DC & CV Lab. CSIE NTU JOKE

DC & CV Lab. CSIE NTU END