1. Digital Camera and Computer Vision Laboratory Department of Computer Science and Information Engineering National Taiwan University, Taipei, Taiwan, R.O.C. Computer and Robot Vision I Chapter 13 Perspective Projection Geometry Presented by: 傅楸善 & 張博思 0911 246 313 r94922093@ntu.edu.tw 指導教授 : 傅楸善 博士

2. DC & CV Lab. CSIE NTU 13.1 Introduction Computer vision problems often involve interpreting the information on a two- dimensional (2D) image of a three-dimensional (3D) world in order to determine the placement of the 3D objects portrayed in the image. To do this requires understanding the perspective transformation governing the geometric way 3D information is projected onto the 2D image.

3. DC & CV Lab. CSIE NTU 13.1 Introduction image formation on the retina, according to Descartes scrape ox eye, observe from darkened room inverted image of scene

4. DC & CV Lab. CSIE NTU Nalwa, Scrape ox eye, observe from darkened room inverted image of scene

5. DC & CV Lab. CSIE NTU 13.2 One-Dimensional Perspective Projection f: focal length of lens u: distance between object and lens center v: distance between image and lens center thin-lens equation: lens law: 1/f=1/u+1/v light passing lens center dose not deflect light parallel to optical axis will pass focus

6. DC & CV Lab. CSIE NTU

7. DC & CV Lab. CSIE NTU

8. DC & CV Lab. CSIE NTU

9. DC & CV Lab. CSIE NTU pinhole camera: infinitesimally small aperture pinhole camera: approximated by lens with aperture adjusted to the smallest pinhole camera: simplest device to form image of 3D scene on 2D surface

10. DC & CV Lab. CSIE NTU 13.2 One-Dimensional Perspective Projection aperture size decreased: image become sharper diameter of aperture is 0.06 inch, 0.015 inch, 0.0025 inch aperture below certain size: diffraction: bending of light rays around edge

11. DC & CV Lab. CSIE NTU

12. DC & CV Lab. CSIE NTU JOKE

13. DC & CV Lab. CSIE NTU 13.2 One-Dimensional Perspective Projection

14. DC & CV Lab. CSIE NTU 13.2 One-Dimensional Perspective Projection f: camera constant (different from above equation) (r, s, 1): homogeneous coordinate system for point (r, s) first linear transformation: translates (r, s, 1) by distance of f

15. DC & CV Lab. CSIE NTU 13.2 One-Dimensional Perspective Projection second linear transformation: takes perspective transformation to image line 1D image line coordinate:

16. DC & CV Lab. CSIE NTU 13.2 One-Dimensional Perspective Projection (Xp,Yp) (X, Y) X Y

17. DC & CV Lab. CSIE NTU 13.2 One-Dimensional Perspective Projection (Xp,Yp) (X, Y) X Y

18. DC & CV Lab. CSIE NTU 13.2 One-Dimensional Perspective Projection lens: at origin and looks down - axis image line: distance f in front of lens and parallel to -axis : the x - y axes rotated anticlockwise by angle

19. DC & CV Lab. CSIE NTU 13.2 One-Dimensional Perspective Projection

20. DC & CV Lab. CSIE NTU 13.2 One-Dimensional Perspective Projection rewriting the relationship in terms of homogeneous coordinate system

21. DC & CV Lab. CSIE NTU 13.3 The Perspective Projection in 3D camera lens: along line parallel to z-axis position of lens: center of perspectivity: (u, v): coordinates of perspective projection of (x, y, z) on image plane

22. DC & CV Lab. CSIE NTU JOKE

23. DC & CV Lab. CSIE NTU 13.3.1 Smaller Appearance of Farther Objects without loss of generality: take center of perspectivity to be origin perspective projection: objects appear smaller the farther they are

24. DC & CV Lab. CSIE NTU

25. DC & CV Lab. CSIE NTU foreshortening: line segments in plane parallel to image has maximum size

26. DC & CV Lab. CSIE NTU 13.3.2 Lines to Lines lines in 3D world transform to lines in the image plane parallel lines in 3D with nonzero z slope: meet in a vanishing point

27. DC & CV Lab. CSIE NTU

28. DC & CV Lab. CSIE NTU 13.3.3 Perspective Projection of Convex Polyhedra are Convex Proofs in textbook, simple but tedious, study as exercise by yourself

29. DC & CV Lab. CSIE NTU 13.3.4 Vanishing Point Perspective projections of parallel 3D lines having nonzero slope along the optic z-axis meet in a vanishing point on the image projection plane.

30. DC & CV Lab. CSIE NTU 13.3.5 Vanishing Line

31. DC & CV Lab. CSIE NTU

32. DC & CV Lab. CSIE NTU 13.3.6 3D Lines-2D perspective Projection Lines There is a relationship between the parameters of a 3D line and the parameters of the perspective projection of the line.

33. DC & CV Lab. CSIE NTU

34. DC & CV Lab. CSIE NTU

35. DC & CV Lab. CSIE NTU JOKE

36. DC & CV Lab. CSIE NTU 13.4 2D to 3D Inference Using Perspective Projection perspective projection on unknown 3D line: provides four of six constraints additional constraints: 3D-world-model information about points, lines

37. DC & CV Lab. CSIE NTU 13.4.1 Inverse Perspective Projection : perspective projection of a point f: image plane distance from camera lens thus : 3D coordinate of the point in image plane camera lens: at the origin line L: inverse perspective projection of the point

38. DC & CV Lab. CSIE NTU 13.4.2 Line Segment with Known Direction Cosines and Known Length known: : line segment length : line segment direction cosine, : perspective projections of endpoints unknown:, : 3D coordinates of endpoints

39. DC & CV Lab. CSIE NTU 13.4.2 Line Segment with Known Direction Cosines and Known Length

40. DC & CV Lab. CSIE NTU 13.4.2 Line Segment with Known Direction Cosines and Known Length

41. DC & CV Lab. CSIE NTU 13.4.3 Collinear Points with Known Interpoint Distances known: : perspective projection of nth collinear points, n = 0, …, N - 1 distance between (n+1)th point and first point unknown: : direction cosine of line, : 3D coordinates of points

42. DC & CV Lab. CSIE NTU 13.4.3 Collinear Points with known Interpoint Distances

43. DC & CV Lab. CSIE NTU 13.4.4 N Parallel Lines known: : perspective projection of nth parallel line unknown: : direction cosine of line

44. DC & CV Lab. CSIE NTU 13.4.4 N Parallel Lines

45. DC & CV Lab. CSIE NTU 13.4.5 N Lines Intersecting at a Point with Known Angles known: : perspective projection of intersecting point : perspective projection of nth intersecting line : known angle between and unknown: : 3D nth intersecting line

46. DC & CV Lab. CSIE NTU 13.4.6 N Lines Intersecting in a Known Plane known: : perspective projection of intersecting point : perspective projection of nth intersecting line : plane equation unknown: : 3D nth intersecting line

47. DC & CV Lab. CSIE NTU 13.4.6 N Lines Intersecting in a Known Plane

48. DC & CV Lab. CSIE NTU 13.4.7 Three Lines in a Plane with One Perpendicular to the Other Two known: : perspective projection of line unknown: three lines in same plane, perpendicular to, : perspective projection of line : since is perpendicular to,

49. DC & CV Lab. CSIE NTU

50. DC & CV Lab. CSIE NTU 13.4.8 Point with Given Distance to a Known Point known: : perspective projection of unknown point : known 3D points : distance between the two points unknown: : direction cosine between two points Inverse perspective projection: u v f

51. DC & CV Lab. CSIE NTU

52. DC & CV Lab. CSIE NTU 13.4.9 Point in a Known Plane known: : perspective projection of unknown point : known plane equation where point lies unknown: : 3D coordinate of the point:

53. DC & CV Lab. CSIE NTU 13.4.9 Point in a Known Plane

54. DC & CV Lab. CSIE NTU 13.4.10 Line in a Known Plane known: : known plane equation where line lies : perspective projection of line unknown: : 3D line A*i + B*j + C*k = 0,

55. DC & CV Lab. CSIE NTU

56. DC & CV Lab. CSIE NTU JOKE

57. DC & CV Lab. CSIE NTU 13.4.11 Angle known: : perspective projection of the unknown line : direction cosine for the known line : angle between the 3D lines unknown: : direction cosine for the unknown line =.

58. DC & CV Lab. CSIE NTU 13.4.11 Angle

59. DC & CV Lab. CSIE NTU 13.4.12 Parallelogram known: perspective projection of four corner points of a parallelogram unknown: : normal to the plane on which the parallelogram lies, : direction cosines of two sides of parallelogram,

60. DC & CV Lab. CSIE NTU 13.4.13 Triangle with One Vertex Known known:,, : perspective projection of three vertices : one known 3D vertex of the three vertices, : known lengths of the triangle in 3D unknown:, : two unknown 3D vertices of the three vertices

61. DC & CV Lab. CSIE NTU 13.4.13 Triangle with One Vertex Known

62. DC & CV Lab. CSIE NTU 13.4.14 Triangle with Orientation of One Leg Known known:,, : perspective projection of three vertices : known direction cosines between the first two vertices, : known lengths of the triangle in 3D unknown:,, : three unknown 3D vertices

63. DC & CV Lab. CSIE NTU

64. DC & CV Lab. CSIE NTU

65. DC & CV Lab. CSIE NTU

66. DC & CV Lab. CSIE NTU 13.4.15 Triangle: three-point spatial resection problem in photogrammetry known:,, : perspective projection of three vertices, : known lengths of the triangle in 3D unknown:,, : three unknown 3D vertices four solutions =, =, =

67. DC & CV Lab. CSIE NTU

68. DC & CV Lab. CSIE NTU 13.4.16 Determining the Principal Point by Using Parallel Lines principle point: point through which the optic axis passes principle point: so far assumes origin of image reference frame known:, n = 1, …, N: perspective projection of nth parallel line unknown: : coordinate of the principal point

69. DC & CV Lab. CSIE NTU

70. DC & CV Lab. CSIE NTU 13.15 Circles known: perspective projection of a circle having known radius unknown: plane on which the circle lies the 3D center of the circle:

71. DC & CV Lab. CSIE NTU JOKE

72. DC & CV Lab. CSIE NTU 13.6 Range from Structured Light structured light: active visual sensing technique upon perspective geometry structured light: controlled light source with regular pattern onto scene regular pattern: stripes, grid, …

73. DC & CV Lab. CSIE NTU

74. DC & CV Lab. CSIE NTU intensity and range images

75. DC & CV Lab. CSIE NTU 13.6 Range from Structured Light Two light sources with cylindrical lenses produce sheets of light that intersect in a line lying on the surface of a conveyor belt. A camera above the belt is aimed so that this line is imaged on a linear array of photo sensors. When there is no object present, all the sensor cells are brightly illuminated. When part of an object interrupts the incident light, the corresponding region on the linear array is darkened. The motion of the belt scans the object past the sensor, generating the second image dimension.

76. DC & CV Lab. CSIE NTU

77. DC & CV Lab. CSIE NTU

78. DC & CV Lab. CSIE NTU

79. DC & CV Lab. CSIE NTU 13.7 Cross-Ratio cross-ratio: of perspective projection of 4 collinear points, takes same value

80. DC & CV Lab. CSIE NTU 13.7.1 Cross-Ratio Definitions and Invariance four collinear points: q, r: centers of perspectivity for two projection images

81. DC & CV Lab. CSIE NTU 13.7.1 Cross-Ratio Definitions and Invariance Let,. by perspective projection equations, cross-ratio: cross-ratio: independent of reference frame, point p, direction cosine b cross-ratio: depends only on directed of collinear points

82. DC & CV Lab. CSIE NTU 13.7.2 Only One Cross-Ratio each of 4! Cross-ratios is a function of cross-ratio

83. DC & CV Lab. CSIE NTU 13.7.3 Cross–Ratio in Three Dimensions The cross-ratio derived from one-dimensional perspective projections in a two-dimensional world can be generalized to two-dimensional perspective projection in a three-dimensional world. five co-planar points : cross-ratio for the line segment and

84. DC & CV Lab. CSIE NTU 13.7.3 Cross–Ratio in Three Dimensions

85. DC & CV Lab. CSIE NTU

86. DC & CV Lab. CSIE NTU 13.7.4 Using Cross-Ratios cross-ratio: to aid in establishing correspondences

87. DC & CV Lab. CSIE NTU END

88. DC & CV Lab. CSIE NTU JOKE

89. DC & CV Lab. CSIE NTU 13.7.4 Using Cross-Ratios cross-ratio: to aid in establishing correspondences Term Project JPEG 2000

90. DC & CV Lab. CSIE NTU 1. Neural Network

91. DC & CV Lab. CSIE NTU 3. Image Compression JPEG, MPEG, H. 264

92. DC & CV Lab. CSIE NTU 5. segmentation based on texture

93. DC & CV Lab. CSIE NTU 6. optical character reading

94. DC & CV Lab. CSIE NTU 7. stereo vision

95. DC & CV Lab. CSIE NTU 7. stereo vision

96. DC & CV Lab. CSIE NTU 8. Handwriting recognition

97. DC & CV Lab. CSIE NTU 8. Handwriting recognition

98. DC & CV Lab. CSIE NTU 9. histogram specification

99. DC & CV Lab. CSIE NTU Term Project

100. DC & CV Lab. CSIE NTU 10. homomorphic filtering

101. DC & CV Lab. CSIE NTU 10. homomorphic filtering

102. DC & CV Lab. CSIE NTU 12. calculating the sizes of stones, cells, cell nucleus.

103. DC & CV Lab. CSIE NTU 13. trademark resemblance, semi-automatic similarity classification

104. DC & CV Lab. CSIE NTU 15. structured light 3-D reconstruction

105. DC & CV Lab. CSIE NTU 15. structured light 3-D reconstruction

106. DC & CV Lab. CSIE NTU 16. object classification with moments invariant to rotation, scaling, translation

107. DC & CV Lab. CSIE NTU 17. photometric stereo

108. DC & CV Lab. CSIE NTU 18. shape from focus

109. DC & CV Lab. CSIE NTU 19. shape from polarization Sec. 12.5, p. 22

110. DC & CV Lab. CSIE NTU 20. shape from shading

111. DC & CV Lab. CSIE NTU 20. shape from shading

112. DC & CV Lab. CSIE NTU 21. shape from texture

113. DC & CV Lab. CSIE NTU 21. shape from texture

114. DC & CV Lab. CSIE NTU 22. solving correspondence problem or optic flow field

115. DC & CV Lab. CSIE NTU 23. motion and shape parameter recovery

116. DC & CV Lab. CSIE NTU 24. segmentation of newspaper, documents into title, figure, caption, …

117. DC & CV Lab. CSIE NTU 25. optical distortion correction

118. DC & CV Lab. CSIE NTU JOKE

119. DC & CV Lab. CSIE NTU Term Project

120. DC & CV Lab. CSIE NTU 26. line labeling of 2D line drawing of 3D objects

121. DC & CV Lab. CSIE NTU 27. Computer Tomography

122. DC & CV Lab. CSIE NTU 27. Computer Tomography

123. DC & CV Lab. CSIE NTU 29. X Ray diagnostic

124. DC & CV Lab. CSIE NTU 30. finger-print validation

125. DC & CV Lab. CSIE NTU 31. face recognition (intensity image, range image)

126. DC & CV Lab. CSIE NTU 33. digital morphing

127. DC & CV Lab. CSIE NTU 33. digital morphing

128. DC & CV Lab. CSIE NTU Term Project

129. DC & CV Lab. CSIE NTU 39. Printed music sheet recognition and translation into MIDI (Musical Instrument Digital Interface) format file

130. DC & CV Lab. CSIE NTU 40. wafer defect inspection

131. DC & CV Lab. CSIE NTU 41. wafer critical dimension measurement

132. DC & CV Lab. CSIE NTU 42. IC pin inspection

133. DC & CV Lab. CSIE NTU 43. IC mark printing inspection

134. DC & CV Lab. CSIE NTU Term Project

135. DC & CV Lab. CSIE NTU JOKE

136. DC & CV Lab. CSIE NTU END