2. Digital Image Processing Additional Material : Imaging Geometry 11 September 2006 Digital Image Processing Additional Material : Imaging Geometry 11 September 2006

3. Imaging Geometry: Perspective Transformation Lens center x,X y,Y (x,y)(x,y) z,Z (X,Y,Z) = world coordinate (x,y,z) = Camera coordinate system (X,Y,Z) = focal length Image plane Eq. 1.1

4. Imaging Geometry: Perspective Transformation (cont.) Question: How can we project the real world object at (X,Y,Z) onto the image plane (such as photographic film)? Answer: Relation between camera coordinate (x,y,z) and world coordinate (X,Y,Z) are given by Since on the image plane z is always zero, z=0, we consider only (x,y) while z is neglected. Eq. 1.2

5. Imaging Geometry: Perspective Transformation (cont.) Equation 1.2 is not linear because of Z in the dividers so we introduce the homogeneous coordinate to solve this problem. Cartesian coordinate Homogeneous coordinate k = nonzero constant To convert from the homogeneous coordinate w h to the Cartesian coordinate w, we divide the first 3 components of w h by the fourth component.

6. Imaging Geometry: Perspective Transformation (cont.) The perspective transformation matrix for the homogeneous coordinate: Perspective transformation becomes: Eq. 1.3

7. Imaging Geometry: Perspective Transformation (cont.) From homogeneous coordinate We get camera coordinate in the image plane:

8. Imaging Geometry: Inverse Perspective Transformation where Eq. 1.4

9. For an image point (x 0,y 0 ), since on the image plane z=0, we have Inverse Perspective Transformation (cont.) We get the world coordinate : Since the perspective transformation maps 3-D coordinates to 2-D Coordinates, we cannot get the inverse transform unless we have additional information.

10. To find the solution, let Inverse Perspective Transformation (cont.) We get Eq. 1.5

11. Inverse Perspective Transformation (cont.) From Eq. 1.5, We get Eq. 1.6 Substituting Eq. 1.6 into Eq.1.5, we get Eq. 1.7 Equations 1.7 show that inverse perspective transformation requires information of at least one component of the world coordinate of the point.

12. Inverse Perspective Transformation (cont.) These equations: show that points on Line L in the world coordinate map to a single point in the image plane. Lens center x,X y,Y z,Z (x,y)(x,y) Image plane (X 2,Y 2,Z 2 ) (X 3,Y 3,Z 3 ) Points (X 1,Y 1,Z 1 ), (X 2,Y 2,Z 2 ), and (X 3,Y 3,Z 3 ) map to Point (x,y) in the image plane. (X 1,Y 1,Z 1 ) Line L

13. y x Stereo Imaging: How we get depth information from 2 eyes Left lens center (x2,y2)(x2,y2) Image 2 y x right lens center B World point (X,Y,Z) Optical axis Image 1 (x1,y1)(x1,y1)

14. Stereo Imaging: How we get depth information from 2 eyes (cont.) Problem: we know camera coordinates of the object on left and right image planes (x 1,y 1 ) and (x 2,y 2 ) and want to how far from the camera the object is located. Note: when y-axis is parallel to the ground, we have y 1 = y 2 (x2,y2)(x2,y2) (x1,y1)(x1,y1) B w Z Image 2 Image 1 Plane of constant Z X Origin of world coordinate system

15. 1. From the inverse perspective transform, we compute X 1 and X 2 : 3. Since left and right lenses are separated by distance B, we have 2. Z 1 and Z 2 must be equal, we get Stereo Imaging: How we get depth information from 2 eyes (cont.) 4. From 1, 2 and 3, we get Solving Z yields

16. Stereo Imaging: How we get depth information from 2 eyes (cont.) We can locate the object if we know positions of the object in left and right image planes using Equation: Question: While the equation is so simple but why it is very difficult to built an automatic stereo vision system that can reconstruct 3-D scene from images obtained from 2 cameras? Answers: for a computer, locating the corresponding points on left and right images is the most difficult task.

17. Imaging Geometry : Affine Transformations 1. Translation 2. Scaling 3. Rotating

18. Image Geometry: Translation of Object Y Z X Displace the object by vector (X 0,Y 0,Z 0 ) with respect to its old position. (X+X0,Y+Y0,Z+Z0)(X+X0,Y+Y0,Z+Z0) (X,Y,Z)(X,Y,Z) (X0,Y0,Z0)(X0,Y0,Z0)

19. Image Geometry: Translation of Frame Y Z X (0,0,0) Translate the origin point of the frame by (X 0,Y 0,Z 0 ) with respect to the old frame Y  ZZ X  (X0,Y0,Z0)(X0,Y0,Z0) The object still stays at the same position. Only the frame is moved.

20. Image Geometry: Scaling Scale by factors S x, S y, S z along X, Y, and Z axes. a b c d e f g A B C DE F G Note: Origin point is unchanged.

21.  Image Geometry: Rotating an object about X-axis Rotate an object about X-axis by  x in a counterclockwise direction. Y Z X=X  xx Note : In this case the object is moved. Only y and z are changed while x stills the same.

22.  Image Geometry: Rotating a frame about X-axis Rotate the frame about X-axis by  x in a counterclockwise direction. Y Z xx Y  ZZ Note : In this case the object is not moved. The frame is rotated instead. X=X 

23. Image Geometry: Rotating an object about Y-axis Rotate an object about Y-axis by  y in a counterclockwise direction. Z X yy Note : In this case the object is moved. Only x and z are changed while y stills the same.  Y= Y 

24. Image Geometry: Rotating a frame about Y-axis Rotate the frame about Y-axis by  y in a counterclockwise direction. Y= Y  Z X yy Note : In this case the object is not moved. The frame is rotated instead.  ZZ X 

25. Image Geometry: Rotating an object about Z-axis Rotate an object about Z-axis by  z in a counterclockwise direction. Y Z X zz Note : In this case the object is moved. Only x and y are changed while z stills the same. 

26. Image Geometry: Rotating a frame about Z-axis Rotate the frame about Z-axis by  z in a counterclockwise direction. Y Z= Z  X zz Y  Note : In this case the object is not moved. The frame is rotated instead. X  

27. X Y Z World Coordinate System How to compute a point on an image plane from the world coordinate Problem: we know the location of the object and want to know where it will be projected on the film (image plane). y z x Camera Coordinate System Answer: 1. Transform the world coordinate to the camera coordinate 2. Perform the perspective transformation

28. Lens center x y (x,y)(x,y) z (x,y,z) = Camera coordinate, (X,Y,Z) = World coordinate Image plane (z=0) Before using the perspective transformation, the world axes X-Y-Z must coincide with the camera axes x-y-z, (we need some transformations). (X,Y,Z) X Z Y How to compute a point on an image plane from the world coordinate (cont.)

29. Compute the camera coordinate fromthe world coordinate X Y Z World Coordinate System y z x Camera Coordinate System 2 3 Gimbal center 1. Translate* by w 0 Note : *perform on the frame 2. Pan the camera* (rotate about Z-axis) 4. Translate* by z = r Steps from Gonzalez’s book 3. Tilt the camera* (rotate about X-axis) 5. Compute the perspective Tr. 1 w0w0 4 r

30. Formula from Gonzalez’s book Perspective tr. Translate to the image plane Center by z = r Translate to the gimbal center w 0 World coordinate Camera coordinate Compute the camera coordinate from the world coordinate (cont.) Rotate by Pan (  z ) and Tilt (  x )

31. X Y Z World Coordinate System y z x Camera Coordinate System 2 3 4 Gimbal center 1. Translate* by w 1 2. Pan the camera* 3. Tilt the camera* Note : *perform on the frame General case 4. Twist the camera* (rotate about Y-axis) 5. Compute the perspective Tr. w1w1 1 Compute the camera coordinate from the world coordinate (cont.)

32. General case Perspective tr. Translate to the image plane center w 1 World coordinate Camera coordinate Rotate by Pan (  z ), Tilt (  x ), Twist (  y ) Compute the camera coordinate from the world coordinate (cont.)