1. CS 691 Computational Photography Instructor: Gianfranco Doretto 3D to 2D Projections

2. Pinhole camera model Pinhole model: –Captures pencil of rays – all rays through a single point –The point is called Center of Projection (COP) –The image is formed on the Image Plane –Effective focal length f is distance from COP to Image Plane

3. Dimensionality Reduction Machine (3D to 2D) 3D world2D image What have we lost? Angles Distances (lengths)

4. Projection can be tricky… Slide source: Seitz

5. Projection can be tricky… Slide source: Seitz

6. Projective Geometry What is lost? Length Which is closer? Who is taller?

7. Lengths can’t be trusted... B’ C’ A’

8. …but humans adopt! http://www.michaelbach.de/ot/sze_muelue/index.html Müller-Lyer Illusion We don’t make measurements in the image plane

9. Projective Geometry What is lost? Length Angles Perpendicular? Parallel?

10. Parallel lines aren’t…

11. Projective Geometry What is preserved? Straight lines are still straight

12. Vanishing points and lines Parallel lines in the world intersect in the image at a “vanishing point”

13. Vanishing points and lines o Vanishing Point o Vanishing Line

14. Vanishing points and lines Vanishing point Vanishing line Vanishing point Vertical vanishing point (at infinity) Credit: Criminisi

15. Vanishing points and lines Photo from online Tate collection

16. Note on estimating vanishing points Use multiple lines for better accuracy … but lines will not intersect at exactly the same point in practice One solution: take mean of intersecting pairs … bad idea! Instead, minimize angular differences

17. Vanishing objects

18. Modeling projection The coordinate system –We will use the pin-hole model as an approximation –Put the optical center (Center Of Projection) at the origin –Put the image plane (Projection Plane) in front of the COP Why? –The camera looks down the negative z axis we need this if we want right-handed-coordinates –

19. Modeling projection Projection equations –Compute intersection with PP of ray from (x,y,z) to COP –Derived using similar triangles (on board) We get the projection by throwing out the last coordinate:

20. Homogeneous coordinates Is this a linear transformation? Trick: add one more coordinate: homogeneous image coordinates homogeneous scene coordinates Converting from homogeneous coordinates no—division by z is nonlinear

21. Homogeneous coordinates Invariant to scaling Point in Cartesian is ray in Homogeneous Homogeneous Coordinates Cartesian Coordinates

22. Perspective Projection Projection is a matrix multiply using homogeneous coordinates: divide by third coordinate This is known as perspective projection The matrix is the projection matrix Can also formulate as a 4x4 divide by fourth coordinate

23. Orthographic Projection Special case of perspective projection –Distance from the COP to the PP is infinite –Also called “parallel projection” –What’s the projection matrix? Image World

24. Scaled Orthographic Projection Special case of perspective projection –Object dimensions are small compared to distance to camera –Also called “weak perspective” –What’s the projection matrix?

25. Spherical Projection What if PP is spherical with center at COP? In spherical coordinates, projection is trivial:  d  Note: doesn’t depend on focal length!

26. Projection matrix x: Image Coordinates: w(u,v,1) K: Intrinsic Matrix (3x3) R: Rotation (3x3) t: Translation (3x1) X: World Coordinates: (X,Y,Z,1) OwOw iwiw kwkw jwjw R,T

27. K Projection matrix Intrinsic Assumptions Unit aspect ratio Principal point at (0,0) No skew Extrinsic Assumptions No rotation Camera at (0,0,0)

28. Remove assumption: known optical center Intrinsic Assumptions Unit aspect ratio No skew Extrinsic Assumptions No rotation Camera at (0,0,0)

29. Remove assumption: square pixels Intrinsic Assumptions No skew Extrinsic Assumptions No rotation Camera at (0,0,0)

30. Remove assumption: non- skewed pixels Intrinsic AssumptionsExtrinsic Assumptions No rotation Camera at (0,0,0) Note: different books use different notation for parameters

31. Oriented and Translated Camera OwOw iwiw kwkw jwjw t R

32. Allow camera translation Intrinsic AssumptionsExtrinsic Assumptions No rotation

33. 3D Rotation of Points, : Rotation around the coordinate axes, counter-clockwise: p p’p’p’p’  y z Slide Credit: Saverese

34. Allow camera rotation

35. Degrees of freedom 56

36. Vanishing Point = Projection from Infinity

37. Lens Flaws

38. Lens Flaws: Chromatic Aberration Dispersion: wavelength-dependent refractive index –(enables prism to spread white light beam into rainbow) Modifies ray-bending and lens focal length: f(λ) color fringes near edges of image Corrections: add ‘doublet’ lens of flint glass, etc.

39. Chromatic Aberration Near Lens Center Near Lens Outer Edge

40. Radial Distortion (e.g. ‘Barrel’ and ‘pin-cushion’) straight lines curve around the image center

41. Radial Distortion Radial distortion of the image –Caused by imperfect lenses –Deviations are most noticeable for rays that pass through the edge of the lens No distortionPin cushionBarrel

42. Radial Distortion

43. Slide Credits This set of sides also contains contributions kindly made available by the following authors – Alexei Efros – Stephen E. Palmer – Steve Seitz – Derek Hoiem – David Forsyth – George Bebis – Silvio Savarese