1. Image Formation: Optics and Imagers
Real world
Optics
Sensor
Acknowledgment: some figures by B. Curless, E. Hecht, W.J. Smith, B.K.P. Horn, and A. Theuwissen

2. Optics
Pinhole camera
Lenses
Focus, aperture, distortion

3. Pinhole Camera “Camera obscura” – known since antiquity Image plane
Object
Pinhole camera
Image

4. Pinhole Camera “Camera obscura” – known since antiquity
First recording in 1826 onto a pewter plate (by Joseph Nicéphore Niepce)
Image plane
Object
Image
Pinhole
Pinhole camera

5. Pinhole Camera Limitations
Aperture too big: blurry image
Aperture too small: requires long exposure or high intensity
Aperture much too small: diffraction through pinhole  blurry image

6. Lenses
Focus a bundle of rays from a scene point onto a single point on the imager
Result: can make aperture bigger

7. Ideal Lenses 1/do + 1/di = 1/f Thin-lens approximation
Gaussian lens law:
Real lenses and systems of lenses may be approximated by thin lenses if only paraxial rays (near the optical axis) are considered
1/do + 1/di = 1/f

8. Camera Adjustments Iris? Focus? Zoom? Changes aperture Changes di
Changes f and sometimes di

9. Zoom Lenses – Varifocal

10. Zoom Lenses – Parfocal

11. Focus and Depth of Field
For a given di, “perfect” focus at only one do
In practice, OK for some range of depths
Circle of confusion smaller than a pixel
Better depth of field with smaller apertures
Better approximation to pinhole camera

12. Field of View Q: What does field of view of camera depend on?
Focal length of lens
Size of imager
Object distance?

13. Computing Field of View
1/do + 1/di = 1/f
tan  /2 = ½ xo / do
xo / do = xi / di xo di xi do 
 = 2 tan-1 ½ xi (1/f1/do)
Since typically do >> f,
  2 tan-1 ½ xi / f
  xi / f

14. Aperture Controls amount of light Affects depth of field
Affects distortion (since thin-lens approximation is better near center of lens)

15. Aperture
Aperture typically given as “f-number” (also “f-stops” or just “stops”)
What is f /4?
Aperture is ¼ the focal length

16. Monochromatic Aberrations
Real lenses do not follow thin lens approximation because surfaces are spherical (manufacturing constraints)
Result: thin-lens approximation only valid iff sin   

17. Monochromatic Aberrations
Consider the next term in the Taylor series, i.e. sin    - 3/3!
“Third-order” theory – deviations from the ideal thin-lens approximations
Called primary or Seidel aberrations

18. Spherical Aberration
Results in blurring of image, focus shifts when aperture is stopped down
Can vary with the way lenses are oriented

19. Coma
Results in changes in magnification with aperture

20. Coma

21. Distortion Pincushion or barrel radial distortion
Varies with placement of aperture

22. Distortion
Varies with placement of aperture

23. Distortion
Varies with placement of aperture

24. Distortion
Varies with placement of aperture

25. First-Order Radial Distortion
Goal: mathematical formula for distortion
If distortion is small, can be approximated by “first-order” formula:
Higher-order models possible
r’ = r (1 +  r2)
r = ideal distance to center of image
r’ = distorted distance to center of image

26. Correcting for Aberrations
Compound lenses use multiple lens elements to “cancel out” aberrations
Lenses of different materials
5-15 elements, more for extreme wide angle

27. Catadioptrics Catadioptric systems use both lenses and mirrors
Motivations:
Systems using parabolic mirrors can be designed to not introduce these aberrations
Easier to make very wide-angle systems with mirrors

28. Wide-Angle Catadioptric System

29. Other Limitations of Lenses
Flare: light reflecting (often multiple times) from glass-air interface
Results in ghost images or haziness
Worse in multi-lens systems
Ameliorated by optical coatings (thin-film interference)

30. Other Limitations of Lenses
Optical vignetting: less power per unit area transferred for light at an oblique angle
Transferred power falls off as cos4 
Result: darkening of edges of image
Mechanical vignetting: due to apertures

31. Sensors
Film
Vidicon
CCD
CMOS

32. Vidicon Best-known in family of “photoconductive video cameras”
Basically television in reverse
   
Scanning Electron Beam
Electron Gun
Lens System
Photoconductive Plate

33. Digression: Gamma
Vidicon tube naturally has signal that varies with light intensity according to a power law: Signal = Eg, g  1/2.5
CRT (televisions) naturally obey a power law with gamma  2.5
Result: standard for video signals has a gamma of 1/2.5

34. MOS Capacitors MOS = Metal Oxide Semiconductor Gate (wire)
SiO2 (insulator)
p-type silicon

35. MOS Capacitors
Voltage applied to gate repels positive “holes” in the semiconductor
+10V
Depletion region
(electron “bucket”)

36. MOS Capacitors Photon striking the material creates electron-hole pair
+10V        +

37. Charge Transfer
Can move charge from one bucket to another by manipulating voltages

38. Charge Transfer
Various schemes (e.g. three-phase-clocking) for transferring a series of charges along a row of buckets

39. CCD Architectures Linear arrays 2D arrays Full frame
Frame transfer (FT)
Interline transfer (IT)
Frame interline transfer (FIT)

40. Linear CCD Accumulate photons, then clock them out
To prevent smear: first move charge to opaque region, then clock it out

41. Full-Frame CCD
Other arrangements to minimize smear

42. Frame Transfer CCD

43. Interline Transfer CCD

44. Frame Interline Transfer CCD

45. CMOS Imagers
Recently, can manufacture chips that combine photosensitive elements and processing elements
Benefits:
Partial readout
Signal processing
Eliminate some supporting chips  low cost

46. Color
3-chip vs. 1-chip: quality vs. cost

47. Chromatic Aberration
Due to dispersion in glass (focal length varies with the wavelength of light)
Result: color fringes near edges of image
Correct by building lens systems with multiple kinds of glass

48. Correcting Chromatic Aberration
Simple way of partially correcting for residual chromatic aberration after the fact: scale R,G,B channels independently

49. Video
Depending on the scene, pictures updated at 15–70 Hz. perceived as “continuous”
Most video cameras use a shutter, so they are capturing for only part of a frame
Short shutter: less light, have to open aperture
Long shutter: more light, but motion blur
Television uses interlaced video

50. Interlacing These rows transmitted first These rows transmitted
1/60 sec later

51. Television US: NTSC standard Fields are 1/60 sec.
2 fields = 1 frame  frames are 1/30 sec.
Each frame has 525 scanlines, of which approximately 480 are visible
No discrete pixels along scanlines, but if pixels were square, there would be about 640 visible

52. Television NTSC standard PAL standard
Thus, an NTSC frame is about 640480
Color at lower resolution than intensity
PAL standard
Lower rate: fields at 50 Hz. (frames at 25 Hz.)
Higher resolution: about 768576