Image Formation: Optics and Imagers Real world Optics Sensor Acknowledgment: some figures by B. Curless, E. Hecht, W.J. Smith, B.K.P. Horn, and A. Theuwissen

Optics Pinhole cameraPinhole camera LensesLenses Focus, aperture, distortionFocus, aperture, distortion

Pinhole Object Pinhole camera Pinhole Camera “Camera obscura” – known since antiquity“Camera obscura” – known since antiquity Image Image plane

Pinhole Camera “Camera obscura” – known since antiquity“Camera obscura” – known since antiquity First recording in 1826 onto a pewter plate (by Joseph Nicéphore Niepce)First recording in 1826 onto a pewter plate (by Joseph Nicéphore Niepce) Pinhole Object Image Pinhole camera Image plane

Pinhole Camera Limitations Aperture too big: blurry imageAperture too big: blurry image Aperture too small: requires long exposure or high intensityAperture too small: requires long exposure or high intensity Aperture much too small: diffraction through pinhole  blurry imageAperture much too small: diffraction through pinhole  blurry image

Lenses Focus a bundle of rays from a scene point onto a single point on the imagerFocus a bundle of rays from a scene point onto a single point on the imager Result: can make aperture biggerResult: can make aperture bigger

Ideal Lenses Thin-lens approximationThin-lens approximation Gaussian lens law:Gaussian lens law: Real lenses and systems of lenses may be approximated by thin lenses if only paraxial rays (near the optical axis) are consideredReal lenses and systems of lenses may be approximated by thin lenses if only paraxial rays (near the optical axis) are considered 1/d o + 1/d i = 1/f

Camera Adjustments Focus?Focus? – Changes d i Zoom?Zoom? – Changes f Iris?Iris? – Changes aperture

Focus and Depth of Field For a given d i, “perfect” focus at only one d oFor a given d i, “perfect” focus at only one d o In practice, focus is OK for some range of depthsIn practice, focus is OK for some range of depths – Circle of confusion smaller than a pixel Better depth of field with smaller aperturesBetter depth of field with smaller apertures – Better approximation to pinhole camera

Field of View Q: What does field of view of camera depend on?Q: What does field of view of camera depend on? – Focal length of lens – Size of imager – Object distance?

Computing Field of View xoxoxoxo didididi xixixixi dodododo 1/d o + 1/d i = 1/f  x o / d o = x i / d i  = 2 tan -1 ½ x i (1/f  1/d o ) Since typically d o >> f,   2 tan -1 ½ x i / f tan  /2 = ½ x o / d o   x i / f

Aperture Controls amount of lightControls amount of light Affects depth of fieldAffects depth of field Affects distortion (since thin-lens approximation is better near center of lens)Affects distortion (since thin-lens approximation is better near center of lens)

Aperture Aperture typically given as “f-number” (also “f-stops” or just “stops”)Aperture typically given as “f-number” (also “f-stops” or just “stops”) What is f /4?What is f /4? – Aperture is ¼ the focal length

Monochromatic Aberrations Real lenses do not follow thin lens approximation because surfaces are spherical (manufacturing constraints)Real lenses do not follow thin lens approximation because surfaces are spherical (manufacturing constraints) Result: thin-lens approximation only valid iff sin   Result: thin-lens approximation only valid iff sin   

Distortion Pincushion or barrel radial distortionPincushion or barrel radial distortion Varies with placement of apertureVaries with placement of aperture

Distortion

Distortion

Distortion

First-Order Radial Distortion Goal: mathematical formula for distortionGoal: mathematical formula for distortion If distortion is small, can be approximated by “first-order” formula:If distortion is small, can be approximated by “first-order” formula: Why (1 +  r 2 ) and not (1 +  r)?Why (1 +  r 2 ) and not (1 +  r)? r’ = r (1 +  r 2 ) r = ideal distance to center of image r’ = distorted distance to center of image

Chromatic Aberration Due to dispersion in glass (focal length varies with the wavelength of light)Due to dispersion in glass (focal length varies with the wavelength of light) Result: color fringes near edges of imageResult: color fringes near edges of image Correct by building lens systems with multiple kinds of glassCorrect by building lens systems with multiple kinds of glass

Correcting for Aberrations High-quality compound lenses use multiple lens elements to “cancel out” distortion and aberrationHigh-quality compound lenses use multiple lens elements to “cancel out” distortion and aberration Often 5-10 elements, more for extreme wide angleOften 5-10 elements, more for extreme wide angle

Sensors FilmFilm VidiconVidicon CCDCCD CMOSCMOS

Vidicon Best-known in family of “photoconductive video cameras”Best-known in family of “photoconductive video cameras” Basically television in reverseBasically television in reverse Electron Gun Photoconductive Plate Lens System + +        Scanning Electron Beam

MOS Capacitors MOS = Metal Oxide SemiconductorMOS = Metal Oxide Semiconductor Gate (wire) SiO 2 (insulator) p-type silicon

MOS Capacitors Voltage applied to gate repels positive “holes” in the semiconductorVoltage applied to gate repels positive “holes” in the semiconductor +10V Depletion region (electron “bucket”)

MOS Capacitors Photon striking the material creates electron-hole pairPhoton striking the material creates electron-hole pair +10V Photon   + +            

Charge Transfer Can move charge from one bucket to another by manipulating voltagesCan move charge from one bucket to another by manipulating voltages

CCD Architectures Linear arraysLinear arrays 2D arrays2D arrays

Linear CCD Accumulate photons, then clock them outAccumulate photons, then clock them out To prevent smear: first move charge to opaque region, then clock it outTo prevent smear: first move charge to opaque region, then clock it out

Full-Frame CCD Other arrangements to minimize smearOther arrangements to minimize smear

CMOS Imagers Recently, can manufacture chips that combine photosensitive elements and processing elementsRecently, can manufacture chips that combine photosensitive elements and processing elements Benefits:Benefits: – Partial readout – Signal processing – Eliminate some supporting chips  low cost

Color 3-chip vs. 1-chip: quality vs. cost3-chip vs. 1-chip: quality vs. cost

Video Depending on the scene, pictures updated at 15–70 Hz. perceived as “continuous”Depending on the scene, pictures updated at 15–70 Hz. perceived as “continuous” Most video cameras use a shutter, so they are capturing for only part of a frameMost video cameras use a shutter, so they are capturing for only part of a frame – Short shutter: less light, have to open aperture – Long shutter: more light, but motion blur Television uses interlaced videoTelevision uses interlaced video

Interlacing These rows transmittedfirst transmitted 1/60 sec later

Television US: NTSC standardUS: NTSC standard – Fields are 1/60 sec. – 2 fields = 1 frame  frames are 1/30 sec. – Each frame has 525 scanlines, of which approximately 480 are visible – No discrete pixels along scanlines, but if pixels were square, there would be about 640 visible

Television NTSC standardNTSC standard – Thus, an NTSC frame is about 640  480 – Color at lower resolution than intensity PAL standardPAL standard – Lower rate: fields at 50 Hz. (frames at 25 Hz.) – Higher resolution: about 768  576