1. 1 Light and Color

2. 2/118 Topics The Human Visual System Displaying Intensity and Luminance Display Using Fixed Intensities Understanding Color Display of Color Color Models

3. 3/118 Structure of the Human Eye Image taken from http://hyperphysics.phy- astr.gsu.edu/hbase/vision/eye.html

4. 4/118 Main Parts of the Eye Cornea

5. 5/118 Main Parts of the Eye Cornea  Provides most refraction

6. 6/118 Main Parts of the Eye Cornea Iris

7. 7/118 Main Parts of the Eye Cornea Iris  Opens and Closes to let in more/less light

8. 8/118 Main Parts of the Eye Cornea Iris  Opens and Closes to let in more/less light  Hole is the pupil

9. 9/118 Main Parts of the Eye Cornea Iris Lens

10. 10/118 Main Parts of the Eye Cornea Iris Lens  Flexible - muscles adjust shape

11. 11/118 Main Parts of the Eye Cornea Iris Lens  Flexible - muscles adjust shape  Allows fine-detail focus

12. 12/118 Main Parts of the Eye Cornea Iris Lens Retina

13. 13/118 Main Parts of the Eye Cornea Iris Lens Retina  Layer of receptor cells at back of eye

14. 14/118 Main Parts of the Eye Cornea Iris Lens Retina  Layer of receptor cells at back of eye  Center of focus is the fovea

15. 15/118 Main Parts of the Eye Cornea Iris Lens Retina  Layer of receptor cells at back of eye  Center of focus is the fovea  Optic nerve collects information from retina, and brings to the back of the brain  Causes a blind spot!  Cephalopods (octopus) do not have blind spots

16. 16/118 Focusing Light For a point in focal plane, all light emitting from that point goes to same point on retina

17. 17/118 Focusing Light For a point not in focal plane, light gets spread across retina

18. 18/118 Display Screens and Focus For the usual (current) display systems, we maintain one focus plane  Even for stereo displays  Even if the image tries to simulate differing focus  This is different than nature…

19. 19/118 Rods Distinguish brightness only Best response to blue-green light Prevalent except at fovea (more peripheral) About 100x more sensitive than cones About 100 million in retina

20. 20/118 Cones Color response Centered around fovea  About 147,000 cones/mm 2 at fovea  2mm away from fovea: 9,500 cones/mm 2 6.3 - 6.8 million in retina

21. 21/118 How We See Individual receptors give response Vision is limited by density of cells in part of brain  High detail, good color at center of vision  Peripheral vision can see dimmer light, but mainly black & white, and low resolution

22. 22/118 Vision in the Brain Signals are carried by optic nerve to brain Brain processes to reconstruct image  Best understood part of brain function, but still many ill-understood parts  Many aspects of vision are “hard-wired”  Can lead to optical effects with significant graphics impact Mach banding. Filling in of “blind spot”

23. 23/118 Topics The Human Visual System Displaying Intensity and Luminance Display Using Fixed Intensities Understanding Color Display of Color Color Models

24. 24/118 Intensity and Luminance Intensity/Luminance: How much light energy there is  The amount of energy carried by photons

25. 25/118 Intensity and Luminance Intensity/Luminance: How much light energy there is  The amount of energy carried by photons Brightness: the perceived intensity  Eye does not respond to equal intensity changes equally  Eye notices the ratio of intensities

26. 26/118 Brightness Levels Intensity changes of 1->2, 2->4, 4->8 appear the same  Example: 3-way lightbulb  50->100 seems like bigger change than 100 -> 150  Display devices might limit the number of discrete intensity levels available

27. 27/118 Brightness in Display Devices Assume:  The maximum intensity of a display is 1.0  The minimum is I 0  It can display n+1 intensity levels Then, to get equal brightness increments, it should display levels:  I 0, rI 0, r 2 I 0, …, r n I 0 =1.0

28. 28/118 Brightness Levels Human eye generally can notice r>1.01  So, to have a “smooth” display, we need to make sure that r<1.01  i.e. we need enough “levels” of display  For a given display, we need 1.01 n I 0 =1.0 1.01 n = 1.0/I 0 n log 1.01 = - log I 0 n = -log I 0 / log 1.01 n = -log 1.01 I 0

29. 29/118 Gamma Correction Designed to compensate for how humans perceive intensity of light  I out = k I in   I in, I out = intensity  k,  are device-specific terms  Typically,  = 2.0 to 2.5 (most displays use 2.2) Combining colors/intensity is not linear! Must convert to linear space, combine, convert back

30. 30/118 Original 1080p

31. 31/118 YouTube 360p

32. 32/118 Gamma Corrected 360p

33. 33/118 Original 1080p

34. 34/118 Gamma and non-CRT Displays Gamma is meant to model CRTs only Other displays (e.g. LCD) may have very different response curves between intensity and value sent  They do not match the gamma curve  Often manufacturers adjust responses to try to mimic CRT behavior  Rarely is any device’s response curve exactly what is desired/modeled This can get much more complicated  Major effort just to compensate for gamma

35. 35/118 Dynamic Range 1/I 0 is called dynamic range  The ratio of maximum to minimum intensity (remember, we set 1.0 = max)  Varies by display device  This is the maximum a device can possibly display vs. the minimum it can display.

36. 36/118 Dynamic Range 1/I 0 is called dynamic range  The ratio of maximum to minimum intensity (remember, we set 1.0 = max)  Varies by display device  This is the maximum a device can possibly display vs. the minimum it can display. Contrast: the maximum vs. minimum it can display at the same time.  i.e. for one image on screen, maximum vs. minimum.

37. 37/118 Topics The Human Visual System Displaying Intensity and Luminance Display Using Fixed Intensities Understanding Color Display of Color Color Models

38. 38/118 Limited Display Levels We can’t always get a continuous range of display levels  Printing: either ink is there or not there  We can adjust amount of ink (i.e. how much space it takes), but not its intensity  Other display might limit the number of levels – e.g. 256 levels of intensity. We need ways to mimic continuous colors with discrete levels

39. 39/118 Example

40. 40/118 Example

41. 41/118 Halftoning/Dithering Idea: Eyes integrate over an area  All light hitting one receptor cell is combined.  Eye only cares about the integrated information from an entire area So, we can get varying intensity by filling in fractions of areas vs.

42. 42/118 Halftoning Subdivide image into blocks of pixels.  You will lose resolution!  e.g. a 100x100 region divided into 4x4 blocks of pixels can only display a 25x25 image. The number of intensity levels is related to the number of pixels in a block  2x2 = 5 intensity levels  3x3 = 10 intensity levels  nxn = n 2 +1 intensity levels

43. 43/118 Example: 2x2 block Level 0: Intensities 0.0 – 0.2 Level 1: Intensities 0.2 – 0.4 Level 2: Intensities 0.4 – 0.6 Level 3: Intensities 0.6 – 0.8 Level 4: Intensities 0.8 – 1.0

44. 44/118 Example Image 0.000.201.000.020.500.20 0.150.050.010.050.080.90 0.200.25 0.450.700.60 0.220.250.600.900.500.80 0.300.500.800.901.000.90 0.400.200.61 0.750.95

45. 45/118 Example Image 0.000.201.000.020.500.20 0.150.050.010.050.080.90 0.200.25 0.450.700.60 0.220.250.600.900.500.80 0.300.500.800.901.000.90 0.400.200.61 0.750.95

46. 0.100.270.42 0.230.550.65 0.350.730.90 46/118 Example Image Average Intensities over BlocksHalftone Image

47. 47/118 2x2 Halftone Example

48. 48/118 2x2 Halftone Example

49. 49/118 3x3 Halftone Example

50. 50/118 Halftoning Patterns The pattern you use to fill makes a difference.  Want a “random” pattern, so that artificial artifacts don’t appear  Brain is very good at recognizing some things, like lines

51. 51/118 Example: 3x3 block Bad pattern: Better pattern:.36.42.45.51.36.42.45.51

52. 52/118 3x3 Bad Halftone Example

53. 53/118 3x3 Good Halftone Example

54. 54/118 Dithering Halftoning loses resolution – this is bad. Dithering:  Keep same resolution  Change halftoning pattern into a probability/threshold function (dither pattern)  Overlay dither pattern on entire image  Fill in a pixel iff it is of higher intensity than the dither value

55. 55/118 Dither Pattern Example Intensities 0.0 – 0.2 Intensities 0.2 – 0.4 Intensities 0.4 – 0.6 Intensities 0.6 – 0.8 Intensities 0.8 – 1.0 Halftoning Patterns

56. 56/118 Dither Pattern Example Intensities 0.0 – 0.2 Intensities 0.2 – 0.4 Intensities 0.4 – 0.6 Intensities 0.6 – 0.8 Intensities 0.8 – 1.0 0.20.6 0.80.4 Cells are filled in only when intensity is larger than the given value

57. 57/118 Dither Pattern Example 0.340.450.560.67 0.420.650.780.88 0.210.330.570.77 0.010.220.440.55 0.20.6 0.80.4 Overlay dither pattern

58. 58/118 Dither Pattern Example 0.340.450.560.67 0.420.650.780.88 0.210.330.570.77 0.010.220.440.55 0.20.6 0.80.4 Fill those cells larger than dither value

59. 59/118 Dither Pattern Example 0.340.450.560.67 0.420.650.780.88 0.210.330.570.77 0.010.220.440.55 0.20.6 0.80.4 Fill those cells larger than dither value

60. 60/118 Dither Pattern Example 0.340.450.560.67 0.420.650.780.88 0.210.330.570.77 0.010.220.440.55 0.2 0.80.4 Fill those cells larger than dither value

61. 61/118 Dither Pattern Example 0.340.450.560.67 0.420.650.780.88 0.210.330.570.77 0.010.220.440.55 0.2 0.4 Fill those cells larger than dither value

62. 62/118 Dither Pattern Example 0.340.450.560.67 0.420.650.780.88 0.210.330.570.77 0.010.220.440.55 0.2 0.4 Fill those cells larger than dither value

63. 63/118 Dither Pattern Example 0.340.450.560.67 0.420.650.780.88 0.210.330.570.77 0.010.220.440.55 0.2 0.4 Repeat for rest of image

64. 64/118 3x3 Halftone Example

65. 65/118 3x3 Dither Example

66. 66/118 Error Diffusion Another way to set levels without changing resolution Assume pixels being visited in some order  Left to right, top to bottom Roundoff error (from filling/not filling) gets diffused to adjacent pixels not visited yet Can be combined with halftoning/dithering

67. 67/118 Error Diffusion Pattern: Example: Assume we can display levels in increments of 50. Error 7/16 3/165/161/16 6677107 14276070 Given:

68. 68/118 Error Diffusion Pattern: Example: Assume we can display levels in increments of 50. Error 7/16 3/165/161/16 6677107 14276070 Round to 50

69. 69/118 Error Diffusion Pattern: Example: Assume we can display levels in increments of 50. Error 7/16 3/165/161/16 50 +16 77107 14276070 Round to 50 Error of 16

70. 70/118 Error Diffusion Pattern: Example: Assume we can display levels in increments of 50. Error 7/16 3/165/161/16 5077+7107 14+327+560+170 Round to 50 Error of 16

71. 71/118 Error Diffusion Pattern: Example: Assume we can display levels in increments of 50. Error 7/16 3/165/161/16 5084107 17326170

72. 72/118 Error Diffusion Pattern: Example: Assume we can display levels in increments of 50. Error 7/16 3/165/161/16 5084107 17326170 Round to 100

73. 73/118 Error Diffusion Pattern: Example: Assume we can display levels in increments of 50. Error 7/16 3/165/161/16 50 100 -16 107 17326170 Round to 100 Error of -16

74. 74/118 Error Diffusion Pattern: Example: Assume we can display levels in increments of 50. Error 7/16 3/165/161/16 50100107-7 1732-361-570-1 Round to 100 Error of -16

75. 75/118 Error Diffusion Pattern: Example: Assume we can display levels in increments of 50. Error 7/16 3/165/161/16 50100 17295669

76. 76/118 3x3 Dither Example

77. 77/118 Error Diffusion Example

78. 78/118 Topics The Human Visual System Displaying Intensity and Luminance Display Using Fixed Intensities Understanding Color Display of Color Color Models

79. 79/118 Seeing Color What is color?

80. Seeing Color http://www.johnsadowski.com/big_spanish_castle.html 80/118

81. 81/118 Light Light is carried by photons, traveling with different wavelengths/frequencies c=  Speed of light = c (assume constant)  Wavelength =  Frequency =  So, Wavelength & Frequency are interchangable, and inverse of each other

82. 82/118 Visible Light Image taken from http://www.andor.com/image_lib/lores/in troduction/introduction%20(light)/intligh t%201%20small.jpg

83. 83/118 Visible Light The response of the cells in our eye let us see from ~400nm (violet) to ~700nm (red). If all light arriving is one wavelength, we see it as one color.

84. 84/118 Visible Light Images taken from http://www.newport.com/Information- on-Oriel-Spectral-Irradiance- Data/383232/1033/catalog.aspx The response of the cells in our eye let us see from ~400nm (violet) to ~700nm (red). If all light arriving is one wavelength, we see it as one color. But, Light usually comes in as a spectrum – different intensities at all the various wavelengths.

85. 85/118 Tristimulus Theory There are 3 kinds of cones in the eye that respond differently to different wavelengths of light.

86. 86/118 Tristimulus Theory There are 3 kinds of cones in the eye that respond differently to different wavelengths of light. Anything that stimulates these cones in the same way appears as the same color.  This is fundamental to our ability to reproduce different colors!  A monitor can stimulate the cones to make it appear that a particular color is being generated.

87. 87/118 Response of Cones 3 Cones, Rods respond to light differently Image taken from http://www.unm.edu/~toolson/human_co ne_response.htm

88. 88/118 Receptor Response Though cones often called “red, green, blue”, the peak responses are actually in the yellow, yellow-green, and violet. Altogether, overall response is Gaussian-like, centered on yellow/green  Sun and plants… Image taken from http://www.unm.edu/~toolson/human_co ne_response.htm

89. Receptor Response 89/118

90. 90/118 Tristimulus Since 3 different cones, the space of colors is 3-dimensional. We need a way to describe color within this 3 dimensional space. We want something that will let us describe any visible color…

91. 91/118 Topics The Human Visual System Displaying Intensity and Luminance Display Using Fixed Intensities Understanding Color Display of Color Color Models

92. 92/118 The CIE XYZ system CIE – Comission Internationale de l’Eclairage  International Commission on Illumination  Sets international standards related to light Defined the XYZ color system as an international standard in 1931 X, Y, and Z are three Primary colors. All visible colors can be defined as a combination of these three colors.  Defines the 3 dimensional color space

93. 93/118 Color Matching Functions Given an input spectrum,, we want to find the X, Y, Z coordinates for that color. Color matching functions,,, and tell how to weight the spectrum when integrating: Image taken from http://upload.wikimedia.org/wikipedia/co mmons/8/87/CIE1931_XYZCMF.png

94. 94/118 XYZ space The visible colors form a “cone” in XYZ space.  For visible colors, X, Y, Z are all positive.  But, X, Y, and Z themselves are not visible colors! Image taken from http://fourier.eng.hmc.edu/e180/handouts /color1/node27.html

95. 95/118 Luminance and Chromaticity The intensity (luminance) is just X+Y+Z.  Scaling X, Y, Z just increases intensity.  We can separate this from the remaining part, chromaticity. Color = Luminance + Chromaticity  Chromaticity is 2D, Luminance is 1D To help us understand chromaticity, we’ll fix intensity to the X+Y+Z=1 plane.

96. 96/118 Chromaticity Diagram Project the X+Y+Z=1 slice along the Z-axis Chromaticity is given by the x, y coordinates Image taken from http://fourier.eng.hmc.edu/e180/handouts /color1/node27.html

97. 97/118 White Point White: at the center of the diagram. Image taken from http://fourier.eng.hmc.edu/e180/handouts /color1/node27.html

98. 98/118 Spectral Colors Visible Spectrum along outside curve Image taken from http://fourier.eng.hmc.edu/e180/handouts /color1/node27.html

99. 99/118 Saturation As you move on line from white to edge, you increase the saturation of that color. Royal blue, red: high saturation Carolina blue, pink: low saturation Image taken from http://fourier.eng.hmc.edu/e180/handouts /color1/node27.html

100. 100/118 Hue Hue is the “direction” from white. Combined with saturation, it gives another way to describe color Also called dominant wavelength Image taken from http://fourier.eng.hmc.edu/e180/handouts /color1/node27.html

101. 101/118 Non-Spectral Colors Non-spectral colors: do not correspond to any wavelength of light.  i.e. not seen in rainbow  e.g. maroon, purple, magenta Image taken from http://fourier.eng.hmc.edu/e180/handouts /color1/node27.html

102. 102/118 If we have two colors, A and B, by varying the relative intensity, we can generate any color on the line between A and B. Combining Two Colors Image taken from http://fourier.eng.hmc.edu/e180/handouts /color1/node27.html

103. 103/118 Complementary Colors Complementary colors are those that will sum to white. That is, white is halfway between them. Image taken from http://fourier.eng.hmc.edu/e180/handouts /color1/node27.html

104. 104/118 Combining Three Colors If we have three colors, A, B, and C, by varying the relative intensity, we can generate any color in the triangle between them. Image taken from http://fourier.eng.hmc.edu/e180/handouts /color1/node27.html

105. 105/118 Gamut Display devices generally have 3 colors (a few have more).  e.g. RGB in monitor The display can therefore display any color created from a combination of those 3. This range of displayable colors is called the gamut of the device.

106. 106/118 Differing Gamuts Different devices have different gamuts  e.g. differing phosphors So, RGB on one monitor is not the same as RGB on another Image taken from http://fourier.eng.hmc.edu/e180/handouts /color1/node27.html

107. 107/118 Device Gamuts For monitors, typically have colors in the Red, Green, Blue areas  Helps cover lots of visible spectrum But, not all (in fact, nowhere close to all) of the visible spectrum is ever represented Since all 3 colors are visible, can’t possibly encompass full visible spectrum!

108. 108/118 Gamuts Red: typical monitor gamut Blue: maximum gamut with 3 phosphors Image taken from http://fourier.eng.hmc.edu/e180/handouts /color1/node27.html

109. What does cyan look like? 109/118

110. 110/118 Topics The Human Visual System Displaying Intensity and Luminance Display Using Fixed Intensities Understanding Color Display of Color Color Models

111. 111/118 Color Models CIE’s XYZ system is a standard, but not very intuitive. As we saw with saturation and hue, there’s more than one way to specify a color. A variety of color models have been developed to help with some specifications.

112. 112/118 RGB Red, Green, Blue Common specifications for most monitors  Tells how much intensity to use for pixels Note: Not standard – RGB means different things for different monitors Generally used in an additive system  Each adds additional light (e.g. phosphor)  Combine all three colors to get white

113. 113/118 CMY Cyan, Magenta, Yellow Commonly used in printing Generally used in a subtractive system:  Each removes color from reflected light  Combine all three colors to get black Conceptually, [C M Y] = [1 1 1] – [R G B]  Complimentary colors to RGB

114. 114/118 CMYK Cyan, Magenta, Yellow, Black Comes from printing process – since CMY combine to form black, can replace equal amounts of CMY with Black, saving ink. K = min(C, M, Y) C = C-K M = M-K Y = Y-K

115. 115/118 YIQ / YUV NTSC, PAL standards for broadcast TV Backward compatible to Black and White TV Y is luminance – only part picked up by Black and White Televisions Y is given most bandwidth in signal I, Q channels (or U,V) contain chromaticity information

116. 116/118 HSV Hue, Saturation, Value Used as a user-friendly way to specify color. Hue – angle around cone Saturation – how far from center Value (luminance) – how high up cone (black at bottom, white at top center). Image taken from http://www.mandelbrot- dazibao.com/HSV/HSV.htm

117. 117/118 Lab and Luv Perceptually-based color spaces (CIE standards) Idea: want the distance between colors in the color space to correspond to intuitive notion of how “similar” colors are Not perfect, but much better than XYZ or RGB color spaces.

118. 118/118 Representing Color Generally, store 3 color channels in equal bits  Not necessary, though, e.g. YIQ  Can sometimes get better mapping of color space for an application by adjusting bits  e.g. 10 bits R, 8 bits G, 6 bits B Color indexing: give each color a numerical identifier, then use that as reference  Good for specifying with a limited palette Note: you can use dithering/halftoning with color channels, similar to how you would for achromatic light.

119. 119/118

120. 120/118 Dynamic Range and Contrast of Various Display Mediums Display TypeDynamic RangeContrast Newsprint10 Printed Photo~100 Slide Photo~1000 CRT>10,00075 - 219 LCD~500 - 700 Plasma~100 - 600~100 - 500 DLP1381274 - 332

121. 121/118 Main Parts of the Eye Cornea  Provides most refraction  Reshaped in refractive surgery  Corrects nearsightedness caused by elongation of the eyeball

122. 122/118 Main Parts of the Eye Cornea Iris Lens  Flexible - muscles adjust shape  Allows fine-detail focus  Toughens with age, leading to reading glasses  Cataracts form here

123. 123/118 Focusing Light For a point not in focal plane, light gets spread across retina  Also happens in nearsightedness  Our brain is good at eliminating these signals

124. 124/118 HLS Hue, Lightness, Saturation A variation on HSV, but with a double cone Lightness: black at base (0), white at top (1) Image taken from http://en.wikipedia.org/wiki/Image:Color_cones.png