1. Color ECE 847: Digital Image Processing Stan Birchfield Clemson University

2. Acknowledgment Many slides are courtesy of Bill Freeman at MIT and David Forsyth at UC Berkeley from http://www-static.cc.gatech.edu/classes/AY2007/cs4495_fall/html/materials.html

3. How it all began

4. An aside Tyrannosaurus Allosaurus Titanosaurus 65 feet (L) 50 feet (L) 40 feet (L) American football field: 300 feet x 160 feet Ark: 450 feet x 75 feet Height: 45 feet http://dinodictionary.comhttp://www.kickoffzone.com/articles/images/ClemsonMemorialStadium02.jpg

5. Visible Spectrum Physically, the colors are linear: electromagnetic (EM) spectrum 380 nm720 nm

6. Question Why then does violet look like red mixed with blue? Red and blue are at extreme ends of the spectrum Should have the least in common 380 nm720 nm

7. Answer Psychologically, the colors are circular: Newton chose 7 colors (ROYGBIV) because 7 is a perfect number 6 colors fits the data better (what is indigo anyway?) This is the famous color wheel

8. Color in music De Clario’s color music code http://home.vicnet.net.au/~colmusic/clario1.htm

9. Color and moods intense peaceful, depressing calm, natural happy, optimistic royal, wealthy attention http://www.infoplease.com/spot/colors1.html http://www.cs.brown.edu/courses/cs092/VA10/HTML/GoethesTriangleExplanation.html Goethe’s color triangle

10. Physics of color Forsyth, 2002 Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995 Illumination spectraReflectance spectra blue skylight tungsten bulb

11. Color names for cartoon spectra 400 500 600 700 nm red green blue 400 500 600 700 nm cyan magenta yellow 400 500 600 700 nm http://groups.csail.mit.edu/graphics/classes/CompPhoto06/html/lecturenotes/Color.ppt

12. Metamer Two colors are metamers if they have different spectral distributions same visual appearance http://escience.anu.edu.au/lecture/cg/Color/Image/metamer.gif

13. Trichromacy Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995 In the human visual system, every color can be obtained as the linear combination of three independent primary colors

14. Color matching experiment 1 http://groups.csail.mit.edu/graphics/classes/CompPhoto06/html/lecturenotes/Color.ppt

15. Color matching experiment 1 p 1 p 2 p 3 The primary color amounts needed for a match

16. Color matching experiment 1 p 1 p 2 p 3 The primary color amounts needed for a match

17. Color matching experiment 1 p 1 p 2 p 3 The primary color amounts needed for a match

18. Color matching experiment 2

19. p 1 p 2 p 3

20. Color matching experiment 2 p 1 p 2 p 3

21. Color matching experiment 2 p 1 p 2 p 3 We say a “negative” amount of p 2 was needed to make the match, because we added it to the test color’s side. The primary color amounts needed for a match: p 1 p 2 p 3

22. Superposition principle Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995

23. Grassman’s Laws For color matches: –symmetry: U=V V=U –transitivity: U=V and V=W => U=W –proportionality: U=V tU=tV –additivity: if any two (or more) of the statements U=V, W=X, (U+W)=(V+X) are true, then so is the third I.e., additive color matching is linear Not true at extreme ends of measurements Forsyth & Ponce where

24. Do two people see the same color? Yes! In the following sense: They will choose the same weights for the three primaries to match the color Not true for color-blind people, of course

25. Rods and cones http://en.wikipedia.org/wiki/Trichromatic_color_vision Young-Helmholtz theory (early 1800s): Color vision is the result of three different photoreceptors Experimentally confirmed (1980s) by measuring the cone response functions from the photoreceptors

26. Tetrachromacy At low light levels, rod cells may contribute to color vision Studies suggest that some people may have four cones Some animals (e.g., shrimp) have more than four cones

27. Measure color by color-matching paradigm Pick a set of 3 primary color lights. Find the amounts of each primary, e 1, e 2, e 3, needed to match some spectral signal, t. Those amounts, e 1, e 2, e 3, describe the color of t. If you have some other spectral signal, s, and s matches t perceptually, then e 1, e 2, e 3 will also match s, by Grassman’s laws. Why this is useful—it lets us: –Predict the color of a new spectral signal –Translate to representations using other primary lights. http://groups.csail.mit.edu/graphics/classes/CompPhoto06/html/lecturenotes/Color.ppt

28. Goal: compute the color match for any color signal for any set of primary colors Examples of why you’d want to do that: –Want to paint a carton of Kodak film with the Kodak yellow color. –Want to match skin color of a person in a photograph printed on an ink jet printer to their true skin color. –Want the colors in the world, on a monitor, and in a print format to all look the same.

29. How to compute the color match for any color signal for any set of primary colors Pick a set of primaries, Measure the amount of each primary, needed to match a monochromatic light, at each spectral wavelength (pick some spectral step size). These are called the color matching functions.

30. Color matching functions for a particular set of monochromatic primaries p 1 = 645.2 nm p 2 = 525.3 nm p 3 = 444.4 nm Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995

31. Using the color matching functions to predict the primary match to a new spectral signal We know that a monochromatic light of wavelength will be matched by the amounts of each primary. And any spectral signal can be thought of as a linear combination of very many monochromatic lights, with the linear coefficient given by the spectral power at each wavelength.

32. Using the color matching functions to predict the primary match to a new spectral signal Store the color matching functions in the rows of the matrix, C Let the new spectral signal be described by the vector t. Then the amounts of each primary needed to match t are:

33. Internal review So, for any set of primary colors, if we are given the spectral color matching functions for a set of primary lights We can calculate the amounts of each primary needed to give a perceptual match to any spectral signal.

34. Suppose you use one set of primaries and I use another? We address this in 2 ways: –Learn how to translate between primaries –Standardize on a few sets of favored primaries.

35. How do you translate colors between different systems of primaries? p 1 = (0 0 0 0 0… 0 1 0) T p 2 = (0 0 … 0 1 0...0 0) T p 3 = (0 1 0 0 … 0 0 0 0) T Primary spectra, P Color matching functions, C p’ 1 = (0 0.2 0.3 4.5 7 …. 2.1) T p’ 2 = (0.1 0.44 2.1 … 0.3 0) T p’ 3 = (1.2 1.7 1.6 …. 0 0) T Primary spectra, P’ Color matching functions, C’ Any input spectrum, t The amount of each primary in P needed to match the color with spectrum t. The spectrum of a perceptual match to t, made using the primaries P’ The color of that match to t, described by the primaries, P. The amount of each P’ primary needed to match t

36. So, how to translate from the color in one set of primaries to that in another: P’ are the old primaries C are the new primaries’ color matching functions C P’ a 3x3 matrix The values of the 3 primaries, in the primed system The values of the 3 primaries, in the unprimed system

37. And, by the way, color matching functions translate like this: But this holds for any input spectrum, t, so… a 3x3 matrix that transforms from the color representation in one set of primaries to that of another. P’ are the old primaries C are the new primaries’ color matching functions C P’ From earlier slide

38. How to use this? Given two sets of primaries, P and P’ Measure color matching functions C and C’ Solve C=FC’ for the 3x3 matrix F F now converts between tristimulus values: e = Fe’

39. Human eye photoreceptor spectral sensitivities Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995 What colors would these look like?

40. Are the color matching functions we observe obtainable from some 3x3 matrix transformation of the human photopigment response curves? (Because that’s how color matching functions translate).

41. Color matching functions (for a particular set of spectral primaries

42. Comparison of color matching functions with best 3x3 transformation of cone responses Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995

43. Internal summary What are colors? –Arise from power spectrum of light. How represent colors: –Pick primaries –Measure color matching functions (CMF’s) –Matrix mult power spectrum by CMF’s to find color as the 3 primary color values. How share color descriptions between people? –Translate colors between systems of primaries –Standardize on a few sets of primaries.

44. CIE 1931 standard colorimetric observer (2 o ) Commission internationale de l'éclairage (CIE) Used primaries: –Red: 700 nm –Green: 546.1 nm –Blue: 435.8 nm 2 o standard observer Now considered out of date, but still widely used – 1964 supplementary standard colorimetric observer (10 o ) Procedure: –Show pure color to observer –Match using primaries  weights for RGB –Transform from RGB to XYZ (XYZ are imaginary primaries; in XYZ, all weights are positive) (using NTSC primaries)

45. CIE Chromaticity diagram Normalize: x = X / (X+Y+Z) y = Y / (X+Y+Z) z = 1 – x – y spectral locus line of purples gamut of device is convex hull of primaries Note: It is misleading to draw colors on the chromaticity diagram (not recommended), but it makes the slide pretty

46. CIE Chromaticity diagram With 3 fixed primaries, any color can be matched (allowing negative weights)

47. CIE Chromaticity diagram But with just 2 primaries, any color can also be matched (if the primaries can be moved)

48. CIE Chromaticity diagram Complementary colors are on opposite sides

49. CIE Chromaticity diagram Natural encoding of color for perception: hue (dominant wavelength) saturation (distance from edge) value (height out of plane) Notice similarity to color wheel R C Y G M B

50. Color spaces RGB –orthogonal axes –usually 8 bits each (0 – 255) –natural for capture and display H/W (cameras, monitors) R G B CCIR Rec. 601 was used for television (0.299 0.587 0.114) CCIR Rec. 709 defines RGB for HDTV and is used by all modern devices (0.2125 0.7154 0.0721)

51. Color spaces Turn the RGB cube on its side Hexagon border is color wheel white / black are out of page R C Y G M B R C Y G M B http://viz.aset.psu.edu/gho/sem_notes/color_2d/html/primary_systems.html

52. Color spaces http://viz.aset.psu.edu/gho/sem_notes/color_2d/html/primary_systems.html R C Y G M B This is HLS (hue lightness saturation)

53. Color spaces http://viz.aset.psu.edu/gho/sem_notes/color_2d/html/primary_systems.html Rescale HLS to get HSV (hue saturation value) Also called HIS How would you describe a color? –dominant wavelength (hue) –purity (saturation) –brightness (value) Transformation RGB  HSV is nonlinear

54. Color spaces Y’CbCr color differences –Y’ is luma component –Cb is blue chroma component –Cr is red chroma component YUV is not well defined Usually YUV means scaled version of Y’CbCr For CCIR Rec. 601, (LumaRed, LumaGreen, LumaBlue) = (0.299 0.587 0.114)

55. Color spaces American analog television transmission (NTSC) uses YIQ color space: –originally black-and-white television (only Y) –IQ added later, modulated on top of Y for backward compatibility European joke: “Never twice the same color”

56. Color spaces We have already seen CIE xyz Based on a direct graph of the original X, Y and Z tristimulus functions Problem: Too much space is allocated to greens http://www.cambridgeincolour.com/tutorials/color-spaces.htm

57. Color spaces Ask subject to match test color All matches fall within ellipse on chromaticity diagram These are MacAdam ellipses They capture “just noticeable difference” Note: Color differences don’t make sense for large distances – Is red more like green or blue? http://en.wikipedia.org/wiki/MacAdam_ellipse

58. Color spaces Solution: CIE Lu’v’ Perceptually uniform space Colors are distributed proportional to their perceived color difference http://www.cambridgeincolour.com/tutorials/color-spaces.htm

59. Color spaces CIE La*b* (1976) transforms the colors so that they extend equally on two axes Color space is now a square Each axis represents an easily recognizable property of color: –red-green blend –blue-red blend –blue-green blend http://www.cambridgeincolour.com/tutorials/color-spaces.htm

60. Int’l Color Consortium (ICC) ICC established in 1993 to create an open, standardized color management system Now used in most computers Systems involves three key concepts: –color profiles –color spaces –translation between color spaces http://www.cambridgeincolour.com/tutorials/color-management1.htm

61. One final color space So far, we have been assuming additive colors (light) Now let us consider subtractive colors (pigments) Pigments work similarly but are highly nonlinear

62. Color spaces CMYK is used for printers Subtractive color Black (K) is separate because it is very difficult to get good black by mixing other colors In theory, But in practice much more complicated

63. Gamut Recall: gamut is the range of colors that a device can display Monitor’s gamut is triangle, because additive colors (light) follow Grassman’s laws More complicated for printers, film http://www.imaging-resource.com/PRINT/PPM200/PPM200vsP400.gif http://www.cse.fau.edu/~maria/COURSES/COP4930-GS/ColorFigs/Mvc-061s.jpg

64. Paint mixing This is why paint stores use many more than three paints for mixing (~12) http://whites-autorepair.com/images/paintmixroom.jpg http://www.albert-tague.com/inc/colour-mixer2.jpg

65. Additive and Subtractive Colors additive subtractive (but pigments are nonlinear) RGB CMY Note: Order of colors is the same in both cases

66. This leads us to an important question

67. What are the primary colors? As children, we learn RYB Then we’re told RGB When asked about the discrepancy, we’re told CMY is the same as RYB

68. Something about this is unsettling Yellow still appears to be pure –Even when you know that green and red make yellow, –It is impossible to believe In fact, red, yellow, blue, and green all appear pure So do black and white Could there be six primary colors?

69. Limitations of component theories So far we have discussed component theories of color Component theories are unsatisfying because they do not describe our subjective experience well Psychologically, –violet looks like a combination of red and blue –yellow does not look like a combination of red and green –black and white do not look like combinations of other colors

70. Opponent colors Opponent-process color theory (Hering 1872) Six primary psychological colors: Each color looks pure No such thing as –reddish green (inevitably becomes yellowish green), or –yellowish blue (inevitably becomes yellowish-green) Every other color is a combination of these six http://en.wikipedia.org/wiki/Opponent_process

71. Both are true Controversy raged for 100 years between –component theories –opponent color theory Both are true (experimentally verified): –three photoreceptors provide components –later cells transform to opponent color space (Ballard and Brown)

72. Final thought Piet Mondrian, Composition with Yellow, Blue, and Red, 1921 http://en.wikipedia.org/wiki/Piet_Mondrian

73. Poynton definitions intensity, brightness, lightness, luma, luminance, white

74. Gamma correction

75. Blackbody radiators

76. Fluorescence http://en.wikipedia.org/wiki/Image:AgarosegelUV.jpg