1. Color Frank Dellaert Many slides by Jim Rehg Some slides by David Forsyth

2. Outline zColor and Radiometry zHuman Color Perception zColor spaces zLightness and Color Constancy zPhysics-based Vision: Specularities

3. Color and Radiometry zWhat is color ?

4. What is Color? zA perceptual attribute of objects and scenes constructed by the visual system zA quantity related to the wavelength of light in the visible spectrum zA box of Crayola crayons zA significant industry with conferences, standards bodies, etc. zA challenge y“There are no second-rate brains in color vision” – Edwin Land

5. Why is Color Important? zIn animal vision yfood vs nonfood yidentify predators and prey yCheck health, fitness, etc. of other individuals. zIn computer vision ySkin finder ySegment an image

6. Reflectance Model

7. Illumination Spectra Blue skylight Tungsten bulb

8. Reflectance Spectra

9. Human Color Perception zWhat is the retinal basis for color perception in humans?

10. Human Photoreceptors FoveaPeriphery

11. Human Cone Sensitivities zSpectral sensitivity of L, M, S cones in human eye

12. Color Names for Cartoon Spectra

13. Additive Color Mixing

14. Subtractive Color Mixing

15. Color Matching Process Basis for industrial color standards

16. Color Matching Experiment 1 Image courtesy Bill Freeman

17. Color Matching Experiment 1 Image courtesy Bill Freeman

18. Color Matching Experiment 1 Image courtesy Bill Freeman

19. Color Matching Experiment 1 Image courtesy Bill Freeman

20. Color Matching Experiment 2 Image courtesy Bill Freeman

21. Color Matching Experiment 2 Image courtesy Bill Freeman

22. Color Matching Experiment 2 Image courtesy Bill Freeman

23. Color Matching Experiment 2 Image courtesy Bill Freeman

24. Principle of Trichromaticity

25. Conclusion from Color Matching zThree primaries are sufficient for most people to reproduce arbitrary colors. Caveats: ySome people use different weights, a consequence of a chromosomal disorder. yElderly and neurologically-impaired may require fewer primaries yRandom variation in sample population

26. Caveat: Context Matters ! Figure courtesy of D. Forsyth

27. Figure courtesy of D. Forsyth Caveat: Context Matters !

28. Figure courtesy of D. Forsyth Caveat: Context Matters !

29. Color Spaces

30. Principle of Univariance zPerceived color depends solely on cone responses From “Foundations of Vision” by B. Wandell

31. Color Spaces zUse color matching functions to define a coordinate system for color. zEach color can be assigned a triple of coordinates with respect to some color space (e.g. RGB). zDevices (monitors, printers, projectors) and computers can communicate colors precisely.

32. Grassman’s Laws

33. Color matching functions zChoose primaries A, B, C zGiven energy function what amounts of primaries will match it? zFor each wavelength, determine how much of A, of B, and of C is needed to match that wavelength z= color matching functions Then our match is:

34. monochromatic 645.2, 526.3, 444.4 nm. negative parts -> some colors can be matched only subtractively. RBG Color Matching Figure courtesy of D. Forsyth

35. CIE XYZ: Color matching functions are positive everywhere, but primaries are imaginary. Usually draw x, y, where x=X/(X+Y+Z) y=Y/(X+Y+Z) CIE XYZ Color Matching Figure courtesy of D. Forsyth

36. Geometry of Color (CIE) zPerceptual color spaces are non-convex zThree primaries can span the space, but weights may be negative. zCurved outer edge consists of single wavelength primaries

37. RGB Color Space Many colors cannot be represented (phosphor limitations)

38. Uniform color spaces zMcAdam ellipses (next slide) demonstrate that differences in x,y are a poor guide to differences in color zConstruct color spaces so that differences in coordinates are a good guide to differences in color.

39. Figures courtesy of D. Forsyth McAdam ellipses

40. Lightness and Color Constancy

41. Color on Mars !

42. Human Color Constancy zColor constancy: hue and saturation zLightness constancy: gray-level zHumans can perceive yColor a surface would have under white light (surface color) yColor of reflected light (separate surface color from measured color) yColor of illuminant (limited)

43. Land’s Mondrian Experiments zSquares of color with the same color radiance yield very different color perceptions Photometer: 1.0, 0.3, 0.3 Audience: “Red”Audience: “Blue” White light Colored light Blue Red Blue Red

44. Basic Model for Lightness Constancy zAssumptions: yPlanar frontal scene yLambertian reflectance yLinear camera response zCamera model: zModeling assumptions for scene yPiecewise constant albedo ySlowly-varying Illumination

45. Algorithm Components zGoal: what surfaces look like in white light zProcess I: relative brightness. zProcess II: absolute reference

46. 1-D Lightness “Retinex” Threshold gradient image to find surface (patch) boundaries Figure courtesy of D. Forsyth

47. 1-D Lightness “Retinex” Integration to recover surface lightness (unknown constant) Figure courtesy of D. Forsyth

48. Extension to 2-D zSpatial issues yIntegration becomes much harder zRecover of absolute reference yBrightest patch is white yAverage reflectance across scene is known yGamut is known yKnown reference (color chart, skin color…)

49. Color Retinex Images courtesy John McCann

50. Physics-based Vision: Specularities

51. Finding Specularities zDielectric materials ySpecularly reflected light has color of source zReflected light has two components: yDiffuse (body reflection) ySpecular (highlight) zSpecularities produce a “Skewed-T” in the color histogram of the object. Figure courtesy of Sing Bing Kang

52. Skewed-T in Histogram A Physical Approach to Color Image Understanding – Klinker, Shafer, and Kanade. IJCV 1990 Figure courtesy of D. Forsyth

53. Figure courtesy of D. Forsyth Skewed-T in Histogram

54. Recent Application to Stereo Motion of camera causes highlight location to change. This cue can be combined with histogram analysis. Synthetic scene: Figure courtesy of Sing Bing Kang

55. Recent Application to Stereo “Real” scene: Figure courtesy of Sing Bing Kang