Color Frank Dellaert Many slides by Jim Rehg Some slides by David Forsyth

Outline zColor and Radiometry zHuman Color Perception zColor spaces zLightness and Color Constancy zPhysics-based Vision: Specularities

Color and Radiometry zWhat is color ?

What is Color? zA perceptual attribute of objects and scenes constructed by the visual system zA quantity related to the wavelength of light in the visible spectrum zA box of Crayola crayons zA significant industry with conferences, standards bodies, etc. zA challenge y“There are no second-rate brains in color vision” – Edwin Land

Why is Color Important? zIn animal vision yfood vs nonfood yidentify predators and prey yCheck health, fitness, etc. of other individuals. zIn computer vision ySkin finder ySegment an image

Reflectance Model

Illumination Spectra Blue skylight Tungsten bulb

Reflectance Spectra

Human Color Perception zWhat is the retinal basis for color perception in humans?

Human Photoreceptors FoveaPeriphery

Human Cone Sensitivities zSpectral sensitivity of L, M, S cones in human eye

Color Names for Cartoon Spectra

Additive Color Mixing

Subtractive Color Mixing

Color Matching Process Basis for industrial color standards

Color Matching Experiment 1 Image courtesy Bill Freeman

Color Matching Experiment 1 Image courtesy Bill Freeman

Color Matching Experiment 1 Image courtesy Bill Freeman

Color Matching Experiment 1 Image courtesy Bill Freeman

Color Matching Experiment 2 Image courtesy Bill Freeman

Color Matching Experiment 2 Image courtesy Bill Freeman

Color Matching Experiment 2 Image courtesy Bill Freeman

Color Matching Experiment 2 Image courtesy Bill Freeman

Principle of Trichromaticity

Conclusion from Color Matching zThree primaries are sufficient for most people to reproduce arbitrary colors. Caveats: ySome people use different weights, a consequence of a chromosomal disorder. yElderly and neurologically-impaired may require fewer primaries yRandom variation in sample population

Caveat: Context Matters ! Figure courtesy of D. Forsyth

Figure courtesy of D. Forsyth Caveat: Context Matters !

Figure courtesy of D. Forsyth Caveat: Context Matters !

Color Spaces

Principle of Univariance zPerceived color depends solely on cone responses From “Foundations of Vision” by B. Wandell

Color Spaces zUse color matching functions to define a coordinate system for color. zEach color can be assigned a triple of coordinates with respect to some color space (e.g. RGB). zDevices (monitors, printers, projectors) and computers can communicate colors precisely.

Grassman’s Laws

Color matching functions zChoose primaries A, B, C zGiven energy function what amounts of primaries will match it? zFor each wavelength, determine how much of A, of B, and of C is needed to match that wavelength z= color matching functions Then our match is:

monochromatic 645.2, 526.3, nm. negative parts -> some colors can be matched only subtractively. RBG Color Matching Figure courtesy of D. Forsyth

CIE XYZ: Color matching functions are positive everywhere, but primaries are imaginary. Usually draw x, y, where x=X/(X+Y+Z) y=Y/(X+Y+Z) CIE XYZ Color Matching Figure courtesy of D. Forsyth

Geometry of Color (CIE) zPerceptual color spaces are non-convex zThree primaries can span the space, but weights may be negative. zCurved outer edge consists of single wavelength primaries

RGB Color Space Many colors cannot be represented (phosphor limitations)

Uniform color spaces zMcAdam ellipses (next slide) demonstrate that differences in x,y are a poor guide to differences in color zConstruct color spaces so that differences in coordinates are a good guide to differences in color.

Figures courtesy of D. Forsyth McAdam ellipses

Lightness and Color Constancy

Color on Mars !

Human Color Constancy zColor constancy: hue and saturation zLightness constancy: gray-level zHumans can perceive yColor a surface would have under white light (surface color) yColor of reflected light (separate surface color from measured color) yColor of illuminant (limited)

Land’s Mondrian Experiments zSquares of color with the same color radiance yield very different color perceptions Photometer: 1.0, 0.3, 0.3 Audience: “Red”Audience: “Blue” White light Colored light Blue Red Blue Red

Basic Model for Lightness Constancy zAssumptions: yPlanar frontal scene yLambertian reflectance yLinear camera response zCamera model: zModeling assumptions for scene yPiecewise constant albedo ySlowly-varying Illumination

Algorithm Components zGoal: what surfaces look like in white light zProcess I: relative brightness. zProcess II: absolute reference

1-D Lightness “Retinex” Threshold gradient image to find surface (patch) boundaries Figure courtesy of D. Forsyth

1-D Lightness “Retinex” Integration to recover surface lightness (unknown constant) Figure courtesy of D. Forsyth

Extension to 2-D zSpatial issues yIntegration becomes much harder zRecover of absolute reference yBrightest patch is white yAverage reflectance across scene is known yGamut is known yKnown reference (color chart, skin color…)

Color Retinex Images courtesy John McCann

Physics-based Vision: Specularities

Finding Specularities zDielectric materials ySpecularly reflected light has color of source zReflected light has two components: yDiffuse (body reflection) ySpecular (highlight) zSpecularities produce a “Skewed-T” in the color histogram of the object. Figure courtesy of Sing Bing Kang

Skewed-T in Histogram A Physical Approach to Color Image Understanding – Klinker, Shafer, and Kanade. IJCV 1990 Figure courtesy of D. Forsyth

Figure courtesy of D. Forsyth Skewed-T in Histogram

Recent Application to Stereo Motion of camera causes highlight location to change. This cue can be combined with histogram analysis. Synthetic scene: Figure courtesy of Sing Bing Kang

Recent Application to Stereo “Real” scene: Figure courtesy of Sing Bing Kang