Hebrew University Image Processing Exercise Class 12: Color Many slides from Freeman and Durand Color Exercise Class 12

Hebrew University Image Processing Why does a visual system need color?

Hebrew University Image Processing An incomplete list: To tell what food is edible. To distinguish material changes from shading changes. To group parts of one object together in a scene. To find people’s skin. Check whether a person’s appearance looks normal/healthy. To compress images.... Why does a visual system need color?

Hebrew University Image Processing Lecture Outline The physical nature of color Human color vision Trichromaticity: color matching experiments Color spaces RGB color histogram and applications

Hebrew University Image Processing Newton’s Prism Experiments (1666) Figures from: Sunlight decomposition into fundamental components: 2. Recovering the sunlight by another prism: Newton’s sketch of (1)

Hebrew University Image Processing Spectral colors The Electromagnetic Spectrum:

Hebrew University Image Processing Spectroradiometer (just like Newton’s diagram…) How measure those spectra? Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995

Hebrew University Image Processing Two illumination spectra Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995 Blue SkylightTungsten Light Bulb

Hebrew University Image Processing Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995 The information from rods (“grayscale” receptors) and cones (“color” receptors) is integrated through connections between nerve cells in the retina. Human Color Vision

Hebrew University Image Processing Human Color Vision There are 3 light sensitive pigments in your cones (L,M,S), each with different spectral response curve. Biological basis of colour blindness – genetic disease. © Pat Hanrahan. λ

Hebrew University Image Processing Human Color Vision

Hebrew University Image Processing Principle of Trichromaticity Color matching experiments imply that three primaries are enough for human color vision: Where P i - primary colors, and w i – weights. Example: P1 = 645nm (R), P2 = 526nm (G), P3 =444nm (B)

Hebrew University Image Processing Color matching experiment Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995

Hebrew University Image Processing Color matching experiment 1

Hebrew University Image Processing Color matching experiment 1 p 1 p 2 p 3

Hebrew University Image Processing Color matching experiment 1 p 1 p 2 p 3

Hebrew University Image Processing Color matching experiment 1 p 1 p 2 p 3 The primary color amounts needed for a match

Hebrew University Image Processing Color matching experiment 2

Hebrew University Image Processing Color matching experiment 2 p 1 p 2 p 3

Hebrew University Image Processing Color matching experiment 2 p 1 p 2 p 3

Hebrew University Image Processing Color matching experiment 2 p 1 p 2 p 3 Adding a “negative” amount of p 2 to the test color’s side: The primary color amounts needed for a match: p 1 p 2 p 3

Hebrew University Image Processing Spectral Matching Functions Match each pure colour in the visible spectrum with the 3 primaries, and record the values of the three as a function of wavelength. © Pat Hanrahan. Note: We need to specify a negative amount of one primary to represent all colours. Red, Green & Blue primaries.

Hebrew University Image Processing Color Matching is linear! Grassman’s Laws 1.Color matching is additive: C 1 +C 2 = (R 1 +R 2 ) + (G 1 +G 2 ) + (B 1 +B 2 ) 2.Scaling the color and the primaries by the same factor preserves the match: αC = αR + αG + αB These statements are true as any biological law. They mean that people behave like linear systems in the color matching experiment.

Hebrew University Image Processing Additivity (Superposition) Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995

Hebrew University Image Processing Scaling (Propotionality) Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995

Hebrew University Image Processing Store the color matching functions in the rows of the matrix, C Let the new spectral signal be described by the vector t. Then the amounts of each primary needed to match t are: How to predict the match to a new spectral signal? λ C(λ)

Hebrew University Image Processing p 1 = ( … 0 1 0) T p 2 = (0 0 … ) T p 3 = ( … ) T Primary spectra, P Color matching functions, C p’ 1 = ( …. 2.1) T p’ 2 = ( … 0.3 0) T p’ 3 = ( …. 0 0) T Primary spectra, P’ Color matching functions, C’ Any input spectrum, t The amount of each primary in P needed to match the color with spectrum t. The spectrum of a perceptual match to t, made using the primaries P’ The color of that match to t, described by the primaries, P. The amount of each P’ primary needed to match t Translation between different systems of primaries

Hebrew University Image Processing P’ are the old primaries C are the new primaries’ color matching functions C P’ a 3x3 matrix The values of the 3 primaries, in the primed system The values of the 3 primaries, in the unprimed system Translation between different systems of primaries

Hebrew University Image Processing Color Metamerism C t C s Two spectra, t and s, perceptually match, if Ct = Cs Where C is the Color Matching Functions of a set of primaries =

Hebrew University Image Processing Color Metamerism

Hebrew University Image Processing Luminance Compare colour source to a grey source Luminance: Y = 0.30R G B Color signal on a B&W TV (Except for gamma correction, of course) Perceptual measure – Lightness: L* = Y 1/3

Hebrew University Image Processing The YIQ & YUV Color Models Recoded RGB for transmission efficiency, compatibility with B/W broadcast TV Y = CIE’s Y (luminance); I,Q encode chromaticity                                 B G R Q I Y

Hebrew University Image Processing CIE-XYZ Color Space For only positive mixing coefficients, the CIE (Commission Internationale d’Eclairage) defined 3 new hypothetical light sources X, Y and Z (as shown) to replace red, green and blue. Primary Y intentionally has same response as luminance response of the eye. The weights X, Y, Z form the 3D CIE-XYZ space.

Hebrew University Image Processing CIE-XYZ Color Space Three Imaginary colors X-Y-Z All weights are positive Represents all visible colors (gamut) Colors are additive No three real colors can represent gamut!

Hebrew University Image Processing Sample: PAL RGB  XYZ [ R ] [ ] [ X ] [ G ] = [ ] * [ Y ] [ B ] [ ] [ Z ] Not all XYZ colors have positive RGB values [ X ] [ ] [ R ] [ Y ] = [ ] * [ G ] [ Z ] [ ] [ B ] All positive RGB values have positive XYZ values

Hebrew University Image Processing Chromaticity Diagram CIE Color Coordinates Normalise by the total amount of light energy. Often convenient to work in 2D colour space, so 3D colour space projected onto the plane X+Y+Z=1 to yield the chromaticity diagram. The projection is shown opposite and the diagram appears on the next slide.

Hebrew University Image Processing CIE Chromaticity Diagram C is “white” and close to x = y = z =1/3 The dominant wavelength of a color, e.g. B, is where the line from C through B meets the spectrum, 580nm for B (tint). A and B can be mixed to produce any colour along the line AB here including white. True for EF (no white this time). True for ijk (includes white) D B C A E F i j k

Hebrew University Image Processing Some device colour “gamuts” The diagram can be used to compare the gamuts of various devices. Note particularly that a colour printer can’t reproduce all the colours of a colour monitor. Note no triangle can cover all of visible space. C

Hebrew University Image Processing MacAdam Ellipses Ellipse: 1 JND from center color Find spaces were ellipses are circles? CIE-Lab

Hebrew University Image Processing CIE-Lab Color Space L represents the luminance (L = Y 1/3 ); a,b contain the color information (chromaticity) Euclidian distance in this space approximates the relative perceptual color change ( only for small differences! ) Common variant: CIE-L*a*b* (1976)

Hebrew University Image Processing Lab Color Space

Hebrew University Image Processing The Color Cube R,G,B model is additive, i.e. we add amounts of 3 primaries to get required color. Visualizing RGB space as a cube, grey values occur on diagonal K to W.

Hebrew University Image Processing Rotating the Color Cube red yellow green cyanblue magenta red yellowgreen cyan bluemagenta

Hebrew University Image Processing Artist Color Specification Tints Pure Pigment Shades Black White Greys Saturated  Tones Artist specification of colors resulting from a pure pigment: Tint – Adding white to a pure pigment Shade – Adding black to a pure pigment. Tone – Add both black and white.

Hebrew University Image Processing HSV Color Space Hue – the color of the pure pigment. Saturation – measure of colorfulness (distance from the gray line). Value – measure of brightness. Maximum saturation occurs when V=1. If V = 0, H is undefined.

Hebrew University Image Processing HSL Color Space Hue – the color of the pure pigment. Saturation – measure of colorfulness (distance from the gray line). Lightness – measure of brightness. Maximum saturation occurs when L=0.5 If L=0 or L=1, H is undefined.

Hebrew University Image Processing CMYK – subtractive color model R = (1-C) (1-K) W G = (1-M) (1-K) W B = (1-Y) (1-K) W K = G(1-max(R,G,B)) C = 1 - R/(1-K) M = 1 - G/(1-K) Y = 1 - B/(1-K)

Hebrew University Image Processing List of Color Spaces CIE - XYZ CIE - L*a*b* CIE - RGB HSV, HSL CMY, CMYK YUV, YIQ… Black – Linear Red - Nonlinear

Hebrew University Image Processing Color Histograms

Hebrew University Image Processing What is a Color Histogram?  Density function, usually R 3 → R  The value of bin B ( r i, g i, b i ) (in an RGB color histogram) is the number of image pixels having RGB values equal to [ r i,g i,b i ]

Hebrew University Image Processing The Properties of Color Histograms  Assuming an 8-bit depth RGB image, the RGB histogram has = bins, which means it is usually very sparse.  Another natural image property that contributes to the sparseness of color histograms is the “piecewise smooth world” property.

Hebrew University Image Processing The Properties of Color Histograms The histogram is sparse, with most of its values located in small areas of the RGB cube

Hebrew University Image Processing Histogram Based Color Image Retrieval  Suppose we want to find images that are similar to a query image  Comparing pixel values is extremely sensitive to changes in the image…  Comparing color histograms is much more robust since we are only comparing the “color palette”

Hebrew University Image Processing Histogram Based Color Image Retrieval Different ways of calculating the distance between two histograms:  Euclidean distance  Intersection distance  EMD - “Earth Mover’s Distance”

Hebrew University Image Processing Histogram Based Color Image Retrieval Query Euclidean DistanceHistogram IntersectionQuadratic Distance HSVHSV HSVHSV RGBRGB RGBRGB

Hebrew University Image Processing Histogram Based Color Image Retrieval  One limitation is computation time, suppose we want to search the web for some images…  We need an efficient histogram indexing  Other types of histograms are also worth mentioning:  Color edges histograms  Texture histograms  Co-occurrence Histograms  Similar applications:  Tracking  Object Recognition/Detection  Texture classification