1. Hebrew University Image Processing - 2009 1 Exercise Class 12: Color Many slides from Freeman and Durand Color Exercise Class 12

2. Hebrew University Image Processing - 2009 2 http://www.hobbylinc.com/gr/pll/pll5019.jpg Why does a visual system need color?

3. Hebrew University Image Processing - 2009 3 An incomplete list: To tell what food is edible. To distinguish material changes from shading changes. To group parts of one object together in a scene. To find people’s skin. Check whether a person’s appearance looks normal/healthy. To compress images.... Why does a visual system need color?

4. Hebrew University Image Processing - 2009 4 Lecture Outline The physical nature of color Human color vision Trichromaticity: color matching experiments Color spaces RGB color histogram and applications

5. Hebrew University Image Processing - 2009 5 Newton’s Prism Experiments (1666) Figures from: http://www.departments.bucknell.edu/History/Carnegie/newton/refraction.html http://astrophys-assist.com/educate/solarobs/ses01p02.htm http://www.bio.indiana.edu/~cryo/eftem.html 1. Sunlight decomposition into fundamental components: 2. Recovering the sunlight by another prism: Newton’s sketch of (1)

6. Hebrew University Image Processing - 2009 6 Spectral colors The Electromagnetic Spectrum:

7. Hebrew University Image Processing - 2009 7 Spectroradiometer (just like Newton’s diagram…) How measure those spectra? Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995

8. Hebrew University Image Processing - 2009 8 Two illumination spectra Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995 Blue SkylightTungsten Light Bulb

9. Hebrew University Image Processing - 2009 9 Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995 The information from rods (“grayscale” receptors) and cones (“color” receptors) is integrated through connections between nerve cells in the retina. Human Color Vision

10. Hebrew University Image Processing - 2009 10 Human Color Vision There are 3 light sensitive pigments in your cones (L,M,S), each with different spectral response curve. Biological basis of colour blindness – genetic disease. © Pat Hanrahan. λ

11. Hebrew University Image Processing - 2009 11 Human Color Vision

12. Hebrew University Image Processing - 2009 12 Principle of Trichromaticity Color matching experiments imply that three primaries are enough for human color vision: Where P i - primary colors, and w i – weights. Example: P1 = 645nm (R), P2 = 526nm (G), P3 =444nm (B)

13. Hebrew University Image Processing - 2009 13 Color matching experiment Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995

14. Hebrew University Image Processing - 2009 14 Color matching experiment 1

15. Hebrew University Image Processing - 2009 15 Color matching experiment 1 p 1 p 2 p 3

16. Hebrew University Image Processing - 2009 16 Color matching experiment 1 p 1 p 2 p 3

17. Hebrew University Image Processing - 2009 17 Color matching experiment 1 p 1 p 2 p 3 The primary color amounts needed for a match

18. Hebrew University Image Processing - 2009 18 Color matching experiment 2

19. Hebrew University Image Processing - 2009 19 Color matching experiment 2 p 1 p 2 p 3

20. Hebrew University Image Processing - 2009 20 Color matching experiment 2 p 1 p 2 p 3

21. Hebrew University Image Processing - 2009 21 Color matching experiment 2 p 1 p 2 p 3 Adding a “negative” amount of p 2 to the test color’s side: The primary color amounts needed for a match: p 1 p 2 p 3

22. Hebrew University Image Processing - 2009 22 Spectral Matching Functions Match each pure colour in the visible spectrum with the 3 primaries, and record the values of the three as a function of wavelength. © Pat Hanrahan. Note: We need to specify a negative amount of one primary to represent all colours. Red, Green & Blue primaries.

23. Hebrew University Image Processing - 2009 23 Color Matching is linear! Grassman’s Laws 1.Color matching is additive: C 1 +C 2 = (R 1 +R 2 ) + (G 1 +G 2 ) + (B 1 +B 2 ) 2.Scaling the color and the primaries by the same factor preserves the match: αC = αR + αG + αB These statements are true as any biological law. They mean that people behave like linear systems in the color matching experiment.

24. Hebrew University Image Processing - 2009 24 Additivity (Superposition) Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995

25. Hebrew University Image Processing - 2009 25 Scaling (Propotionality) Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995

26. Hebrew University Image Processing - 2009 26 Store the color matching functions in the rows of the matrix, C Let the new spectral signal be described by the vector t. Then the amounts of each primary needed to match t are: How to predict the match to a new spectral signal? λ C(λ)

27. Hebrew University Image Processing - 2009 27 p 1 = (0 0 0 0 0… 0 1 0) T p 2 = (0 0 … 0 1 0...0 0) T p 3 = (0 1 0 0 … 0 0 0 0) T Primary spectra, P Color matching functions, C p’ 1 = (0 0.2 0.3 4.5 7 …. 2.1) T p’ 2 = (0.1 0.44 2.1 … 0.3 0) T p’ 3 = (1.2 1.7 1.6 …. 0 0) T Primary spectra, P’ Color matching functions, C’ Any input spectrum, t The amount of each primary in P needed to match the color with spectrum t. The spectrum of a perceptual match to t, made using the primaries P’ The color of that match to t, described by the primaries, P. The amount of each P’ primary needed to match t Translation between different systems of primaries

28. Hebrew University Image Processing - 2009 28 P’ are the old primaries C are the new primaries’ color matching functions C P’ a 3x3 matrix The values of the 3 primaries, in the primed system The values of the 3 primaries, in the unprimed system Translation between different systems of primaries

29. Hebrew University Image Processing - 2009 29 Color Metamerism C t C s Two spectra, t and s, perceptually match, if Ct = Cs Where C is the Color Matching Functions of a set of primaries =

30. Hebrew University Image Processing - 2009 30 Color Metamerism

31. Hebrew University Image Processing - 2009 31 Luminance Compare colour source to a grey source Luminance: Y = 0.30R + 0.59G + 0.11B Color signal on a B&W TV (Except for gamma correction, of course) Perceptual measure – Lightness: L* = Y 1/3

32. Hebrew University Image Processing - 2009 32 The YIQ & YUV Color Models Recoded RGB for transmission efficiency, compatibility with B/W broadcast TV Y = CIE’s Y (luminance); I,Q encode chromaticity                                 B G R Q I Y 31.052.021.0 32.028.060.0 11.059.030.0

33. Hebrew University Image Processing - 2009 33 CIE-XYZ Color Space For only positive mixing coefficients, the CIE (Commission Internationale d’Eclairage) defined 3 new hypothetical light sources X, Y and Z (as shown) to replace red, green and blue. Primary Y intentionally has same response as luminance response of the eye. The weights X, Y, Z form the 3D CIE-XYZ space.

34. Hebrew University Image Processing - 2009 34 CIE-XYZ Color Space Three Imaginary colors X-Y-Z All weights are positive Represents all visible colors (gamut) Colors are additive No three real colors can represent gamut!

35. Hebrew University Image Processing - 2009 35 Sample: PAL RGB  XYZ [ R ] [ 3.240479 -1.537150 -0.498535 ] [ X ] [ G ] = [ -0.969256 1.875992 0.041556 ] * [ Y ] [ B ] [ 0.055648 -0.204043 1.057311 ] [ Z ] Not all XYZ colors have positive RGB values [ X ] [ 0.412453 0.357580 0.180423 ] [ R ] [ Y ] = [ 0.212671 0.715160 0.072169 ] * [ G ] [ Z ] [ 0.019334 0.119193 0.950227 ] [ B ] All positive RGB values have positive XYZ values

36. Hebrew University Image Processing - 2009 36 Chromaticity Diagram CIE Color Coordinates Normalise by the total amount of light energy. Often convenient to work in 2D colour space, so 3D colour space projected onto the plane X+Y+Z=1 to yield the chromaticity diagram. The projection is shown opposite and the diagram appears on the next slide.

37. Hebrew University Image Processing - 2009 37 CIE Chromaticity Diagram C is “white” and close to x = y = z =1/3 The dominant wavelength of a color, e.g. B, is where the line from C through B meets the spectrum, 580nm for B (tint). A and B can be mixed to produce any colour along the line AB here including white. True for EF (no white this time). True for ijk (includes white) D B C A E F i j k

38. Hebrew University Image Processing - 2009 38 Some device colour “gamuts” The diagram can be used to compare the gamuts of various devices. Note particularly that a colour printer can’t reproduce all the colours of a colour monitor. Note no triangle can cover all of visible space. C

39. Hebrew University Image Processing - 2009 39 MacAdam Ellipses Ellipse: 1 JND from center color Find spaces were ellipses are circles? CIE-Lab

40. Hebrew University Image Processing - 2009 40 CIE-Lab Color Space L represents the luminance (L = Y 1/3 ); a,b contain the color information (chromaticity) Euclidian distance in this space approximates the relative perceptual color change ( only for small differences! ) Common variant: CIE-L*a*b* (1976)

41. Hebrew University Image Processing - 2009 41 Lab Color Space

42. Hebrew University Image Processing - 2009 42 The Color Cube R,G,B model is additive, i.e. we add amounts of 3 primaries to get required color. Visualizing RGB space as a cube, grey values occur on diagonal K to W.

43. Hebrew University Image Processing - 2009 43 Rotating the Color Cube red yellow green cyanblue magenta red yellowgreen cyan bluemagenta

44. Hebrew University Image Processing - 2009 44 Artist Color Specification Tints Pure Pigment Shades Black White Greys Saturated  Tones Artist specification of colors resulting from a pure pigment: Tint – Adding white to a pure pigment Shade – Adding black to a pure pigment. Tone – Add both black and white.

45. Hebrew University Image Processing - 2009 45 HSV Color Space Hue – the color of the pure pigment. Saturation – measure of colorfulness (distance from the gray line). Value – measure of brightness. Maximum saturation occurs when V=1. If V = 0, H is undefined.

46. Hebrew University Image Processing - 2009 46 HSL Color Space Hue – the color of the pure pigment. Saturation – measure of colorfulness (distance from the gray line). Lightness – measure of brightness. Maximum saturation occurs when L=0.5 If L=0 or L=1, H is undefined.

47. Hebrew University Image Processing - 2009 47 CMYK – subtractive color model R = (1-C) (1-K) W G = (1-M) (1-K) W B = (1-Y) (1-K) W K = G(1-max(R,G,B)) C = 1 - R/(1-K) M = 1 - G/(1-K) Y = 1 - B/(1-K)

48. Hebrew University Image Processing - 2009 48 List of Color Spaces CIE - XYZ CIE - L*a*b* CIE - RGB HSV, HSL CMY, CMYK YUV, YIQ… Black – Linear Red - Nonlinear

49. Hebrew University Image Processing - 2009 49 Color Histograms

50. Hebrew University Image Processing - 2009 50 What is a Color Histogram?  Density function, usually R 3 → R  The value of bin B ( r i, g i, b i ) (in an RGB color histogram) is the number of image pixels having RGB values equal to [ r i,g i,b i ]

51. Hebrew University Image Processing - 2009 51 The Properties of Color Histograms  Assuming an 8-bit depth RGB image, the RGB histogram has 256 3 = 16777216 bins, which means it is usually very sparse.  Another natural image property that contributes to the sparseness of color histograms is the “piecewise smooth world” property.

52. Hebrew University Image Processing - 2009 52 The Properties of Color Histograms The histogram is sparse, with most of its values located in small areas of the RGB cube

53. Hebrew University Image Processing - 2009 53 Histogram Based Color Image Retrieval  Suppose we want to find images that are similar to a query image  Comparing pixel values is extremely sensitive to changes in the image…  Comparing color histograms is much more robust since we are only comparing the “color palette”

54. Hebrew University Image Processing - 2009 54 Histogram Based Color Image Retrieval Different ways of calculating the distance between two histograms:  Euclidean distance  Intersection distance  EMD - “Earth Mover’s Distance”

55. Hebrew University Image Processing - 2009 55 Histogram Based Color Image Retrieval Query Euclidean DistanceHistogram IntersectionQuadratic Distance HSVHSV HSVHSV RGBRGB RGBRGB http://scien.stanford.edu/class/psych221/projects/02/sojeong/

56. Hebrew University Image Processing - 2009 56 Histogram Based Color Image Retrieval  One limitation is computation time, suppose we want to search the web for some images…  We need an efficient histogram indexing  Other types of histograms are also worth mentioning:  Color edges histograms  Texture histograms  Co-occurrence Histograms  Similar applications:  Tracking  Object Recognition/Detection  Texture classification