1. ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale Differential Structure Geusebroek et al, LNCS 1852, 459-464, 1999

2. ter Haar Romeny, FEV The color of an object depends on color of the illuminating light illumination intensity sensor sensitivity direction of surface normal surface reflectance properties Assumptions: Scene is uniformly illuminated light source is colored surface has Lambertian reflectance

3. ter Haar Romeny, FEV What causes color ? Lamp object color spectral color

4. ter Haar Romeny, FEV Object reflectance function for the observed spectrum for a resp. 2500K, 6500K and 10,000K light source: Spectrum reflected from an arbitrary object Emission spectrum of black body radiator

5. ter Haar Romeny, FEV Color receptive fields 0 1 2

6. ter Haar Romeny, FEV Self-organization: receptive fields from Eigenpatches (12x12 pixels)

7. ter Haar Romeny, FEV Blakemore C, Cooper GF (1970), Development of the brain depends on the visual environment. Nature 228, 477 - 478

8. ter Haar Romeny, FEV Colour receptive fields from Eigenpatches

9. ter Haar Romeny, FEV GD Field et al. Nature 467, 673-677 (2010) doi:10.1038/nature09424

10. ter Haar Romeny, FEV Distribution of cone cells in the fovea of an individual with normal color vision (left), and a color blind (protanopic) retina. Note that the center of the fovea holds very few blue-sensitive cones. [Wikipedia]

11. ter Haar Romeny, FEV GD Field et al. Nature 467, 673-677 (2010) doi:10.1038/nature09424 Full functional sampling of cone lattice by four RGC types.

12. ter Haar Romeny, FEV The color opponency model Hering E, 1964. Outlines of a Theory of the Light Sense. Harvard University Press, Cambridge, Mass.

13. ter Haar Romeny, FEV Hering basis 0102030405060708090 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 Idea Koenderink: Gaussian derivatives of zero, first and second order in the wavelength domain wavelength RF sensitivity How can we analyse color differential structure?

14. ter Haar Romeny, FEV Taylor color model Luminance Blue-yellowness Purple-greenness 300400500600700800900 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 L M S Cone sensitivity

15. ter Haar Romeny, FEV Spatial color Color scale-space starts by probing this space. s Energy densities cannot be measured at a point, … … one probes a certain volume

16. ter Haar Romeny, FEV Reflectance of light Lamp object color spectra l color What are invariant properties?

17. ter Haar Romeny, FEV Reflectance model

18. ter Haar Romeny, FEV Transparent materials

19. ter Haar Romeny, FEV The reflected spectrum is: v = viewing direction n = surface patch normal s = direction of illumination  f = Fresnel front surface reflectance coefficient in v R  = body reflectance

20. ter Haar Romeny, FEV Because of projection of the energy distribution on the image plane the vectors n, s and v will depend on the position at the imaging plane. So the energy at a point x is then related to: We assume an illumination with a locally constant color:

21. ter Haar Romeny, FEV Aim: describe material changes independent of the illumination. Both equations have many common terms

22. ter Haar Romeny, FEV The normalized differential determines material changes independent of the viewpoint, surface orientation, illumination direction, illumination intensity and illumination color!

23. ter Haar Romeny, FEV The derivative jet to x and forms a complete family of geometric invariants: These are observed properties, so we convolve with Gaussian derivatives

24. ter Haar Romeny, FEV Color edges can be defined as the thresholding of the spatial gradient (color-invariant equivalent of L w ): Color invariants

25. ter Haar Romeny, FEV Spatial color model and tracing color edges in microscopy Influence of illumination color temperature on edge strength, scale  is 3.0 px. Skin tissue section illuminated by a halogen bulb at 4000 K (top) and 2600 K (bottom) color temperature.

26. ter Haar Romeny, FEV Color-invariant multi-scale structural operators

27. ter Haar Romeny, FEV Total edge strength

28. ter Haar Romeny, FEV Some color differential invariants

29. ter Haar Romeny, FEV Feulgen stain, red-green edges Paramecium caudatum, Feulgen and Fast green stain Color canny, red-green normalized edges, scale 3

30. ter Haar Romeny, FEV Hematoxylin eosin stain Pituitary gland, sheep, adenohypophysis 40x Cell: E 0, scale 1.0 Nuclei: E 0, E +E < 0, scale 3.0 additional constraint added to refine selection

31. ter Haar Romeny, FEV Safranin O stain E > 0, E > 0, scale sigma 1.0 Safranin O stain for proteoglycans (mouse knee joint) Courtesy of Koen Gijbels and Paul Stoppie

32. ter Haar Romeny, FEV Oil red O stain Oil red O stain of fat emboli in lung E > 0, E > 0, scale 1.5

33. ter Haar Romeny, FEV PAS stain L ww > 0, L vv L ww -L vw 2 > 0, E -E > 0, scale sigma 2.0 P.A.S. stain for carbohydrates (goblet cells, gut) carbohydrates stain magenta - elliptic patches

34. ter Haar Romeny, FEV Blood smear Blood smear, Giemsa stain, 100x, JPEG compression RBC: E > 0, E +E > 0, scale 0.5 Leucocytes: E < 0, scale 12 Leucocyte nuclei: E 0, scale 3

35. ter Haar Romeny, FEV Blue-yellow edges Note the complete absence of detection of black-white edges.

36. ter Haar Romeny, FEV Color edges can also be defined as the zero-crossings of the second order derivative in the spatial gradient direction (color-invariant equivalent of L ww ): Second order color invariants

37. ter Haar Romeny, FEV Color invariant edge detection Luminance gradient edge detection

38. ter Haar Romeny, FEV

39. Conclusions Color ‘scale-space’ compatible with classical luminance scale- space The model enables the design of practical image analysis ‘color reasoning’ solutions, e.g. invariance for illumination The color-scale invariant differential operators are building blocks for a differential geometry on color images