Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University

2 What’s the colour2grey problem? A black and white image has one channel. Ok, so make from RGB: use some form of Luminance – e.g., L*, Y(IQ), Y(UV), Y(CbCr),… or use PCA to find main direction What about a multi-spectral satellite image: 200 channels? Well, start with RGB colour images – how about averaging, say:

3 Oops – equi-luminous images: Very common situation, from graphics, so need more sophistication.

4 The problem geometrically: say in L*a*b*: Obviously, very different colours project to identical grey values.

5 Many approaches: [Bala & Eschbach, CIC2004] -- a component related to edges of the chromatic 2-vector is added to the brightness: spatially adaptive. [Gooch et al., SIGGRAPH05] – Color2Gray: iterative optimization (with multiple local minima!) to map colours to grey maintaining chromatic plus luminance difference as well as possible. [Grundland & Dodgson, Patt.Rec. 2007] – a single achromatic channel to optimize chromatic contrast: a combination of luminance and a global “predominant” chromatic change coordinate  must set several parameters, need user input. E.g.,

6 Color2Gray is likely the most well-known. Again, needs several parameters to be set by the user: -- so-called “aesthetic decisions”: choose a colour direction for “cooler” to “hotter” colours choose a window size for the optimization choose a clamping value to clip the effect of a*,b* on L* Algorithm is very slow (unconstrained optimization over thousands of variables)… Also,

Parameters make trouble: This paper Input Gooch Lum. Uh-oh! For this image (© Jay Neitz) only colorblind people should see the “45”!

8 Another example: Lum. This paper Gooch

9 We could of course improve the Gooch et al. output by fixing the parameters manually… [and, some aspects of these algorithms are indeed compelling and could be added to the method presented here!] But, can we devise a parameter-free method? (and also, let’s not forget higher-D colour than RGB) Basic tenet of such algorithms is that we wish to map colour contrast into greyscale contrast. But what is contrast ? --

10 [Socolinsky & Wolff, IEEE Trans. Image Proc., 2002] – use math definition of what is gradient of an n-dimensional image: the colour-channel gradient 2-vectors ▽ R, ▽ G, ▽ B, combine to induce a metric for the underlying geometry. The resulting greyscale gradient is in the direction of maximal change. Contrast: Details: [Di Zenzo, 1986]: Define a 2x2 structure tensor (first fundamental form, in math) --- main eigenvector points in the direction of maximal rate of change.

11 What does this actually look like: * ▽ g based on strong mathematical guarantee on results *

12 Details: In the case of a 3-band color image   = {R, G, B}, we consider the 2x3 array of color-channel gradient 2-vectors ▽  = { ▽ R, ▽ G, ▽ B}. Now form the  2x2 array Structure Tensor Z: 200 Let’s not forget satellite images… 200

Back to RGB: Z is the matrix of outer products of the 2D color-channel gradient components: (2x3) * (3x2) Z is real symmetric, so its eigenvectors form an orthogonal matrix V, The (normalized) eigenvectors of Z point in the direction of minimum- and maximum-contrast, for an underlying grayscale image with metric induced by the structure tensor.

14  Colour to Grey-Gradient: Let V={u, v} with 2-vectors u  v. Then the maximum-eigenvalue direction v is associated with maximum contrast in the grey image, with norm  Grey-Gradient to Grey : Re-integrating gray-gradient -- take another derivative and solve Poisson’s equation:

15 Done ? Problem: eigenvectors are defined only up to sign: ± If we get sign wrong, we have integrability error:

If we make a ± mistake, leads to bends, folds, halos! Wolff solution: use sign of gradient of Luminance. Problem: what if image is  iso-luminant (common issue).

If we make a ± mistake, leads to bends, folds, halos! Problem: Iso-luminant: Wolff RGB This paper

18 Our solution: use max’s of ▽ R, ▽ G, ▽ B, in each compass direction: No need to consider ± signs!

19 Learn the weights  ={ , , ,  } by optimizing closeness of colour Z to resulting grey Z: …

20 Median weights  ={ 0.530, 0.430, 0.537, }; Can we make faster? Use Frankot-Chellappa algorithm in Fourier space to solve for grey g from gradient ▽ g: Takes a 2 nd derivative, by multiplying by the transform of the derivative operator in the frequency domain, and solves the resulting Poisson equation by going back to the spatial domain. Uses  ={1,0,1,0}. Can use FFTW – fast! and then solving for g:

21 Results: Input Lum.Fast Colour2Grey

22 …Results Input Lum. This paper

23 …Results Input Lum. This paper

24 …Results Input Lum. This paper Lum.

25 …Results InputLum. WolffThis paper

26 Thanks! To Natural Sciences and Engineering Research Council of Canada