1. Color2Gray: Salience-Preserving Color Removal
Amy Gooch
Sven Olsen
Jack Tumblin
Bruce Gooch
Define Luminance & Chrominance

2. New Algorithm
Color
Grayscale

3. Problem
Volume of displayable CIE L*a*b* Colors

4. Isoluminant Colors
Color
Grayscale

5. Converting to Grayscale…
In Color Space
Linear
Nonlinear
In Image Space
Pixels (RGB)
Using colors in the image
Different gray for different color
Relative difference
Using colors in the image and their position in image space
Colors can map to same gray…..

6. Traditional Methods: Luminance Channels
Problem can not be solved by
simply switching to a different space
CIE CAM 97
Photoshop LAB
CIE XYZ
YCrCb

7. Traditional Methods Contrast enhancement & Gamma Correction
Doesn’t help with isoluminant values
Photoshop Grayscale
PSGray + Auto Contrast
New Algorithm

8. Simple Linear Mapping
Luminance Axis

9. Principal Component Analysis (PCA)
Luminance Axis

10. Problem with PCA
Worst case:
Isoluminant Colorwheel

11. Non-linear mapping

12. Contemporaneous Research
Rasche et al. [2005, IEEE CG&A and EG]
On the Rasche et al.'s method (in EG, at least) performs relaxation to find one fixed mapping from each input image color to one and only one output gray level; it tries to maintain distances measured ONLY in color space, and it does not have any position dependence at all. Two pixels with the same color in the input image will always get converted to the same grayscale in the output image, no matter how close together or far apart those two pixels are in screen space (xy). It is a many-to-one mapping, and loss of detail may be inevitable, as the number of colors and the number of pixels is far greater than the number of distinguishable gray levels.
Our method considers spatial distance between pixels as well as color-space distance. In short, two widely separated pixels with the same color often will not be assigned to the same grayscale value; we have a many-to-many mapping that allows us to express locally visible changes that would be impossible in the Raschle method.
Color Image
Luminance Only
Rasche et al.'s Method

13. Goals Dimensionality Reduction
From tristimulus values to single channel
Loss of information
Maintain salient features in color image
Human perception

14. Relative differences Color Illusion by Lotto and Purves
SAME gray value in multiple places but look diff
Can compute these pixel differences (vector of differences)
Perceived color vs RGB values
Color Illusion by
Lotto and Purves

15. Challenge 1: Influence of neighboring pixels

16. Challenge 2: Dimension and Size Reduction
120, 120
100
-120, -120

17. Challenge 3: Many Color2Gray Solutions
Original .

18. Algorithm Intuition . color2gray Look at DC i = 1, j = 2 luminance
For first pixel 1 2
Look
at DC
L1 + e1,2 L2 L1 L2
L1 + e1,2
L2 + e2,1 .
i = 1, j = 2
luminance
For Nth pixel L1 L2 L1
L2 + e2,1 L1 L2
Look
at DC

19. Algorithm Overview Convert to Perceptually Uniform Space
CIE L*a*b*
Initialize image, g, with L channel
For every pixel
Compute Luminance distance
Compute Chrominance distance
dij
Adjust g to incorporate both luminance and chrominance differences

20. Optimization: min S S ( (gi - gj) - di,j )2 Color2Grey Algorithm i+m
j=i-m
Use conjugate gradient iterations to descend to a minimum

21. Parameters m : Radius of neighboring pixels a : Max chrominance offset
q :
Map chromatic difference to increases or decreases in luminance values

22. m : Neighborhood Size m = 2 m = 16 = entire image = 300o a = 10
butterfly_NewGrayRGB_CW_039_Crun10_N_FULL_LAB.png
cB45_NewGrayRGB_CW_135_Crun10_N_FULL_LAB.png
mapIslandCrop_NewGrayRGB_CW_030_Crun10_N_FULL_LAB.png
MonetP_NewGrayRGB_CW_139_Crun10_N_FULL_LAB.png
voiture_NewGrayRGB_CW_200_Crun08_N_FULL_LAB.png
m = 2
m = 16
= entire image
= 49o
a = 10

23. m : Neighborhood Size
m = 16
= entire image

24. a: Chromatic variation maps to luminance variation
crunch(x) = a * tanh(x/a)
a = 5
a = 10
a = 25

25. Problem: ||DCij|| is unsigned
Perceptual Distance
Luminance Distance:
DLij = Li - Lj
Chrominance Distance: ||DCij||
Problem: ||DCij|| is unsigned

26. Map chromatic difference to increases or decreases in luminance values
+b* C2
-a*
+a*
Color
Space C1

27. - - + + Color Difference Space +DC1,2 vq = (cos q, sin q) vq q
+Db*
+DC1,2
vq = (cos q, sin q) + - vq q
-Da*
+Da* + -
sign(||DCi,j||) = sign(DCi,j . vq )
-Db*

28. q = 45
q = 225
Photoshop Grayscale

29. q = 135
q = 45
q = 0
Grayscale

30. How to Combine Chrominance and Luminance
dij =
DLij
(Luminance)

31. How to Combine Chrominance and Luminance
DLij
(Luminance)
if |DLij| > ||DCij||
dij =
||DCij||
(Chrominance)

32. How to Combine Chrominance and Luminance
DLij
if |DLij| > crunch(||DCij||)
d (a) ij =
crunch(||DCij||)
-120, -120
120, 120
100 a -a
crunch(x) = a * tanh(x/a)

33. How to Combine Chrominance and Luminance
DLij
if |DLij| > crunch(||DCij||)
d(a,q)ij =
crunch(||DCij||)
if DCij . nq ≥ 0
crunch(-||DCij||)
otherwise
Grayscale .

34. Optimization: min S S ( (gi - gj) - di,j )2 Color2Grey Algorithm i+m i
j=i-m
Use conjugate gradient iterations to descend to a minimum
If dij == DL then ideal image is g
Otherwise, selectively modulated by DCij

35. Results
Photoshop
Grey
Color2Grey
+ Color
Original
Color2Grey

36. Original
PhotoshopGrey
Color2Grey+Color
Color2Grey

37. Original
PhotoshopGrey
Color2Grey

38. Original
PhotoshopGrey
Color2Grey

39. Implementation Performance
Image of size S x S
O(m2 S2) or O(S4) for full neighborhood case
12.7s 100x100 image
65.6s 150x150 image
204.0s 200x200 image
GPU implementation
O(S2) ideal, really O(S3)
2.8s 100x100
9.7s 150x150
25.7s 200x200
Athlon CPU
NVIDIA GeForce GT6800

40. Future Work Faster Smarter Animations/Video Multiscale
Remove need to specify q
New optimization function designed to match both signed and unsigned difference terms
Image complexity measures
Animations/Video

41. Validate "Salience Preserving"
Original
PhotoshopGrey
Color2Grey
Apply Contrast Attention model by Ma and Zhang 2003

42. Validate "Salience Preserving"
Original
PhotoshopGrey
Color2Grey

43. Thank you SIGGRAPH Reviewers NSF Helen and Robert J. Piros Fellowship
SIGGRAPH Reviewers
NSF
Helen and Robert J. Piros Fellowship
Northwestern Graphics Group
MidGraph2004 Participants
especially Feng Liu
(sorry I spelled your name wrong in the acknowledgements)

45. Original
Color2Grey
Color2Grey+Color

46. Original
Color2Grey
Color2Grey+Color

47. Original
Color2Grey
Color2Grey+Color

48. Original
PhotoshopGrey
Color2Grey

49. Original
Color2Grey
Color2Grey+Color

50. Original
PhotoshopGrey
Color2Grey

51. Original
Color2Grey
Color2Grey+Color

52. Photoshop Grayscale

53. Photoshop Grayscale

54. Rasche et al.
Photoshop Grayscale

55. Rasche et al.
Photoshop Grayscale

56. Rasche et al.
Photoshop Grayscale

57. Parameter a a = 5 a = 15 a = 25 a = 35 a = 45 a = 55 a = 65 a = 75