Color2Gray: Salience-Preserving Color Removal Amy Gooch Sven Olsen Jack Tumblin Bruce Gooch Define Luminance & Chrominance

New Algorithm Color Grayscale

Problem Volume of displayable CIE L*a*b* Colors

Isoluminant Colors Color Grayscale

Converting to Grayscale… In Color Space Linear Nonlinear In Image Space Pixels (RGB) Using colors in the image Different gray for different color Relative difference Using colors in the image and their position in image space Colors can map to same gray…..

Traditional Methods: Luminance Channels Problem can not be solved by simply switching to a different space CIE CAM 97 Photoshop LAB CIE XYZ YCrCb

Traditional Methods Contrast enhancement & Gamma Correction Doesn’t help with isoluminant values Photoshop Grayscale PSGray + Auto Contrast New Algorithm

Simple Linear Mapping Luminance Axis

Principal Component Analysis (PCA) Luminance Axis

Problem with PCA Worst case: Isoluminant Colorwheel

Non-linear mapping

Contemporaneous Research Rasche et al. [2005, IEEE CG&A and EG] On the Rasche et al.'s method (in EG, at least) performs relaxation to find one fixed mapping from each input image color to one and only one output gray level; it tries to maintain distances measured ONLY in color space, and it does not have any position dependence at all. Two pixels with the same color in the input image will always get converted to the same grayscale in the output image, no matter how close together or far apart those two pixels are in screen space (xy). It is a many-to-one mapping, and loss of detail may be inevitable, as the number of colors and the number of pixels is far greater than the number of distinguishable gray levels. Our method considers spatial distance between pixels as well as color-space distance. In short, two widely separated pixels with the same color often will not be assigned to the same grayscale value; we have a many-to-many mapping that allows us to express locally visible changes that would be impossible in the Raschle method. Color Image Luminance Only Rasche et al.'s Method

Goals Dimensionality Reduction From tristimulus values to single channel Loss of information Maintain salient features in color image Human perception

Relative differences Color Illusion by Lotto and Purves SAME gray value in multiple places but look diff Can compute these pixel differences (vector of differences) Perceived color vs RGB values Color Illusion by Lotto and Purves http://www.lottolab.org

Challenge 1: Influence of neighboring pixels

Challenge 2: Dimension and Size Reduction 120, 120 100 -120, -120

Challenge 3: Many Color2Gray Solutions Original .

Algorithm Intuition . color2gray Look at DC i = 1, j = 2 luminance For first pixel 1 2 Look at DC L1 + e1,2 L2 L1 L2 L1 + e1,2 L2 + e2,1 . i = 1, j = 2 luminance For Nth pixel L1 L2 L1 L2 + e2,1 L1 L2 Look at DC

Algorithm Overview Convert to Perceptually Uniform Space CIE L*a*b* Initialize image, g, with L channel For every pixel Compute Luminance distance Compute Chrominance distance dij Adjust g to incorporate both luminance and chrominance differences

Optimization: min S S ( (gi - gj) - di,j )2 Color2Grey Algorithm i+m j=i-m Use conjugate gradient iterations to descend to a minimum

Parameters m : Radius of neighboring pixels a : Max chrominance offset q : Map chromatic difference to increases or decreases in luminance values

m : Neighborhood Size m = 2 m = 16 = entire image = 300o a = 10 butterfly_NewGrayRGB_CW_039_Crun10_N_FULL_LAB.png cB45_NewGrayRGB_CW_135_Crun10_N_FULL_LAB.png mapIslandCrop_NewGrayRGB_CW_030_Crun10_N_FULL_LAB.png MonetP_NewGrayRGB_CW_139_Crun10_N_FULL_LAB.png voiture_NewGrayRGB_CW_200_Crun08_N_FULL_LAB.png m = 2 m = 16 = entire image = 49o a = 10

m : Neighborhood Size m = 16 = entire image

a: Chromatic variation maps to luminance variation crunch(x) = a * tanh(x/a) a = 5 a = 10 a = 25

Problem: ||DCij|| is unsigned Perceptual Distance Luminance Distance: DLij = Li - Lj Chrominance Distance: ||DCij|| Problem: ||DCij|| is unsigned

Map chromatic difference to increases or decreases in luminance values +b* C2 -a* +a* Color Space C1

- - + + Color Difference Space +DC1,2 vq = (cos q, sin q) vq q +Db* +DC1,2 vq = (cos q, sin q) + - vq q -Da* +Da* + - sign(||DCi,j||) = sign(DCi,j . vq ) -Db*

q = 45 q = 225 Photoshop Grayscale

q = 135 q = 45 q = 0 Grayscale

How to Combine Chrominance and Luminance dij = DLij (Luminance)

How to Combine Chrominance and Luminance DLij (Luminance) if |DLij| > ||DCij|| dij = ||DCij|| (Chrominance)

How to Combine Chrominance and Luminance DLij if |DLij| > crunch(||DCij||) d (a) ij = crunch(||DCij||) -120, -120 120, 120 100 a -a crunch(x) = a * tanh(x/a)

How to Combine Chrominance and Luminance DLij if |DLij| > crunch(||DCij||) d(a,q)ij = crunch(||DCij||) if DCij . nq ≥ 0 crunch(-||DCij||) otherwise Grayscale .

Optimization: min S S ( (gi - gj) - di,j )2 Color2Grey Algorithm i+m i j=i-m Use conjugate gradient iterations to descend to a minimum If dij == DL then ideal image is g Otherwise, selectively modulated by DCij

Results Photoshop Grey Color2Grey + Color Original Color2Grey

Original PhotoshopGrey Color2Grey+Color Color2Grey

Original PhotoshopGrey Color2Grey

Original PhotoshopGrey Color2Grey

Implementation Performance Image of size S x S O(m2 S2) or O(S4) for full neighborhood case 12.7s 100x100 image 65.6s 150x150 image 204.0s 200x200 image GPU implementation O(S2) ideal, really O(S3) 2.8s 100x100 9.7s 150x150 25.7s 200x200 Athlon 64 3200 CPU NVIDIA GeForce GT6800

Future Work Faster Smarter Animations/Video Multiscale Remove need to specify q New optimization function designed to match both signed and unsigned difference terms Image complexity measures Animations/Video

Validate "Salience Preserving" Original PhotoshopGrey Color2Grey Apply Contrast Attention model by Ma and Zhang 2003

Validate "Salience Preserving" Original PhotoshopGrey Color2Grey

Thank you SIGGRAPH Reviewers NSF Helen and Robert J. Piros Fellowship www.color2gray.info SIGGRAPH Reviewers NSF Helen and Robert J. Piros Fellowship Northwestern Graphics Group MidGraph2004 Participants especially Feng Liu (sorry I spelled your name wrong in the acknowledgements)

Original Color2Grey Color2Grey+Color

Original Color2Grey Color2Grey+Color

Original Color2Grey Color2Grey+Color

Original PhotoshopGrey Color2Grey

Original Color2Grey Color2Grey+Color

Original PhotoshopGrey Color2Grey

Original Color2Grey Color2Grey+Color

Photoshop Grayscale

Photoshop Grayscale

Rasche et al. Photoshop Grayscale

Rasche et al. Photoshop Grayscale

Rasche et al. Photoshop Grayscale

Parameter a a = 5 a = 15 a = 25 a = 35 a = 45 a = 55 a = 65 a = 75