1. Purdue University Page 1 Color Image Fidelity Assessor Color Image Fidelity Assessor * Wencheng Wu (Xerox Corporation) Zygmunt Pizlo (Purdue University) Jan P. Allebach (Purdue University) * Research supported by HP Company while Wencheng Wu was at Purdue

2. Purdue University Page 2 Outline Introduction Spatial color descriptor: chromatic difference Structure of Color Image Fidelity Assessor (CIFA) Psychophysical experiment and its results Test examples Conclusion

3. Purdue University Page 3 Introduction (Motivation) Image fidelity assessment is important in the development of imaging systems and image processing algorithms  Create visually lossless reproduction  Allocate efforts on most visible area Subjective evaluation is expensive and slow.

4. Purdue University Page 4 Introduction (Prior work) Simple but not working  Root-Mean-Square Error Consider structure of HVS and perceptual process  Achromatic: Daly’s VDP, Lubin’s VDM, Taylor’s Achromatic IFA (IFA)  Color: Jin’s CVDM (Daly’s VDP + Wandell’s Spatial CIE Lab)

5. Purdue University Page 5 Introduction (CVDM vs. CIFA) Both operate along opponent-color coordinates Both incorporate results from electrophysiological and psychophysical exp. They differ in a similar way as VDP vs. IFA  CIFA has closer link between the structure of the model and the psychophysical data used by the model CIFA normalize the chromatic responses  This discounts luminance effect in chromatic channels  This reduces the dimension of psychometric LUT

6. Purdue University Page 6 Introduction (Overview of CIFA) Color extension of Taylor’s achromatic IFA The model predicts perceived image fidelity  Assesses visible differences in the opponent channels  Explains the nature of visible difference (luminance change vs. color shift) Color Image Fidelity Assessor (CIFA) Ideal Rendered Viewing parameters Image maps of predicted visible differences

7. Purdue University Page 7 Chromatic difference (Definition) Objective: evaluate the spatial interaction between colors First transform CIE XYZ to opponent color space (O 2,O 3 ) * * X. Zhang and B.A. Wandell, “A SPATIAL EXTENSION OF CIELAB FOR DIGITAL COLOR IMAGE REPRODUCTION”, SID-97 Then normalize to obtain opponent chromaticities (o 2,o 3 ) Define chromatic difference (analogous to luminance contrast c 1 ) Luminance  Red-Green  Blue-Yellow 

8. Purdue University Page 8 Opponent color representation (13.3,o 2,0.17) (13.3,0.24,o 3 ) (Y,0.24,0.17) (Y,o 2,o 3 )

9. Purdue University Page 9 Chromatic difference (illustration) Chromatic difference is a measure of chromaticity variation Chromatic difference is a spatial feature derived from opponent chromaticity that has little dependence upon luminance 0.1 0.20.05 Chromatic difference is the amplitude of the sinusoidal grating

10. Purdue University Page 10 CIFA Ideal Y Image Rendered Y Image Ideal O 2 Image Rendered O 2 Image Ideal O 3 Image Rendered O 3 Image Blue - yellow IFA Red - green IFA Achromatic* IFA Chromatic IFAs * Previous work of Taylor et al (Y,O 2,O 3 ): Opponent representation of an image Multi-resolution Y images Image map of predicted visible luminance differences Image map of predicted visible blue-yellow differences Image map of predicted visible red-green differences

11. Purdue University Page 11 Psychometric LUT  (f,o 2,c 2 ) Chromatic diff. discrimination Red-green IFA Psychometric Selector Channel Response Predictor Limited Memory Prob. Sum. Lowpass Pyramid Lowpass Pyramid Chromatic Diff. Decomposition Chromatic Diff. Decomposition  +–+– Adaptation level Contrast Decomposition Contrast Decomposition Achromatic IFA Psychometric LUT  (f,Y,c 1 ) Lum. contrast discrimination Contrast: luminance contrast & chromatic difference

12. Purdue University Page 12 IFA components Psychometric LUT  Results from psychophysical experiment  Stored in the form of Lookup-Table:  (f,Y,c 1 ),  (f,o 2,c 1 ),  (f,o 3,c 1 )  Time consuming, but it is done off-line Image processing:  Lowpass pyramid: create 5 multi-resolution images »Lowpass filtering +  2 in horizontal and vertical direction »Normalized by Y images if it is a chromatic IFA  Signal decomposition: create 8 orientation-specific contrast or chromatic- difference images at each resolution  Lowpass pyramid + Signal decomposition: 40 (5 levels  8 orientations) visual channels for each image pixel

13. Purdue University Page 13 IFA components (cont’d) Image processing (continued):  Psychometric selector: for each pixel at each visual channel, find discrimination threshold by choosing appropriate data from LUT  Channel response predictor: for each pixel at each visual channel, convert chromatic difference to discrimination probability  Limited memory probability summation: for each pixel, combine discrimination probability across all 40 visual channel

14. Purdue University Page 14 Estimating parameters of LUT (Stimulus: Isoluminant Gabor patch) Red-green (O 2 or o 2 ) stimulus  Keep Y, O 3 (o 3 ) constant  Let O 2 =Yo 2 +Yc 2 cos(.)e (.) or equivalently o 2 ’ =o 2 +c 2 cos(.)e (.) (Y,o 2,o 3 ) specifies the background color, c 2 is the chromatic difference Gabor patch  f, o 2, c 2

15. Purdue University Page 15 Estimating parameters of LUT (Psychophysical method) Red-green stimulus: (Y,o 2,o 3 ) specifies the background color, c 2 is the ref. chromatic difference Which stimulus has less chromatic difference?

16. Purdue University Page 16 Subject WW’s responses probability Estimating parameters of LUT (Data analysis) Fit subject’s responses to a Normal distribution using probit analysis Record the standard deviation as the discrimination threshold  LUT:  rg (f,o 2,c 2 ) 

17. Purdue University Page 17 Estimating parameters of LUT (List of experimental conditions)   indicate spatial frequency of 1, 2, 4, 8, 16 cpd

18. Purdue University Page 18 Representative results Results for f = 16, 8, 4, 2, 1 cycle/deg are drawn in red, green, blue, yellow, and black. Threshold is not affected strongly by the reference chromatic difference Chromatic channels function like low-pass filters Reference c 3 Reference c 2 Threshold  Red-green discrimination at RG1:(Y,o 2,o 3 )=(5,0.2,-0.3) Blue-yellow discrimination at BY1:(Y,o 2,o 3 )=(5,0.3,0.2)

19. Purdue University Page 19 CIFA output for example distortions (Hue change) LuminanceR-GB-Y

20. Purdue University Page 20 CIFA output for example distortions (Blurring) Luminance R-G B-Y

21. Purdue University Page 21 CIFA output for example distortions (Limited gamut) LuminanceR-GB-Y

22. Purdue University Page 22 Conclusion CIFA provides good assessment of the perceived visible differences over a range of image contents and distortion types Chromatic difference describes the color percept of HVS efficiently Suggestions on future directions  Add DC component in the LUT in chromatic IFAs  Subjective validation  Improve spatial localization  Take dependency between visual channels into account (in prob. Sum. stage)

23. Purdue University Page 23 CIFA output for example distortions (Limited color quantization) LuminanceR-GB-Y

24. Purdue University Page 24 CIFA output for example distortions (Limited gamut) LuminanceR-GB-Y

25. Purdue University Page 25 CIFA output for example distortions (Increased saturation) LuminanceR-GB-Y