1 Marco Carli VPQM /01/2007 ON BETWEEN-COEFFICIENT CONTRAST MASKING OF DCT BASIS FUNCTIONS Nikolay Ponomarenko (*), Flavia Silvestri(**), Karen Egiazarian (***), Marco Carli (**), Jaakko Astola (***) and Vladimir Lukin (*) (*) National Aerospace University, Kharkov, Ukraine (**) University of Rome "Roma TRE", Rome, Italy (***) Tampere University of Technology, Tampere, Finland

2 Marco Carli VPQM /01/2007 Outline 1.Introduction 2.Proposed model of between-coefficient contrast masking of DCT basis functions 3.Modification of PSNR using a new masking model 4.MATLAB implementation of the proposed measure 5.A set of test images for comparative analysis for taking into account the masking effect in quality metrics 6.Subjective experiment to test quality measures 7.Results of the experiment 8.Examples of quality assessment of test images 9.Example of use of the proposed model to masking noise on a real image 10.Summary and Conclusion

3 Marco Carli VPQM /01/2007 Introduction Human visual sensitivity varies as a function of several key image properties, such as: Light level Spatial frequency Color Local image contrast Eccentricity Temporal frequency Goal of the research: Efficient accounting for local image contrast using a model of between- coefficient contrast masking of DCT basis functions Masking model can be used in : Image and video compression Image filtering Digital watermarking Validation of effectiveness of image processing methods Requirements to the model: Images compressed (filtered or processed) with accounting the model can be visualized in unknown illumination conditions, monitor brightness, distance to the monitor, viewing angle, etc. Thus such model should operate by only some averaged parameters of image visualization

4 Marco Carli VPQM /01/2007 Proposed model of between-coefficient contrast masking of DCT basis functions Let us denote a weighted energy of DCT coefficients of an image block 8x8 as E w (X): (1) where Xij is a DCT coefficient with indices i,j, Cij is a correcting factor determined by the CSF. The DCT coefficients X and Y are visually undistinguished if E w (X-Y) < max(E w (X)/16, E w (Y)/16), where E w (X)/16 is a masking effect E m of DCT coefficients X (normalizing factor 16 has been selected experimentally). Reducing of the masking effect due to an edge presence in the analyzed image block: we propose to reduce a masking effect for a block D proportionally to the local variances V(.) in blocks D1, D2, D3, D4 in comparison to the entire block: Em(D) = Ew(D)δ(D)/16, (2) where δ(D) = (V(D1)+V(D2)+V(D3)+V(D4))/4V(D), V(D) is the variance of the pixel values in block D.

5 Marco Carli VPQM /01/2007 Proposed model of between-coefficient contrast masking of DCT basis functions i\j Values of C ij JPEG Quantization table of Y component Values of C ij have been obtained using the quantization table for the color component Y of JPEG (the values of quantization table JPEG have been normalized by 10 and squared)

6 Marco Carli VPQM /01/2007 Modification of PSNR using a new masking model A basis of the proposed metric is a PSNR-HVS ( Egiazarian K., Astola J., Ponomarenko N., Lukin V., Battisti F., Carli M. “New full-reference quality metrics based on HVS”, CD-ROM Proceedings of the Second International Workshop on Video Processing and Quality Metrics, Scottsdale, USA, 2006, 4 p ). Flow-chart of PSNR-HVS-M calculation Reduction by value of contrast masking in accordance to the proposed model is carried out in the following manner. First, the maximal masking effect E max is calculated as max(E m (X e ), E m (X d )) where X e and X d are the DCT coefficients of a original image block and a distorted image block, respectively. Then, the visible difference between Xe and X d is determined as: X ∆ij = where E norm is.

7 Marco Carli VPQM /01/2007 MATLAB implementation of the proposed measure The MATLAB implementation of PSNR-HVS-M is available on

8 Marco Carli VPQM /01/2007 A set of test images for comparative analysis for taking into account the masking effect in quality metrics While creating an image test set we took into consideration the following: Such set should contain images with both spatially uncorrelated and correlated noise (the latter one is typical for images formed by digital cameras and is more visible for humans); The set should contain images with noise distributed spatially uniformly and with noise which is masked or unmasked (concentrated in regions with maximal and minimal masking properties, respectively); The set is to be maximally simple for visual comparison by humans (because of this in our set we used only three values of noise variance σ 2 and a total number of distorted test images was 2x3x3 = 18 images). Original test images having a lot of different type regions with high masking effect

9 Marco Carli VPQM /01/2007 Subjective experiment to test quality measures Result of the experiment: the test image set ordered according to subjective visual quality. Number of observers: 155 (45 from Finland, 43 from Italy, 67 from Ukraine). Number of comparisons of visual appearance of test images: 8192 (on average 53 for each observer). 17” or 19” Monitor Resolution: 1152x864 pixels. Number of experiments carried out using CRT monitors: 128. Number of experiments carried out using LCD monitors: 27. Group of observersSpearman correlationKendall correlation Finland – Italy Finland – Ukraine Italy - Ukraine CRT - LCD Cross correlation factors

10 Marco Carli VPQM /01/2007 Results of the experiment MeasureReference Spearman correlation Kendall correlation PSNR-HVS-MThis paper PSNR-HVS Egiazarian K., Astola J., Ponomarenko N., Lukin V., Battisti F., Carli M. “New full- reference quality metrics based on HVS”, CD-ROM Proceedings of the Second Intern. Workshop on Video Processing and Quality Metrics, Scottsdale, USA, 2006, 4 p NQM Damera-Venkata N., Kite T., Geisler W., Evans B. and Bovik A. "Image Quality Assessment Based on a Degradation Model", IEEE Trans. on Image Processing, Vol. 9, 2000, pp DCTune Solomon J. A., Watson A. B., and Ahumada A. “Visibility of DCT basis functions: Effects of contrast masking”. Proc. of Data Compression Conf., 1994, pp DCTune 2.0 page UQI Wang Z., Bovik A. “A universal image quality index”, IEEE Signal Processing Letters, vol. 9, March, 2002, pp. 81– PSNRPeak Signal to Noise Ratio VQMXiao F. “DCT-based Video Quality Evaluation”, Final Project for EE392J, SSIM Wang Z., Bovik A., Sheikh H., Simoncelli E. “Image quality assessment: from error visibility to structural similarity”, IEEE Trans. on Image Proc., vol.13, 2004, pp VIF Sheikh H. R. and Bovik A. C., "Image Information and Visual Quality", IEEE Transactions on Image Processing, vol. 15, February, 2006, pp PQS Miyahara, M., Kotani, K., Algazi, V.R. ”Objective picture quality scale (PQS) for image coding”, IEEE Transactions on Communications, vol. 46, issue 9, 1998, pp

11 Marco Carli VPQM /01/2007 Examples of quality assessment of test images DCTune = 24.9, PSNR-HVS-M = dB PSNR-HVS-M says: “This is better!” DCTune = 24.5, PSNR-HVS-M = dB DCTune says: “This is better!”

12 Marco Carli VPQM /01/2007 Examples of quality assessment of test images SSIM = 0.80, PSNR-HVS-M = dB SSIM says: “This is better!” SSIM = 0.79, PSNR-HVS-M = dB PSNR-HVS-M says: “This is better!”

13 Marco Carli VPQM /01/2007 Example of use of the proposed model to masking noise on a real image Original test image Baboon The image with masked noise, PSNR=26.18 dB, MSE=158, PSNR-HVS=34.43 dB, PSNR-HVS-M=51.67 dB

14 Marco Carli VPQM /01/2007 Summary and Conclusion Summary A simple and efficient model of between-coefficient contrast masking of DCT basis functions is proposed; A modification of PSNR that takes into account this masking model is proposed; Subjective experiments on comparison of known quality metrics are carried out; Conclusions The proposed measure based on the designed masking model has demonstrated the best correspondence to the results of the subjective experiments. However for providing more reliable conclusions on efficiency of the proposed model it is necessary to carry out additional more extensive experiments and research. The proposed test set has allowed to demonstrate drawbacks of many well known metrics that do not fully or even badly correspond to human visual perception.