1. 1 Marco Carli VPQM 2006 26/01/2007 ON BETWEEN-COEFFICIENT CONTRAST MASKING OF DCT BASIS FUNCTIONS Nikolay Ponomarenko (*), Flavia Silvestri(**), Karen Egiazarian (***), Marco Carli (**), Jaakko Astola (***) and Vladimir Lukin (*) (*) National Aerospace University, Kharkov, Ukraine (**) University of Rome "Roma TRE", Rome, Italy (***) Tampere University of Technology, Tampere, Finland

2. 2 Marco Carli VPQM 2006 26/01/2007 Outline 1.Introduction 2.Proposed model of between-coefficient contrast masking of DCT basis functions 3.Modification of PSNR using a new masking model 4.MATLAB implementation of the proposed measure 5.A set of test images for comparative analysis for taking into account the masking effect in quality metrics 6.Subjective experiment to test quality measures 7.Results of the experiment 8.Examples of quality assessment of test images 9.Example of use of the proposed model to masking noise on a real image 10.Summary and Conclusion

3. 3 Marco Carli VPQM 2006 26/01/2007 Introduction Human visual sensitivity varies as a function of several key image properties, such as: Light level Spatial frequency Color Local image contrast Eccentricity Temporal frequency Goal of the research: Efficient accounting for local image contrast using a model of between- coefficient contrast masking of DCT basis functions Masking model can be used in : Image and video compression Image filtering Digital watermarking Validation of effectiveness of image processing methods Requirements to the model: Images compressed (filtered or processed) with accounting the model can be visualized in unknown illumination conditions, monitor brightness, distance to the monitor, viewing angle, etc. Thus such model should operate by only some averaged parameters of image visualization

4. 4 Marco Carli VPQM 2006 26/01/2007 Proposed model of between-coefficient contrast masking of DCT basis functions Let us denote a weighted energy of DCT coefficients of an image block 8x8 as E w (X): (1) where Xij is a DCT coefficient with indices i,j, Cij is a correcting factor determined by the CSF. The DCT coefficients X and Y are visually undistinguished if E w (X-Y) < max(E w (X)/16, E w (Y)/16), where E w (X)/16 is a masking effect E m of DCT coefficients X (normalizing factor 16 has been selected experimentally). Reducing of the masking effect due to an edge presence in the analyzed image block: we propose to reduce a masking effect for a block D proportionally to the local variances V(.) in blocks D1, D2, D3, D4 in comparison to the entire block: Em(D) = Ew(D)δ(D)/16, (2) where δ(D) = (V(D1)+V(D2)+V(D3)+V(D4))/4V(D), V(D) is the variance of the pixel values in block D.

5. 5 Marco Carli VPQM 2006 26/01/2007 Proposed model of between-coefficient contrast masking of DCT basis functions i\j01234567 000.82641.00000.39060.17360.06250.03840.0269 10.6944 0.51020.27700.14790.02970.02780.0331 20.51020.59170.39060.17360.06250.03080.02100.0319 30.51020.34600.20660.11890.03840.01320.01560.0260 40.30860.20660.07300.03190.02160.00840.00940.0169 50.17360.08160.03310.02440.01520.00920.00780.0118 60.04160.02440.01640.01320.00940.00680.00690.0098 70.01930.01180.01110.01040.00800.01000.00940.0102 Values of C ij 1611101624405161 12 141926586055 1413162440576956 1417222951878062 182237566810910377 243555648110411392 49647887103121120101 7292959811210010399 JPEG Quantization table of Y component Values of C ij have been obtained using the quantization table for the color component Y of JPEG (the values of quantization table JPEG have been normalized by 10 and squared)

6. 6 Marco Carli VPQM 2006 26/01/2007 Modification of PSNR using a new masking model A basis of the proposed metric is a PSNR-HVS ( Egiazarian K., Astola J., Ponomarenko N., Lukin V., Battisti F., Carli M. “New full-reference quality metrics based on HVS”, CD-ROM Proceedings of the Second International Workshop on Video Processing and Quality Metrics, Scottsdale, USA, 2006, 4 p ). Flow-chart of PSNR-HVS-M calculation Reduction by value of contrast masking in accordance to the proposed model is carried out in the following manner. First, the maximal masking effect E max is calculated as max(E m (X e ), E m (X d )) where X e and X d are the DCT coefficients of a original image block and a distorted image block, respectively. Then, the visible difference between Xe and X d is determined as: X ∆ij = where E norm is.

7. 7 Marco Carli VPQM 2006 26/01/2007 MATLAB implementation of the proposed measure The MATLAB implementation of PSNR-HVS-M is available on www.cs.tut.fi/~ponom/psnrhvsm.htm

8. 8 Marco Carli VPQM 2006 26/01/2007 A set of test images for comparative analysis for taking into account the masking effect in quality metrics While creating an image test set we took into consideration the following: Such set should contain images with both spatially uncorrelated and correlated noise (the latter one is typical for images formed by digital cameras and is more visible for humans); The set should contain images with noise distributed spatially uniformly and with noise which is masked or unmasked (concentrated in regions with maximal and minimal masking properties, respectively); The set is to be maximally simple for visual comparison by humans (because of this in our set we used only three values of noise variance σ 2 and a total number of distorted test images was 2x3x3 = 18 images). Original test images having a lot of different type regions with high masking effect

9. 9 Marco Carli VPQM 2006 26/01/2007 Subjective experiment to test quality measures Result of the experiment: the test image set ordered according to subjective visual quality. Number of observers: 155 (45 from Finland, 43 from Italy, 67 from Ukraine). Number of comparisons of visual appearance of test images: 8192 (on average 53 for each observer). 17” or 19” Monitor Resolution: 1152x864 pixels. Number of experiments carried out using CRT monitors: 128. Number of experiments carried out using LCD monitors: 27. Group of observersSpearman correlationKendall correlation Finland – Italy0.9960.895 Finland – Ukraine0.9960.935 Italy - Ukraine0.9970.961 CRT - LCD0.9980.922 Cross correlation factors

10. 10 Marco Carli VPQM 2006 26/01/2007 Results of the experiment MeasureReference Spearman correlation Kendall correlation PSNR-HVS-MThis paper 0.9840.948 PSNR-HVS Egiazarian K., Astola J., Ponomarenko N., Lukin V., Battisti F., Carli M. “New full- reference quality metrics based on HVS”, CD-ROM Proceedings of the Second Intern. Workshop on Video Processing and Quality Metrics, Scottsdale, USA, 2006, 4 p 0.8950.712 NQM Damera-Venkata N., Kite T., Geisler W., Evans B. and Bovik A. "Image Quality Assessment Based on a Degradation Model", IEEE Trans. on Image Processing, Vol. 9, 2000, pp. 636-650 0.8570.673 DCTune Solomon J. A., Watson A. B., and Ahumada A. “Visibility of DCT basis functions: Effects of contrast masking”. Proc. of Data Compression Conf., 1994, pp. 361-370 http://vision.arc.nasa.gov/dctune/ - DCTune 2.0 page 0.8290.712 UQI Wang Z., Bovik A. “A universal image quality index”, IEEE Signal Processing Letters, vol. 9, March, 2002, pp. 81–84 0.5500.438 PSNRPeak Signal to Noise Ratio 0.5370.359 VQMXiao F. “DCT-based Video Quality Evaluation”, Final Project for EE392J, 2000 0.4410.281 SSIM Wang Z., Bovik A., Sheikh H., Simoncelli E. “Image quality assessment: from error visibility to structural similarity”, IEEE Trans. on Image Proc., vol.13, 2004, pp.600-612 0.4060.358 VIF Sheikh H. R. and Bovik A. C., "Image Information and Visual Quality", IEEE Transactions on Image Processing, vol. 15, February, 2006, pp. 430-444 0.3770.255 PQS Miyahara, M., Kotani, K., Algazi, V.R. ”Objective picture quality scale (PQS) for image coding”, IEEE Transactions on Communications, vol. 46, issue 9, 1998, pp. 1215-1226 0.3020.242

11. 11 Marco Carli VPQM 2006 26/01/2007 Examples of quality assessment of test images DCTune = 24.9, PSNR-HVS-M = 33.20 dB PSNR-HVS-M says: “This is better!” DCTune = 24.5, PSNR-HVS-M = 29.31 dB DCTune says: “This is better!”

12. 12 Marco Carli VPQM 2006 26/01/2007 Examples of quality assessment of test images SSIM = 0.80, PSNR-HVS-M = 25.50 dB SSIM says: “This is better!” SSIM = 0.79, PSNR-HVS-M = 31.29 dB PSNR-HVS-M says: “This is better!”

13. 13 Marco Carli VPQM 2006 26/01/2007 Example of use of the proposed model to masking noise on a real image Original test image Baboon The image with masked noise, PSNR=26.18 dB, MSE=158, PSNR-HVS=34.43 dB, PSNR-HVS-M=51.67 dB

14. 14 Marco Carli VPQM 2006 26/01/2007 Summary and Conclusion Summary A simple and efficient model of between-coefficient contrast masking of DCT basis functions is proposed; A modification of PSNR that takes into account this masking model is proposed; Subjective experiments on comparison of known quality metrics are carried out; Conclusions The proposed measure based on the designed masking model has demonstrated the best correspondence to the results of the subjective experiments. However for providing more reliable conclusions on efficiency of the proposed model it is necessary to carry out additional more extensive experiments and research. The proposed test set has allowed to demonstrate drawbacks of many well known metrics that do not fully or even badly correspond to human visual perception.