1. H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and Video Engi., Dept. of ECE Univ. of Texas at Austin

2. Outline  Introduction of Image Quality Assessment  Visual Information Fidelity  Experiments and Results  Conclusion

3. Quality Assessment (QA)  For testing, optimizing, bench-marking, and monitoring applications. Quality?

4. Three Broad QA Categories  Full-Reference (FR) QA Methods  Non-Reference (NR) QA Methods  Reduced-Reference (RR) QA Methods Reference Image Distorted Image FR QA Quality

5. PSNR  Simple but not close to human visual quality Contrast enhancement Blurred JPEG compressed VIF = 1.10 VIF = 0.07 VIF = 0.10

6. Prior Arts  Image Quality Assessment based on Error Sensitivity CSF: Contrast Sensitivity Function Channel Decomposition: DCT or Wavelet Transform Error Normalization: Convert the Error into Units of Just Noticeable Difference (JND) Error Pooling:

7. Problems of Error-Sensitivity Approaches  The Quality Definition Problem  The Suprathreshold Problem  The Natural Image Complexity Problem  The Decorrelation Problem  The Cognitive Interaction Problem

8. Visual Information Fidelity Natural Image Source Channel (Distortion) Channel (Distortion) HVS CD F E Reference Image Test Image Human Visual System Reference Image Information Human Visual System Test Image Information

9. Definition of VIF Natural Image Source Channel (Distortion) Channel (Distortion) HVS CD F E

10. Source Model  The natural images are modeled in the wavelet domain using Gaussian scale mixtures (GSMs). The subband coefficients are partitioned into nonoverlapping blocks of M coefficients each

11. Gaussian scale mixture (GSM)  Wavelet coefficient => non-Gaussian The variance is proportional to the squared magnitudes of coefficients at spatial positions. V. Strela, J. Portilla, and E. Simoncelli, “Image denoising using a local Gaussian scale mixture model in the wavelet domain,” Proc. SPIE, vol. 4119, pp. 363–371, 2000.

12. Implementation Issues  Assumption about the source model: V. Strela, J. Portilla, and E. Simoncelli, “Image denoising using a local Gaussian scale mixture model in the wavelet domain,” Proc. SPIE, vol. 4119, pp. 363–371, 2000.

13. Distortion Model

14. Distorted Images Synthesized versions

15. Distorted Images

16. Human Visual System (HVS) Model Natural Image Source Channel (Distortion) Channel (Distortion) HVS CD F E

17. Visual Information Fidelity Criterion (IFC) Natural Image Source Channel (Distortion) Channel (Distortion) HVS CD F E CE Mutual Information I(C;E)

18. Mutual Information  Assuming that G, and are known C E Mutual Information I(C;E)

19. Mutual Information

20. Implementation Issues  Assumption about the source model: V. Strela, J. Portilla, and E. Simoncelli, “Image denoising using a local Gaussian scale mixture model in the wavelet domain,” Proc. SPIE, vol. 4119, pp. 363–371, 2000.

21. Implementation Issues  Assumption about the distortion model: use B x B window centered at coefficient i to estimate and at i  Assumption about the HVS model: Hand-optimize the value of (by linear regression)

22. Definition of VIF Natural Image Source Channel (Distortion) Channel (Distortion) HVS CD F E

23. Experiments  Twenty-nine high-resolution(768x512) 24- bits/pixel RGB color images  Five distortion types: JPEG 2000, JPEG, white noise in RGB components, Gaussian blur, and transmission errors  20-25 human observers  Perception of quality: “Bad,” “Poor,” “Fair,” “Good,” and “Excellent”  Scale to 1-100 range and obtain the difference mean opinion score (DMOS) for each distorted image  Data base: http://live.ece.utexas.edu/research/quality/ http://live.ece.utexas.edu/research/quality/

24. Scatter Plots for Four Objective Quality Criteria (x) JPEG2000, (+) JPEG, (o) white noise in RGB space, (box) Gaussian blur, and (diamond) transmission Errors in JPEG2000 stream over fast-fading Rayleigh channel

25. Scatter Plots for the Quality Prediction

26. Validation Scores THE VALIDATION CRITERIA ARE: CORRELATION COEFFICIENT (CC), MEAN ABSOLUTE ERROR (MAE), ROOT MEAN-SQUARED ERROR (RMS), OUTLIER RATIO(OR), AND SPEARMAN RANK-ORDER CORRELATION COEFFICIENT (SROCC) Two version of VIF: VIF using the finest resolution at all orientations and Using the horizontal and vertical orientations only

27. Cross-Distortion Performance

28. (dark solid) JPEG2000, (dashed) JPEG, (dotted) white noise, (dash-dot) Gaussian blur, and (light solid) transmission errors in JPEG2000 stream over fast-fading Rayleigh channel

29. Dependence on the HVS Parameter Dependence of VIF performance on the parameter. (solid) VIF, (dashed) PSNR, (dash-dot) Sarnoff JNDMetrix 8.0, and (dotted) MSSIM.

30. Conclusion  A VIF criterion for full-reference image QA is presented.  The VIF was demonstrated to be better than a state-of-the-art HVS-based method, the Sarnoff’s JND-Metrix, as well as a state-of-the-art structural fidelity criterion, the SSIM index  The VIF provides the ability to predict the enhanced image quality by contrast enhancement operation.

31. Reference 1. H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006. 2. H. R. Sheikh, A. C. Bovik, and G. de Veciana, “An information fidelity criterion for image quality assessment using natural scene statistics,” IEEE Trans. Image Process., vol. 14, no. 12, pp. 2117–2128, Dec. 2005. 3. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error measurement to structural similarity,” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600–612, Apr. 2004. 4. V. Strela, J. Portilla, and E. Simoncelli, “Image denoising using a local Gaussian scale mixture model in the wavelet domain,” Proc. SPIE, vol. 4119, pp. 363–371, 2000.