1. Just Noticeable Difference Estimation For Images with Structural Uncertainty WU Jinjian Xidian University

2. Outline Concept of JND Existing Works Inspiration JND Estimation Experimental Results Conclusion

3. Original image JND noise (PSNR=26.4) contaminated image

4. Concept of Just Noticeable Difference Original image JND noise (PSNR=26.4) contaminated image +

5. Concept of Just Noticeable Difference Original image JND noise (PSNR=26.4) contaminated image +

6. Concept of Just Noticeable Difference Original image JND noise (PSNR=26.4) contaminated image  Just-noticeable difference (JND): below which the change cannot be detected by the majority (e.g., 75%) of viewers;  JND is useful in perceptual based signal processing, e.g., image/video coding, watermarking, quality assessment, etc.

7. Factors determine JND threshold ：  Luminance adaptation: the HVS presents different resolutions for different background illuminations ； Existing works

8. Factors determine JND threshold ：  Luminance adaptation: the HVS presents different resolutions for different background illuminations ； Existing works

9. Factors determine JND threshold ：  Luminance adaptation: the HVS presents different resolutions for different background illuminations ； Existing works

10. Factors determine JND threshold ：  Luminance adaptation: the HVS presents different resolutions for different background illuminations ； Existing works

11. Factors determine JND threshold ：  Luminance adaptation: the HVS presents different resolutions for different background illuminations ； Existing works

12. Factors determine JND threshold ：  Contrast masking: the spatial non-uniformity of the background luminance causes masking effect;

13. Existing works Factors determine JND threshold ：  Contrast masking: the spatial non-uniformity of the background luminance causes masking effect;

14. Existing works Factors determine JND threshold ：  Contrast masking: the spatial non-uniformity of the background luminance causes masking effect;

15. contaminated Existing works Shortcoming of the existing works ：  Overestimate the orderly regions (edge);  Underestimate the disorderly regions (texture); original

16. contaminated Existing works Shortcoming of the existing works ：  Overestimate the orderly regions (edge);  Underestimate the disorderly regions (texture); Too much noise original

17. contaminated Existing works Shortcoming of the existing works ：  Overestimate the orderly regions (edge);  Underestimate the disorderly regions (texture); original

18. contaminated Existing works Shortcoming of the existing works ：  Overestimate the orderly regions (edge);  Underestimate the disorderly regions (texture); Too little noise original

19. Luminance Adaptation Luminance Adaptation JND Threshold JND Threshold Factors determine JND threshold ： Existing works Contrast Masking Contrast Masking

20. Luminance Adaptation Luminance Adaptation JND Threshold JND Threshold Factors determine JND threshold ： Existing works Contrast Masking Contrast Masking ? ? There must be another factor, which also determines JND

21. Inspiration Intuitive perception ：  The HVS is sensitive to image regions with simple/regular structures(with small JND); Regular structures

22. Inspiration Intuitive perception ：  The HVS is sensitive to image regions with simple/regular structures(with small JND);  The HVS is insensitive to image regions with complex/irregular structures (with large JND); Regular structuresirregular structures

23. Theoretical Support: (Free-Energy Principle)  Recent perceptual science findings indicate that the HVS possesses an internal generative mechanism (IGM); Inspiration

24. Inspiration

25. Inspiration

26. By adjusting the internal structure of neurons By adjusting the internal structure of neurons Actively predicts the visual signal, while ignores the residual Actively predicts the visual signal, while ignores the residual When Perceiving an input scene Inspiration

27. Inspiration Theoretical Support: (Free-Energy Principle)  According to the free-energy principle, orderly structures are easily to be predicted, and the HVS is sensitive to them; Orderly structure

28. Inspiration Theoretical Support: (Free-Energy Principle)  According to the free-energy principle, orderly structures are easily to be predicted, and the HVS is sensitive to them;  Disorderly structures possess abundant uncertainty, and the HVS is insensitive to them. Orderly structure Disorderly structures

29. Inspiration Theoretical Support: (Free-Energy Principle)  According to the free-energy principle, orderly structures are easily to be predicted, and the HVS is sensitive to them;  Disorderly structures possess abundant uncertainty, and the HVS is insensitive to them.  Structural uncertainty determines the HVS sensitivity. Orderly structure Disorderly structures

30. Luminance Adaptation Luminance Adaptation JND Threshold JND Threshold Factors determine JND threshold ： Contrast Masking Contrast Masking Structural Uncertainty Structural Uncertainty Inspiration

31. Luminance Adaptation Luminance Adaptation JND Threshold JND Threshold Factors determine JND threshold ： Contrast Masking Contrast Masking Structural Uncertainty Structural Uncertainty How to calculate structural uncertainty? Inspiration

32. JND threshold estimation Structural Uncertainty:  Represent the disorderly degree of the visual structure;

33. JND threshold estimation Structural Uncertainty:  Represent the disorderly degree of the visual structure;  Decompose disorderly content from orderly content; Disorderly content Acquisition Disorderly content Acquisition AR prediction Based decomposition AR prediction Based decomposition TheoryTechnology

34. JND threshold estimation Structural Uncertainty:  Represent the disorderly degree of the visual structure;  Decompose disorderly content from orderly content;  Analyze the visual structure of the disorderly content; Disorderly content Acquisition Disorderly content Acquisition AR prediction Based decomposition AR prediction Based decomposition TheoryTechnology Visual Structure Characteristic Visual Structure Characteristic Local Binary Pattern (LBP) Local Binary Pattern (LBP)

35. JND threshold estimation Structural Uncertainty:  Represent the disorderly degree of the visual structure;  Decompose disorderly content from orderly content;  Analyze the visual structure of the disorderly content;  Calculate the structural uncertainty. Disorderly content Acquisition Disorderly content Acquisition AR prediction Based decomposition AR prediction Based decomposition TheoryTechnology Visual Structure Characteristic Visual Structure Characteristic Local Binary Pattern (LBP) Local Binary Pattern (LBP) Structural Uncertainty Structural Uncertainty Shannon Entropy Shannon Entropy

36. JND threshold estimation AR based disorderly content decomposition:  AR based prediction:

37. JND threshold estimation AR based disorderly content decomposition:  AR based prediction: The correlation between the central pixel and its i-th neighbor The correlation between the central pixel and its i-th neighbor

38. JND threshold estimation AR based disorderly content decomposition:  AR based prediction:  Disorderly content decomposition: The correlation between the central pixel and its i-th neighbor The correlation between the central pixel and its i-th neighbor

39. Input AR Pred. AR based disorderly content decomposition: JND threshold estimation

40. Input Orderly content Disorderly content AR Pred. AR based disorderly content decomposition: JND threshold estimation

41. Structure analysis on the disorderly content:  Local binary pattern (LBP) is successful in structure description;  The structure of a pixel is described as the correlations with its surround neighbors:  By assigning the binomial factors, LBP form is acquired: JND threshold estimation

42. Structural uncertainty computation: JND threshold estimation  Shannon Entropy is adopted;  Structural uncertainty is computed as the entropy of LBP on the disorderly content; The LBP distribution on a local region centered at The LBP distribution on a local region centered at

43. JND threshold estimation Structural Uncertainty: Original Image Structural Uncertainty

44. Pattern masking: JND threshold estimation Contrast Masking Affection of Structural Uncertainty

45. Pattern masking: JND threshold estimation Contrast Masking Affection of Structural Uncertainty

46. JND threshold estimation Pattern masking : parameter setting  for contrast masking function Linear Fitting The ground truth (the circle points) from subjective contrast masking testing

47. Pattern masking : parameter setting  for structural uncertainty function The ground truth comes from subjective viewing testing JND threshold estimation

48. JND computation model JND threshold estimation Luminance Adaptation Luminance Adaptation JND Threshold JND Threshold Pattern Masking Pattern Masking Luminance Contrast Luminance Contrast Structural Uncertainty Structural Uncertainty

49. JND computation model JND threshold estimation Luminance Adaptation Luminance Adaptation JND Threshold JND Threshold Pattern Masking Pattern Masking Luminance Contrast Luminance Contrast Structural Uncertainty Structural Uncertainty

50. Experimental Results Pattern Masking VS. Contrast Masking Contrast MaskingPattern Masking Original

51. Experimental Results Pattern Masking VS. Contrast Masking Contrast MaskingPattern Masking Original

52. Experimental Results Pattern Masking VS. Contrast Masking Contrast MaskingPattern Masking Original

53. Comparison among JND models ： Wu2013Proposed Yang2005 Liu2011 Original Experimental Results

54. Comparison among JND models ： Wu2013Proposed Yang2005 Liu2011 Original Experimental Results

55. Comparing with the existing JND models (Positive values means the proposed JND outperforms the compared model) Experimental Results

56. Comparing with the existing JND models (Positive values means the proposed JND outperforms the compared model) Experimental Results Subjective quality comparison (38 subjects are invited)

57. Conclusion The shortcoming of the existing JND model is analyzed; We suggest structural uncertainty is another factor which determines JND; Structural uncertainty is defined and calculated; Novel JND model is proposed; Experimental results demonstrate the effectiveness;

58. Related Works J. Wu, W. Lin, G. Shi, etc., “Pattern Masking Estimation in Image with Structural Uncertainty,” IEEE TIP. J. Wu, G. Shi, W. Lin, etc., “Just Noticeable Difference Estimation For Images with Free-Energy Principle,” IEEE TMM. J. Wu, F. Qi, and G. Shi. “Self-Similarity Based Structural Regularity for Just Noticeable Difference Estimation,” JVCI. J. Wu, W. Lin, and G. Shi. “Visual Masking Estimation Based On Structural Uncertainty,” ISCAS2013.(Best student paper).