1. Perceptual Watermarks for Digital Image and Video ECE 738 paper presentation Pei Qi ECE at UW-Madison pqi@cae.wisc.edu

2. What is ‘ perceptual ’ watermark Prior knowledge Prior knowledge Perceptual watermark Perceptual watermark

3. Prior knowledge Additive watermark Additive watermark Ideal watermark Ideal watermark Three principles Three principles - Transparency or imperceptibility - Transparency or imperceptibility - Robustness - Robustness - Capacity - Capacity Challenging problem Challenging problem - Conflicts - Conflicts - Tradeoff between transparency and robustness - Tradeoff between transparency and robustness

4. Prior knowledge Human visual system Human visual system Three properties of the human visual system Three properties of the human visual system 1. Frequency sensitivity 1. Frequency sensitivity What’s freq. sensitivity What’s freq. sensitivity Freq. sensitivity describes the human eye’s sensitivity to sine wave gratings at various freq. Given that the minimum viewing distance is fixed, it’s possible to determine a static just noticeable difference threshold for each freq. band. Freq. sensitivity describes the human eye’s sensitivity to sine wave gratings at various freq. Given that the minimum viewing distance is fixed, it’s possible to determine a static just noticeable difference threshold for each freq. band. JND threshold JND threshold The JND threshold is such that changes in the frequency content in the image in the particular frequency band below the threshold are not noticeable The JND threshold is such that changes in the frequency content in the image in the particular frequency band below the threshold are not noticeable

5. Prior knowledge Human visual system Human visual system Three properties of the human visual system Three properties of the human visual system 2. Luminance sensitivity 2. Luminance sensitivity What’s luminance sensitivity What’s luminance sensitivity Luminance sensitivity measures the effects of the detectability threshold of noise on a constant background, which is a nonlinear function and depends on local image characteristics. Luminance sensitivity measures the effects of the detectability threshold of noise on a constant background, which is a nonlinear function and depends on local image characteristics. 3. Contrast masking 3. Contrast masking Contrast masking allows more dynamic control of the JND threshold levels. Contrast masking refers to the detectability of one signal in the presence of another signal. Contrast masking allows more dynamic control of the JND threshold levels. Contrast masking refers to the detectability of one signal in the presence of another signal.

6. Prior knowledge Summary Summary What is our goal to introduce human visual system in watermarking application? What is our goal to introduce human visual system in watermarking application? 1. Determine if a watermark inserted into a image is invisible or not 1. Determine if a watermark inserted into a image is invisible or not 2. We are always trying to insert the maximum strength and maximum length watermarks into an image, SINCE more watermarks are inserted 2. We are always trying to insert the maximum strength and maximum length watermarks into an image, SINCE more watermarks are inserted - more robust to attacks - more robust to attacks - more likely to be detected - more likely to be detected Make use of properties of human visual system to adjust the watermark so that it’s perfect for both robustness and transparency Make use of properties of human visual system to adjust the watermark so that it’s perfect for both robustness and transparency JNDs JNDs JNDs generated from different properties provide the quantized thresholds for embedding watermarks. JNDs generated from different properties provide the quantized thresholds for embedding watermarks. - upper bounds on watermark strength levels - upper bounds on watermark strength levels - upper bounds on watermark length (capacity) - upper bounds on watermark length (capacity) Note: JND thresholds are NOT a fixed value, which depend on different images and approaches Note: JND thresholds are NOT a fixed value, which depend on different images and approaches

7. Perceptual watermark techniques Image-Independent watermark Image-Independent watermark Image-dependent or Image-adaptive watermark Image-dependent or Image-adaptive watermark

8. Image-Independent watermark A typical method (Cox approach) A typical method (Cox approach)  Key points Place watermark in perceptually significant components (low frequency) (for robustness)Place watermark in perceptually significant components (low frequency) (for robustness) –Modify by a small amount below Just-noticeable-difference (JND) Use long random vector as watermark to avoid artifactsUse long random vector as watermark to avoid artifacts Any difference if using other watermark instead (w-b images, logo) (for imperceptibility & robustness)Any difference if using other watermark instead (w-b images, logo) (for imperceptibility & robustness)  Embedding v’ i = v i +  v i w i = v i (1+  w i ) Perform DCT on entire image and embed watermark in DCT coefficientsPerform DCT on entire image and embed watermark in DCT coefficients Choose N=1000 largest AC coeff. and scale {vi} by a random factorChoose N=1000 largest AC coeff. and scale {vi} by a random factor  Detection

9. Block diagram of Cox ’ s scheme 2D DCTsor t v’=v (1+  w) IDCT & normalize Original image N largest coeff. other coeff. marke d image random vector generator wm k seed

10. Implementation Avoiding to change the corresponding location of each coefficient in the image, when you sort the vector projected from matrix of DCT coefficients

11. Challenging problem How to improve Cox approach How to improve Cox approach Global scaling factor is not suitable for all coefficientsGlobal scaling factor is not suitable for all coefficients - Maybe beyond the threshold in some areas of image, especially obvious in the smooth background area - Maybe beyond the threshold in some areas of image, especially obvious in the smooth background area More explicitly compute Just-noticeable-differenceMore explicitly compute Just-noticeable-difference –JND ~ max amount each frequency coefficient can be modified imperceptibly –Use  i for each coefficients  finely tune watermark strength OverheadOverhead - Cost of computation of thresholds for each coefficient - Cost of computation of thresholds for each coefficient

12. Image-dependent or Image-adaptive watermark Block-based DCT approach Block-based DCT approach Wavelet DWT approach Wavelet DWT approach

13. Image-Adaptive watermark General Image-Adaptive watermark scheme  X* u,v : The watermarked image  X u,v : The original image  W u,v : The sequence of watermark values  J u,v : The computed JND for each coefficient Question: Why Xu,v > Ju,v (from local image, considering properties of HVS)

14. Block-based DCT approach Nonoverlapping 8x8 blocks DCT applied to each block independently X u,v,b : The DCT coefficients X * u,v,b : The watermarked DCT coefficients W u,v,b : The sequence of watermark values t C u,v,b : The computed JND calculated from the visual model Key points Block-by-block DCT How to derive t C u,v,b

15. Block-based DCT approach t F u,v : a frequency threshold value, which is an 8x8 matrix values for each DCT basis function t L u,v,b : Luminance sensitivity estimated by the formula. X 0,0,b : DC coeff. for block b X 0,0,b : DC coeff. for block b X 0,0 : DC coeff. Corresponding to the mean luminance of the display X 0,0 : DC coeff. Corresponding to the mean luminance of the display a: parameter controlling the degree of luminance sensitivity (empirical value=0.649) a: parameter controlling the degree of luminance sensitivity (empirical value=0.649) t C u,v,b :Contrast masking threshold, where w between 0 and 1, a empirical value for w is 0.7

16. Block diagram of IA-DCT approach Calculate JNDs Calculate JNDs Watermark Insertion Watermark Insertion DCT Original image X(i,j)X(u,v) Watermark sequence W(u,v) Watermarked image X*(u,v) J(u,v)

17. Wavelet DWT approach Key point Key point  Hierarchy Decomposition  The upper left corner: Lowest frequency band.  l: resolution level 1, 2, 3, 4  F: frequency orientation 1, 2, 3  Much simpler than DCT app. - Cost of computing JNDs - Cost of computing JNDs

18. Wavelet DWT approach X u,v,l,f : wavelet coefficient at position(u,v) in resolution level l and frequency orientation f X * u,v,l,f : watermarked wavelet coefficient W u,v,l,f : watermark sequence t F l,f : computed frequency weight at level l and frequency orientation f, which could be further refined by adding image-dependent components like DCT approach

19. Detection Detection scheme for Block-based DCT 1. Based on classical detection theory as SS detection (Cox) - Original image is subtract from watermarked image and correlation between the signal difference and the watermark sequence is determined - The correlation value is compared to a threshold to determine whether the received image contains the watermark.

20. Testing IA-DCT without original image Key points 1. Assume original image has been JPEG compressed 2. Feature vector {X f }, X D is greater than ½ of its corresponding quantization table value Q 3. W is only inserted in {X f } 4. A correlation measure c is found between {Z f } and W 5. A threshold test is performed on c to determine if the W under test is present in Z

21. Detection Detection for Wavelet Detection for Wavelet 1. First, the correlation is performed separately at each level 1. First, the correlation is performed separately at each level 2. Second, We calculate the average for each resolution level l and freq. orientation f 2. Second, We calculate the average for each resolution level l and freq. orientation f 3. At last, we choose the maximum correlation value over all the possible levels as well as freq. locations 3. At last, we choose the maximum correlation value over all the possible levels as well as freq. locations

22. Comparison Image-Independent vs Image-Adaptive Image-Independent vs Image-Adaptive Image Quality Image Quality All acceptable, but SS watermark is most visible in the smooth background area. All acceptable, but SS watermark is most visible in the smooth background area. Robustness to Compression and Cropping Robustness to Compression and Cropping Winner: IA-W Winner: IA-W Robustness to Scaling Robustness to Scaling Winner: IA-W again Winner: IA-W again Robustness to shift Robustness to shift Only IA-W survives Only IA-W survives

23. Video watermarks Unique requirements for watermarks Unique requirements for watermarks Extension of the IA-DCT Technique to Video Extension of the IA-DCT Technique to Video Watermarking of MPEG-2 Watermarking of MPEG-2 Scene-Adaptive Video Watermarking Scene-Adaptive Video Watermarking Watermarking Standards Watermarking Standards

24. Key points in paper What ’ s the perceptual watermark What ’ s the perceptual watermark How does HVS work for watermark applications How does HVS work for watermark applications Three typical watermarking techniques Three typical watermarking techniques

25. Papers Perceptual Watermarks for Digital Image and video Perceptual Watermarks for Digital Image and video RAYMOND B. WOLFGANG RAYMOND B. WOLFGANG CHRISTINE I. PODILCHUK AND EDWARD J. DELP CHRISTINE I. PODILCHUK AND EDWARD J. DELP Image-Adaptive Watermarking Using Visual Models Image-Adaptive Watermarking Using Visual Models CHRISTINE I. PODILCHUK AND WENJUN ZENG CHRISTINE I. PODILCHUK AND WENJUN ZENG

26. Thank you for your attention PPT file and papers can be downloaded from website PPT file and papers can be downloaded from website http://www.cae.wisc.edu/~pqi/ece738/presentation/ http://www.cae.wisc.edu/~pqi/ece738/presentation/ http://www.cae.wisc.edu/~pqi/ece738/presentation/ Contact info: Contact info: Name: Qi, Pei Email: pqi@cae.wisc.edu Name: Qi, Pei Email: pqi@cae.wisc.edu