1. Sensor Transforms to Improve Metamerism-Based Watermarking Mark S. Drew School of Computing Science Simon Fraser University mark@cs.sfu.ca Raja Bala Xerox Research Center Webster, Xerox Corp. Raja.Bala@xerox.com

2. 2 Is it possible to use colour to digitally hide a watermark? i.e., embed information into a document in a way that is – invisible, – easily seen under a special environment meant to reveal it, and – difficult to remove Here, we’re concentrating on visible watermarking, as opposed to purely digital watermarking == steganography [i.e., bit-flipping]

3. 3 A brief history: Bala et al. [ CIC 2007 “Substrate Fluorescence: Bane or Boon?” ] – use fluorescent property of paper substrate to make watermark visible under a portable UV lamp: under D50under UV

4. 4 more:

5. 5 … brief history… Can this be done without UV – using just visible light? Bala et al. [CIC 2009 “Watermark Encoding and Detection using Narrowband Illumination”] – use narrowband LED lights – consider a pair of inks, with reflectance spectra that are metameric under D50: max K min K So use an LED illuminant to emphasize difference == break metamerism:

6. 6 Ok, how does this LED-light approach do? “Watermark example for the strongest watermark signal. Top shows image photographed under daylight illumination. Bottom shows the same image photographed under the LED illumination”

7. 7 So, not bad… Another example: but could we not do better?

8. 8 So far, we used illuminant metamerism, in order to see watermark. We could use observer metamerism as well, to further break apart the colours, by interposing a camera+display system. +

9. 9 … brief history… Combating cinema piracy in video by using observer metamerism to generate a watermark if a camcorder illegally shoots a film: Doyen et al. [CIC 2009 “Description and Evaluation of the Variability of Human Color Vision in an Anti-Piracy Context”] – use 3-primary projector for part of film, and 4 primaries for watermark : camcorder sees the difference – but, observer variability is substantial, and would have to be taken into consideration in determining a watermark/disturbance signal that is invisible in the cine theater to all "normal" observers, and yet different enough to show up when captured by a camcorder

10. 10 For this paper [  printed materials, not video ], since we have a camera and hence digital image processing, we go on to transform into a new colour space: so as to optimally disambiguate the foreground (the watermark) from the background 3 x 3

11. 11 optimize on M optimize on LED lighting as well

12. 12 We start with 6 inks == 3 pairs that are metameric under D50: Again, these are max-K, min-K metamer pairs

13. 13 Again, we see max-differences: Differences between spectral pairs Start off by using fixed set of LEDs LEDs chosen 500580660

14. 14 These inks are metamers under D50: 1212 3434 5656 illuminant metamerism: D65  illuminant metamerism: LEDs 

15. 15 Add a camera: D50 D65 LEDs 

16. 16 Ok, let’s go to a transformed colour space: matrix with M -positivity of transformed colours -Maximize sum of squared differences between members of metamer pairs -set a normalization, to constrain M Requirements of an optimization:  normalize image to max brightness

17. 17 Result of optimization Camera, LED lighting transformed Progress over optimization:

18. 18 Ok, for all pairs, optimization improves discriminability. But, could we not do better by optimizing on the LED lights as well? Suppose we have available 31 narrowband LED illuminants maxima

19. 19 Let the (float) weights over the set of LEDs be w Define 93 x 93 matrix W as Our optimization now reads: Also define R = 31 x 6 x 3 set of RGB values under each of the LEDs, split into R 1 and R 2

20. 20 optimized weights w

21. 21 Optimized weights w no matrixing optimized matrixing

22. 22 But, we may have only binary weights available: result is almost as good − 4 nonzero contributions from float weights w >0.2 re-optimized matrix M for these LED lights

23. 23 Compare objective function: … but, using non-perceptual differences

24. 24 Conclusions: -matrixing has a dramatic effect -float weights are best, but optimizing for floats and then binarizing is almost as effective

25. 25 -choice of inks: still metamers, but optimized blends along iso-colour loci -ensure watermark invisible over normal lighting and over observers -optimize on perceptual differences (could just use Jacobian of CIELAB transform) -some metamer pairs don’t separate as much  weight some pairs more -model LEDs better -could design lights just for this task [ ACM Transactions on Graphics 2009] -include more metamer pairs Future work:

26. 26 Thanks! Natural Sciences and Engineering Research Council of Canada Xerox Corporation