1. Computational Photography Prof. Feng Liu Spring 2015 http://www.cs.pdx.edu/~fliu/courses/cs510/ 04/13/2015

2. Last Time  Filters  De-noise 2

3. Today  Color  Color to Gray 3

4. Light and Color  The frequency, , of light determines its “color” Wavelength, , is related: Energy also related  Describe incoming light by a spectrum Intensity of light at each frequency A graph of intensity vs. frequency  We care about wavelengths in the visible spectrum: between the infra-red (700nm) and the ultra-violet (400nm) 4

5. Color and Wavelength 5

6. Color Spaces  The principle of trichromacy means that the colors displayable are all the linear combination of primaries  Taking linear combinations of R, G and B defines the RGB color space the range of perceptible colors generated by adding some part each of R, G and B  If R, G and B correspond to a monitor’s phosphors (monitor RGB), then the space is the range of colors displayable on the monitor 6

7. RGB Color Space 7

8. L*a*b* Color Space  RGB Perceptually non-uniform  L*a*b* More perceptually uniform Look into Opencv or Matlab http://en.wikipedia.org/wiki/Lab_color_space 8

9.  The eye contains rods and cones Rods work at low light levels and do not see color  That is, their response depends only on how many photons, not their wavelength Cones come in three types (experimentally and genetically proven), each responds in a different way to frequency distributions Seeing in Color 9

10. Color receptors 10  Each cone type has a different sensitivity curve Experimentally determined in a variety of ways  For instance, the L-cone responds most strongly to red light  “Response” in your eye means nerve cell firings  How you interpret those firings is not so simple …

11. Color Perception 11  How your brain interprets nerve impulses from your cones is an open area of study, and deeply mysterious  Colors may be perceived differently: Affected by other nearby colors Affected by adaptation to previous views Affected by “state of mind”  Experiment: Subject views a colored surface through a hole in a sheet, so that the color looks like a film in space Investigator controls for nearby colors, and state of mind

12. The Same Color? 12

13. The Same Color? 13

14. Color Deficiency 14  Some people are missing one type of receptor Most common is red-green color blindness in men Red and green receptor genes are carried on the X chromosome - most red-green color blind men have two red genes or two green genes  Other color deficiencies Anomalous trichromacy, Achromatopsia, Macular degeneration Deficiency can be caused by the central nervous system, by optical problems in the eye, injury, or by absent receptors

15. Color Deficiency 15

16. Color Transformation  Re-coloring  Color to Gray 16

17. Color2Gray: Salience-Preserving Color Removal Amy Gooch Sven Olsen Jack Tumblin Bruce Gooch

18. Color Grayscale New Algorithm

19. Isoluminant Colors ColorGrayscale

20. CIE CAM 97Photoshop LAB CIE XYZYCrCb Traditional Methods: Luminance Channels Problem can not be solved by simply switching to a different space

21. Traditional Methods Contrast enhancement & Gamma Correction – Doesn’t help with isoluminant values Photoshop Grayscale PSGray + Auto ContrastNew Algorithm

22. Goals Dimensionality Reduction – From tristimulus values to single channel Loss of information Maintain salient features in color image – Human perception

23. Relative differences Color Illusion by Lotto and Purves http://www.lottolab.org

24. Challenge 1: Influence of neighboring pixels

25. Challenge 2: Dimension and Size Reduction -120, -120 120, 120 0 100

26. Original Challenge 3: Many Color2Gray Solutions......

27. Algorithm Overview Adjust g to incorporate both luminance and chrominance differences  ij For every pair of pixel i and j – Compute Luminance distance – Compute Chrominance distance Initialize image, g, with L channel Convert to Perceptually Uniform Space – CIE L*a*b*

28. Color2Grey Algorithm Optimization: min      (g i - g j ) -  i,j   i j=i-  i+ 

29. Parameters Map chromatic difference to increases or decreases in luminance values  Max chrominance offset  Radius of neighboring pixels 

30.  = 2  = 16  = entire image  = 300 o  = 10  = 49 o  = 10  : Neighborhood Size

31.  = 16  = entire image

32. Luminance Distance: Problem: ||  C ij || is unsigned  L ij = L i - L j Perceptual Distance Chrominance Distance: ||  C ij ||

33. C2C2 C1C1 Map chromatic difference to increases or decreases in luminance values Color Space +b* +a*-a*

34.  C 1,2  vv sign ( ||  C i,j || ) = sign (  C i,j. v  ) Color Difference Space v  = (cos , sin  ) +  b* +  a*-  a* - - + + -  b*

35. Photoshop Grayscale  = 225  = 45

36. Grayscale  = 45  = 135  = 0

37. How to Combine Chrominance and Luminance  ij  (Luminance)  L ij

38. How to Combine Chrominance and Luminance  ij   L ij ||  C ij || (Luminance) if |  L ij | > ||  C ij || (Chrominance)

39. ...... How to Combine Chrominance and Luminance  ij   L ij ||  C ij || if  C ij.  ≥ 0 if |  L ij | > ||  C ij || -||  C ij || otherwise Grayscale

40. ...... How to Combine Chrominance and Luminance  ij   L ij crunch(||  C ij ||) if  C ij.  ≥ 0 if |  L ij | > crunch(||  C ij ||) crunch(-||  C ij ||) otherwise Grayscale

41.  : Chromatic variation maps to luminance variation  = 5  = 10  = 25   crunch(x) =  * tanh(x/  )

42. Color2Grey Algorithm Optimization: min      (g i - g j ) -  i,j   j=i-  i i+ 

43. OriginalColor2Grey Photoshop Grey Results Color2Grey + Color

44. OriginalPhotoshopGreyColor2Grey

45. OriginalPhotoshopGrey Color2Grey+Color

46. OriginalPhotoshopGreyColor2Grey

47. OriginalPhotoshopGreyColor2Grey

48. Implementation Performance Image of size S x S – O(  2 S 2 ) or O(S 4 ) for full neighborhood case 12.7s 100x100 image 65.6s 150x150 image 204.0s 200x200 image – GPU implementation O(S 2 ) ideal, really O(S 3 ) – 2.8s 100x100 – 9.7s 150x150 – 25.7s 200x200 Athlon 64 3200 CPU NVIDIA GeForce GT6800

49. Future Work Animations/Video Faster – Multiscale Smarter – Remove need to specify  New optimization function designed to match both signed and unsigned difference terms – Image complexity measures

50. Validate "Salience Preserving" Apply Contrast Attention model by Ma and Zhang 2003 OriginalPhotoshopGreyColor2Grey

51. Validate "Salience Preserving" OriginalPhotoshopGreyColor2Grey

52. Next Time  Re-lighting 52

53. Thank you SIGGRAPH Reviewers NSF Helen and Robert J. Piros Fellowship Northwestern Graphics Group MidGraph2004 Participants – especially Feng Liu (sorry I spelled your name wrong in the acknowledgements)

55. OriginalColor2GreyColor2Grey+Color

56. OriginalColor2GreyColor2Grey+Color

57. Original Color2Grey Color2Grey+Color

58. OriginalPhotoshopGreyColor2Grey

59. OriginalColor2GreyColor2Grey+Color

60. OriginalPhotoshopGreyColor2Grey

61. OriginalColor2GreyColor2Grey+Color

62. Next Time  Re-lighting 62