2. Computer Science 121 Scientific Computing Winter 2014 Chapter 14 Images

3. Background: Images ● Signal (sound, last chapter) is a single (one- dimensional) quantity that varies over time. ● Image (picture) can be thought of as a two- dimensional quantity that varies over space. ● Otherwise, the computations for sounds and images are about the same!

4. 14.1 Black-and-White Images

7. ● Eventually we get down to a single point (pixel, “picture element”) whose color we need to represent. ● For black-and-white images, two basic options –Halftone: Black (0) or white (1); having lots of points makes a region look gray –Grayscale: Some set of values (2 N, N typically = 8) representing shades of gray.

8. 14.1 Black-and-White Images ● To turn a b/w image into a grid of pixels, imagine running a stylus horizontally along the image:

9. 14.1 Black-and-White Images ● This gives us a one-dimensional function showing gray level at each point along the horizontal axis:

10. 14.1 Black-and-White Images ● Doing this repeatedly with lots of evenly-spaced scan lines gives us a grayscale map of the image: ● Height = grayscale value ● Spacing between lines = spacing within line

11. 14.2 Color ● Light comes in difference wavelengths (frequencies):

12. 14.2 Color ● Like a sound, energy from a given light source (e.g., star) can be described by its wavelength (frequency) spectrum : the amount of each wavelength present in the source. Spectrum of vowel “ee” Frequency (Hz) Energy (dB) Spectrum of star ICR3287 Wavelength (nm) Intensity

13. 14.2 Color ● Light striking our eyes is focused by a lens and projected onto the retina. ● The retina contains photoreceptors (sensors) called cones that respond to different wavelengths of light: red, green, and blue. ● So retina can be thought of as a transducer that inputs light and outputs a three-dimensional value for each point in the image: [R, G, B], where R, G, and B each fall in the interval (0,1). ● So we can represent any color image as three “grayscale” images:

14. 14.2 Color =, R, G B

15. 14.3 Digital Sampling of Images ● Same questions arise for sampling images as for sound: –Sampling Frequency (how often to sample) –Quantization (# of bits per sample) ● With images, “how often” means “how many times per linear unit (inch, cm) – a.k.a. resolution ● Focus on quantization –Each sample is an index into a color map –With too few bits, we lose gradual shading

16. Color Maps ● With 8 bits per color, 3 colors per pixel, we get 24 bits per pixel: 2 24 ≈ 100,000,000 distinct colors ● We can actually get away with far fewer, using color maps ● Each pixel has a value that tells us what row in the color map to use ● Color map rows are R, G, B values:

17. Color Maps >> colormap ans = 0 0 0.5625 0 0 0.6250... 0.4375 1.0000 0.5625 0.5000 1.0000 0.5000... 0.5625 0 0 0.5000 0 >> size(colormap) ans = 64 3

18. Color Maps Matlab provides us with some default color maps: >> drawMandelbrot([-2.5 2.5], [-2 2])

19. Color Maps >> colormap(hot)

20. Color Maps >> colormap(flag)

21. Quantization Problems With too few bits, we lose gradual shading: 5 bits 1 bit 2 bits 3 bits 4 bits 6 bits 7 bits 8 bits

22. 14.4 Sampling and Storing Images in Files Scanners usually output images in one of several formats –JP(E)G (Joint Photographic Experts Group) –GIF (Graphics Interchange Format) –PNG (Portable Network Graphics) –TI(F)F (Tagged Image File Format) As with sound formats, main issues in image formats are compression scheme (how images are stored and transmitted to save space/time) and copyright We'll focus on Matlab and mathematical issues....

23. 14.4 Sampling and Storing Images in Files Matlab imread commands lets us read in files in several formats (automagically figures out which): >> a = imread('sakidehli.png'); % b/w >> size(a) ans = 269 176 >> b = imread('monaLisaLouvre.jpg'); % color >> size(b) ans = 864 560 3

24. 14.4 Sampling and Storing Images in Files Then use image to display image: >> image(b)

25. 14.4 Sampling and Storing Images in Files

26. Then use image to display image: >> image(b) imwrite writes it back out: >> imwrite(b, 'monaLisaCopy.jpg')

27. 14.4 Sampling and Storing Images in Files Recall formula for sound signal: P(t) = Σ i A i sin(2πf i t + φ i ) For images, we just add another dimension: P(x, y) = Σ j,k A j,k sin(2π( f j x + f k y + φ j,k )) This means that the issues/algorithms for images are similar to those for sounds Aliasing Frequency transforms, filters (next lecture)

28. Spatial Frequency: Intuitive Version x : ~18 color changes y : ~6 color changes

29. 14.4 Sampling and Storing Images in Files As with sound, aliasing becomes an issue if sampling frequency is inadequate:

30. 14.4 Sampling and Storing Images in Files As with sound, aliasing becomes an issue if sampling frequency is inadequate:

31. 14.4 Sampling and Storing Images in Files As with sound, aliasing becomes an issue if sampling frequency is inadequate:

32. 14.4 Sampling and Storing Images in Files As with sound, aliasing becomes an issue if sampling frequency is inadequate: 0.04 mm samples

33. 14.4 Sampling and Storing Images in Files As with sound, aliasing becomes an issue if sampling frequency is inadequate: 0.04 mm samples 0.25 mm samples

34. 14.5 Manipulating and Synthesizing Images Probably want to use PhotoShop or another image-processing package instead of Matlab. But Matlab allows us to see the math behind the processing: >> b = imread('monaLisaLouvre.jpg'); >> red = b(:,:,1); >> green = b(:,:,2); >> blue = b(:,:,3); >> image(red) >> colormap(gray) % just show intensity

35. 14.5 Manipulating and Synthesizing Images >> image(red)

36. 14.5 Manipulating and Synthesizing Images >> image(green)

37. 14.5 Manipulating and Synthesizing Images >> image(blue)

38. 14.5 Manipulating and Synthesizing Images Put them back together with cat : >> image(cat(3, red, green, blue))

39. 14.5 Manipulating and Synthesizing Images Put them back together with cat : >> image(cat(3, red, green, blue))

40. 14.5 Manipulating and Synthesizing Images >> max(max(a/3)) ans = 68 >> image(68-a/3), colormap(gray) Make a negative:

41. 14.5 Manipulating and Synthesizing Images >> max(max(a/3)) ans = 68 >> image(68-a/3), colormap(gray) Make a negative:

42. 14.6 Example: Image Restoration Mona Lisa has faded and yellowed over 500 years. Can we get an idea of what it looked like originally? Basic idea: Balance out R, G, B

43. >> a = imread('monaLisaLouvre.jpg'); >> mona = image2double(a); % kaplan >> min(min(min(mona))) ans = 0 >> max(max(max(mona))) ans = 1 >> r = mona(:,:,1); >> hist(r(:), 50) % histogram

44. >> a = imread('monaLisaLouvre.jpg'); >> mona = image2double(a); % kaplan >> min(min(min(mona))) ans = 0 >> max(max(max(mona))) ans = 1 >> r = mona(:,:,1); >> hist(r(:), 50) % histogram

46. >> newr = equalize(r); % kaplan >> hist(newr(:), 50)

47. >> newr = equalize(r); % kaplan >> hist(newr(:), 50)

48. >> newr = equalize(r); >> newg = equalize(g); >> newb = equalize(b); >> newmona = cat(3, newr, newg, newb); >> image(newmona)

49. >> newr = equalize(r); >> newg = equalize(g); >> newb = equalize(b); >> newmona = cat(3, newr, newg, newb); >> image(newmona)

50. >> image(newmona*.5 + mona*.5);