1. 2-0 2.1. Image representations, types and concepts 2.2. Image digitization 2.3. Digital image properties 2.4. Color images 2.5. Cameras Ch. 2 – The image, its representation, type, and properties

2. 2-1 ○ Function 2.1. Image representations, types and concepts Continuous – A, B: continuous Discrete – A: discrete, B: continuous Digital – A, B: discrete Digital image f(x,y): a 2-D digital function ○ Scene g(x,y,z): a 3-D continuous function Pixel: picture element Gray level: pixel value Origin ○

3. 2-2 Pin-hole camera ○ Image Formation

4. 2-3 ○ Perspective Projection

5. 2-4 Drawbacks of pinhole camera: (i) Big pinhole - Averaging rays blurs image Small pinhole - Diffraction effect blurs image 2 mm 1 mm 0.35 mm 0.15 mm0.07 mm (ii) Images are relatively dark Solution : ＊ Pinholes Lenses Lenses: gather light sharp focus

6. 2-5 Intensity (Gray Scale) Image ○ Types of Images Binary Image

7. 2-6 Color Image Indexed (or Palette) Color Image

8. 2-7 Multispectral Image

9. 2-8 X-Ray Image Gamma-Ray Image

10. 2-9 Ultraviolet Image Radio Image

11. 2-10 Ultrasound Image

12. 2-11 X-Ray Transmission Computerized Tomography Image

13. 2-12 Range Image

14. 2-13 Moire Image

15. 2-14 ○ Biological Visual Systems human cattle, horse bird, chicken, duck bat snake insect, fly, bee

16. 2-15 Camera Image array 2.2. Image Digitization Types of resolutions: Spatial (space), Radiometric (intensity), Spectral (color), Temporal (time). Sensor array

17. 2-16 2.2.1. Spatial Resolution

18. 2-17 False contours 2.2.2. Radiometric Resolution

19. 2-18 Non-uniform resolution -- to reduce storage while preserve as much information as possible ＊ Radiometric resolution (quantization): Busy area – fine, dense Inactive region – coarse, sparse ＊ Spatial resolution (sampling): Sharp region – fine, dense Smooth area – coarse, sparse Non-uniform ＊ Spectral resolution (quantization): ＊ Time resolution (sampling):

20. 2-19 Uniform sampling Non-uniform sampling Non-Uniform Sampling 128 by 128 64 by 64

21. 2-20 Adaptive mesh Adapted mesh Gradient image Adaptive sampling Input image Spring model Sampled image

22. 2-21 2.3. Digital Image Properties ○ Metric Properties Euclidean City-block Chess-board Quasi-Euclidean (i) Distances

23. 2-22 (ii) Adjacency 4-neighbors8-neighbors (iii) Distance Transform (Chamfering)

24. 2-23

25. 2-24

26. 2-25 (i) Crack edges: each pixel has 4 crack edges (ii) Inner, outer borders (iii) Convex, concave (vi) Convex hull: the smallest convex region containing the input region ○ Geometrical Properties

27. 2-26 ○ Topological properties (i) Rubber Sheet Transform (RST): preserves the contiguity of object parts and the number of holes in regions (no cut and joint). e.g., Imagine a rubber balloon with an object painted on it (ii) Euler-Poincare Characteristic: Euler number (#regions - #holes) is invariant under RST (iii) Topological components:

28. 2-27 2.3.2. Histograms Histogram Probability distribution

29. 2-28 Information theory -- amount of information Physical system -- measure of disorder Confidence theory -- amount of uncertainty Image -- degree of redundancy 2.3.3. Entropy

30. 2-29 2.3.4. Visual Perception (i) Perceptual Grouping (Organization)

31. 2-30

32. 2-31 (iii) Overshoot and Undershoot (iv) Subjective Objects (ii) Simultaneous Contrast Mack band pattern

33. The human brain tricks us whenever it can! (v) Optical Illusion and Visual Phenomena 2-32 Are the lines parallel or not?

34. 2-33 Coil or circle?

35. If something‘s rotating – go home, you need a break! 2-34

36. It doesn‘t move! 2-35

37. It doesn‘t move too! 2-36

38. It doesn‘t move too! 2-37

39. Concentrate on the cross in the middle, after a while you will notice that this moving purple dot will turn green! Look at the cross a bit longer and you‘ll notice that all dots except the green one will disappear. 2-38

40. 2-39 2.3.5. Image Quality Image degradation results from (i) noise, (ii) error, (iii) distortion, (iv) blurring where g(x,y): input image, f(x,y): degraded image H: degradation process, n(x,y): noise Degradation model:

41. 2-40 Linearity: Additivity: Homogeneity: Position (or space) invariance: Degradation processes H

42. 2-41 Image: If H is linear, If H is homogeneous, Let : impulse response of H : point spread function (PSF) Then, Degraded image:

43. 2-42 If H is position invariant, Then, Simplified degradation model: In frequency domain,

44. 2.3.6. Noise -- Originating from image acquisition, digitization, or transmission ○ White noise: the noise whose Fourier spectrum is constant e.g., Television raster, film material e.g., Physical processes FT 2-43 Additive: Multiplicative:

45. 2-44 ○ Random noise : Objectives: i) examine the performance of a developed algorithm ii) help to remove noise from an image (i) Salt-and-pepper (impulse) noise

46. 2-45 (ii) Uniform noise

47. 2-46

48. 2-47 (iii) Rayleigh noise (iv) Erlang (gamma) noise

49. 2-48 (v) Exponential noise (vi) Gaussian noise

50. Method 1: 2-49

51. 2-50 Method 2:

52. pdf cdf inverse 若 p 為均勻分佈，則 F -1 (p) 為常態分佈 Method3:

53. Definition: By Taylor expansion: Error Function Erf(x)

54. 2-53 ◎ Estimation of noise parameters Gaussian Rayleigh Uniform Image segmentation (i) manually or (ii) automatically

55. 2-54 Steps: 1. Segment the image 2. Choose a uniform image region 3. Compute its histogram 4. Compute the mean and variance of the histogram 5. Determine the probability distribution from the shape of the histogram 6. Estimate the parameters of the probability distribution of the histogram

56. 2-55 Example: Rayleigh noise: From

57. 2-56 1. Signal-to-Noise Ratio (SNR) (logarithmic scale) 2. Peak Signal-to-Noise Ratio (PSNR) Measures of Image Quality where I: input image, K: output image

58. 2.4. Color Images 2.4.1. Physics of Color -- The world is colorless. Human color perception is carried out by the nervous system, which interprets differently to distinct electromagnetic wavelengths. -- Electromagnetic spectrum 2-57

59. 2-58 -- Observed color of objects Surface reflection rebounds incoming energy, whose spectrum remains the same as that of the light source Body reflection: The incoming energy diffuses into surface and reflects from its internal pigment -- Visible spectral colors

60. 2-59 2.4.2. Human Color Perception Human Eye FOV: Field Of View (FOV) width × height = 160 deg × 135 deg

61. 2-60 Eyeball Camera

62. 2-61 Retina is composed of photoreceptors retina photoreceptors neural fiber light 2 types of photo- receptors: rods and cones

63. 2-62 Rods : sensitive to intensity Cones: sensitive to color Types of cones: L(red), M(green), S(blue)

64. 2-63 ○ RGB Color Space : color matching functions may be negative 2.4.3. Color spaces – in which colors are made up R, G, B: primary colors, real of different amounts of primary colors

65. 2-64 ○ CIE XYZ Color Space CIE (Commission Internationale d’Eclairage): an organization responsible for color standard X,Y,Z: not real primaries, Y: luminance 。 The volume of visible colors in CIE XYZ space is a cone Their color matching functions are positive everywhere

66. 2-65 。 The relationship between RGB and XYZ

67. 2-66 ○ CIE xy Color Space -- The section intersects the XYZ space with the plane Since x + y + z = 1, a color can be specified by x and y alone. Chromaticity Diagram

68. 2-67 。 Chromaticity Diagram (i) Spectral locus: the curved boundary along which colors of single wavelengths are viewed (ii) Hue changes as one moves around the spectral locus (iv) Colors that lie farther away from the neutral point are more saturated (iii) Neutral point: the color whose weights are equal for all three primaries

69. 2-68 。 RGB Gamut – The colors correspond to positive matching values

70. 2-69 Gamut which can be displayed by display devices CRT monitor Color film Color printer

71. 2-70 ○ HSV (Hue, Saturation, Value) Color Space Hue: varies from red  green Saturation: varies from red  pink Value: varies from black  white

72. 2-71 ○ (i) RGB HSV If R = V, then If G = V, then If B = V, then If H ends up being negative, add 1 If (R,G,B) = (0,0,0), then (H,S,V) = (0,0,0)

73. 2-72 。 Example: (R, G, B) = (0.2, 0.4, 0.6)

74. 2-73 (ii) HSV RGB

75. 2-74 。 Example: (H, S, V) = (0.5833, 0.6667, 0.6)

76. 2-75 ○ YIQ Color Space – Used for TV and video Y : luminance information I, Q : color information

77. 2-76 ○ Uniform Color Space 。 Macadam ellipse -- the noticeable difference of a color forms the boundary of the color in a color space and can be fitted with an ellipse 。 Noticeable difference – the difference when modifying a color until one can tell it has changed

78. 2-77 The color difference in CIE xy space is poor: The distance in the space is a guide to color difference (a) the ellipses at the top are larger than those at the bottom (b) the ellipses rotate as they move up

79. 2-78 ○ CIE u’v’ Color Space – a more uniform space than the CIE xy color space

80. 2-79 ○ CIE L*a*b* Color Space – another substantial uniform space where : the XYZ coordinates of a reference white patch

81. 2-80 ○ Summary ○ Secondary colors (primaries of pigments): Magenta (purple) = R + B = W - G Cyan = G + B = W - R Yellow = R + G = W - B

82. 2.4.5. Color and Lightness Constancies Surfaces have different colors and brightness when viewed under different lights 2-81

83. 2-82 Color constancy – Intensity-independent description of color Lightness constancy – Color-independent description of brightness

84. 2-83 2.5. Cameras 2.5.1. Photosensitive Sensors Types: CCD (charge-coupled devices) CMOS (complementary metal oxide semiconductor) CCD uses a rectangular grid of electron collection sites laid over a silicon wafer.

85. ○ Color CCD: uses the same chip as black and white CCD except that successive rows or columns are made sensitive to R, G or B light using a filter coating that blocks the complementary light. Bayer pattern: a filter pattern including a mosaics of 2 by 2 blocks form by two G, one R, and one B receptors 2-84

86. 2-85 Perspective Projection Summary Biological images in addition to engineering images Types of resolutions: Spatial (space), Radiometric (intensity), Spectral (color), Temporal (time).

87. 2-86 Euclidean City-block Chess-board Metric functions Non-uniform resolution -- to reduce storage while preserve as much information as possible

88. 2-87 Image degradation model: Chamfer distance Simplified model: In frequency domain,

89. Bayer pattern: 2-88 Color spaces: Color and brightness constancies: Noise generation and estimation