1. CSC 8610 & 5930 Multimedia Technology Lecture 2 Digital Image Representation

2. 2 Chapter 1 Today in class (1/30) 6:15Recap, Reminders 6:25Lecture – Digital Image Representation 7:30Break 7:40Workshop 1 discussion & presentations 8:20Workshop 2 – Digital Image Exploration 9:00Wrap

3. 3 ANALOG VS. DIGITAL

4. 4 Analog Information Examples: time, weight temperature line length width and length of a sheet of paper sound loudness light brightness color saturation and hue Continuous information An infinite number of divisions exist between any two measurements

5. 5 What is the length of the pencil?

6. 6

7. 7 What is the temperature?

8. 8 Analog Thermometer vs. Digital Thermometer digital thermometer analog thermometer

9. 9 Analog vs. Digital Analog information – continuous – made up of infinite number of data points Digital data – discrete

10. 10 Discrete Data Examples: number of persons There is no in-between one person and two persons. choices in multiple-choice questions There is no in-between choice A and choice B.

11. 11 Analog vs. Digital Therometers and Scales What are the limitations of these analog and digital devices? What are the advantages of these analog and digital devices?

12. 12 Analog vs. Digital Sight and sound we peceive in our natural world are analog information--continuous and infinite number of points between any two points. Computers handle discrete digital data. In addition, the amount of data has to be finite. Sight and sound must be converted into finite discrete digital data in order for the computer to handle.

13. 13 MONITORING A PUPPY'S WEIGHT IN HIS FIRST YEAR

14. 14 Suppose you use an analog scale to weigh the puppy

15. 15 Now, what is the weight you would note down for this puppy?

16. 16 See the problem in picking a number to represent an analog measurement?

17. 17 Number of Decimal Places In recording the weight, you must decide the number of decimal places to use. This determines the precision or exactness of the measurement. How many will give an exact measurement? How many is enough? How many is too many?

18. 18 Using More Decimal Places Pros : – increase the precision in general (But how many is meaningful?) – Will allow finer distinction between values (will explain in the next slide) Cons: – Require more paper and paperwork. – Take longer to read through and interpret the numbers.

19. 19 Distinction Between Values With one decimal place: – You can have 10 different values between say 2 and 3: 2.1, 2.2,...3.0 You can distinct between 2.5 and 2.8. But 2.5 and 2.8 would have been rounded to the same value of 3 the values do not allow decimal places.

20. 20 Distinction Between Values Suppose the allowable weight read outs are these 10 levels: 0, 5, 10, 15, 20, 25, 30, 35, 40, 45 Then, 2 pounds: rounded to 0 pound 3 pounds: rounded to 5 pounds The difference between 2 and 3 pounds is 1 pound. But now, it become 5 pounds if we use these levels.

21. 21 How many measurements to make? A. once a year B. once a month C. every two weeks D. every week E. every day F. every hour G. every minute H. every second Now, how often would you weigh the puppy to produce a "good" monitoring of his weight over his first year?

22. 22 What are your considerations in deciding how often to weigh the puppy?

23. 23 Considerations What happens if you weigh the puppy not often enough? What happens if you weigh the puppy too often? Is there one right answer? Will you use the same weighing schedule to monitor the weight of an adult dog?

24. 24 DIGITIZATION: SAMPLING AND QUANTIZATION Back to the Computer

25. 25 Digitization To convert analog information into digital data that computers can handle 2-step process: 1. sampling 2. quantization

26. 26 Sampling Analogous to weighing and recording the puppy's weight During the sampling step, you need to set a sampling rate. Sampling rate: how often you take a data

27. 27 Suppose this is the true timeline of the puppy's first-year growth

28. 28 Suppose you weigh the puppy once a month

29. 29 You get these data points

30. 30 You then interpolate the points

31. 31 You would miss the changes that occur during the first month

32. 32 But the rest matches with the true growth pretty well

33. 33 What about weighing the puppy once a week?

34. 34 You get these data points

35. 35 The data is catching the changes occurring in the first month better

36. 36 But is it exactly?

37. 37 Now for the rest of the year, the data points seem too many

38. 38 Sampling Rate Weighing Puppy Scenario Digitization high (i.e. taking data often) Pros: can catch more weight changes Cons: produce more paperwork and thus take longer to read through all the data Pros: can capture details (e.g. some changes of color within a small region in a picture or amplitude changes in sound within a short period of time) Cons: produce larger file and thus take longer to process low (i.e. taking data infrequently) Pros: less paperwork and thus take shorter time to read through all the data Cons: may miss weight changes Pros: produce smaller file and thus take shorter time to process Cons: may miss details (e.g. color changes in a picture or changes in sound)

39. 39 Quantization Analogous to rounding the weight to fix number of digits in the weighing puppy scenario During the quantization step, you need to set bit depth. Bit depth refers to the number of allowable levels you map (or round) the values to.

40. 40 Example: 10 levels of weight For 10 discrete levels, you may have the 10 allowable values as 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, and 2.9 0, 5, 10, 15, 20, 25, 30, 35, 40, 45 2, 4, 6, 8, 10, 12, 14, 16, 18, 20 1, 2, 3, 4, 5, 6, 7, 8, 9, 10... and so forth

41. 41 Suppose you choose 2.0, 2.1,..., 2.9 For 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9 Any weight data below 2.0 will be recorded as 2.0. Any weight data higher than 2.9 will be capped at 2.9. It works well if the puppy's weight falls in this range. But it does not seem to be the case.

42. 42 Suppose you choose 0, 5,..., 45 For 0, 5, 10, 15, 20, 25, 30, 35, 40, 45 A weight of 2 pounds would be rounded to 0 and a weight of 3 pounds to 5. Cons: For example, the difference between 2 and 3 pounds is altered after they are mapped to the allowable value on this 10-level scale. The difference becomes 5 pounds not 1 pound. Pros: Wider range. Again, it works well if the puppy's weight falls in this range.

43. 43 Well, what if we choose this: 2.0, 2.1, 2.2,..., 44.8, 44.9, 45.0

44. 44 Suppose you choose 2.0, 2.1,...,44.9, 45.0 You have increased the number of levels from 10 to 431. Pros: – Increase precision compared to using 0, 5, 10, 15, 20, 25, 30, 35, 40, 45 – Increase range compared to using 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9

45. 45 Increase Number of Allowable Levels There does not seem to be any cons in the weighing puppy scenario. However, for digitization, increasing the number of allowable levels (i.e. increasing bit depth) will increase the file size.

46. 46 Sampling and Quantization Digitizing media involves sampling and quantization regardless of the type of media: – images – video – audio

47. 47 Overview of how sampling rate and bit depth affect digital media file quality Sampling rate is related to: Bit depth is related to: digital imagesimage resolution, or number of pixels number of allowable colors in an image digital videonumber of pixels in the video, frame rate number of allowable colors digital audiosampling rate of the audio (it limits how high the pitch of the audio can be captured) number of allowable levels of amplitude Details will be covered in chapters for each media type.

48. 48 BITMAPS & DIGITIZATION

49. 49 Pegboard Analogy A 10 holes  10 holes pegboard

50. 50 Pegboard Analogy Suppose you want to copy this music note graphic on the pegboard.

51. 51 Pegboard Analogy Place one peg.

52. 52 Pegboard Analogy 2 pegs.

53. 53 Pegboard Analogy 3 pegs.

54. 54 Pegboard Analogy Suppose we only put peg in a hole if more than half of its area is covered by the musical note graphic.

55. 55 Pegboard Analogy Now remove the musical note overlay.

56. 56 Pegboard Analogy Details are lost because the grid is too coarse for this musical note.

57. 57 How would you improve the details of the musical notes on the pegboard?

58. 58 Using a pegboard with more holes

59. 59 Using a pegboard with more holes Now it looks closer to the original musical note.

60. 60 Pixels Each peg hole on the pegboard is a sample point. The sample points are discrete. In digital imaging, each of these discrete sample points is called a picture element, or pixel for short.

61. 61 Pixel Dimensions Refer to an image’s width and height in pixels In the pegboard analogy, the dimension of this pegboard would be 10 holes  10 holes.

62. 62 Sampling Step

63. 63 Let's look at the sampling step of digitizing a natural scene as if we are taking a digital photo of a natural scene.

64. 64 A natural scene Look up and let your eyes fall on the scene in front of you. Draw an imaginary rectangle around what you see. This is your “viewfinder.” Imagine that you are going to capture this view on a pegboard.

65. 65 Sample into a grid of 25  20 discrete samples Suppose you are going to sample the scene you see in the "viewfinder" into a pegboard with 25  20 holes.

66. 66 One color for each peg hole. Each peg hole takes only one peg. Suppose each peg has one solid color. Suppose the color of each of these discrete samples is determined by averaging the corresponding area.

67. 67 This sampled image looks blocky. Details are lost because the grid is too coarse for this image.

68. 68 For 25  20 sample points, it means you get a digitized image of 25  20 pixels.

69. 69 Let's try a different grid size.

70. 70 Sample into a grid of 100  80 discrete samples Suppose you are going to sample the scene you see in the "viewfinder" into a pegboard with 100  80 holes.

71. 71 Again, one color for each peg hole.

72. 72 For 100  80 sample points, it means you get a digitized image of 100  80 pixels.

73. 73 Sampling Rate Refers to how frequent you take a sample For an image, sampling frequency refers to how close neighboring samples are in a 2-D image plane.

74. 74 Sampling Rate For example, when we change the grid from 25  20 to 100  80, we say that we increase the sampling rate. – You are sampling more frequently within the same spatial distance.

75. 75 Resolution In digital imaging, increasing the sampling rate is equivalent to increasing the image resolution.

76. 76 Consequences of Higher Resolution With higher resolution, You have more sample points (pixels) to represent the same scene, i.e., the pixel dimensions of the captured image are increased. The file size of the digitized image is larger. You gain more detail from the original scene.

77. 77 Resolution of Digital Photos Note that 25  20 and 100  80 pixels are by no means realistic pixel dimensions in digital photography. They are only for illustration purposes here. Most digital cameras can capture images in the range of thousand pixels in each dimension—for example, 3000 pixels  2000 pixels.

78. 78 A Pixel is not a Square Block A pixel is a sample point. It does not really have a physical dimension associated with it.

79. 79 A Pixel is not a Square Block When you zoom in on a digital image in an image editing program, you often see the pixels represented as little square blocks. This is simply an on-screen representation of a sample point of an digitized image.

80. 80 COLORS

81. 81 Problems A natural image is colored in continuous tones, and thus it theoretically has an infinite number of colors. The discrete and finite language of the computer restricts the reproduction of an infinite number of colors and shades.

82. 82 Quantization Step To encode an infinite number of colors and shades with a finite list. Quantizing the sampled image involves mapping the color of each pixel to a discrete and precise value.

83. 83 Quantization Step First, you need to consider how many possible colors you want to use in the image. To illustrate this process, let’s return to the example of the 100  80 sampled image.

84. 84 The sampled 100  80 image

85. 85 Say, we want to map the color of each sample points into one of these four colors: Mapping colors

86. 86 Quantized with 4 Colors

87. 87 Quantized with 8 Colors

88. 88 Consequences of Quantization Reduce the number of allowed colors in the image. When we reduce the colors, different colors from the original may bemapped to the same color on the palette. This causes the loss of the image fidelity and details. The details that rely on the subtle color differences are lost during quantization.

89. 89 The area outlined in red is made up of many different green colors. The same area in the 4-color image now has only one color.

90. 90 Bit Depth The number of colors used for quantization is related to the color depth or bit depth of the digital image. A bit depth of n allows 2 n different colors. Examples: – A 2-bit digital image allows 2 2 (i.e., 4) colors in the image. – An 8-bit digital image allows 2 8 (i.e., 256) colors in the image. The most common bit depth is 24. A 24-bit image allows 2 24 (i.e., 16,777,216) colors.

91. 91 Increasing colors It depends, and in most cases, can be yes. The number of colors or the bit depth is not the only determining factor for image fidelity in quantizing an image. The choice of colors for the quantization also plays an important role in the reproduction of an image. Will increasing the number of colors in the palette improve the image fidelity?

92. 92 Quantized with 8 Different Colors

93. 93 Effect of Bit Depth on File Size Higher bit depth means more bits to represent a color. Thus, an image with a higher bit depth has a larger file size than the same image with a lower bit depth.

94. 94 VECTOR GRAPHICS

95. 95 Vector Graphics Examples: graphics created in – Adobe Flash – Adobe Illustrator

96. 96 Vector Graphics Characteristics Generated mathematically, i.e. instructions not pixel-based Resolution independent

97. 97 Vector Graphics Instruction Example A simple postscript example: %! newpath 200 200 moveto 300 200 lineto stroke showpage

98. 98 Bitmap Images vs. Vector Graphics An analogy: Driving Directions: A visual map vs. a written instruction Bitmap ~~> map Vector Graphics ~~> a written instruction

99. 99 Bitmap Images vs. Vector Graphics An analogy: Driving Directions: A visual Triptik map vs. a written instruction Which one takes up more storage space? Triptik map ~~> bitmap Which one takes you more time to translate the direction into a mental image? written instruction ~~> vector graphics (take more computation to display on computer because it is mathematically generated)

100. 100 Bitmap Images vs. Vector Graphics An analogy: Driving Directions: A visual Triptik map vs. a written instruction If you are going to draw out a map based on a written instruction, how big can you draw? as big as you physically can ~~> the instruction is resolution independent

101. 101 Bitmap Images vs. Vector Graphics 1111111111111111111111011111101111110111111 011111101111111111111 %! newpath 2 1 moveto 6 5 lineto stroke showpage vector graphic bitmap image

102. 102 Vector Graphics %! newpath 2 1 moveto 6 5 lineto stroke showpage vector graphic The unit is arbituary, i.e. when you print out an image, you may set one unit as an inch or a foot. This means vector graphic is resolution independent.

103. 103 Printing Bitmap Images bitmap image print bigger print smaller have the same amount of data, i.e. same level of details

104. 104 Printing Vector Graphics vector graphics print bigger print smaller

105. 105 Bitmap Images vs. Vector Graphics Example (a) Vector graphics:A line defined by two end points. (b) Vector graphics: The same line is stroked with a certain width. (c) & (d) The line is converted to a bitmap.

106. 106 Curve Drawing in Vector Graphics Programs defined by a set of points; we call them anchor points the direction handles or tangent handles of a point controls the tangent at that point on the curve

107. 107 Rasterizing Vector Graphics Raster means convert vector graphics into pixel-based images. Most vector graphics programs let you rasterize vector graphics. Need to specify a resolution for rasterizing, that is, how coarse or how fine the sampling.

108. 108 ALIASING Blurriness, Blockiness, Moire Patterns, Jagged Edges

109. 109 Aliasing The rasterized image will appear jagged. This jagged effect is a form of aliasing caused by undersampling (which means insufficient sampling rate.) Recall the musical note on a pegboard example. Original vector graphics Rastered vector graphics without anti-aliasing

110. 110 Anti-aliasing Techniques To soften the jaggedness by coloring the pixels with intermediary shades in the areas where the sharp color changes occur. Original vector graphics Rastered vector graphics without anti-aliasing Rastered vector graphics with anti-aliasing

111. 111 FREQUENCY IN DIGITAL IMAGES

112. 112 Function over spatial domain Any image can be represented as a function Frequency – the rate at which color values change over the space the image occupies

113. 113 DISCRETE COSINE TRANSFORM

114. 114 Discrete Cosine Transform Transforming the image data (function) from the spatial domain to the frequency domain We won’t explore the mathematics Awareness of functions and transform is goal

115. 115 COLOR MODELS

116. 116 Color Models Used to describe colors numerically, usually in terms of varying amounts of primary colors. Common color models: – RGB – CMYK – HSB – CIE and their variants.

117. 117 RGB Color Model Primary colors: – red – green – blue Additive Color System

118. 118 Additive Color System

119. 119 Additive Color System of RGB Full intensities of red + green + blue = white Full intensities of red + green = yellow Full intensities of green + blue = cyan Full intensities of red + blue = magenta Zero intensities of red, green, and blue = black Same intensities of red, green, and blue = some kind of gray

120. 120 Color Monitors From a standard CRT monitor screen

121. 121 Color Monitors From a SONY Trinitron monitor screen

122. 122 Color Monitors From a LCD screen

123. 123 RGB Color Model depicted graphically as a cube defined by three axes in 3-D space The maximum value on eachaxis is normalized to 1.

124. 124 RGB Color Model x-axis: red values y-axis: green values z-axis: blue values

125. 125 RGB Color Model The coordinates within the color cube represent the relative intensities of red, green, and blue colors.

126. 126 RGB Color Model origin (0,0,0): black

127. 127 RGB Color Model (1,0,0): full intensity of red

128. 128 RGB Color Model (0,1,0): full intensity of green

129. 129 RGB Color Model (0,0,1): full intensity of blue

130. 130 RGB Color Model (1,1,1): white

131. 131 RGB Color Model (1,1,0): full red + full green = full yellow

132. 132 RGB Color Model (1,0,1): full red + full blue = full magenta

133. 133 RGB Color Model (0,1,1): full green+ full blue = full cyan

134. 134 Color Picker So where is the color whose RGB values are (150, 200, 100) in the RGB cube?

135. 135 Correlating RGB Color Cube with Color Picker This is a 2-D slice containing all the colors with red = 150.

136. 136 Correlating RGB Color Cube with Color Picker This is a 2-D slice containing all the colors with green = 200.

137. 137 Correlating RGB Color Cube with Color Picker This is a 2-D slice containing all the colors with blue = 100.

138. 138 Correlating RGB Color Cube with Color Picker The color (150, 200, 100) is located in the 3-D space of the RGB color cube. It is at the intersection of the three 2-D slices.

139. 139 CMYK Color Model Primary colors: – cyan – magenta – yellow – black Subtractive Color System

140. 140 Subtractive Color System of CMY

141. 141 Subtractive Color System of CMY Full intensities of cyan + magenta + yellow = black (theoretically, but in practice with inks, it is not full black) Full intensities of cyan + magenta = blue Full intensities of cyan + yellow = green Full intensities of magenta + yellow = red Zero intensities of cyan, magenta, and yellow = white

142. 142 HSB Color Model Hue: – basic color – 0 o to 360 o : the location on a color wheel – in the order of colors in a rainbow Saturation: – purity of the color – how far away from the neutral gray of the same brightness Brightness

143. 143 HSB Color Model

144. 144 HSB Color Model

145. 145 HSB Color Model A slice of the color wheel from the HSB model

146. 146 HSB Color Model Matches well with the way humans intuitively think about colors For example, how would you describe this color? Would you think of it in terms of how much red, green, and blue? Would you first think of it as a yellowish color and then figure out the lightness and how washed out the color is?

147. 147 Problems with RGB and CMYK Color Space Do not encompass all the colors human can see

148. 148 CIE XYZ Color Model Primaries: – X – Y – Z Primaries are not physical colors Its color space encompasses all the colors human can see. Normally not used in digital image editing because monitors and printers cannot reproduce all the colors in the CIE XYZ color space anyway.

149. 149 Color Gamuts Refers to the range of colors of a specific system or device can produce or capture

150. 150 Color Gamuts a) Colors that human can see b) RGB color gamut of typical CRT monitors c) CMYK color gamut of typical inkjet printers

151. 151 Color Gamuts Monitors and inkjet printers cannot reproduce all the colors that human can see Some of the colors that monitors can reproduce cannot be reproduced by inkjet printers. Most of these colors lie at the corners of the color gamut of the monitor, which means these are highly saturated colors.

152. 152 Identifying Out-of-Gamut Colors in Images Out-of-gamut colors will not be printed correctly. In digital image editing programs such as Adobe Photoshop, you can tell whether a color is out of gamut based on your CMYK setting. Click on the icon to use the closest color in gamut for printing.

153. 153 Identifying Out-of-Gamut Colors in Images Out-of-gamut colors will not be printed correctly. In digital image editing programs such as Adobe Photoshop, you can tell whether a color is out of gamut based on your CMYK setting.

154. 154 Identifying Out-of-Gamut Colors in Images

155. 155 Identifying Out-of-Gamut Colors in Images Click on the icon to replace the out-of-gamut color with the closest color in gamut for printing.

156. 156 Identifying Out-of-Gamut Colors in Images The out-of-gamut color is replaced with the closest color in gamut for printing.

157. 157 Difficulties in Reproducing Colors in Digital Images Digital devices cannot produce all of the colors visible to human Difficulties exist in reproducing color across devices – different devices have different color gamuts – additive color system for screen display vs. subtractive color system for printing

158. 158 Indexed Color

159. 159 Indexed Color