IT-101 Section 001 Lecture #7 Introduction to Information Technology

 Overview Chapter 5: From the real world to Images and Video Introduction to visual representation and display Converting images to gray scale Color representation Video Image Compression

 Introduction to Visual Representation and Display Images play a fundamental role in the representation, storage and transmission of information In the previous chapters we learned how to represent information that was in the form of numbers and text In this chapter we will learn how to represent still and time varying images with binary digits “A picture is worth ten thousand words” But it takes a whole lot more than that!

Image Issues The world we live in is analog, or continuously varying There are problems involved in digitizing, or making discrete, so we make some approximations and determine tradeoffs involved While digitizing, we need to consider the following facts: We are producing information for human use Human vision has limitations Take advantage of this fact Produce displays that are “good enough”

Digital Information for Humans Many digital systems take advantage of human limitations (visual, aural, etc) Human gray scale acuity is 2% of full brightness Or: Most people can detect at most 50 gray levels (6 bits) The human eye can resolve about 60 “lines per degree of visual arc”- a measure of the ability of the eye to resolve fine detail When we look at a 8.5 x 11” sheet of paper at 1 foot (landscape) the viewing angles are degrees for the horizontal dimension, and 39 degrees for the vertical dimension We can therefore distinguish : degrees x 60 = 2955 horizontal lines 39 degrees x 60 = 2340 vertical lines These numbers give us a clue about the length of the code needed to capture images

“lines per degree of visual arc” Image brought closer to the eye, we can resolve more detail Humans can resolve 60 lines per degree of visual arc A line requires two strings of pixels – one black, one white Pixel – The smallest unit of representation for visual information Visual Arc

Pixels A pixel is the smallest unit of representation for visual information Each pixel in a digitized image represents one intensity (brightness) level (gray scale or color) 13 x 13 grid = 169 pixels Gray scaleColor

To form a black line, you need two rows of pixels (one black and one white) to give a visual clue of the transition from black to white For our paper example: Number of pixels needed to represent total image on a page: (2 x 2955) x (2 x 2340) = 27,658,800 pixels per page This number of pixels would be sufficient to represent any image on the page with no visible degradation compared to a perfect (unpixelized) image at a distance of one foot As the number of pixels that form an image (spatial resolution) decreases, the amount of data that needs to be transmitted, stored or processed decreases as well. However, the tradeoff is that the quality of the image degrades as a result

When dealing with printers we often quote the resolution in terms of dots per inch (dpi), which corresponds to pixels per inch in our example It is popular to set laser or ink-jet printer settings to 600 dpi of resolution However, if we hold the paper closer, we would need a greater resolution printer for example: 720 dpi, 1200 dpi or greater A note about printer resolution

How many pixels should be used If too few pixels used, image appears “coarse” 16 x16 (256 pixels) 64 x 64 (4096 pixels)

 Digitizing Images (gray scale) The first step to digitize a “black and white” image composed of an array of gray shades, is to divide the image into a number of pixels, depending on the required spatial resolution The number of brightness levels to be represented by each pixel is assigned next If we wish to use for example, 6 bits for the brightness level of each pixel, then each pixel can represent 64 different brightness levels (shades of gray, from black to white) Then, each pixel would have a 6-bit number associated with it, representing the brightness level (shade) that is closest to the actual brightness level at that pixel

This process is known as quantization (we will learn more about this later in the course)-It is the process of rounding off actual continuous values so that they can be represented by a fixed number of binary digits As a result of the operations just described, the analog image is digitized and represented by a string of binary digits…

6-bit image (64 gray levels) In the figures below, each pixel in the image is represented by 6 bits, 3 bits and 1 bit. The effect of varying the number of bits used to represent each pixel is evident 3-bit image (8 gray levels)

1-bit image (black and white)

How much storage is needed? Total number of bits required for storage = total number of pixels * number of bits used per pixel For example – Black and white photo 64 x 64 pixels Use 32 gray levels (5 bits) 64 x 64 x 5 = 20,480 bits = 2560/1024 bytes = 2.5KB Remember data storage is in bytes KB represents 2 10 or 1024 bytes

Another example Black and White photo 256 x 256 pixel 6 bits (64 gray levels) How much storage is needed? 256 x 256 x 6 = 393,216 bits 393,216/8 = 49,152 bytes 49,152/1024 = 48 KB

A note about resolution Since the total number of bits required for storage = total number of pixels * number of bits used per pixel, there are two ways to reduce the number of bits needed to represent an image: Reduce total number of pixels Reduce number of bits used per pixel Applying these however, reduces the quality of the image. The first results in low spatial resolution (image appears coarse). The second results in poor brightness resolution, as seen by the previous couple of slides. The amount of storage can, however be reduced by applying Image Compression

Recall that any color can be created by adding the right proportions of red, green and blue light If we wish to digitize a color image, we must first divide the image into pixels We must then determine the amount of red, green and blue (RGB) that comprises the color at each pixel location Finally, we must convert these three levels to a binary number of a predefined length For example: If we use 3 bits for each color value, we would be able to represent 8 intensity levels each of red, green and blue This representation would require 9 bits per pixel This would give us 512 different colors per pixel  Digitizing Images (color)

Example Color photo 256x256 pixel 9 bits per pixel (3 bits each for red, green and blue) 256x256x9=589,826 bits 589,826/8=73,728 bytes 73,728/1024=72 KB of storage is needed to store this color photo

Another approach to color representation of images is Hue, Luminance and Saturation (HLS) This system does not represent colors by combinations of other colors, but it still uses 3 numerical values Hue: Represents where the pure color component falls on a scale that extends across the visible light spectrum Luminance: Represents how light or dark a pixel is Saturation: How “pure” the color is, i.e. how much it is diluted by the addition of white (100% saturation means no dilution with white ) Let us see at how this system works with the power point color palette on this box: Hue Luminance Saturation

 Video Human perception of movement is slow Studies show that humans can only take in 20 different images per second before they begin to blur together If these images are sufficiently similar, then the blurring which takes place appears to the eye to resemble motion, in the same way we discern it when an object moves smoothly in the real world. We can detect higher rates of flicker, but only to about 50 per second This phenomenon has been used since the beginning of the 20th century to produce ``moving pictures,'' or movies. Movies show 24 frames per second TV works similarly, but instead of a frame, TV refreshes in lines across the tube This same phenomenon can be used to create digitized video--a video signal stored in binary form

Video We have already discussed how individual images are digitized; digital video simply consists of a sequence of digitized still images, displayed at a rate sufficiently high to appear as continuous motion to the human visual system. The individual images are obtained by a digital camera that acquires a new image at a fast enough rate (say,60 times per second), to create a time-sampled version of the scene in motion Because of human visual latency, these samples at certain instants in time are sufficient to capture all of the information that we are capable of taking in!

Adding up the bits Assume a screen that is 512x512 pixels--about the same resolution as a good TV set. Assume 3 bits per color per pixel, for a total of 9 bits per pixel Let's say we want the scene to change 60 times per second, so that we don't see any flicker or choppiness. This means we will need 512 x 512 pixels x 9 bits per pixel x 60 frames per second x 3600 seconds = 500 billion bits per hour--just for the video. Francis Ford Coppola's The Godfather, at over 3 hours, would require nearly 191 GB--over 191 billion bytes--of memory using this approach. This almost sounds like an offer we can refuse. But, do films actually require this much storage? – Fortunately, no.. The reason we can represent video with significantly fewer bits than in this example is due to compression techniques, which take advantage of certain predict abilities and redundancies in video information to reduce the amount of information to be stored.

 Image Compression Near Photographic Quality Image 1,280 Rows of 800 pixels each, with 24 bits of color information per pixel Total = 24,576,000 bits 56 Kbps modem 56,000 bits/sec How long does it take to download? 24,576,000/56,000 = 439 seconds/60 = 7.31 minutes Obviously image compression is essential.

Images are well suited for compression Images have more redundancy than other types of data. Images contain a large amount of structure. Human eye is very tolerant of approximation error. 2 types of image compression Lossless coding Every detail of original data is restored upon decoding Examples – Run Length Encoding, JPEG, GIF Lossy coding Portion of original data is lost but undetectable to human eye Good for images and audio Examples - JPEG

JPEG -Joint Photographic Experts Group 29 distinct coding systems for compression, 2 for Lossless compression Lossless JPEG uses a technique called predictive coding to attempt to identify pixels later in the image in terms of previous pixels in that same image Lossy JPEG consists of image simplification, removing image complexity at some loss of fidelity GIF – Graphics Interchange Format Developed by CompuServe Lossless image compression system. Application of Lempel-Ziv-Welch (LZW) The two compressed image formats most often encountered on the Web are JPEG and GIF.

Digital Video Compression (MPEG) MPEG is a series of techniques for compressing streaming digital information DVDs use MPEG coding MPEG achieves compression results on the order of 1/35 of original If we examine two still images from a video sequence of images, we will almost always find that they are similar This fact can be exploited by transmitting only the changes from one image to the next Many pixels will not change from one image to the next. Called IMAGE DIFFERENCE CODING Motion Picture Expert Group (MPEG) standard for video compression.