Multimedia Specification Design and Production 2012 / Semester 1 / L2 Lecturer: Dr. Nikos Gazepidis

2 Digital Media  In computers, audio, image and video are stored as files just like other text files.  For images, these files can have an extension like: BMP, JPG, GIF, TIFF, PNG, etc.  For audios, the file extensions include: WAV, MP3, …  The videos files usually have extensions: AVI, MOV, MKV Digital Audio, Image & Video

3 An Example of Digital Image  Let’s open an image file is its “raw” format: Digital Audio, Image & Video * P6: (this is a ppm image) * Resolution: 512x512 * Depth: 255 (8bits per pixel in each channel)

4 An Example of Digital Image (cont.) Digital Audio, Image & Video Color depth or Bit depth is the number of bits used to indicate the color of a single pixel in a bitmapped image or video frame buffer.  1-bit color (21 = 2 colors): monochrome  2-bit color (22 = 4 colors): CGA, gray-scale  3-bit color (23 = 8 colors): many early home computers with TV displays  4-bit color (24 = 16 colors): least common denominator VGA  5-bit color (25 = 32 colors)  6-bit color (26 = 64 colors)  8-bit color (28 = 256 colors): most early color Unix workstations, VGA at low-res  12-bit color (212 = 4096 colors): some Silicon Graphics systems The HDMI 1.3 specification defines bit depths of 30 bits (1.073 billion colors), 36 bits (68.71 billion colors), and 48 bits (281.5 trillion colors) …& transparency

5 An Example of Digital Image (cont.) Digital Audio, Image & Video 1 bit (2 colors)2 bits (4 colors)4 bits (16 colors) 8 bits (256 colors)24 bits (16,777,216 colors, "truecolor")

6 An Example of Digital Image (cont.) Digital Audio, Image & Video 8 bits 16 bits 24 bits

7 An Example of Digital Image (cont.) Digital Audio, Image & Video An image contains a header and a bunch of (integer) numbers.

8 Digital Media Capturing Digital Audio, Image & Video  To get a digital image, an audio or a video clip, we need some media capturing device such as: a digital camera or a scanner, a digital audio recorder, or a digital camcorder.  All these devices have to complete tasks: Sampling: To convert a continuous media into discrete formats. Digitization: To convert continuous samples into finite number of digital numbers. There are probably some further compression process.

9 An Audio Signal Digital Audio, Image & Video

10 Sampling for an Audio Signal Digital Audio, Image & Video Sampling period Ts, fs =1/Ts Signal Period T, f = 1/T The sampling frequency or sampling rate fs is defined as: the number of samples obtained in one second, or fs = 1/T …and is measured in Hertz

11 Sampling for an Audio Signal Digital Audio, Image & Video fs = 2.5f fs = 1.67f Original signal A new component is added This is denoted as aliasing.

12 Sampling for an Audio Signal Digital Audio, Image & Video fs = 2f There are infinite number of possible sin waves going through the sampling points

13 Frequency Decomposition Digital Audio, Image & Video  We can use “Fourier Transform” to compute these frequency components.  Nyquist Theorem

14 Image Sampling Digital Audio, Image & Video  The sampling theorem applies to 2D signal (images) too  Nearest point interpolation.

15 Image Sampling (original image) Digital Audio, Image & Video

16 Image Sampling (Aliasing due to sampling) Digital Audio, Image & Video

17 Digitization Digital Audio, Image & Video  The samples are continuous and have infinite number of possible values.  The digitization process approximates these values with a fixed number of numbers.  To represent N numbers, we need log 2 N bits.

18 Digital Audio Digital Audio, Image & Video  You often hear that an audio is 16bits at 44kHz.  44KHz is the sampling frequency. Music has more high frequency components than speech. 8kHz sampling is good enough for telephone quality speech.  16bits means each sample is represented as a 16bit integer.  Digital audio could have more than one channels.

19 Digital Images Digital Audio, Image & Video An image contains 2D samples of a surface, which can be represented as matrices. Each sample in an image is called a pixel.

20 Types of Digital Images Digital Audio, Image & Video  Grayscale image Usually we use 256 levels for each pixel. Thus we need 8bits to represent each pixel (2^8 == 256) Some images use more bits per pixel, for example MRI (Magnetic resonance imaging) images could use 16bits per pixel. A 8bit grayscale Image.

21 Binary Image Digital Audio, Image & Video  A binary image has only two values (0 or 1).  Binary image is quite important in image analysis and object detection applications.

22 Binary Image Digital Audio, Image & Video [ b7 b6 b5 b4 b3 b2 b1 b0] Each bit plane is a binary image. Most Significance Bitplane - MSB Less Significance Bitplane - LSB b7

23 Dithering Digital Audio, Image & Video A technique to represent a grayscale image with a binary one. Convert image to 4 levels: I’ = floor(I/64) 0  1  2  3 

24 Dithering Matrix Digital Audio, Image & Video 0  1  2  3  The dithering matrix is: Definition A square matrix of threshold values that is repeated as a regular array to provide a threshold pattern for an entire image in the dither method of image representation

25 Color Image Digital Audio, Image & Video r g b There are other color spaces: YUV, HSV etc. 24 bit image

26 Color Table Digital Audio, Image & Video Image with 256 colors r g b Clusters of colors It is possible to use much less colors To represent a color image without much degradation.

27 Human Vision Digital Audio, Image & Video Human eye has two kinds of light sensitive cells. 1.The rods and 2.The cones Rods response curve (black and white vision) Cones response curve (color vision) R = s E( ) S r ( )d G = s E( ) S g ( )d B = s E( ) S b ( )d

28 Colors Digital Audio, Image & Video Colorimeter experiment  Color matching function

29 CIE Color Matching Functions Digital Audio, Image & Video  The amounts of R, G, B lighting sources to form single wavelength light forms the color matching curves. CIE color matching curvesCIE standard color matching functions CIE = International Commission on Illumination

30 Gamma Correction Digital Audio, Image & Video Linearly increasing intensity Without gamma correction Linearly increasing intensity with gamma correction

31 Video (Analog Video) Digital Audio, Image & Video Even frameOdd Frame Definition Analog video is a video signal transferred by an analog signal. An analog color video signal contains luminance, brightness (Y) and chrominance (C) of an analog television image. When combined in to one channel, it is called composite video as is the case, among others with NTSC, PAL and SECAM. Analog video may be carried in separate channels, as in two channel S-Video (YC) and multi-channel component video formats.

32 Color System in Video Digital Audio, Image & Video  YUV was used in PAL (an analog video standard) and also used for digital video.  Y is the luminance component (brightness) Y = R G B  U and V are color components U = B – Y V = R - Y

33 PAL vs NTSC vs SECAM Digital Audio, Image & Video  PAL (Phase Alternating Line), is an analogue television colour encoding system used in broadcast television systems in many countries.  NTSC (National Television System Committee) receivers have a tint control to perform colour correction manually. If this is not adjusted correctly, the colours may be faulty. The PAL standard automatically cancels hue errors by phase reversal, so a tint control is unnecessary.  SECAM (Sequential Color with Memory) is an earlier attempt at compatible colour television which also tries to resolve the NTSC hue problem. It does so by applying a different method to colour transmission, namely alternate transmission of the U and V vectors and frequency modulation, while PAL attempts to improve on the NTSC method. SECAM transmissions are more robust over longer distances than NTSC or PAL.

34 PAL vs NTSC vs SECAM Digital Audio, Image & Video