Audio & Video Representation CS105

Data Representation Types of data: – Numbers – Text – Images – Audio & Video

What is sound? A continuous wave created by oscillations of pressure through any material (solid, liquid & gas) For a computer: Analog to Digital – Specialized Hardware Analog signal

Digitizing Sound Voltage Time Discrete time (sampling) & discrete voltage (quantization) Reasonable sound production: 40,000 times per sec Quantization: Dividing vertical axis into pieces – process of mapping a continuous range of values by a small finite set of values 8 bit quantization = 256 levels, 16 bit = levels Quantization Sampling

Digitizing Sound Quantization into levelsSampling time intervals How many bits are needed to represent per sample? - 3 bits - Power of 2 How many bits are needed to digitize this signal? - 3 x 13 = 39 bits

Bit Depth and Bit Rate Bit Depth: Number of bits per sample Bit Rate: Number of bits that are sent per unit of time  Each tick on the time axis (x-axis) is 1 sec. All levels are represented by 3 bits. What is the bit rate? - 3 bits/s  Every 4 ticks on the time axis is 1 sec i.e. you sample 4 times a second. What is the bit rate now? - 12 bits/s 1 sec

Audio Formats Wav, AU, AIFF, VQF and MP3 MP3 (MPEG1 or 2) – dominant format for compressing audio data – Psychoacoustic models – Form of Huffman encoding – Lossy compression Bit rates: – Uncompressed audio: 1411 kbit/s – Compressed audio (such as mp3) : 128 to 320 kbit/s

Practice Digitizing Signals!

Name: Date: Time Voltage Digitize this graph in the sampling intervals using 2 bits Level 3 Level 1 Level 2 Level 4 S1 S3 S2 S4 S5 S6

Video Three dimensional array of color pixels – Succession of images – Each image in the video is called a frame Video: Duration, Frame Size with a bit depth, frame rate (fps) – Example video: Duration: 3600s (1 hr), Frame Size: 640W*480H with 24 bits for each pixel, 25 fps Frame: Gb, Every sec: Gb, Every hour: ~83 Gb – Human eye and frame rate: film and video games

Video Codecs Video files are big! – Typically gigabytes of data for an hour of video Codec: Compressor/Decompressor – Employs lossy compression – Temporal & Spatial compression Temporal compression: Differences between frames – Keyframe is used to compare differences between frames and encode only the differences Spatial compression: Similar regions within frames and groups them – Run length encoding

Container formats Encapsulates both audio, video and metadata (needed to synchronize audio and video streams) Stores the data, does not encode it – For access: program that opens the container and decodes the data inside AVI, MOV

Video Data MPEG 1, 2 and 4 lossy compression – MPEG 1 and 2 – deals with differences in frames – MPEG 4 : Better video quality than MPEG 1 and 2 Widely used Instead of dealing with each individual frame, it tackles objects within each frame Bit rate: – 1.25 Mbit/s – VCD quality ( MPEG-1 video compression ) – 5 Mbit/s – DVD quality ( MPEG-2 compression ) – 8 to 15 Mbit/s – HDTV quality ( MPEG-4 compression ) – 40 Mbit/s max – Blu-ray Disc