Audio/Video compression An introduction Alain Bouffioux December, 20, 2006

AV Compression / Alain Bouffioux 2 Agenda Introduction - The evolution of Audio/Video consumer products and the role of compression techniques. Audio & Video compression principles Audio demonstration Video demonstration

December, 20, 2006 AV Compression / Alain Bouffioux 3 Introduction - The evolution of Audio/Video consumer products and the role of compression techniques. Audio & Video compression principles Audio demonstration Video demonstration Agenda

December, 20, 2006 AV Compression / Alain Bouffioux 4 Moore’s law Number of transistors per square inch doubles every 18 months

December, 20, 2006 AV Compression / Alain Bouffioux 5 Moore’s law today Cost of a transistor divided by one million in 30 years

December, 20, 2006 AV Compression / Alain Bouffioux 6 Moore’s law today (2) “Self-fulfulling prophecy” = roadmap for the semiconductor industry Itanium 2 (2004, 592,000,000)

December, 20, 2006 AV Compression / Alain Bouffioux 7 Moore’s law today (3) Roadmap for semiconductor industry = only certainty in the current undefined future Progress in semiconductors = fuels the innovation = fuels the software revolution = fuels the wireless revolution (WLAN, WPAN, WBAN, …) Examples: WBAN & sensors, RFID applications, camera to swallow, flexible display… New products & related needs motivate semiconductor industry (  Self-fulfilling prophecy) Moore’s law will continue to apply: 10 years, 20 years ? –Economical limitation ? Investment (fixed) cost / globalisation –Power consumption (Moore’s low in reverse direction) –Architectural gap between IP-blocks & application (middleware still more complex…)

December, 20, 2006 AV Compression / Alain Bouffioux 8 The evolution of CE products (1) Past: every CE product was analogue ~1983: Music becomes digital (CD players) Early 90ies: microprocessor enters CE devices (Early DVD players incorporated processing power equivalent to their comtemporary PC) Late 90ies: Communication features incorporated within a CE devices. ConsumerComputerCommunication CD DVD Receiver Digital TV or Residential gateway

December, 20, 2006 AV Compression / Alain Bouffioux 9 The evolution of CE products (2) The Residential Gateway (Set-Top-Box, ADSL modem?) as the link between the home and the world-wide information infrastructure. RG Home Network World-wide communication infrastructure

December, 20, 2006 AV Compression / Alain Bouffioux 10 The evolution of CE products (3) The Residential gateway (in home) as the gateway to various services. Local Server provides 2 kind of services: –Broadcast Analogue & digital TV, NVOD, PPV –Point-to-point (Home to local server) Home shopping, VOD, , Web browsing, PC connection... Network Local server Internet Local server About 1000 homes

December, 20, 2006 AV Compression / Alain Bouffioux 11 The evolution of CE products (4) The Residential gateway as a key element of the home network Home Network Residential Gateway To telephone Network To satellite Network To cable Network Computer DVD Jukebox Television Disk Recorder

December, 20, 2006 AV Compression / Alain Bouffioux 12 The evolution of CE products (5) Everything will become digital: audio, telephone, video, photography, newspaper… The question is not if a selected product will really become digital; the question is when? Consumer/Computer/Communication Convergence is progressive New products combine all 3 domains (e.g. New GSM devices – Television on mobile) Products always more and more complex Products have always new features Lifetime of products is always shorter

December, 20, 2006 AV Compression / Alain Bouffioux 13 Compression is one among the various factors (all powered by semiconductor progresses) that enable multimedia technologies. Factors enabling such evolution

December, 20, 2006 AV Compression / Alain Bouffioux 14 BUT !! Convergence of technologies (consumer, communication, computer) All products combine all three technologies BUT ! Divergence of applications –Home consumer, Multimedia phone, Camera, PDA, Office computer, Automotive… –High number of potential products Technology push  Market pull (user centric approach)

December, 20, 2006 AV Compression / Alain Bouffioux 15 Introduction - The evolution of Audio/Video consumer products and the role of compression techniques. Audio & Video compression principles Audio demonstration Video demonstration Agenda

December, 20, 2006 AV Compression / Alain Bouffioux 16 Compression in first A/V CE Products (1) First Audio/Video products made compression without knowing it was compression. How ? By removal of irrelevancies (sampling rate, quantization) Audio and Video characteristics

December, 20, 2006 AV Compression / Alain Bouffioux 17 Compression in first A/V CE Products (2) Audio products From 2 to 7.1 channels are enough to provide the spatial resolution. Video products Three colours (RGB) are enough to provide the spectral resolution.

December, 20, 2006 AV Compression / Alain Bouffioux 18 Audio: Compression needed in spectral domain Bitrate of a stereo audio source (CD-DA encoding) Sampling frequency : 44.1 kHz Stereo 16-bit per sample Bitrate = * 2 * 16 = 1.41 Mbit/sec The need for more compression (1/5)

December, 20, 2006 AV Compression / Alain Bouffioux 19 Video: Compression needed in spatial domain Bitrate of a video source (CCIR Hz countries) 25 images per second YUV coding (Y: luminance - U,V : Chrominance) Y: 8 bit per pixel - U,V: 1 pixel on 2 coded, 8 bit per pixel Bitrate = (576*720)*25*16 = 166 Mbit/sec The need for more compression (2/5)

December, 20, 2006 AV Compression / Alain Bouffioux 20 The need for more compression (3/5) Channels availables for AV transmission –Analog television channel (compatibility) Cable (bandwidth = 8 MHz) Satellite (Bandwidth = MHz)  Capacity around 40 Mbit/sec –Compact disc (CD) For 74 min. play time : 1.41 Mbit/sec

December, 20, 2006 AV Compression / Alain Bouffioux 21 The need for more compression (4/5) MPEG-1 target (Moving Picture Expert Group) (Video-CD : 74 min. constraints) But quality was judged too poor (about VHS quality)

December, 20, 2006 AV Compression / Alain Bouffioux 22 The need for more compression (5/5) MPEG-2 target –Program stream (DVD) –Transport stream (DVB)

December, 20, 2006 AV Compression / Alain Bouffioux 23 Principles of compression (1/2) Compression (or source coding) is achieved by suppressing information : –redundant information –irrelevant information Suppression of redundant information  lossless compression example: PCM to DPCM,DCT The original signal and the one obtained after encoding and decoding are identical

December, 20, 2006 AV Compression / Alain Bouffioux 24 Principles of compression (2/2) Suppression of irrelevant information  lossy compression Example: bandwidth limitation, masking in audio The original signal and the one obtained after encoding and decoding are different but are perceived as identical

December, 20, 2006 AV Compression / Alain Bouffioux 25 Introduction - The evolution of Audio/Video consumer products and the role of compression techniques. Audio & Video compression principles Audio demonstration Video demonstration Agenda

December, 20, 2006 AV Compression / Alain Bouffioux 26 Audio Demonstration From “Borderline” Madonna - Stereo - 16 bit/channel Compression used AAC Compression Decompression Original - 32 kbps 128 kbps 64 kbps 16 kbps 705 kbps

December, 20, 2006 AV Compression / Alain Bouffioux 27 MOS scale (1/2) Signal distortion is not a good measure of the performance of a lossy compression method  an other method is necessary: MOS scale (Mean Opinion Score) The five-grade CCIR impairment scale (Rec.562) 1(Very annoying), 2(Annoying), 3(Slightly annoying), 4(Perceptible but not annoying), 5(Imperceptible) Example:Double blind test

December, 20, 2006 AV Compression / Alain Bouffioux 28 MOS scale (2/2)

December, 20, 2006 AV Compression / Alain Bouffioux 29 Introduction - The evolution of Audio/Video consumer products and the role of compression techniques. Audio & Video compression principles Audio demonstration Video demonstration Agenda

December, 20, 2006 AV Compression / Alain Bouffioux 30 Compression to VBR or to CBR CBR (Constant Bit Rate) vs VBR (Variable Bit Rate) Scene more complex  Higher bit rate for same quality CBR  variable quality (example : Video CD artefact) Constant quality  VBR necessary (e.g.: DVD-Video)

December, 20, 2006 AV Compression / Alain Bouffioux 31 Video demonstration MPEG-1 Example Video CD standard (288*352)video – CBR 1.4 Mbps MPEG-2 Example DVD standard (576*720)video – VBR ~3Mbps

December, 20, 2006 AV Compression / Alain Bouffioux 32 The compression trade-off Compression techniques are still making progress Trade-off Complexity/Quality/Bit Rate New technique may result in new trade-off Quality Bitrate Complexity MPEG Layer 1 MPEG Layer 2 MPEG Layer 3 MPEG AAC Other Technique Speech coding

December, 20, 2006 AV Compression / Alain Bouffioux 33