1. Audio/Video compression An introduction Alain Bouffioux December, 20, 2006

2. AV Compression / Alain Bouffioux 2 Agenda Introduction - The evolution of Audio/Video consumer products and the role of compression techniques. Audio & Video compression principles Audio demonstration Video demonstration

3. December, 20, 2006 AV Compression / Alain Bouffioux 3 Introduction - The evolution of Audio/Video consumer products and the role of compression techniques. Audio & Video compression principles Audio demonstration Video demonstration Agenda

4. December, 20, 2006 AV Compression / Alain Bouffioux 4 Moore’s law Number of transistors per square inch doubles every 18 months

5. December, 20, 2006 AV Compression / Alain Bouffioux 5 Moore’s law today Cost of a transistor divided by one million in 30 years

6. December, 20, 2006 AV Compression / Alain Bouffioux 6 Moore’s law today (2) “Self-fulfulling prophecy” = roadmap for the semiconductor industry Itanium 2 (2004, 592,000,000)

7. December, 20, 2006 AV Compression / Alain Bouffioux 7 Moore’s law today (3) Roadmap for semiconductor industry = only certainty in the current undefined future Progress in semiconductors = fuels the innovation = fuels the software revolution = fuels the wireless revolution (WLAN, WPAN, WBAN, …) Examples: WBAN & sensors, RFID applications, camera to swallow, flexible display… New products & related needs motivate semiconductor industry (  Self-fulfilling prophecy) Moore’s law will continue to apply: 10 years, 20 years ? –Economical limitation ? Investment (fixed) cost / globalisation –Power consumption (Moore’s low in reverse direction) –Architectural gap between IP-blocks & application (middleware still more complex…)

8. December, 20, 2006 AV Compression / Alain Bouffioux 8 The evolution of CE products (1) Past: every CE product was analogue ~1983: Music becomes digital (CD players) Early 90ies: microprocessor enters CE devices (Early DVD players incorporated processing power equivalent to their comtemporary PC) Late 90ies: Communication features incorporated within a CE devices. ConsumerComputerCommunication CD DVD Receiver Digital TV or Residential gateway

9. December, 20, 2006 AV Compression / Alain Bouffioux 9 The evolution of CE products (2) The Residential Gateway (Set-Top-Box, ADSL modem?) as the link between the home and the world-wide information infrastructure. RG Home Network World-wide communication infrastructure

10. December, 20, 2006 AV Compression / Alain Bouffioux 10 The evolution of CE products (3) The Residential gateway (in home) as the gateway to various services. Local Server provides 2 kind of services: –Broadcast Analogue & digital TV, NVOD, PPV –Point-to-point (Home to local server) Home shopping, VOD, e-mail, Web browsing, PC connection... Network Local server Internet Local server About 1000 homes

11. December, 20, 2006 AV Compression / Alain Bouffioux 11 The evolution of CE products (4) The Residential gateway as a key element of the home network Home Network Residential Gateway To telephone Network To satellite Network To cable Network Computer DVD Jukebox Television Disk Recorder

12. December, 20, 2006 AV Compression / Alain Bouffioux 12 The evolution of CE products (5) Everything will become digital: audio, telephone, video, photography, newspaper… The question is not if a selected product will really become digital; the question is when? Consumer/Computer/Communication Convergence is progressive New products combine all 3 domains (e.g. New GSM devices – Television on mobile) Products always more and more complex Products have always new features Lifetime of products is always shorter

13. December, 20, 2006 AV Compression / Alain Bouffioux 13 Compression is one among the various factors (all powered by semiconductor progresses) that enable multimedia technologies. Factors enabling such evolution

14. December, 20, 2006 AV Compression / Alain Bouffioux 14 BUT !! Convergence of technologies (consumer, communication, computer) All products combine all three technologies BUT ! Divergence of applications –Home consumer, Multimedia phone, Camera, PDA, Office computer, Automotive… –High number of potential products Technology push  Market pull (user centric approach)

15. December, 20, 2006 AV Compression / Alain Bouffioux 15 Introduction - The evolution of Audio/Video consumer products and the role of compression techniques. Audio & Video compression principles Audio demonstration Video demonstration Agenda

16. December, 20, 2006 AV Compression / Alain Bouffioux 16 Compression in first A/V CE Products (1) First Audio/Video products made compression without knowing it was compression. How ? By removal of irrelevancies (sampling rate, quantization) Audio and Video characteristics

17. December, 20, 2006 AV Compression / Alain Bouffioux 17 Compression in first A/V CE Products (2) Audio products From 2 to 7.1 channels are enough to provide the spatial resolution. Video products Three colours (RGB) are enough to provide the spectral resolution.

18. December, 20, 2006 AV Compression / Alain Bouffioux 18 Audio: Compression needed in spectral domain Bitrate of a stereo audio source (CD-DA encoding) Sampling frequency : 44.1 kHz Stereo 16-bit per sample Bitrate = 44100 * 2 * 16 = 1.41 Mbit/sec The need for more compression (1/5)

19. December, 20, 2006 AV Compression / Alain Bouffioux 19 Video: Compression needed in spatial domain Bitrate of a video source (CCIR 601 - 50 Hz countries) 25 images per second YUV coding (Y: luminance - U,V : Chrominance) Y: 8 bit per pixel - U,V: 1 pixel on 2 coded, 8 bit per pixel Bitrate = (576*720)*25*16 = 166 Mbit/sec The need for more compression (2/5)

20. December, 20, 2006 AV Compression / Alain Bouffioux 20 The need for more compression (3/5) Channels availables for AV transmission –Analog television channel (compatibility) Cable (bandwidth = 8 MHz) Satellite (Bandwidth = 30-40 MHz)  Capacity around 40 Mbit/sec –Compact disc (CD) For 74 min. play time : 1.41 Mbit/sec

21. December, 20, 2006 AV Compression / Alain Bouffioux 21 The need for more compression (4/5) MPEG-1 target (Moving Picture Expert Group) (Video-CD : 74 min. constraints) But quality was judged too poor (about VHS quality)

22. December, 20, 2006 AV Compression / Alain Bouffioux 22 The need for more compression (5/5) MPEG-2 target –Program stream (DVD) –Transport stream (DVB)

23. December, 20, 2006 AV Compression / Alain Bouffioux 23 Principles of compression (1/2) Compression (or source coding) is achieved by suppressing information : –redundant information –irrelevant information Suppression of redundant information  lossless compression example: PCM to DPCM,DCT The original signal and the one obtained after encoding and decoding are identical

24. December, 20, 2006 AV Compression / Alain Bouffioux 24 Principles of compression (2/2) Suppression of irrelevant information  lossy compression Example: bandwidth limitation, masking in audio The original signal and the one obtained after encoding and decoding are different but are perceived as identical

25. December, 20, 2006 AV Compression / Alain Bouffioux 25 Introduction - The evolution of Audio/Video consumer products and the role of compression techniques. Audio & Video compression principles Audio demonstration Video demonstration Agenda

26. December, 20, 2006 AV Compression / Alain Bouffioux 26 Audio Demonstration From “Borderline” Madonna - Stereo - 16 bit/channel Compression used AAC Compression Decompression Original - 32 kbps 128 kbps 64 kbps 16 kbps 705 kbps

27. December, 20, 2006 AV Compression / Alain Bouffioux 27 MOS scale (1/2) Signal distortion is not a good measure of the performance of a lossy compression method  an other method is necessary: MOS scale (Mean Opinion Score) The five-grade CCIR impairment scale (Rec.562) 1(Very annoying), 2(Annoying), 3(Slightly annoying), 4(Perceptible but not annoying), 5(Imperceptible) Example:Double blind test

28. December, 20, 2006 AV Compression / Alain Bouffioux 28 MOS scale (2/2)

29. December, 20, 2006 AV Compression / Alain Bouffioux 29 Introduction - The evolution of Audio/Video consumer products and the role of compression techniques. Audio & Video compression principles Audio demonstration Video demonstration Agenda

30. December, 20, 2006 AV Compression / Alain Bouffioux 30 Compression to VBR or to CBR CBR (Constant Bit Rate) vs VBR (Variable Bit Rate) Scene more complex  Higher bit rate for same quality CBR  variable quality (example : Video CD artefact) Constant quality  VBR necessary (e.g.: DVD-Video)

31. December, 20, 2006 AV Compression / Alain Bouffioux 31 Video demonstration MPEG-1 Example Video CD standard (288*352)video – CBR 1.4 Mbps MPEG-2 Example DVD standard (576*720)video – VBR ~3Mbps

32. December, 20, 2006 AV Compression / Alain Bouffioux 32 The compression trade-off Compression techniques are still making progress Trade-off Complexity/Quality/Bit Rate New technique may result in new trade-off Quality Bitrate Complexity MPEG Layer 1 MPEG Layer 2 MPEG Layer 3 MPEG AAC Other Technique Speech coding

33. December, 20, 2006 AV Compression / Alain Bouffioux 33