1. MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal

2. MPEG-4 standard 2 Agenda Overview Audio coding Video coding System

3. MPEG-4 standard 3 Definition of MPEG-4 applications “A coded, streamable representation of audio-visual objects and their associated time-variant data along with a description of how they are combined”

4. MPEG-4 standard 4 History MPEG-4 is an ISO/IEC standard developed by the Moving Picture Experts Group. MPEG-4 was finalized in October 1998 and became an International Standard in the first months of 1999 Fully backward compatible extensions under the title of MPEG-4 Version 2 were frozen at the end of 1999, to acquire the formal International Standard Status early in 2000.

5. MPEG-4 standard 5 Applications for MPEG-4 Create content once, play on any network 3 levels of quality –Multimedia database, video games –Video phone, Multimedia authoring –Mobile videophone, wireless LAN

6. MPEG-4 standard 6 Examples of possible scenes (0,1,0) Z Y X 2D inside a 3D plane

7. MPEG-4 standard 7 Examples of possible scenes Animated Text+ Video + Still Images

8. MPEG-4 standard 8 Main characteristics Allows more interaction by the user Scalable Object based, allows scenes to be composed of natural and synthetic objects Very different from MPEG-1&2

9. MPEG-4 standard 9 MPEG-4 audio coders MPEG 4 audiosystemvideo SA Natural codingSynthetic coding GA CELPParametric TTS

10. MPEG-4 standard 10 GA Stands for General Audio The input signal is first decomposed into a time/frequency spectral representation by means of an analysis filter bank Then subsequently quantized and coded with AAC and TwinVQ coders

11. MPEG-4 standard 11 GA coder scheme

12. MPEG-4 standard 12 AAC High quality Higher bit rate (above 32 kbp/s) Twice more compressed than MP3 for same quality

13. MPEG-4 standard 13 TwinVQ Transform-domain Weighted Interleave Vector Quantization (lower bit rate)

14. MPEG-4 standard 14 MPEG-4 audio coders MPEG 4 audiosystemvideo SA Natural codingSynthetic coding GA CELP Parametric TTS

15. MPEG-4 standard 15 CELP Code Excited Linear Predictive Voice coding technique Used with bitrates between 6-24 kbit/s 2 sampling rates: 8 and 16 kHz

16. MPEG-4 standard 16 MPEG-4 audio coders MPEG 4 audiosystemvideo SA Natural codingSynthetic coding CELP Parametric TTS GA

17. MPEG-4 standard 17 Parametric coders Very low bit rate (2 to 16 kbit/s ) Decompose the input signal into components which are described by appropriate source models and represented by model parameters Certain aspects of the coded representation can be manipulated independently HVXC (Harmonic Vector Excitation Coding) for speech HILN (Harmonic and Individuals plus Noise) for music

18. MPEG-4 standard 18 Parametric-HVXC Harmonic coding of LPC residual signals for voiced segments Vector eXcitation Coding for unvoiced segments 2.0 and 4.0 kbit/s of fixed bit rate mode Less than 2.0 kbit/s of variable rate mode

19. MPEG-4 standard 19 HVXC decoder scheme

20. MPEG-4 standard 20 Parametric-HILN Models parameters: –Harmonics lines: fundamental freq+ amplitudes of the harmonics components –Individual Lines: frequency and amplitude of each individual line –Noise: spectral shape of the noise (gotten by LPC method) Bit rate : 6-16 kbit/s

21. MPEG-4 standard 21 HILN coder/decoder scheme

22. MPEG-4 standard 22 Natural Audio Coders Quality Cellular Telephone AM FM CD 248163264 kbit/s Parametric speech (HVXC) High quality speech (CELP) General audio (AAC, TwinVQ) Parametric audio (HILN)

23. MPEG-4 standard 23 MPEG-4 audio coders MPEG 4 audiosystemvideo SA Natural codingSynthetic coding CELPParametric TTS GA

24. MPEG-4 standard 24 SA Structured Audio (use the structural redundancy in the creation of the sounds) SAOL (Structured Audio Orchestra Language) software-synthesis language for any kind of synthesis (FM, sampling, physical-modeling,…) SASL (Structured Audio Score Language) coding of the note desired for SAOL, the time of occurrence, and the parameters controlling the differentiating algorithm (how loud the sound is, how long it is, how it varies) SASBF (Structured Audio Sample Bank Format) format for efficiently transmitting banks of sound samples

25. MPEG-4 standard 25 MPEG-4 audio coders MPEG 4 audiosystemvideo SA Natural codingSynthetic coding CELPParametric TTS GA

26. MPEG-4 standard 26 TTS Text To Speech TTS coders bit rate range from 200 bit/s to 1.2 Kbit/s From a text or a text with prosodic parameters (pitch contour, phoneme duration, and so on) it generates intelligible synthetic speech. Lip synchronization control with phoneme information. Trick mode functionality: pause, resume, jump forward/backward. International language and dialect support for text. (i.e., it can be signaled in the bitstream which language and dialect should be used) International symbol support for phonemes, and support for specifying age, gender, speech rate of the speaker

27. MPEG-4 standard 27 Scalable audio SNR / NMR (Noise to Mask Ratio) Scalability Audio Bandwidth Scalability Restriction of Generality Ex: CELP + AAC Implementation Complexity Core layer easier to decode

28. MPEG-4 standard 28 MPEG-4 Video MPEG 4 audio systemvideo

29. MPEG-4 standard 29 Video coding-Main features Coding and animation of synthetic and natural hybrid video object Same approach as MPEG1/2 algorithms –Macro-block based DCT motion compensation –Uses I,P,B frames and variable length codes Wide range of bit rate 5 kbit/s to 5 Mbit/s available Wide range of resolutions available (from a few pel per line to TV resolution) Supports the coding of arbitrary object shape (non rectangular) Allowed face and body animation Coding of 2D and 3D Meshes with Implicit Structure Supports coding of SPRITE objects

30. MPEG-4 standard 30 MPEG-4 system MPEG 4 audio systemvideo BIFS Obj Descrip DMIF

31. MPEG-4 standard 31 BIFS Binary Format for Scene

32. MPEG-4 standard 32 BIFS Tree scene description

33. MPEG-4 standard 33 BIFS Features VRML concepts : set of nodes to represent the primitive scene objects to be composed, the behavior and interactivity Integration of streams Integration of 2D and 3D video and audio objects Advanced Audio Features Update protocol to modify the scene in time Compression efficiency

34. MPEG-4 standard 34 MPEG-4 system MPEG 4 audio systemvideo Obj Descrip BIFSDMIF

35. MPEG-4 standard 35 Object descriptors Contain pointers to : –Scalably coded content streams –Alternate quality content streams –Object Content Information (locations, transparency,…) –IPR Information Sub descriptors for : –Decoder Configuration –Sync. Layer Header Configuration –Quality of Service Information –Extension Information

36. MPEG-4 standard 36 Object DescriptorID (OD_ID) ObjectDescriptor { OD_ID_1 List of { Elementary- Stream- Descriptors } ES_Descriptor { ES_ID_1....... } ES_Descriptor { ES_ID_2....... } ES_Descriptor { ES_ID_3....... } Object descriptors

37. MPEG-4 standard 37 MPEG-4 system MPEG 4 audio systemvideo DMIF Obj DescripBIFS

38. MPEG-4 standard 38 DMIF Delivery Multimedia Integration Framework It is the interface between the MPEG4 application and the transport network Irrespective of whether the peer is a remote interactive peer, broadcast or local storage media Open the different channels for the elementary streams with different bandwidth and QoS Use of different networks (IP, ATM, narrowband, mobile,…)

39. MPEG-4 standard 39 System scheme

40. MPEG-4 standard 40 Conclusion MPEG-4 provides a lot of tools to code audio and video objects for a whole range of applications In addition to this set of tools, MPEG-4 is a structure to manipulate interactively these objects MPEG-4 has been evolving (more audio and video coders)

41. MPEG-4 standard 41 Main references N1683 MPEG4 Overview N1695 MPEG4 Systems FAQ http://garuda.imag.fr/MPEG4/syssite/syspub/main.html http://www.chiariglione.org/mpeg/faq/mp4-aud/mp4- aud.htm http://www.tnt.uni- hannover.de/project/mpeg/audio/faq/mpeg4.html http://sound.media.mit.edu/mpeg4/sa-tech.html http://faac.sourceforge.net/wiki/index.php?page=HVXC http://wwwam.hhi.de/mpeg-video/standards/mpeg- 4.htm#E11E16

42. MPEG-4 standard 42 Questions????? Let’s go yellow jackets!!!!!!!!!