1. Roberta Eklund Consultant MPEG-4 AUDIO OVERVIEW

2. MPEG-4 Audio Overview Y Y Natural Audio Y Y T/F Y YCELP Y Y PARA Y Y Structured Audio Y YSAOL Y YSASL Y YSASBF Y YMIDI-DLS-version 2 Y YTTS Y Y Cross Tool(Algorithm) Functionality Y Y Pitch/tempo change Y Y Bitrate scalability Y Y Computation complexity scalability Y Y Error robustness Y Y Audio related effects Y Y Acoustic virtualization

3. Different Tools for Bitrates/Application

4. MPEG-4 Audio Tools PROFILES n Profile - defines the syntax of the bitstream for one single Object, that can represent a meaningful entity in the Audio or Visual scene. Elementary bitstream n Object Profile - defines the syntax of the bitstream for one single Object, that can represent a meaningful entity in the Audio or Visual scene. Elementary bitstream n Profile - defines which different Object Profiles can be combined in the Audio or Visual scene. Combinations of Elementary bitstreams. n Composition Profile - defines which different Object Profiles can be combined in the Audio or Visual scene. Combinations of Elementary bitstreams.

5. OBJECT PROFILES

6. Combination Profiles

7. MPEG-4 Encoder Structure

8. Encoder Configuration MPEG-4 T/F Encoder Configuration

9. MPEG-4 T/F Decoder Configuration

10. Block Diagram of CELP Encoder

11. Excitation signal generator: l codebook l regular pulse excitation (RPE) l multi-pulse excitation (MPE) Block Diagram of CELP Decoder

12. Block Diagram of PARA Encoder

13. Block Diagram of PARA Decoder

14. Two operating modes l l harmonic and noise components (HVXC) – –for speech coding at 2...4 kbps l l harm. & indiv. sinusoidal comp. + noise (HILN) – –for coding of music signals with low complexity content (e.g. single instruments) at 4...16 kbps l l combination of both modes – –support by syntax, defined transition – –automatic mode selector – –cross fade from one signal to another one PARA is Two Codecs in One

15. Text-to-Speech n Phonemic (language-independent) syntax n Prosody, timing cues n Language, dialect, gender, age parameters n Automatic synchronization with FBA n Exact TTS synthesis non-normative; only interface is specified

16. Structured Audio n Structured Audio - Sound coding using structured descriptions n Structured Audio decoder - music and sound-effect synthesis n MMA, Microsoft, EMU now collaborating on MIDI DLS-version 2 in MPEG4

17. SAOL n Downloadable BNF synthesis grammar n Header contains description of several synthesizers and effects processors control algorithms and routing instructions for audio flow of control n SAOL has 100 primitive processing instructions, signal generators and operators which fill wavetables with data.

18. SASL and MIDI n New format for describing control parameters - Basically a scheduler of audio events - Basically a scheduler of audio events - Designed to interface well with SAOL - Designed to interface well with SAOL - New Control Language Similar to MIDI - New Control Language Similar to MIDI n MIDI (Musical Instrument Digital Interface) –Simpler format for describing control –Included as alternate control method –Leverages existing authoring tools –Gives “backwards compatibility” to SA

19. DLS Level 2 n Aims at consistent synthetic audio playback across wide range of platforms n Defines a simple wavetable synthesizer n Bitstream includes sound samples n Score expressed in MIDI n Growing support from both software and hardware developers –DLS Part of DirectMusic in Microsoft’s DirectX 6.0

20. DLS-2 synthesizer model n Simple yet powerful structure much alike to many existing synthesizers in the market (eg in PC soundcards) –Uses loopable samples as sound sources (wavetable) –variable routing of control sources n 2 envelopes for amplitude control n 2 low frequency oscillators n 1-pole dynamic low-pass filter –Standardized response to MIDI controllers

21. Audio Bifs AudioSource Piano (SA) Finger snaps (Parametric) BIFS stuff Audio channels Bass (SA) AudioSource AudioMix AudioFX Synchronization with Visual! AudioFX AudioDelay AudioMix HRTF

22. Demo Audio BIFS

23. Conclusion n MPEG-4 Audio attempts to offer solutions to all spectra of sound. n Some of the tools are more stable, while others are still in Research and Development. n MPEG2-AAC is the best multi- channel lossy audio compression standard to date.

24. Acknowledgements I would like to thank the authors from the references for providing the material presented here today. I would like to thank the authors from the references for providing the material presented here today.

25. Definitions l T/F Time/Frequency (MDCT transform) l AAC Advanced Audio Coding l PARA Parametric l CELP Code Excited Linear Prediction l SA Structured Audio l PNS Perceptual Noise Substitution l HVXC Harmonic Vector eXcitation Coding l HILN Harmonic and Individual Line + Noise l SAOL Structured Audio Orchestra Language l SASL Structured Audio Score Language l MIDI Musical Instrument Digital Interface l TTS Text to Speech

26. More Definitions l CD Committee Draft l IS13818-7 Advanced Audio Coding l LC Low Complexity l BSAC Bit Sliced Arithmetic Coding l SSR Scalable Sample Rate l PNS Perceptual Noise Substitution l VBRVariable Bit Rate l TLSSTools for Large Step Scalability l SNHCSynthetic/Natural Hybrid Coding l DLSDownloadable Samples

27. Natural Audio Complexity

28. AAC Decoder Complexity Evaluation MPEG AAC DecoderComplexity MPEG AAC DecoderComplexity 2-channel Main Profile40% of 133 MHz Pentium 2-channel Low Complexity25% of 133 MHz Pentium 5-channel Main Profile90 sq. mm die, 0.5 micron CMOS 5-channel Low Complexity60 sq.mm die, 0.5 micron CMOS

29. AAC Test Results n Test at BBC and NHK according to ITU-R BS.1116 –triple-stimulus/hidden-reference/double-blind –ITU-R 5-point impairment scale –95% Confidence Intervals n MPEG AAC provides “indistinguishable” quality at 320 kb/s per five channels n MPEG AAC at 320 kb/s outperforms MPEG BC Layer II at 640 kb/s per five channels n Recent Stereo Tests at NHK Showed MPEG AAC provides “indistinguishable” quality at 128 kb/s per two channels

30. References n M. Bosi, E. Schrierer, B. Edler, Peter G. Schreiner MPEG-4 Seminar, Fribourg, Switzerland 1997 n S. Quackenbush, “Coding of Natural Audio in MPEG-4”, Proc IEEE ICASSP, Seattle, 1998 n B. Grill, B. Edler, I. Kaneko, Y. Lee, M. Nishiguichi, E. Scheirer, and M. Väänänen (Eds). ISO 14496-4(MPEG-4 Audio) Committee Draft. MPEG document N1903 n E. Schrier, “The MPEG-4 Structured Audio Standard”, Proc IEEE ICASSP, Seattle, 1998 Juergen Herre, “Updated Description for Perceptual Noise Substitution Tool”, MPEG Document M2692 n n E. Scheirer, R. Väänänen, J. Huopaniemi, “AudioBIFS: The MPEG-4 Standard for Effects Processing”, AES, SF, 1998 n n Overview: http://www.cselt.it/mpeg/standards/mpeg-4/mpeg-4.htm