1. The ISO/MPEG standardization process Requirements Call for proposals Evaluation Core experiments Draft specification National bodies agree

2. Some major MPEG-4 requirements Initial goal – utmost compression efficiency In 1994 – refocussing on ____________-based functionalities – _______-based representation of audio-visual data – ________-side composition of AV objects – Allow user ___________ with AV objects – Allow downloading and streaming – Error __________ bitstream syntax – Provide ___________ coding – …

3. The structure of the MPEG-4 Standard (1/4) Part 1 : Systems (IS 14496-1:2001 – w3850) Part 2 : Visual(IS 14496-2:1999 – w3056) Part 3 : Audio(IS 14496-3:2001 – w3798) Part 4 : Conformance(IS 14496-4:2001 – w4024) Part 5 : Reference Software(IS 14496-5:2001 – w4025) Part 6 : Delivery Multimedia Integration Framework (DMIF) (IS 14496-6:2000– w3713) Part 7 : Optimized Visual Reference Software (PDTR 14496-7– w4057) Part 8 : Carriage of MPEG-4 contents over IP networks (CD 14496-8 – w4081)

4. The structure of the MPEG-4 Standard (2/4) Part 2 : Visual – Still texture coding – Arbitrary _________ video coding – Synthetic coding tools (Meshes, Face & Body) – Fine Grain Scalability – … Part 3 : Audio – General audio coding (transform coder) – Speech coding (CELP, Parametric) – Text-to-speech interface – _________ audio coding – … Part 4 : Conformance – Conformance can only be tested for a _________ & __________ combination Per Profile & Level Test bit streams Additional test information

5. The structure of the MPEG-4 Standard (3/4) Part 5 : Reference Software – Contains all _____________ elements of Part 1 – 3 (decoders) – “Informative” part contains encoders & tools Part 6 : DMIF - ________ Multimedia Integration Framework – Mainly the abstraction of the Delivery (Transport) Layer – The MPEG-4 world ends (begins :-) at the ____(DMIF Application Interface) – Actual delivery adaptations mostly _________ MPEG (-4) (MPEG-4 over IP; MPEG-4 over MPEG-2) Part 7 : Optimized Visual Reference Software – Non-standard tools to document state of the art Part 8 : Carriage of MPEG-4 contents over IP networks – Umbrella for specifications done by _____ (e.g. RTP payloads, SDP extensions)

6. The structure of the MPEG-4 Standard (4/4) Part 1 : Systems Audiovisual Scene Coded Representation 14496-1 Natural and Synthetic Audio Information Coded Representation 14496-2 Management & Synchronization of Audiovisual Information 14496-1 Natural and Synthetic Visual Information Coded Representation 14496-2

7. An example of MPEG4 scene

8. Object-based compression and delivery Comp- ress Comp- ress Comp- ress Comp- ress D E L I V E R Y I N T E R F A C E (DAI) Comp- ress Scene Des. decomp- ress decomp- ress decomp- ress decomp- ress D E L I V E R Y I N T E R F A C E (DAI) decomp- ress COMPOSITORCOMPOSITOR

9. Linking elementary streams to the scene

10. Linking streams into the scene ObjectdescriptorID

11. Linking streams into the scene ObjectdescriptorID

12. Linking streams into the scene ObjectdescriptorID ES_Descriptor { ES_ID_4....... } ES_Descriptor { ES_ID_5....... } ObjectDescriptor { OD_ID_1 List of { Elementary..... ObjectDescriptor { OD_ID_2 List of { Elementary Stream Descriptors }

13. Linking streams into the scene ObjectdescriptorID ES_Descriptor { ES_ID_1....... } ES_Descriptor { ES_ID_2....... } ObjectDescriptor { OD_ID_1 List of { Elementary Stream Descriptors } ObjectDescriptor { OD_ID_2 List of { Elementary Stream ………. ES_Descriptor { ES_ID_3....... }

14. Linking streams into the scene ObjectdescriptorID ObjectDescriptor { OD_ID_1 List of { elementary stream descriptor } ObjectDescriptor { OD_ID_2 List of { elementary stream descriptor } URLs in ESD point to remote stream All description local ES_Descriptor { URL....... } ES_Descriptor { URL....... }

15. Linking streams into the scene An object descriptor contains ES descriptors pointing to: – _________ coded content streams – Alternate _____ content streams – Object content __________ – ____ information ES descriptors have subdescriptors for: – ________ configuration – _______ layer configuration – ____________ information – Future / private extensions Allows the terminal to select suitable streams “stream type” “stream header” Configures the flexible SL syntax Mainly for heteroge- neous networks

16. Describing scalable content

17. Describing alternate content versions

18. Decoder configuration info in older standards Cfg = configuration information (“stream headers”)

19. Decoder configuration information in MPEG-4

20. The Initial Object Descriptor Derived from the generic object descriptor – Contains additional elements to signal ________ and ______ Profile and level indications are the default way of content selection – The terminal reads the P&L indications and knows whether it has capability to process the presentation Profiles are signaled in multiple separate dimensions – ______________ – Graphics – Object descriptors – Audio – Visual The “first” object descriptor for an MPEG-4 presentation is always an initial object descriptor

21. Transport of object descriptors Object descriptors are encapsulated in OD _____________s – ObjectDescriptorUpdate / ObjectDescriptorRemove – ES_DescriptorUpdate / ES_DescriptorRemove OD commands are conveyed in their own object descriptor _________ in a synchronized manner with time stamps – Objects / streams may be announced during a presentation There may be __________ OD & scene description streams – A partitioning of a large scene becomes possible Name scopes for identifiers (OD_ID, ES_ID) are defined – Resource management for sub scenes can be distributed A propose “resource management” – If the location of streams is changed, only the ___s need modification. Not the scene description

22. Initial OD pointing to scene and OD stream

23. Initial OD pointing to a scalable scene

24. Ancilliary streams IPMP streams – Information for Intellectual Property Management and Protection – Structured in (time stamped) messages. – Content is defined by ____________ IPMP systems – Complemented by IPMP ___________s OCI (Object Content Information) streams – Meta data for an object. – Structured descriptors conveyed in (time stamped) _________s – Content author, date, keywords, description, language,... – Some OCI descriptors may be directly in __s or ___s ES_Descriptors pointing to such streams may be attached to any _________ descriptor – This scopes the _____ or ___ stream. – An IPMP stream attached to the object descriptor stream is valid for ___ streams

25. Adding an OCI stream to an Audio stream

26. Adding OCI descriptors to Audio streams

27. Linking streams to the scene - including “upstreams” ObjectdescriptorID ObjectDescriptor { OD_ID_2....... ES downstream ES upstream }