The ISO/MPEG standardization process Requirements Call for proposals Evaluation Core experiments Draft specification National bodies agree

Some major MPEG-4 requirements Initial goal – utmost compression efficiency In 1994 – refocussing on ____________-based functionalities – _______-based representation of audio-visual data – ________-side composition of AV objects – Allow user ___________ with AV objects – Allow downloading and streaming – Error __________ bitstream syntax – Provide ___________ coding – …

The structure of the MPEG-4 Standard (1/4) Part 1 : Systems (IS :2001 – w3850) Part 2 : Visual(IS :1999 – w3056) Part 3 : Audio(IS :2001 – w3798) Part 4 : Conformance(IS :2001 – w4024) Part 5 : Reference Software(IS :2001 – w4025) Part 6 : Delivery Multimedia Integration Framework (DMIF) (IS :2000– w3713) Part 7 : Optimized Visual Reference Software (PDTR – w4057) Part 8 : Carriage of MPEG-4 contents over IP networks (CD – w4081)

The structure of the MPEG-4 Standard (2/4) Part 2 : Visual – Still texture coding – Arbitrary _________ video coding – Synthetic coding tools (Meshes, Face & Body) – Fine Grain Scalability – … Part 3 : Audio – General audio coding (transform coder) – Speech coding (CELP, Parametric) – Text-to-speech interface – _________ audio coding – … Part 4 : Conformance – Conformance can only be tested for a _________ & __________ combination Per Profile & Level Test bit streams Additional test information

The structure of the MPEG-4 Standard (3/4) Part 5 : Reference Software – Contains all _____________ elements of Part 1 – 3 (decoders) – “Informative” part contains encoders & tools Part 6 : DMIF - ________ Multimedia Integration Framework – Mainly the abstraction of the Delivery (Transport) Layer – The MPEG-4 world ends (begins :-) at the ____(DMIF Application Interface) – Actual delivery adaptations mostly _________ MPEG (-4) (MPEG-4 over IP; MPEG-4 over MPEG-2) Part 7 : Optimized Visual Reference Software – Non-standard tools to document state of the art Part 8 : Carriage of MPEG-4 contents over IP networks – Umbrella for specifications done by _____ (e.g. RTP payloads, SDP extensions)

The structure of the MPEG-4 Standard (4/4) Part 1 : Systems Audiovisual Scene Coded Representation Natural and Synthetic Audio Information Coded Representation Management & Synchronization of Audiovisual Information Natural and Synthetic Visual Information Coded Representation

An example of MPEG4 scene

Object-based compression and delivery Comp- ress Comp- ress Comp- ress Comp- ress D E L I V E R Y I N T E R F A C E (DAI) Comp- ress Scene Des. decomp- ress decomp- ress decomp- ress decomp- ress D E L I V E R Y I N T E R F A C E (DAI) decomp- ress COMPOSITORCOMPOSITOR

Linking elementary streams to the scene

Linking streams into the scene ObjectdescriptorID

Linking streams into the scene ObjectdescriptorID

Linking streams into the scene ObjectdescriptorID ES_Descriptor { ES_ID_ } ES_Descriptor { ES_ID_ } ObjectDescriptor { OD_ID_1 List of { Elementary..... ObjectDescriptor { OD_ID_2 List of { Elementary Stream Descriptors }

Linking streams into the scene ObjectdescriptorID ES_Descriptor { ES_ID_ } ES_Descriptor { ES_ID_ } ObjectDescriptor { OD_ID_1 List of { Elementary Stream Descriptors } ObjectDescriptor { OD_ID_2 List of { Elementary Stream ………. ES_Descriptor { ES_ID_ }

Linking streams into the scene ObjectdescriptorID ObjectDescriptor { OD_ID_1 List of { elementary stream descriptor } ObjectDescriptor { OD_ID_2 List of { elementary stream descriptor } URLs in ESD point to remote stream All description local ES_Descriptor { URL } ES_Descriptor { URL }

Linking streams into the scene An object descriptor contains ES descriptors pointing to: – _________ coded content streams – Alternate _____ content streams – Object content __________ – ____ information ES descriptors have subdescriptors for: – ________ configuration – _______ layer configuration – ____________ information – Future / private extensions Allows the terminal to select suitable streams “stream type” “stream header” Configures the flexible SL syntax Mainly for heteroge- neous networks

Describing scalable content

Describing alternate content versions

Decoder configuration info in older standards Cfg = configuration information (“stream headers”)

Decoder configuration information in MPEG-4

The Initial Object Descriptor Derived from the generic object descriptor – Contains additional elements to signal ________ and ______ Profile and level indications are the default way of content selection – The terminal reads the P&L indications and knows whether it has capability to process the presentation Profiles are signaled in multiple separate dimensions – ______________ – Graphics – Object descriptors – Audio – Visual The “first” object descriptor for an MPEG-4 presentation is always an initial object descriptor

Transport of object descriptors Object descriptors are encapsulated in OD _____________s – ObjectDescriptorUpdate / ObjectDescriptorRemove – ES_DescriptorUpdate / ES_DescriptorRemove OD commands are conveyed in their own object descriptor _________ in a synchronized manner with time stamps – Objects / streams may be announced during a presentation There may be __________ OD & scene description streams – A partitioning of a large scene becomes possible Name scopes for identifiers (OD_ID, ES_ID) are defined – Resource management for sub scenes can be distributed A propose “resource management” – If the location of streams is changed, only the ___s need modification. Not the scene description

Initial OD pointing to scene and OD stream

Initial OD pointing to a scalable scene

Ancilliary streams IPMP streams – Information for Intellectual Property Management and Protection – Structured in (time stamped) messages. – Content is defined by ____________ IPMP systems – Complemented by IPMP ___________s OCI (Object Content Information) streams – Meta data for an object. – Structured descriptors conveyed in (time stamped) _________s – Content author, date, keywords, description, language,... – Some OCI descriptors may be directly in __s or ___s ES_Descriptors pointing to such streams may be attached to any _________ descriptor – This scopes the _____ or ___ stream. – An IPMP stream attached to the object descriptor stream is valid for ___ streams

Adding an OCI stream to an Audio stream

Adding OCI descriptors to Audio streams

Linking streams to the scene - including “upstreams” ObjectdescriptorID ObjectDescriptor { OD_ID_ ES downstream ES upstream }