MPEG-4 Applications Interactive TV (broadcast) Streaming media on the web (internet) Multimedia titles (CD-ROM) Network games Mobile multimedia (picture phone)

Technical Capabilities Define media objects –Audio and visual –Natural and synthetic Define composition architecture –Scene graph extended/borrowed from VRML System level –Streams: Multiplexing, synchronization, buffering –Programmability? Simple interaction

Structured Video Natural representation: Exposure sheets –Layers and elements, alpha compositing –Chroma keying and blue-screen mattes Compression –Standard video compression Trouble with motion and object boundaries Trouble with graphics: text, graphics –Object coding/Computer graphics models Compact format: e.g. cels Resolution independent But beware! Be clever!

Video Stream VS > VO > VOL > GOV > VOP Video object –Shape + Motion + Texture Sprites –Texture + Projective transformation

Shape Coding Bounding box + shape-adaptive macroblocks Alpha / Shape –Binary Transparent or opaque Context-based arithmetic encoder Motion predicted –Grey (coded as textures) Texture –Shape-based DCT –Padding algorithm

Scalable Video Layered video –Spatial scalability: base + enhancement –Temporal scalability: IPI + B = IBPBI Scalable content –Quality improves with decoder technology –Inherent resolution independence (text,midi)

2D Meshes Type –Uniform rectangular –Triangular (Delauney) Geometry –Intra: Predict position compression –Inter: Compressed motion vector Texture –Piecewise affine texture map –Wavelet compression ala mipmaps

Face Facial Definition Parameter or user supplied model/texture Facial Animation Parameter plus Amplification and Filters

References