Multimedia Communication Systems Techniques, Standards, and Networks Chapter 5 Multimedia Communication Standards

Standards A compromise between the theoretically possible & technologically feasible A compromise between the theoretically possible & technologically feasible Require a balanced cost/performance ratio Require a balanced cost/performance ratio Many companies have different approaches to standardization Many companies have different approaches to standardization Provide interoperability between vendors Provide interoperability between vendors MPEG MPEG MPEG Homepage MPEG Homepage MPEG Homepage MPEG Homepage MPEG Industry Forum MPEG Industry Forum MPEG Industry Forum MPEG Industry Forum

Coder Example Coding is standardized Coding is standardized Encourages widespread use of the system Encourages widespread use of the system Decoding is not standardized Decoding is not standardized Encourages innovation Encourages innovation

MPEG A working group of ISO/IEC Established in Meetings per year Large group ~350 experts ~200 companies ~20 counties

MPEG Standards MPEG-1 Video CD and MP3 MPEG-1 MPEG-2 DTV set top boxes and DVD MPEG-2 MPEG-4 multimedia for the web MPEG-4 MPEG-7 description and search of audio & visual content MPEG-7 MPEG-21 the Multimedia Framework. MPEG-21

MPEG-1 Part 1 Part 1 Combining data streams Combining data streams Part 2 Part 2 Video coding & compression Video coding & compression Part 3 Part 3 Audio coding & compression Audio coding & compression Part 4 Part 4 Testing/verification Testing/verification Part 5 Part 5 Report on software implementation Report on software implementation

MPEG I Picture I (intra or independent) picture I (intra or independent) picture Coded without reference to any other picture Coded without reference to any other picture Creates random access points Creates random access points Least amount of compression Least amount of compression

MPEG P Picture P (predicted) picture P (predicted) picture Based on a previous I picture Based on a previous I picture Increased amount of compression Increased amount of compression

MPEG B Picture B (bi-directional or bi-predictive) picture B (bi-directional or bi-predictive) picture Generally require 2 or more pictures to be decoded before they can be reconstructed Generally require 2 or more pictures to be decoded before they can be reconstructed Highest amount of compression Highest amount of compression

Picture Sequence Streaming and display order are different Streaming and display order are different

MPEG Data Hierarchy - Blocks A digital image is reduced to a series of 8x8 pixel blocks A digital image is reduced to a series of 8x8 pixel blocks

MPEG Data Hierarchy – Macro Blocks Blocks are combined into macro blocks Blocks are combined into macro blocks Each macro block contains: Each macro block contains: 4 Luminance blocks 4 Luminance blocks 2 Chromance blocks 2 Chromance blocks

Motion Compensation Motion Compensation Coding between two pictures Coding between two pictures Used in B & P pictures Used in B & P pictures Work at the macro block level Work at the macro block level Compressed file contents: Compressed file contents: Motion vector – the spatial difference between the reference macro block and the coded macro block Motion vector – the spatial difference between the reference macro block and the coded macro block Error terms –the content difference between the two macro blocks Error terms –the content difference between the two macro blocks

Intra-Picture Coding Coding within a picture Coding within a picture Consists of three stages: Consists of three stages: DCT – Discrete Cosine Transform DCT – Discrete Cosine TransformDiscrete Cosine TransformDiscrete Cosine Transform Transforms the spatial domain to the frequency domain Transforms the spatial domain to the frequency domain Quantization Quantization Quantization Reduces the frequency coefficients to a limited set of values Reduces the frequency coefficients to a limited set of values Run-length coding Run-length coding Consists of several stages Consists of several stages Organizes the data into run-amplitude pairs Organizes the data into run-amplitude pairs

Run Length Coding Run Length Coding The DCT arranges the coefficients The DCT arranges the coefficients Low to high frequency (left to right and top to bottom) Low to high frequency (left to right and top to bottom) The coefficients are gathered in a zigzag pattern The coefficients are gathered in a zigzag pattern Entropy coded using VLC tables Entropy coded using VLC tables Entropy coded Entropy coded

Video Coding References References MPEG Overview MPEG Overview MPEG Overview MPEG Overview MPEG-1 Video Coder MPEG-1 Video Coder MPEG-1 Video Coder MPEG-1 Video Coder Beginners Guide to MPEG-2 Beginners Guide to MPEG-2 Beginners Guide to MPEG-2 Beginners Guide to MPEG-2 MPEG-2 Tutorial MPEG-2 Tutorial MPEG-2 Tutorial MPEG-2 Tutorial MPEG Tutorials MPEG Tutorials MPEG Tutorials MPEG Tutorials

MPEG-2 Video MPEG-2 Video Backward compatible with MPEG-1 video Backward compatible with MPEG-1 video Defines Profiles Defines Profiles Increases flexibility/applications Increases flexibility/applications Broadcast satellite, cable TV, interactive TV Broadcast satellite, cable TV, interactive TV Adapts to bandwidth/network considerations Adapts to bandwidth/network considerations

GOPs I, P, & B pictures are reordered for transmission I, P, & B pictures are reordered for transmission P pictures are 30-50% the size of I pictures P pictures are 30-50% the size of I pictures B pictures are 50% the size of P pictures B pictures are 50% the size of P pictures GOPs are time stamped to support editing GOPs are time stamped to support editing

GOP Time Stamps Reference time stamps Reference time stamps PES (packetized elementary stream) syntax PES (packetized elementary stream) syntax PES Program syntax Program syntax Transport syntax Transport syntax Decoding time stamp Decoding time stamp Synchronizes audio and video decoding Synchronizes audio and video decoding SMPTE time codes SMPTE time codes SMPTE time codes SMPTE time codes

New Image Components Slices Slices Slices A group of macroblocks in an image A group of macroblocks in an imagemacroblocks Supports random access within an image Supports random access within an image The quantization is controlled by a scale factor The quantization is controlled by a scale factorquantization Makes it possible to support constant bit rate video streams Makes it possible to support constant bit rate video streams Frame & field pictures Frame & field pictures Frame & field pictures Frame & field pictures Field pictures are used in interlaced scanning Field pictures are used in interlaced scanning Fields are called top/bottom instead of even/odd Fields are called top/bottom instead of even/odd Are motion compensated (estimated) Are motion compensated (estimated)

Motion Estimation (Prediction) Prediction Occurs at the macroblock level Occurs at the macroblock level If 2 identical (or very similar) macroblocks occur in 2 pictures: If 2 identical (or very similar) macroblocks occur in 2 pictures: The difference between the macroblocks is calculated (content & location) The difference between the macroblocks is calculated (content & location) If the content and location are the same, the coding is skipped If the content and location are the same, the coding is skipped A DCT is performed on the content difference A DCT is performed on the content differenceDCT The result is variable length coded The result is variable length coded The motion (location) vector is added The motion (location) vector is added

Scalability Provides interoperability between different services & receivers Provides interoperability between different services & receivers 5 profiles are defined 5 profiles are defined 5 profiles are defined 5 profiles are defined Spatial scalability Spatial scalability Different display resolutions Different display resolutions Interworking between HDTV and standard definition TV (SDTV) Interworking between HDTV and standard definition TV (SDTV) Temporal scalability Temporal scalability Different picture rates in the same video stream Different picture rates in the same video stream

Video Layering One video layer contains the base or reference information One video layer contains the base or reference information Other layers include more details Other layers include more details Applications Applications Stereoscopic video (CRC website) Stereoscopic video (CRC website) Stereoscopic videoCRC website Stereoscopic videoCRC website Left & right pictures (requires shutter glasses) Left & right pictures (requires shutter glasses) SNR scalability SNR scalability The picture quality adjusts as a function of background noise The picture quality adjusts as a function of background noise Provides error concealment Provides error concealment Very similar to data partitioning Very similar to data partitioning

Chroma Chroma sub-sampling Chroma sub-sampling Chroma sub-sampling Chroma sub-sampling

MPEG-2 Audio Backward compatible with MPEG-1 audio Backward compatible with MPEG-1 audio Backward compatible Backward compatible MPEG-1 is somewhat forward compatible with MPEG-2 MPEG-1 is somewhat forward compatible with MPEG-2forward compatibleforward compatible Employs 3 compression layers Employs 3 compression layers

Audio Layers Each layer increases the number of encoded subbands Each layer increases the number of encoded subbands encoded subbands encoded subbands Other published bit rates Other published bit rates Other published bit rates Other published bit rates Layer Compression Ratio ~ Bit Rate (Kbps) ~ Processing Delay (mSec) 14: : :164150

Audio Subbands Layer 1 Layer 1 32 subbands 32 subbands 12 frequency domain samples 12 frequency domain samples Layer 2 Layer 2 32 subbands 32 subbands 36 frequency domain samples 36 frequency domain samples Layer 3 Layer or 576 subbands 192 or 576 subbands

PsychoacousticPsychoacoustic Model Psychoacoustic The SMR (signal to mask ratio) is calculated for all subbands The SMR (signal to mask ratio) is calculated for all subbands Sounds which cannot be perceived are eliminated Sounds which cannot be perceived are eliminated

Multichannel Variable sampling rates (bandwidth) Variable sampling rates (bandwidth) 16, 22.05, 24 KHz (7.5, 10.3, KHz) 16, 22.05, 24 KHz (7.5, 10.3, KHz) Applications Applications Commentaries/multilingual (7 per program) Commentaries/multilingual (7 per program) Surround sound Surround sound Surround sound Surround sound MPEG-2 ACC now MPEG-4 part 3 MPEG-2 ACC now MPEG-4 part 3MPEG-4 part 3MPEG-4 part 3

MPEG-2 Single Transport MPEG-2 packets are 188 octets MPEG-2 packets are 188 octets Fits into 4 ATM cells Fits into 4 ATM cells Audio & video packets are combined Audio & video packets are combined

MPEG-2 Multiple Transport Requires additional signaling tables Requires additional signaling tables Slide show by Karir Slide show by Karir Slide show by Karir Slide show by Karir DSM CC DSM CC DSM CC DSM CC

MPEG-4 References References MPEG-4 by Apple MPEG-4 by Apple MPEG-4 by Apple MPEG-4 by Apple MPEG Industry Forum MPEG Industry Forum MPEG Industry Forum MPEG Industry Forum Streamingmedia.com Streamingmedia.com Streamingmedia.com Version 1 Overview, audio version 2 Version 1 Overview, audio version 2 Version 1 Overviewaudio version 2 Version 1 Overviewaudio version 2

MPEG-4 Applications Digital TV Digital TV Satellite HDTV Satellite HDTVHDTV Multimedia streaming Multimedia streaming DSL video services DSL video services Animation Animation Animation 3G wireless videotelephony 3G wireless videotelephony 3G wireless 3G wireless Optical Disks Optical Disks HD-DVD (abandoned) HD-DVD (abandoned) HD-DVD BluRay BluRay BluRay

MPEG-4 Objects Objects Objects A multimedia scene component A multimedia scene component Have spatial & temporal attributes Have spatial & temporal attributes Scene descriptors (BIFS) Scene descriptors (BIFS)BIFS Uses MPEG-J (a Java variant) Uses MPEG-J (a Java variant) Composition occurs after decoding rather than before encoding Composition occurs after decoding rather than before encoding Allows user interaction Allows user interaction Change color of a car, broadcast logo, etc. Change color of a car, broadcast logo, etc. Tag player in a sporting event Tag player in a sporting event Personalize video programming Personalize video programming

Media Objects A Guide to MPEG-4

MPEG Profiles Groups of tools Groups of tools Application dependent Application dependent

MPEG-4 Attributes Interoperability Interoperability Not platform specific Not platform specific Transport Independent Transport Independent Service provider chooses the transport Service provider chooses the transport Rich Media Compression Rich Media Compression Low & mid bit-rate compression and transmission of rich media Low & mid bit-rate compression and transmission of rich media Interactivity Interactivity Authors & viewers can choose how to interact with a stream Authors & viewers can choose how to interact with a stream Scalability Scalability Decoding & content resolution adapts to network and receiver Decoding & content resolution adapts to network and receiver Profiles Profiles A variety of environments are provided to support different applications A variety of environments are provided to support different applications

MPEG-4 Devices

Video Bit Rate

MPEG-7 A standard description of multimedia content A standard description of multimedia content Used for: Used for: Indexing, cataloging, searching, selecting Indexing, cataloging, searching, selecting ‘Smart reasoning’ about content ‘Smart reasoning’ about content Examples Examples Music & video downloads Music & video downloads Video editing Video editing Personal video recorders Personal video recorders

MPEG-21 The ‘big picture’ The ‘big picture’ Managing content & access Managing content & access How different elements build delivery infrastructure How different elements build delivery infrastructure Digital rights management Digital rights management Digital rights management Digital rights management

Other Standards Bodies ITU ITU ITU A UN organization A UN organization Developed the H (and other) series standards Developed the H (and other) series standards ISOC comprised of: ISOC comprised of: ISOC IETF IETF IETF Develops Internet standards Develops Internet standards IAB IAB IAB Architecture and administration Architecture and administration Others Others Others

ITU Video H.261, H.262, H.263, H.264 H.261, H.262, H.263, H.264 H.261H.262H.263H.264 H.261H.262H.263H.264 Also called Px64 Also called Px64 The first practical digital video coding standard The first practical digital video coding standard Used in many subsequent standards Used in many subsequent standards Overview Overview Overview

ITU Control H.221, H.223, H.225 H.221, H.223, H.225 H.221H.223H.225 H.221H.223H.225 Multimedia synchronization & multiplexing standards Multimedia synchronization & multiplexing standards H.245 H.245 H.245 Common control protocol Common control protocol Terminal to terminal signaling Terminal to terminal signaling

ITU PSTN Videoconferencing H.320 H.320 H.320 The first videoconferencing standard The first videoconferencing standard Uses H.261 compression Uses H.261 compression ISDN 64 Kbps – 2 Mbps ISDN 64 Kbps – 2 Mbps Most widely used circuit switched technique Most widely used circuit switched technique Cost effective for business Cost effective for business

ITU LAN Video H.323 H.323 H.323 Video telephony on nonguaranteed QoS LANs Video telephony on nonguaranteed QoS LANs Ethernet, FDDI, & Token ring Ethernet, FDDI, & Token ring Significant delay Significant delay Not optimized for real-time Not optimized for real-time

The Internet References References The Living Internet The Living Internet The Living Internet The Living Internet Ipv6 Portal Ipv6 Portal Ipv6 Portal Ipv6 Portal IPv6 Insights IPv6 Insights IPv6 Insights IPv6 Insights IPv4 vs. IPv6 IPv4 vs. IPv6 IPv4IPv6 IPv4IPv6 IPv6 completed in 1997 (deployed in 2004) IPv6 completed in 1997 (deployed in 2004) Deployed by Korea & Japan, the west lags behind Deployed by Korea & Japan, the west lags behind Deployed by Korea & Japan, the west lags behind Deployed by Korea & Japan, the west lags behind

IPv6 Advantages Adds QoS Adds QoS Supports video streaming and VoIP Supports video streaming and VoIP Addresses latency, jitter, and bandwidth Addresses latency, jitter, and bandwidth Supports multicast & unicast Supports multicast & unicast Increases Security Increases Security Provides authentication Provides authentication Facilitates e-commerce Facilitates e-commerce Reduces spam & viruses Reduces spam & viruses