Multimedia Communication Systems Techniques, Standards, and Networks Chapter 6 Multimedia Communication Across Networks

Audio/Video Nguyen Slides Nguyen Slides Nguyen Slides Nguyen Slides Digitized versions are bursty Digitized versions are bursty Can be statistically multiplexed Can be statistically multiplexed Can be packetized Can be packetized Continuously connected circuits waste resources Continuously connected circuits waste resources

Packet Voice Mostly silence Mostly silence Only active 35% - 45% of the time Only active 35% - 45% of the time Delay Delay Acceptable range 100 – 600 mSec Acceptable range 100 – 600 mSec Packet size Packet size 200 – 700 speech bits 200 – 700 speech bits Packet headers 4 – 8 bytes Packet headers 4 – 8 bytes Speech content Speech content 10 – 50 mSec 10 – 50 mSec

Packet Voice Additional information Additional information Possibly required Possibly required Time stamps Time stamps Sequence number Sequence number Flow control Flow control Not required Not required Error detection Error detection Acknowledgement Acknowledgement Retransmission Retransmission

Timing Reconstruction NTI – Null Timing Information NTI – Null Timing Information No time stamps No time stamps An arbitrary fixed transmission delay is assumed An arbitrary fixed transmission delay is assumed Essentially a fixed buffer Essentially a fixed buffer Processing delay Processing delay All subsequent packets are assumed to arrive after a shorter networking processing delay All subsequent packets are assumed to arrive after a shorter networking processing delay CTI – Complete Timing Information CTI – Complete Timing Information Each packet delay is measured Each packet delay is measured Can be relative or global timing Can be relative or global timing Suitable for highly variable delay networks Suitable for highly variable delay networks

Packet Video ATM ATM Circuit switched (fixed delay, jitter) Circuit switched (fixed delay, jitter) Small cells (48 octet payload and 5 octet header) Small cells (48 octet payload and 5 octet header) IP IP Packet switched (Route is not predetermined) Packet switched (Route is not predetermined) Variable length packets (Max 65,535 octets) Variable length packets (Max 65,535 octets) IPv4 No QoS IPv4 No QoS Variable queuing delays, out of order sequencing Variable queuing delays, out of order sequencing IPv6 QoS IPv6 QoS 25 bit flow identifier 25 bit flow identifier

Video Source Highly variable bit rate & delay Highly variable bit rate & delay Compression algorithm Compression algorithm Scene complexity Scene complexity Network traffic Network traffic Packet switched networks Packet switched networks Difficult to create statistical models Difficult to create statistical models Hard to negotiate QoS at setup Hard to negotiate QoS at setup UPC & NPC (usage/network parameter control) UPC & NPC (usage/network parameter control) UPC & NPC UPC & NPC

Policing Mechanisms Rate Control – the amount of information is regulated Rate Control – the amount of information is regulated Rate Shaping – controls when the information is sent Rate Shaping – controls when the information is sent Leaky Bucket Leaky Bucket Leaky Bucket Leaky Bucket Multiresolution encoding (layering) Multiresolution encoding (layering) Coarse resolution – high priority stream Coarse resolution – high priority stream Fine detail – low priority stream (prone to discard) Fine detail – low priority stream (prone to discard)

Simulcast Layers Multiple independent layers Multiple independent layers Different bit rates Different bit rates

Dependent Layering Layers can be dropped to support a constant bit rate Layers can be dropped to support a constant bit rate congestion control, etc. congestion control, etc.

Transmission Errors FEC – allows the receiver to correct errors FEC – allows the receiver to correct errors FEC RVLC – reversible variable length coding RVLC – reversible variable length coding RVLC Inserts resynchronization markers Inserts resynchronization markers Error resilient entropy coding Error resilient entropy coding Error resilient entropy coding Error resilient entropy coding Rearranges variable length blocks into fixed length slots Rearranges variable length blocks into fixed length slots Attaches a prefix code Attaches a prefix code Temporal/spatial errors Temporal/spatial errors Periodically inert I pictures Periodically inert I pictures Segment the data domain Segment the data domain i.e. even/odd frames i.e. even/odd frames

Error Concealment Replace a damaged macroblock with the previous correct one Replace a damaged macroblock with the previous correct one Interpolate the block from surrounding pixels Interpolate the block from surrounding pixels POCS Projection onto convex sets POCS Projection onto convex sets POCS An iterative technique, not suitable for real-time An iterative technique, not suitable for real-time

Rate Control How to cope with bandwidth fluctuations How to cope with bandwidth fluctuations Traffic shaping Traffic shaping Traffic shaping Traffic shaping Leaky bucket Leaky bucket Leaky bucket Leaky bucket Token bucket Token bucket Token bucket Token bucket SRC – scaleable rate control SRC – scaleable rate control SRC The amount of compression is adjusted The amount of compression is adjusted Complex math Complex math Complex math Complex math

Internet Video Transport Download mode Download mode The entire clip is downloaded before viewing The entire clip is downloaded before viewing Streaming mode Streaming mode The video clip may be viewed while transmission is in progress The video clip may be viewed while transmission is in progress

Video Streaming Architecture Video compression Video compression Application layer QoS Application layer QoS Adapts the video bit rate to the network Adapts the video bit rate to the network Continuous media distribution services Continuous media distribution services An extranet or internet An extranet or internet Streaming servers Streaming servers Retrieves the streaming media from storage Retrieves the streaming media from storage Media synchronization Media synchronization Audio, video, data etc. Audio, video, data etc. Media protocols Media protocols

Video Streaming Architecture Streaming media systems

Video Compression Scaleable Scaleable Deals gracefully with bandwidth fluctuations Deals gracefully with bandwidth fluctuations A compromise between efficiency, flexibility, & complexity A compromise between efficiency, flexibility, & complexity Compresses data into multiple substreams Compresses data into multiple substreams Each DCT coefficient bit can be assigned to a different stream (i.e. MSB = coarse) Each DCT coefficient bit can be assigned to a different stream (i.e. MSB = coarse) FGS PFGS FGS PFGS FGSPFGS FGSPFGS Non Scalable Non Scalable

Streaming Constraints Streaming Constraints Requires bounded end-to-end delay Requires bounded end-to-end delay Rate control is needed to avoid congestion Rate control is needed to avoid congestion Receiver buffering is required Receiver buffering is required Must deal with packet loss Must deal with packet loss Simple decoding is needed for low power receivers Simple decoding is needed for low power receivers PDAs, cell phones, etc. PDAs, cell phones, etc. PDAs

Application Layer QoS Avoid Congestion (Rate Control) Avoid Congestion (Rate Control) Source based Source based The rate is determined by a Probe or Model The rate is determined by a Probe or Model Receiver based Receiver based The receiver adds/drops channels The receiver adds/drops channels Hybrid Hybrid Both the source & receiver adjust the bit rate Both the source & receiver adjust the bit rate Maximize video quality Maximize video quality

Continuous Media Services Use network filters Use network filters Frame dropping for congestion control Frame dropping for congestion control Content replication Content replication Caching (local copies) Caching (local copies) Mirroring (duplicated servers) Mirroring (duplicated servers)

Streaming Servers Streaming Servers Consists of Consists of Communicator Communicator Applications and transport layer protocols Applications and transport layer protocols Operating system Operating system Storage system Storage system Support VCR-like controls Support VCR-like controls Flash Media Server Flash Media Server Flash Media Server Flash Media Server

Media Synchronization Three levels Three levels Intrastream Intrastream Maintains continuity of a logical stream Maintains continuity of a logical stream i.e. Voice or video i.e. Voice or video Interstream Interstream Maintains timing between streams Maintains timing between streams i.e. Voice and video i.e. Voice and video Interobject Interobject Layers within streams Layers within streams

Streaming Protocols Streaming Protocols Network Network IP IP IP Transport Transport UDP, TCP, RTP, RTCP UDP, TCP, RTP, RTCP UDPTCPRTPRTCP UDPTCPRTPRTCP Session Session RTSP, SIP RTSP, SIP RTSPSIP RTSPSIP

Streaming over ATM Internet video streaming Internet video streaming Internet video streaming Internet video streaming