Download presentation
Presentation is loading. Please wait.
Published byDiana Hall Modified over 9 years ago
1
Multimedia Networking Ashwin Bharambe Carnegie Mellon University 15-441 Networking, Spring 2003 Tuesday, April 22, 2003
2
Ashwin BharambeCarnegie Mellon University What media are we talking about? Audio and/or Video This lecture concerned about:
3
Ashwin BharambeCarnegie Mellon University Lecture Overview Multimedia Applications Application Requirements / Challenges Media Encodings Case Studies Streaming Stored Media Real Time Interactive Media (Voice over IP) Protocols Real-Time Transport Protocol (RTP) Real-Time Transport Control Protocol (RTCP)
4
Ashwin BharambeCarnegie Mellon University Application Classes Streaming Stored Media Pipeline reception of stored files over the network No need to wait for entire file before viewing Example: On-demand video Unidirectional Real-Time Media Similar, but data is generated on the fly Non-interactive: just listen/view, no fast-forward! Example: news broadcast Interactive Real-Time Media Example: video conference
5
Ashwin BharambeCarnegie Mellon University Streaming Media Applications Characteristics Interactive: Users can play, pause Even rewind/fast-forward Requirements Delay: from client request until display start can be 1 to 10 seconds Time is used to buffer data to reduce chance of stalling Delay constraints on data delivery to maintain continuous playout. On demand Video
6
Ashwin BharambeCarnegie Mellon University Unidirectional Real-Time Apps Characteristics By storing media: can pause, rewind Often used combined with multicast Requirements As with streaming media, delay constraints on data delivery to maintain continuous playout News Broadcast
7
Ashwin BharambeCarnegie Mellon University Real-Time Interactive Applications Requirements Very stringent delay requirements because of real-time interactive nature Limit to how much people are willing to adapt to the delay Video: < 150 msec acceptable (one-way) Audio: < 150 msec not perceived, < 400 msec acceptable (one-way) Internet Telephony
8
Ashwin BharambeCarnegie Mellon University Roadmap Multimedia Applications Application Requirements / Challenges Media Encodings Case Studies Streaming Stored Media Real Time Interactive Media (Voice over IP) Protocols Real Time Protocol (RTP) Real Time Control Protocol (RTCP)
9
Ashwin BharambeCarnegie Mellon University Review: Network Basics Source Dest Throughput = Amount of data arriving per unit time. Jitter = Variance of interpacket arrival times Interpacket Arrival Time Delay Loss Best-effort delivery!!
10
Ashwin BharambeCarnegie Mellon University End-to-End Delay Interactive applications Video conferencing Human communication starts to break down at ~500 msec delays Feedback loops Delay affected by encoding/decoding Today’s Internet Best effort service and end-to-end mechanisms Provide little opportunity for controlling delay Longer term (only?) solution: Quality of Service
11
Ashwin BharambeCarnegie Mellon University Packet Jitter Real-Time systems must deal with this Jitter is hard to bound even with QoS Partial solution: Buffer management Buffers are used to smooth jitter, but how big should the buffer be? Unexpected jitter can lead to starvation or buffer overflow if buffer is too small But over-provisioning buffers increases delay
12
Ashwin BharambeCarnegie Mellon University Throughput Estimating throughput is problematic Media coding may be tunable, but changes in quality are generally perceptually annoying Throughput will often vary faster than feedback delay which makes estimation hard Multicast makes this worse Possible Solutions: QoS mechanisms Scalable encoding combined with congestion control (i.e., dynamically adapt compression to bandwidth)
13
Ashwin BharambeCarnegie Mellon University Packet Loss Effect of loss is media and encoding specific Loss of one packet in a video stream may result in the loss of multiple frames Other packets received can be rendered useless Retransmission often not feasible (no time) Possible Solutions: Encoding that alleviates packet loss effects Forward Error Correction (FEC)
14
Ashwin BharambeCarnegie Mellon University Roadmap Multimedia Applications Application Requirements / Challenges Media Encodings Case Studies Streaming Stored Media Real Time Interactive Media (Voice over IP) Protocols Real Time Protocol (RTP) Real Time Control Protocol (RTCP)
15
Ashwin BharambeCarnegie Mellon University Media Encoding Digitization essential, of course! Sample signal at some frequency 8KHz, for example Quantize represent each sample as some fixed #bits Compress: make number of bits small Audio: GSM, G.729, G.723.3, MP3, … Video: MPEG 1/2/4, H.261, … Then: Send -> Receive, decompress/convert, play
16
Ashwin BharambeCarnegie Mellon University Audio Encoding Traditional telephone quality encoding 8KHz samples of 8 bits each 64kbps CD quality encoding: 44.1 KHz of 16 bits 1.41 Mbs uncompressed MP3 compression Frequency ranges that are divided in blocks, which are converted using DCT, quantized, and encoded RangeRatio Layer 1384 kbps4 Layer 2192 kbps8 Layer 3128 kbps12
17
Ashwin BharambeCarnegie Mellon University Image Encoding: JPEG Divide digitized image in 8 x 8 pixel blocks The DCT phase converts the pixel block into a block of frequency coefficients Discrete Cosine Transform – similar to FFT The quantization phase limits the precision of the frequency coefficient This is based on a quantization table, which controls the degree of compression The encoding phase packs this information in a dense fashion
18
Ashwin BharambeCarnegie Mellon University Video Encoding Once frames are captured ("raw" video) resulting file is very large: 320 x 240 x 24 -bit color = 230,400 bytes/frame 15 frames/second = 3,456,000 bytes/second 10 seconds takes around 30 Mbytes! (no audio) Commonly-used encodings: AVI, MPEG Exploit spatial locality inside a single image Temporal locality across images Layered encoding
19
Ashwin BharambeCarnegie Mellon University MPEG Encoding of Video Three frame types: I frame: independent encoding of the frame (JPEG) P frame: encodes difference relative to I-frame (predicted) B frame: encodes difference relative to interpolated frame Note that frames will have different sizes Complex encoding, e.g. motion of pixel blocks, scene changes,.. Decoding is easier than encoding. MPEG often uses fixed-rate encoding. IBBPBBPBBIBBPBB
20
Ashwin BharambeCarnegie Mellon University MPEG System Streams Combine one more MPEG video and audio streams in a single synchronized stream. Consists of a hierarchy with meta data at every level describing the data. System level contains synchronization information Video level is organized as a stream of group of pictures Group of pictures consists of pictures Pictures are organized in slices …
21
Ashwin BharambeCarnegie Mellon University Roadmap Multimedia Applications Application Requirements / Challenges Media Encodings Case Studies Streaming Stored Media Real Time Interactive Media (Voice over IP) Protocols Real Time Protocol (RTP) Real Time Control Protocol (RTCP)
22
Ashwin BharambeCarnegie Mellon University Streaming Media Applications Important and growing Reduction of storage costs High-speed access networks (e.g., DSL) Enhancements to network caching of video using content distribution networks Operation Segment audio/video file; send over UDP/TCP Real-Time Transport Protocol (RTP) User-Interactive Control Real-Time Transport Control Protocol (RTCP)
23
Ashwin BharambeCarnegie Mellon University Streaming Helper Application Displays content Typically requested via a Web browser Example: RealPlayer Typical functions: GUI for user control Decompression Jitter removal Error correction
24
Ashwin BharambeCarnegie Mellon University Streaming From Web Servers (1) Video sent as HTTP object(s) Options: Interleaved audio and video in one file Two separate files with client synchronizing the display
25
Ashwin BharambeCarnegie Mellon University Streaming From Web Servers (2) Alternative Set up connection between server and player, then download
26
Ashwin BharambeCarnegie Mellon University Using a Streaming Server Application layer protocol can be better tailored to streaming Gets us around HTTP, allows a choice of UDP vs. TCP
27
Ashwin BharambeCarnegie Mellon University Streaming Media Application Client Side Issues
28
Ashwin BharambeCarnegie Mellon University Quick Question What protocol does RealPlayer or QuickTime use?
29
Ashwin BharambeCarnegie Mellon University Tricky Question What protocol does RealPlayer or QuickTime use?
30
Ashwin BharambeCarnegie Mellon University Real Time Streaming Protocol (RTSP) For user to control transmission of stream Display: rewind, fast forward, pause, resume, etc… Think: remote control for your VCR Out-of-band protocol (Port 554) RFC 2326
31
Ashwin BharambeCarnegie Mellon University RTSP at Work
32
Ashwin BharambeCarnegie Mellon University Roadmap Multimedia Applications Application Requirements / Challenges Media Encodings Case Studies Streaming Stored Media Real Time Interactive Media (Voice over IP) Protocols Real Time Protocol (RTP) Real Time Control Protocol (RTCP)
33
Ashwin BharambeCarnegie Mellon University Voice Over IP (VoIP) Internet phone applications generate packets during talk spurts No traffic during silence Operation Sample, quantize, encapsulate and send Protocol choices UDP: some packets may be lost (up to ~20% loss is tolerable) TCP: eliminates loss but creates variance in delay TCP slow start can also create periods of very small throughput
34
Ashwin BharambeCarnegie Mellon University Voice Over IP (VoIP): Issues Delay End-to-end delays > 400 msec cannot be tolerated Packets that are that delayed are ignored (i.e., lost) Jitter Use timestamps, sequence numbers Delay playout at receivers (fixed or adaptive) Packet loss Forward Error Correction (FEC) Interleaving Receiver-based repair
35
Ashwin BharambeCarnegie Mellon University Compensating for Jitter: VoIP with Fixed Playout Delay Playout delay
36
Ashwin BharambeCarnegie Mellon University Compensating for Jitter: Adaptive Playout Delay Disadvantages of fixed playout delay More packet loss (delayed packets), if too small High latency, if too large Solution: track network delay performance Estimate average delay and delay variance Similar to RTT estimation in TCP Adjust playout delay during silent periods Silent periods get shortened/expanded Hardly noticeable!
37
Ashwin BharambeCarnegie Mellon University Recovery from Packet Loss: Forward Error Correction (FEC) Send redundant encoded chunk every n chunks (XOR original n chunks) If 1 packet in this group lost, can reconstruct If > 1 packet lost, cannot recover Disadvantages: The smaller the group size, larger the overhead Playout delay increased
38
Ashwin BharambeCarnegie Mellon University Recovering from Packet Loss: Piggybacking Lo-fidelity Stream
39
Ashwin BharambeCarnegie Mellon University Recovering from Packet Loss: Interleaving Divide 20 msec of audio data into smaller units of 5 msec each and interleave Upon loss, have a set of partially filled chunks
40
Ashwin BharambeCarnegie Mellon University Recovering from Packet Loss: Receiver-based Repair Exploit temporal similarity Replicate packets Smart solutions Interpolation
41
Ashwin BharambeCarnegie Mellon University Roadmap Multimedia Applications Application Requirements / Challenges Media Encodings Case Studies Streaming Stored Media Real Time Interactive Media (Voice over IP) Protocols Real Time Protocol (RTP) Real Time Control Protocol (RTCP)
42
Ashwin BharambeCarnegie Mellon University Real Time Transport Protocol (RTP) Recall: Multimedia apps. append headers to audio/video chunks Format, sequence numbers, timestamps, … RTP: public domain standard for such headers Chunks encapsulated in RTP packets before being sent to UDP Real Time Control Protocol (RTCP) for control RTP logically extends UDP. Sits between UDP and application Thin Protocol
43
Ashwin BharambeCarnegie Mellon University RTP: What does it do? Framing Multiplexing “Real-time delivery” (sort’a) Synchronization Feedback
44
Ashwin BharambeCarnegie Mellon University RTP Packet Format Source/Payload type Different formats assigned different codes E.g., GSM -> 3, MPEG Audio -> 14 Sequence numbers Time stamps Synchronization source ID Miscellaneous fields
45
Ashwin BharambeCarnegie Mellon University Timestamp vs. Sequence Number Timestamp relates packet to real time Timestamp values sampled from a media specific clock Sequence number relates packet to other packets Allows many packets to have the same timestamp but different sequence numbers
46
Ashwin BharambeCarnegie Mellon University Audio silence example Consider audio data: what do you want to do during silence? Not send anything Why might this cause problems? Other side needs to distinguish between loss and silence How does the timestamp/seq# mechanism help? Big jump in timestamps, but correct next sequence number Silence!
47
Ashwin BharambeCarnegie Mellon University SSRC: Synchronization Source Identifies the stream Not just the participant All packets with the same SSRC go together Picked at random Why do we need this? No assumption of stream association from underlying network mechanism Possible problems? Collision
48
Ashwin BharambeCarnegie Mellon University Real Time Control Protocol (RTCP) RTCP packets transmitted by each participant in RTP session to all others using multicast Distinct port number from RTP Reports on: Loss rate Inter-arrival jitter Identity of receivers Delay, (indirectly).
49
Ashwin BharambeCarnegie Mellon University Control Bandwidth Scaling Needed for scalability Consider one speaker and 2 receivers as compared to one speaker with 20 or 200 receivers Recommendation: 5% of data bandwidth Sending delay should be randomized Should account for overhead of underlying network service (i.e., UDP and IP)
50
Ashwin BharambeCarnegie Mellon University Summary Supporting multimedia applications in today’s Internet is difficult Several tricks to alleviate problems posed by best-effort delivery In future, perhaps bandwidth will be extremely plentiful everywhere Or – we will need Quality of Service some day or the other! Bunch of Real-Time protocols standardizing some aspects of real-time communication
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.