IT 424 Networks2 IT 424 Networks2 Ack.: Slides are adapted from the slides of the book: “Computer Networking” – J. Kurose, K. Ross Chapter 4: Multimedia Networks (ch 7 in book) Part 1: Streaming Stored Multimedia

Multimedia & QoS Introduction to Audi and Video Streaming Stored Media Streaming Protocols Overview 2 Streaming Live – Interactive Apps

Learning Outcomes 1 To describe the characteristics and requirements of multimedia applications 2 To explain the problem of multimedia transmission over the Internet 3 To define the challenges of streaming stored video/audio 4 To differentiate between the different schemes and protocols for streaming multimedia 5 To understand and analyze the RTSP protocol 6 To understand the difference between streaming stored and live multimedia

Multimedia & QoS Introduction to Audi and Video Streaming Stored Media Streaming Protocols Overview 4 Streaming Live – Interactive Apps

Multimedia & QoS Multimedia and Quality of Service: What Is It? multimedia : network audio and video (“continuous media”) network provides application with level of performance needed for application to function. QoS

Multimedia & QoS - Applications MM Networking Applications MM Application Types: 1.Streaming, stored audio, video  Streaming: can begin playout before downloading entire file  Stored (at server): which allows pause, rewind or fast-forward (implies storing/buffering at client)  E.G., Youtube, netflix, hulu 2.Streaming live audio, video  E.G., Live sporting event (football) 3.Conversational voice/video over IP  This type allows people to use audio/video to communicate with each other in real time  Interactive nature of human-to-human conversation limits delay tolerance  E.G., Skype

Multimedia & QoS - Applications MM Networking Applications Fundamental characteristics:  Delay sensitive  End-to-end delay  Delay jitter  Loss tolerant: infrequent losses cause minor glitches  Different from data, which are loss intolerant but delay tolerant. Jitter is the variability of packet delays within the same packet stream

Multimedia & QoS Introduction to Audio and Video Streaming Stored Media Streaming Protocols Overview 8 Streaming Live – Interactive Apps

Analog audio signal is converted to a digital signal as follows:  Analog audio signal sampled at constant rate  Telephone: 8,000 samples/sec  CD music: 44,100 samples/sec  Each sample quantized, i.e., Rounded  E.G., 2 8 =256 possible quantized values  Each quantized value represented by fixed number of bits,  e.g., 8 bits for 256 values Multimedia: Audio Introduction to Audio and Video - Audio time audio signal amplitude analog signal quantized value of analog value quantization error sampling rate (N sample/sec)

 Example: 8,000 samples/sec, 256 quantized values, what is the bit rate? Answer: 64,000 bps  Receiver converts bits back to analog signal:  Some quality reduction example rates  CD: Mbps  MP3: 96, 128, 160 kbps  Internet telephony: 5.3 kbps and up Multimedia: Audio Introduction to Audio and Video - Audio time audio signal amplitude analog signal quantized value of analog value quantization error sampling rate (N sample/sec)

 Video: sequence of images displayed at constant rate  E.G. 24 images/sec  Digital image: consists of array of pixels  Each pixel represented by bits  Coding: use redundancy within and between images to decrease # bits used to encode image  Spatial (within image)  Temporal (from one image to next) Multimedia: Video ……………………...… spatial coding example: instead of sending N values of same color (all purple), send only two values: color value (purple) and number of repeated values (N) ……………………...… frame i frame i+1 temporal coding example: instead of sending complete frame at i+1, send only differences from frame i Introduction to Audio and Video - Video

 CBR: (constant bit rate): video encoding rate fixed  VBR: (variable bit rate): video encoding rate changes as amount of spatial, temporal coding changes  Examples:  MPEG 1 (CD-ROM) 1.5 Mbps  MPEG2 (DVD) 3-6 Mbps  MPEG4 (often used in Internet, < 1 Mbps) Multimedia: Video Introduction to Audio and Video - Video ……………………...… spatial coding example: instead of sending N values of same color (all purple), send only two values: color value (purple) and number of repeated values (N) ……………………...… frame i frame i+1 temporal coding example: instead of sending complete frame at i+1, send only differences from frame i

Multimedia Over Today’s Internet Introduction to Audio and Video - Video TCP/UDP/IP: “best-effort service” no guarantees on delay, loss Today’s Internet multimedia applications use application-level techniques to mitigate (as best possible) effects of delay, loss But you said multimedia apps requires QoS and level of performance to be effective! ? ? ?? ? ? ? ? ? ? ?

Multimedia & QoS - Applications How Should The Internet Evolve To Better Support Multimedia? Integrated services philosophy:  Fundamental changes in internet so that apps can reserve end-to-end bandwidth  Requires new, complex software in hosts & routers Laissez-faire  No major changes  More bandwidth when needed  Content distribution, application-layer multicast  Application layer Differentiated services philosophy:  Fewer changes to internet infrastructure, yet provide 1st and 2nd class service What’s your opinion?

Multimedia & QoS Introduction to Audio and Video Streaming Stored Media Streaming Protocols Overview 15 Streaming Live – Interactive Apps

Streaming Stored Multimedia Stored streaming:  Media stored at source  Transmitted to client  Streaming: client playout begins before all data has arrived  Timing constraint for still-to-be transmitted data: in time for playout Streaming Stored Video - Challenges

Streaming Stored Video 1.video recorded (e.g., 30 frames/sec) 2. video sent Cumulative data Streaming: at this time, client playing out early part of video, while server still sending later part of video network delay (fixed in this example) time 3. video received, played out at client (30 frames/sec)

Streaming Stored Multimedia: Interactivity  VCR-like functionality: client can pause, rewind, FF, push slider bar  10 sec initial delay OK  1-2 sec until command effect OK  Timing constraint for still-to-be transmitted data: in time for playout Streaming Stored Multimedia - Interactivity

Streaming Stored Video: Challenges  Continuous playout constraint: once client playout begins, playback must match original timing  … but network delays are variable (jitter), so will need client-side buffer to match playout requirements  Other challenges:  Client interactivity: pause, fast-forward, rewind, jump through video  Video packets may be lost, retransmitted Streaming Stored Video - Challenges

 Client-side buffering and playout delay: compensate for network-added delay, delay jitter Streaming Stored Video: Client Buffering Streaming Stored Video constant bit rate video transmission Cumulative data time variable network delay client video reception constant bit rate video playout at client client playout delay buffered video

Streaming Stored Video: Buffering Client-Side Buffering, Playout variable fill rate, x(t) client application buffer, size B drain rate d to decompression and playout, e.g., CBR r buffer fill level, Q(t) video server client  Client-side buffering and playout delay: compensate for network-added delay, delay jitter

Client-Side Buffering, Playout Streaming Stored Video: Buffering variable fill rate, x(t) client application buffer, size B playout rate, e.g., CBR r buffer fill level, Q(t) video server client 1. Initial fill of buffer until playout begins at t p 2. Playout begins at t p, 3. Buffer fill level varies over time as fill rate x(t) varies and playout rate r is constant

Streaming Stored Video  Application-level streaming techniques for making the best out of best effort service:  Client-side buffering  Use of UDP versus TCP  Multiple encodings of multimedia Streaming Stored Video - Challenges  Jitter removal  Decompression  Error concealment  Graphical user interface w/ controls for interactivity Media Player

Internet Multimedia: Simplest Approach  Audio or video stored in file  Files transferred as HTTP object  Received in entirety at client  Then passed to player Audio, video not streamed:  No, “pipelining,” long delays until playout! Streaming Stored Video - Challenges

Internet Multimedia: Streaming Approach  Browser gets metafile  Browser launches player, passing metafile  Player contacts server  Server streams audio/video to player Streaming Stored Video - Challenges

Streaming from a Streaming Server  HTTP is not optimized for streaming  This scheme allows for non-HTTP protocol between server, media player  It also allows for UDP or TCP for step (3), more shortly Streaming Stored Video - Challenges

Multimedia & QoS Introduction to Audio and Video Streaming Stored Media Streaming Protocols Overview 27 Streaming Live – Interactive Apps

Streaming Protocols Streaming Multimedia: UDP or TCP? UDP  Server sends at rate appropriate for client (oblivious to network congestion !)  Often send rate = encoding rate = constant rate  Then, fill rate = constant rate - packet loss  Short playout delay (2-5 seconds) to remove network jitter  Error recover: time permitting TCP  Send at maximum possible rate under TCP  Fill rate fluctuates due to TCP congestion control  Larger playout delay: smooth TCP delivery rate  HTTP/TCP passes more easily through firewalls

Q: How to handle different client receive rate capabilities?  28.8 Kbps dialup  100 Mbps Ethernet A: Server stores, transmits multiple copies of video, encoded at different rates Streaming Multimedia: Client Rate(s) Streaming Protocols 1.5 Mbps encoding 28.8 Kbps encoding

Streaming Protocols - RTSP User Control of Streaming Media: RTSP HTTP  Does not target multimedia content  No commands for fast forward, etc. RTSP: RFC 2326  Client-server application layer protocol  User control: rewind, fast forward, pause, resume, repositioning, etc…  Out-of-band control  RTSP control messages use different port numbers than media stream: out-of-band - port 554 What it doesn’t do:  Doesn’t define how audio/video is encapsulated for streaming over network  Doesn’t restrict how streamed media is transported (UDP or TCP possible)  Doesn’t specify how media player buffers audio/video

RTSP Example Scenario:  Metafile communicated to web browser  Browser launches player  Player sets up an RTSP control connection, data connection to streaming server Streaming Protocols - RTSP

Metafile Example Twister <track type=audio e="PCMU/8000/1" src = "rtsp://audio.example.com/twister/audio.en/lofi"> <track type=audio e="DVI4/16000/2" pt="90 DVI4/8000/1" src="rtsp://audio.example.com/twister/audio.en/hifi"> <track type="video/jpeg" src="rtsp://video.example.com/twister/video">

RTSP Operation Streaming Protocols - RTSP

Multimedia & QoS Introduction to Audio and Video Streaming Stored Media Streaming Protocols Overview 34 Streaming Live – Interactive Apps

Streaming Live Multimedia Examples:  Internet radio talk show  Live sporting event Streaming (as with streaming stored multimedia)  Playback buffer  Playback can lag tens of seconds after transmission  Still have timing constraint Interactivity  Fast forward impossible  Rewind, pause possible! Multimedia Applications

Real-time Interactive Applications PC-2-PC phone Skype PC-2-phone Dialpad Net2phone Skype Video conference with webcams Skype Polycom Going to look at Voice-over- IP in detail Multimedia Applications

Conclusion  Internet provides best effort service only which is not suitable for multimedia applications  MM applications can be classified into three types:  Streaming, stored audio, video  Conversational voice/video over IP  Streaming live audio, video  Multimedia traffic is bursty in nature and requires stringent delay guarantees  Streaming mulimedia can be improved using UDP protocol, buffering content, adapt compression level and send redundant packets. Conclusion

References  Computer Networking: A Top-Down Approach Featuring the Internet by James Kurose and Keith Ross, Addison Wesley, 2012 (chapter 7 )Computer Networking  Dynamic Adaptive Streaming over HTTP – Design Principles and Standard Dynamic Adaptive Streaming over HTTP