What is Multimedia? Function: noun plural but singular or plural in construction Date: 1950 : a technique (as the combining of sound, video, and text) for expressing ideas (as in communication, entertainment, or art) in which several media are employed; also : something (as software) using or facilitating such a technique

What is Multimedia? Our Focus: Streaming Stored video/audio Streaming Live events Real-time interactive

Issues to deal with: Delay Sensitive: Voice delays over VOIP Delayed actions for online games Delayed video playback

More issues to deal with: Loss Tolerant: Slight glitch in audio Lost pixels or frames of video Lost actions

One More issue to deal with: Packet Jitter: Variability of packet delays within the same packet stream

Streaming Stored Multimedia Streaming: The ability for playback without downloading the entire media. Netflix Youtube Fox.com Syfy.com

Streaming Stored Multimedia Streaming: Stored at source (or multiple sources) Playback begins after an Application/system designed delay -Buffering

Streaming Stored Multimedia Functionality: Pause – Streaming may continue Rewind – Streaming may stop Fast Forward – Buffering issues

Streaming Stored Multimedia Boondock Saints 2 Or Netflix

Streaming “Live” Multimedia Still not live due to some buffering delays and a congested network Radio – PandoraPandora TV - wwiTVwwiTV News Etc…..

Streaming “Live” Multimedia Functionality: No Fast Forward (well maybe if you paused) Can Rewind & Pause (maybe)

Real Time Interactive Multimedia (phew) Voice over IP – Video Conferencing – Distributed interactive worlds –

Real Time Interactive Multimedia (phew) Delay Sensitive Audio < 150 msec good < 400 msec OK > Not so good * No Rewind, play, or fast forward functionalities

TCP/UDP?!?!? OPP – Other peoples problems? Best-effort service Application deals with it

Evolution of Multimedia over the Internet…….next steps? Hard Guarantee – QoS (Quality of Service) is given as requested Soft Guarantee – QoS is given with high probability

Evolution of Multimedia over the Internet…….next steps? Integrated Service Philosophy: - Big change to underlying network - End-to-end bandwidth reservations

Evolution of Multimedia over the Internet…….next steps? Laissez-Faire - deal with it - Expand bandwidth as needed

Evolution of Multimedia over the Internet…….next steps? Laissez-Faire (continued) - Content Distributed Networks (CDNs) - Multicast Overlay Networks

Evolution of Multimedia over the Internet…….next steps? Differentiated services philosophy: - Some changes - 1 st and 2 nd class services

Audio Sampling done at a constant rate - 8,000 samples/second – Telephone - 44,1000 samples/second – CD * Each sample is 8 bits (give or take) Ex)8,000samples/second x 8bits/sample 64,000bits/second

Audio Compression Techniques: GSM – 13 Kbps G Kpbs G – 6.4 & 5.3 Kpbs MP3 – 96, 128, 160 Kpbs complex - psychoacoustic masking - redundancy reduction - bit reservoir buffering

Video Def.: A sequence of images displayed at a constant rate e.g frames/second

Video Compression Redundancy Checking: Spatial

Video Compression Redundancy Checking: Temporal

Video Compression Formats MPEG1 MPEG2 MPEG4 And many more…..

Streaming Stored Multimedia More Details How to deal with best effort service? - Client Buffering - UDP or TCP - Multiple sources of media User’s Media Player deals with: - Jitter removal - Decompression - Error Concealment - Play, pause, FF, Rewind

How to stream using HTTP -Get Metafile -Pass Metafile to Media player -Media player contacts server -Server streams to media player

How to stream using a Streaming Server -Similar…. -Allows for non-HTTP protocols

Client Buffering

TCP/UDP Buffering Pros/Cons UDP - Client determines incoming rate -Short Delay -Error Recovery…… TCP -Max TCP rate -Congestion Control causes fill rate fluxuations -Longer delays -Easier to pass through firewalls

Real-Time Streaming Protocol (RTSP) HTTP: - Does not target multimedia content -No media controlers RTSP -Client Server application layer protocol

Real-Time Streaming Protocol (RTSP) User controls are “out-of- band” -uses port 554 for play, FF, and Rewind The Media stream is considered “in-band”

Real-Time Streaming Protocol (RTSP) RTSP Does not do: -Compression -Packet Encapsulation -Choose UDP or TCP -Control Buffering

RTSP Example Scenario:  metafile communicated to web browser  browser launches player  player sets up an RTSP control connection, data connection to streaming server

Metafile Example Twister <track type=audio e="PCMU/8000/1" src = "rtsp://audio.example.com/twister/audio.en/lofi"> <track type=audio e="DVI4/16000/2" pt="90 DVI4/8000/1" src="rtsp://audio.example.com/twister/audio.en/hifi"> <track type="video/jpeg" src="rtsp://video.example.com/twister/video">

7: Multimedia Networking7-36 RTSP Operation

7: Multimedia Networking7-37 RTSP Exchange Example C: SETUP rtsp://audio.example.com/twister/audio RTSP/1.0 Transport: rtp/udp; compression; port=3056; mode=PLAY S: RTSP/ OK Session 4231 C: PLAY rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 Range: npt=0- C: PAUSE rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 Range: npt=37 C: TEARDOWN rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 S: OK

Sources Main: [1] J. Kuros and K. Rose, Computer Networking: A Top-Down Approach, 5 th ed., Pearson, [2] K. Abernathy and T. Allen, Exploring the Digital Domain: An Introduction to Computing with Multimedia and Networking, California: Brooks/Cole, [3]Netflix. [Online]. Available: [Accessed: Nov. 30, 2009]. [4] IMDB, Boondock Saints II: All Saints Day. [Online]. Available: [Accessed: Nov. 30, 2009]. [5]Pandora. [Online]. Available: [Accessed: Nov.30, 2009]. [6]wwiTV. [Online]. Available: [Accessed: Nov. 30, 2009]. [7]J. Lendino, “Streaming Video to Your Living-Room PC,” PC Magazine, vol. 26, issue 25, p , December [Online]. Available: EBSCOhost, [Accessed: Nov. 29, 2009].

[1]Available: collection.jpg. [Accessed Nov. 27, 2009]. collection.jpg [2] Available: [Accessed Nov. 27, 2009]. [3] Available: [Accessed Nov. 29, 2009]. [4] Alberto Bertapelle, Brainbug: Nightmare [MP3], [5] Available: [Accessed Nov. 27, 2009].ttp://s2.b3ta.com/host/creative/46786/ /TortoiseRace.gif [6] Available: [Accessed Nov. 27, 2009]. [7] Available: hatsnew/1_6/optipustics/magic- waterfall.jpg&imgrefurl= tipustics/index.html&h=430&w=369&sz=9&tbnid=nalj_bl6dL4ykM:&tbnh=126&tb nw=108&prev=/images%3Fq%3DWater%2Banimated%2Bgif&hl=en&usg=__rOHn0 aXBWpgIfwVrDoFY5mMrH0s=&ei=4VwQS6fqOcaMlAe4r4GPBA&sa=X&oi=image_r esult&resnum=4&ct=image&ved=0CBEQ9QEwAw. [Accessed Nov. 27, 2009]. [8] Available: gallery/albums/userpics//poster_boondock-saints-2.jpg. [Accessed Nov. 27, 2009]. Sources Main (Secondary):

[9] Available: racer-animation-expensive.jpg. [Accessed Nov. 29, 2009]. racer-animation-expensive.jpg [10] Available: [Accessed Nov. 29, 2009]. [11] Available: [Accessed Nov. 29, 2009]. [12] Available: [Accessed Nov. 29, 2009]. [13] Available: [Accessed Nov. 29, 2009]. [14] Available: content/uploads/2009/05/clickadam.jpg. [Accessed Nov. 29,2009]. content/uploads/2009/05/clickadam.jpg [15] Available: [Accessed Nov. 29, 2009]. [16] Available: [Accessed Nov. 29, 2009]. [17] Available: [Accessed Nov. 29, 2009]. [18]Available: [Accessed Nov. 29, 2009]. [19] Available: content/uploads/2009/08/farmtown1.jpg. [Accessed Nov. 29, 2009]. content/uploads/2009/08/farmtown1.jpg

[20] Available: 2.jpg. [Accessed Nov. 29, 2009]. 2.jpg [21] Available: [Accessed Nov. 29, 2009]. [22] Available: YFc/s400/500x500_sucker2.jpg. [Accessed Nov. 29, 2009]. YFc/s400/500x500_sucker2.jpg [23] Available: content/uploads/2009/09/evolution.jpg. [Accessed Nov. 29, 2009]. [24] Available: details jpg. [Accessed Nov. 29, 2009]. [25] Available: content/uploads/2009/07/musiplication.jpg. [Accessed Nov. 29, 2009]. content/uploads/2009/07/musiplication.jpg [26] Available: [Accessed Nov. 29, 2009]. [27] Available: [Accessed Nov. 29, 2009]. [28] Available: member3/NT.jpg. [Accessed Nov. 29, 2009]. [29] Available: x-Example.svg.png. [Accessed Nov. 29, 2009].