2. What is Multimedia? Function: noun plural but singular or plural in construction Date: 1950 : a technique (as the combining of sound, video, and text) for expressing ideas (as in communication, entertainment, or art) in which several media are employed; also : something (as software) using or facilitating such a technique

3. What is Multimedia? Our Focus: Streaming Stored video/audio Streaming Live events Real-time interactive

4. Issues to deal with: Delay Sensitive: Voice delays over VOIP Delayed actions for online games Delayed video playback

5. More issues to deal with: Loss Tolerant: Slight glitch in audio Lost pixels or frames of video Lost actions

6. One More issue to deal with: Packet Jitter: Variability of packet delays within the same packet stream

7. Streaming Stored Multimedia Streaming: The ability for playback without downloading the entire media. Netflix Youtube Fox.com Syfy.com

8. Streaming Stored Multimedia Streaming: Stored at source (or multiple sources) Playback begins after an Application/system designed delay -Buffering

9. Streaming Stored Multimedia Functionality: Pause – Streaming may continue Rewind – Streaming may stop Fast Forward – Buffering issues

10. Streaming Stored Multimedia Boondock Saints 2 Or Netflix

11. Streaming “Live” Multimedia Still not live due to some buffering delays and a congested network Radio – PandoraPandora TV - wwiTVwwiTV News Etc…..

12. Streaming “Live” Multimedia Functionality: No Fast Forward (well maybe if you paused) Can Rewind & Pause (maybe)

13. Real Time Interactive Multimedia (phew) Voice over IP – Video Conferencing – Distributed interactive worlds –

14. Real Time Interactive Multimedia (phew) Delay Sensitive Audio < 150 msec good < 400 msec OK > Not so good * No Rewind, play, or fast forward functionalities

15. TCP/UDP?!?!? OPP – Other peoples problems? Best-effort service Application deals with it

16. Evolution of Multimedia over the Internet…….next steps? Hard Guarantee – QoS (Quality of Service) is given as requested Soft Guarantee – QoS is given with high probability

17. Evolution of Multimedia over the Internet…….next steps? Integrated Service Philosophy: - Big change to underlying network - End-to-end bandwidth reservations

18. Evolution of Multimedia over the Internet…….next steps? Laissez-Faire - deal with it - Expand bandwidth as needed

19. Evolution of Multimedia over the Internet…….next steps? Laissez-Faire (continued) - Content Distributed Networks (CDNs) - Multicast Overlay Networks

20. Evolution of Multimedia over the Internet…….next steps? Differentiated services philosophy: - Some changes - 1 st and 2 nd class services

21. Audio Sampling done at a constant rate - 8,000 samples/second – Telephone - 44,1000 samples/second – CD * Each sample is 8 bits (give or take) Ex)8,000samples/second x 8bits/sample 64,000bits/second

22. Audio Compression Techniques: GSM – 13 Kbps G.729- 8 Kpbs G.723.3 – 6.4 & 5.3 Kpbs MP3 – 96, 128, 160 Kpbs complex - psychoacoustic masking - redundancy reduction - bit reservoir buffering

23. Video Def.: A sequence of images displayed at a constant rate e.g. 24-30 frames/second

24. Video Compression Redundancy Checking: Spatial

25. Video Compression Redundancy Checking: Temporal

26. Video Compression Formats MPEG1 MPEG2 MPEG4 And many more…..

27. Streaming Stored Multimedia More Details How to deal with best effort service? - Client Buffering - UDP or TCP - Multiple sources of media User’s Media Player deals with: - Jitter removal - Decompression - Error Concealment - Play, pause, FF, Rewind

28. How to stream using HTTP -Get Metafile -Pass Metafile to Media player -Media player contacts server -Server streams to media player

29. How to stream using a Streaming Server -Similar…. -Allows for non-HTTP protocols

30. Client Buffering

31. TCP/UDP Buffering Pros/Cons UDP - Client determines incoming rate -Short Delay -Error Recovery…… TCP -Max TCP rate -Congestion Control causes fill rate fluxuations -Longer delays -Easier to pass through firewalls

32. Real-Time Streaming Protocol (RTSP) HTTP: - Does not target multimedia content -No media controlers RTSP -Client Server application layer protocol

33. Real-Time Streaming Protocol (RTSP) User controls are “out-of- band” -uses port 554 for play, FF, and Rewind The Media stream is considered “in-band”

34. Real-Time Streaming Protocol (RTSP) RTSP Does not do: -Compression -Packet Encapsulation -Choose UDP or TCP -Control Buffering

35. RTSP Example Scenario:  metafile communicated to web browser  browser launches player  player sets up an RTSP control connection, data connection to streaming server

36. Metafile Example Twister <track type=audio e="PCMU/8000/1" src = "rtsp://audio.example.com/twister/audio.en/lofi"> <track type=audio e="DVI4/16000/2" pt="90 DVI4/8000/1" src="rtsp://audio.example.com/twister/audio.en/hifi"> <track type="video/jpeg" src="rtsp://video.example.com/twister/video">

37. 7: Multimedia Networking7-36 RTSP Operation

38. 7: Multimedia Networking7-37 RTSP Exchange Example C: SETUP rtsp://audio.example.com/twister/audio RTSP/1.0 Transport: rtp/udp; compression; port=3056; mode=PLAY S: RTSP/1.0 200 1 OK Session 4231 C: PLAY rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 Range: npt=0- C: PAUSE rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 Range: npt=37 C: TEARDOWN rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 S: 200 3 OK

39. Sources Main: [1] J. Kuros and K. Rose, Computer Networking: A Top-Down Approach, 5 th ed., Pearson, 2009. [2] K. Abernathy and T. Allen, Exploring the Digital Domain: An Introduction to Computing with Multimedia and Networking, California: Brooks/Cole, 1999. [3]Netflix. [Online]. Available: www.netflix.com. [Accessed: Nov. 30, 2009].www.netflix.com [4] IMDB, Boondock Saints II: All Saints Day. [Online]. Available: http://www.imdb.com/title/tt1300851/. [Accessed: Nov. 30, 2009]. http://www.imdb.com/title/tt1300851/ [5]Pandora. [Online]. Available: www.pandora.com. [Accessed: Nov.30, 2009].www.pandora.com [6]wwiTV. [Online]. Available: www.wwiTV.com. [Accessed: Nov. 30, 2009].www.wwiTV.com [7]J. Lendino, “Streaming Video to Your Living-Room PC,” PC Magazine, vol. 26, issue 25, p. 83-84, December 2007. [Online]. Available: EBSCOhost, http://web.ebscohost.com/ehost/. [Accessed: Nov. 29, 2009]. http://web.ebscohost.com/ehost/

40. [1]Available: http://www.qweas.com/downloads/desktop/icons/scr-multimedia-icon- collection.jpg. [Accessed Nov. 27, 2009].http://www.qweas.com/downloads/desktop/icons/scr-multimedia-icon- collection.jpg [2] Available: http://www.merriam-webster.com/dictionary/Multimedia. [Accessed Nov. 27, 2009].http://www.merriam-webster.com/dictionary/Multimedia [3] Available: http://www.wadehone.com/icons/zWolf-anim-large.gif. [Accessed Nov. 29, 2009].http://www.wadehone.com/icons/zWolf-anim-large.gif [4] Alberto Bertapelle, Brainbug: Nightmare [MP3], 1997. [5] Available: http://s2.b3ta.com/host/creative/46786/1249032511/TortoiseRace.gif. [Accessed Nov. 27, 2009].ttp://s2.b3ta.com/host/creative/46786/1249032511/TortoiseRace.gif [6] Available: http://www.gifandgif.eu/animated_gif/Water/#link. [Accessed Nov. 27, 2009].http://www.gifandgif.eu/animated_gif/Water/#link [7] Available: http://www.google.com/imgres?imgurl=http://www.thebest3d.com/dogwaffle/w hatsnew/1_6/optipustics/magic- waterfall.jpg&imgrefurl=http://www.thebest3d.com/dogwaffle/whatsnew/1_6/op tipustics/index.html&h=430&w=369&sz=9&tbnid=nalj_bl6dL4ykM:&tbnh=126&tb nw=108&prev=/images%3Fq%3DWater%2Banimated%2Bgif&hl=en&usg=__rOHn0 aXBWpgIfwVrDoFY5mMrH0s=&ei=4VwQS6fqOcaMlAe4r4GPBA&sa=X&oi=image_r esult&resnum=4&ct=image&ved=0CBEQ9QEwAw. [Accessed Nov. 27, 2009]. [8] Available: http://www.moviesonline.ca/movie- gallery/albums/userpics//poster_boondock-saints-2.jpg. [Accessed Nov. 27, 2009]. Sources Main (Secondary):

41. [9] Available: http://roflrazzi.files.wordpress.com/2009/05/celebrity-pictures-speed- racer-animation-expensive.jpg. [Accessed Nov. 29, 2009].http://roflrazzi.files.wordpress.com/2009/05/celebrity-pictures-speed- racer-animation-expensive.jpg [10] Available: http://it.tinypic.com/view.php?pic=28rnkh3&s=3. [Accessed Nov. 29, 2009].http://it.tinypic.com/view.php?pic=28rnkh3&s=3 [11] Available: http://www.beatcanvas.com/pics/pause_sign_go.jpg. [Accessed Nov. 29, 2009].http://www.beatcanvas.com/pics/pause_sign_go.jpg [12] Available: http://www.netweed.com/prohiphop/graf/bekindrewindmosdef.jpg. [Accessed Nov. 29, 2009].http://www.netweed.com/prohiphop/graf/bekindrewindmosdef.jpg [13] Available: http://allyho.files.wordpress.com/2009/04/ff-fast-forward-pola.jpg. [Accessed Nov. 29, 2009].http://allyho.files.wordpress.com/2009/04/ff-fast-forward-pola.jpg [14] Available: http://adamsandler-information.com/wp- content/uploads/2009/05/clickadam.jpg. [Accessed Nov. 29,2009].http://adamsandler-information.com/wp- content/uploads/2009/05/clickadam.jpg [15] Available: http://www.skype.com/. [Accessed Nov. 29, 2009].http://www.skype.com/ [16] Available: http://m.www.yahoo.com/. [Accessed Nov. 29, 2009].http://m.www.yahoo.com/ [17] Available: https://www.cordia.us/. [Accessed Nov. 29, 2009].https://www.cordia.us/ [18]Available: http://www.tentonhammer.com/system/files/images/wow_logo.jpg. [Accessed Nov. 29, 2009].http://www.tentonhammer.com/system/files/images/wow_logo.jpg [19] Available: http://farmtownblog.influxpc.com/wp- content/uploads/2009/08/farmtown1.jpg. [Accessed Nov. 29, 2009].http://farmtownblog.influxpc.com/wp- content/uploads/2009/08/farmtown1.jpg

42. [20] Available: http://gameaxis.com/wp-content/uploads/2009/10/modern-warfare- 2.jpg. [Accessed Nov. 29, 2009].http://gameaxis.com/wp-content/uploads/2009/10/modern-warfare- 2.jpg [21] Available: http://media.photobucket.com/image/Time/mclelun/time.jpg. [Accessed Nov. 29, 2009].http://media.photobucket.com/image/Time/mclelun/time.jpg [22] Available: http://4.bp.blogspot.com/_gcA0ZuKGkI8/SgzcZVnQGDI/AAAAAAAAA7A/q2HmPqxL YFc/s400/500x500_sucker2.jpg. [Accessed Nov. 29, 2009]. http://4.bp.blogspot.com/_gcA0ZuKGkI8/SgzcZVnQGDI/AAAAAAAAA7A/q2HmPqxL YFc/s400/500x500_sucker2.jpg [23] Available: http://www.buildamovement.com/blog/wp- content/uploads/2009/09/evolution.jpg. [Accessed Nov. 29, 2009]. [24] Available: http://mos.totalfilm.com/images/h/harold-ramis-gives-ghostbusters-3- details-800-75.jpg. [Accessed Nov. 29, 2009]. [25] Available: http://flaglermagazine.com/wp- content/uploads/2009/07/musiplication.jpg. [Accessed Nov. 29, 2009].http://flaglermagazine.com/wp- content/uploads/2009/07/musiplication.jpg [26] Available: http://www.fbi.gov/hq/lab/fsc/backissu/jan2003/images/image006.jpg. [Accessed Nov. 29, 2009].http://www.fbi.gov/hq/lab/fsc/backissu/jan2003/images/image006.jpg [27] Available: http://www.thg.ru/video/20000913/images/mpeg-4-video.jpg. [Accessed Nov. 29, 2009].http://www.thg.ru/video/20000913/images/mpeg-4-video.jpg [28] Available: http://media.photobucket.com/image/Real%20time%20streaming/downarchive2/ member3/NT.jpg. [Accessed Nov. 29, 2009]. [29] Available: http://upload.wikimedia.org/wikipedia/commons/thumb/8/84/Example.svg/600p x-Example.svg.png. [Accessed Nov. 29, 2009].