1. NUS.SOC.CS5248 Ooi Wei Tsang Systems Support for Continuous Media OOI WEI TSANG

2. NUS.SOC.CS5248 OOI Wei Tsang Scope of this class Systems Support: Application Middleware Operating System Continuous Media: Video, audio, animation

3. NUS.SOC.CS5248 OOI Wei Tsang Local Media Applications Watch Movie Listen to Music Video Editing : (Much Less Interesting …)

4. NUS.SOC.CS5248 OOI Wei Tsang Network Media Applications Live Webcast Pre-recorded Webcast Video Conferencing Video on Demand

5. NUS.SOC.CS5248 OOI Wei Tsang Categories Interactive Non- Interactive Live Video Conferencing NUSLive, NUSCast Pre- recorded Lecture/Video on Demand

6. NUS.SOC.CS5248 OOI Wei Tsang Streaming Media Requires Reliable networks Low latency Bounded latency Plenty of bandwidth Not in today’s Internet.

7. NUS.SOC.CS5248 OOI Wei Tsang Overview Network Encoder Sender Middlebox Receiver Decoder

8. NUS.SOC.CS5248 OOI Wei Tsang We will NOT Design new codec Study media processing CS6212/CS5240 Attempt to improve the Internet EE5910

9. NUS.SOC.CS5248 Ooi Wei Tsang Summary of Topics Covered

10. NUS.SOC.CS5248 OOI Wei Tsang Sample of Topics Covered Review of Media Compression Network Communication Model Network Encoder Sender Middlebox Receiver Decoder

11. NUS.SOC.CS5248 OOI Wei Tsang Sample of Topics Covered Adaptation Video on Demand Network Encoder Sender Middlebox Receiver Decoder

12. NUS.SOC.CS5248 OOI Wei Tsang Sample of Topics Covered Buffering Audio/Video Synchronization Packet Loss Recovery Network Encoder Sender Middlebox Receiver Decoder

13. NUS.SOC.CS5248 OOI Wei Tsang Sample of Topics Covered Caching Application-Level Multicast Network Encoder Sender Middlebox Receiver Decoder

14. NUS.SOC.CS5248 OOI Wei Tsang Lecture Philosophy Cover the breadth, rather than depth Classic papers, rather than recent papers

15. NUS.SOC.CS5248 Ooi Wei Tsang Class Organization

16. NUS.SOC.CS5248 OOI Wei Tsang Goals Learn about existing techniques and systems Learn about current research efforts Learn about how to do research

17. NUS.SOC.CS5248 OOI Wei Tsang Philosophy Research oriented Students are expected to be mature and independent What you learned is more important than grade

18. NUS.SOC.CS5248 OOI Wei Tsang Please don’t ask.. “Is this equation important?” “Is this equation examinable?” “Do I have to memorize this header format?” “Post answer to quiz/survey/review”

19. NUS.SOC.CS5248 OOI Wei Tsang Please do ask.. “What is the effect of changing k in the equation?” “Why did the designer added this bit in the header?”

20. NUS.SOC.CS5248 OOI Wei Tsang Academic Honesty No copying among students No copying from published work ZERO TOLERENCE to Plagiarism

21. NUS.SOC.CS5248 OOI Wei Tsang Discussion? Strongly encouraged.. but must acknowledge all contributions write up solutions independently

22. NUS.SOC.CS5248 OOI Wei Tsang Gilligan’s Island Rule Discuss with friends Throw notes away Go watch “Gilligan’s Island” (or “Phua Chu Kang”, or “Super Sunday”) Write down your solution

23. NUS.SOC.CS5248 Ooi Wei Tsang Continuous Assessments

24. NUS.SOC.CS5248 OOI Wei Tsang Continuous Assessments Project (50%) Paper Survey (20%) Paper Review (15%) One of Programming Assignment Random Quizzes Class Participation Mid-term

25. NUS.SOC.CS5248 OOI Wei Tsang Research Project Define a problem Propose a solution Implement the solution Evaluate the solution

26. NUS.SOC.CS5248 OOI Wei Tsang Types of Project New problem + solution Existing problem + new solution Existing problem + existing solutions + comparisons + new insights See website for more..

27. NUS.SOC.CS5248 OOI Wei Tsang Variations Combine with your own research subjected to your advisor’s approval Combine with another class project subjected to the other instructor’s approval must be more significant

28. NUS.SOC.CS5248 OOI Wei Tsang Research Project Deliverables: 10-12 page paper poster presentation demo (if applicable) In a group of 1 to n students

29. NUS.SOC.CS5248 OOI Wei Tsang Survey Paper In-depth study of a topic 4-10 papers expected Suggested list of topics available (can be the same topic as your project!)

30. NUS.SOC.CS5248 OOI Wei Tsang Survey Paper summarize recent results organize, generalize, categorize add understanding

31. NUS.SOC.CS5248 OOI Wei Tsang Survey Paper Deliverables: 10-12 page survey paper Individual work

32. NUS.SOC.CS5248 OOI Wei Tsang Paper Reviews Read, summarize, and criticize 10 papers One is assigned almost weekly Deliverables: One page report per paper

33. NUS.SOC.CS5248 OOI Wei Tsang Expected Workload Read 3-4 papers per week Some hours of coding per week

34. NUS.SOC.CS5248 OOI Wei Tsang Office Hours Saturday 11 am – 12 noon SOC1 04-20 or email for appointment or just pop in

35. NUS.SOC.CS5248 OOI Wei Tsang Web Sites Main Website http://www.comp.nus.edu.sg/~cs5248 IVLE Website

36. NUS.SOC.CS5248 Ooi Wei Tsang Questions

37. NUS.SOC.CS5248 Ooi Wei Tsang Review of Media Compression

38. NUS.SOC.CS5248 OOI Wei Tsang You are Here Network Encoder Sender Middlebox Receiver Decoder

39. NUS.SOC.CS5248 OOI Wei Tsang Why compress? “Bandwidth Not Enough” “Disk Space Not Enough” Size of Uncompressed DVD Movie =

40. NUS.SOC.CS5248 OOI Wei Tsang Why compress? “Bandwidth Not Enough” “Disk Space Not Enough” Size of Uncompressed DVD Movie = 720 x 576 x 3 x 25 x 60 x 120 = 208.6GB

41. NUS.SOC.CS5248 Ooi Wei Tsang JPEG Compression

42. Original Image (1153KB) 1:1

43. Original Image (1153KB) 3.5:1

44. Original Image (1153KB) 17:1

45. Original Image (1153KB) 27:1

46. Original Image (1153KB) 72:1

47. Original Image (1153KB) 192:1

48. NUS.SOC.CS5248 OOI Wei Tsang Compression Ratio QualitySizeRatio Raw TIFF1153KB1:1 Zipped TIFF982KB1.2:1 Q=100331KB3.5:1 Q=7067KB17:1 Q=4043KB27:1 Q=1016KB72:1 Q=16KB192:1

49. NUS.SOC.CS5248 OOI Wei Tsang Magic of JPEG Throw away information we cannot see Color information “Low-frequency signals” Rearrange data for good compression Use standard compression

50. NUS.SOC.CS5248 OOI Wei Tsang Throw away color information Y VU

51. NUS.SOC.CS5248 OOI Wei Tsang Color Sub-sampling 4:4:4 4:2:2 4:2:0 4:1:1

52. Original Image (1153KB)

55. NUS.SOC.CS5248 OOI Wei Tsang Discrete Cosine Transform DCT demo applet and source code is available on the web.

56. NUS.SOC.CS5248 OOI Wei Tsang Quantization 24265 -54-10 235 -4-2 136 21 35 -2 88 88 88 816 88 8 32 64 308 -6 20 00 10 00 00 00 /=

57. NUS.SOC.CS5248 OOI Wei Tsang Differential Coding 308 6 20 00 10 00 00 00 253 21 10 00 40 00 10 00 273 21 10 00 40 00 10 00 308 6 20 00 10 00 00 00 -53 21 10 00 40 00 10 00 23 21 10 00 40 00 10 00

58. NUS.SOC.CS5248 OOI Wei Tsang Zig-zag ordering 273 21 10 00 40 00 10 00 27, 3, 2, 4, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0

59. NUS.SOC.CS5248 OOI Wei Tsang Run-Length Encoding 273 21 10 00 40 00 10 00 27, 3, 2, 4, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0 (27, 1) (3, 1) (2, 1), (4, 1), (1, 2), (0, 5), (1, 1), (0, 4)

60. NUS.SOC.CS5248 OOI Wei Tsang Idea: Motion JPEG Compress every frame in a video as JPEG DVD-quality video = 208.6GB Reduction ratio = 27:1 Final size = 7.7GB

61. NUS.SOC.CS5248 Ooi Wei Tsang Video Compression

63. NUS.SOC.CS5248 OOI Wei Tsang Temporal Redundancy

64. NUS.SOC.CS5248 OOI Wei Tsang Motion Estimation

65. NUS.SOC.CS5248 OOI Wei Tsang Bi-directional Prediction

66. NUS.SOC.CS5248 OOI Wei Tsang H.261 I-Frame P-Frame

67. NUS.SOC.CS5248 OOI Wei Tsang MPEG-1 B-Frame

68. NUS.SOC.CS5248 OOI Wei Tsang MPEG Frame Pattern Example display sequence: IBBPBBP … Example encoding sequence: IPBBPBB

69. NUS.SOC.CS5248 OOI Wei Tsang Compression Ratio Frame TypeTypical Ratio I10:1 P20:1 B50:1

70. NUS.SOC.CS5248 OOI Wei Tsang Sequence sequence header: width height frame rate bit rate :

71. NUS.SOC.CS5248 OOI Wei Tsang GOP: Group of Picture gop header: time :

72. NUS.SOC.CS5248 OOI Wei Tsang Picture pic header: number type (I,P,B) :

73. NUS.SOC.CS5248 OOI Wei Tsang Picture

74. NUS.SOC.CS5248 OOI Wei Tsang Slice

75. NUS.SOC.CS5248 OOI Wei Tsang Macroblock

76. NUS.SOC.CS5248 OOI Wei Tsang Block YY YY U V 1 Macroblock =

77. NUS.SOC.CS5248 OOI Wei Tsang MPEG in daily life MPEG Standards Bit-rateUsage MPEG-11.5MbpsVCD MPEG-210-40 Mbps DVD, SVCD, HDTV MPEG-4Scalable QuickTime, DivX;)

78. NUS.SOC.CS5248 OOI Wei Tsang Impact on Systems Design How to package data into packets? How to deal with packet loss? How to deal with bursty traffic? How to predict decoding time? :