1. Adaptive Content-Aware Scaling for Improved Video Streaming. Avanish Tripathi Advisor: Mark Claypool Reader: Bob Kinicki

2. Outline Introduction  Motivation  Related Work Methodology Experiments Results Conclusions and Future Work

3. Motivation Internet disseminates enormous amounts of information TCP is the de facto standard… but TCP is not ideal for multimedia and… 77% of all Web traffic is Multimedia, of this about 33% is streaming content. [Chandra, Ellis ’99]

4. Multimedia Flows… …tend to use UDP with no congestion control Other network protocols are being developed: TFRC: smooth reduction in rates as against abrupt drops in TCP [Floyd et. al. ’00] RAP: Architecture for delivery of layered encoded streams. [Rejaie et. al. ’99] MPEG-TFRCP: Mapping MPEG to TFRC Protocol [Miyabayashi et. al. ’00] Idea-rate based with smooth increase and decrease

5. Multimedia issues Generally very high bandwidth requirements Random packet drop by routers during congestion is detrimental to perceptual quality due to interdependencies between packets Need application level solution…

6. …Media Scaling Need media scaling: Application level data- rate reduction Scaling types:  Temporal  Quality  Spatial “Content of the stream should influence the choice of scaling mechanism” To the best of our knowledge this idea has not yet been employed

7. Related Work Quality Scaling: Receiver-driven Layered Multicast [McCanne ’96] Temporal Scaling:  Player for adaptive MPEG Streaming [Walpole et. al. ‘97]  Better Behaved Better Performing MM networking [Chung, Claypool ‘00] Content based forwarding for differentiated networks: use priorities based on MPEG characteristics [Shin et. al. ’00] Filtering System: used for media scaling of MPEG streams. [Yeadon ’96]

8. Outline Introduction  Motivation  Related Work Methodology Experiments Results Conclusions and Future Work

9. Methodology: Content-Aware Scaling Develop and verify motion measurement mechanism Define temporal and quality scaling levels Evaluate the potential impact of content- aware scaling Build system to do content-aware scaling adaptively Evaluate the practical impact of the full system

10. MPEG Overview Three kinds of pictures  I- Intra encoding  P- Predictive encoding  B- Bi-directional predictive encoding Subdivided into Macroblocks  Intra, predictive, interpolated macroblocks Motion vectors are used for motion compensation

11. Motion Measurement Higher percentage of interpolated macroblocks means low motion Lower percentage of interpolated macroblocks means high motion Conducted a pilot study to verify our hypothesis Divide frame into 16 sub-blocks Count the number of blocks that have motion Correlate that with the percentage of Interpolated macroblocks.

12. Pilot Study Result: Motion Measurement

13. Motion Computation Keep latency low so that the system is sufficiently reactive

14. Methodology: Content-Aware Scaling Develop and verify motion measurement mechanism Define temporal and quality scaling levels Evaluate the potential impact of content- aware scaling Build system to do content-aware scaling adaptively Evaluate the practical impact of the full system

15. Filtering We extend the system developed at Lancaster university  Frame dropping filter (Temporal Scaling)  Requantization filter (Quality Scaling)

16. User Study Details 22 graduate and undergraduate students in the department Platform: 3 Pentium III machines with 128MB RAM running Linux Clips were on local hard drives Four ~10 second clips (2 high motion, 2 low motion) Users rated the clips with numbers from 0 - 100

17. User Study Details Five versions of each clip: Perfect, Temporal Level 1, Temporal Level 2, Quality Level 1, Quality Level 2

18. Methodology: Content-Aware Scaling Develop and verify motion measurement mechanism Define temporal and quality scaling levels Evaluate the potential impact of content- aware scaling Build system to do content-aware scaling adaptively Evaluate the practical impact of the full system

19. Results Four men sitting at a bar  Low Motion ( 70 % interpolated macroblocks)

20. Results A girl walks across a room while talking on the phone  Low Motion (57% interpolated Macroblocks)

21. Results Rodeo scene where a man on horseback tries to rope a bull  High Motion (27% interpolated macroblocks)

22. Results Car commerical  High Motion (20% interpolated macroblocks)

23. Methodology: Content-Aware Scaling Develop and verify motion measurement mechanism Define temporal and quality scaling levels Evaluate the potential impact of content- aware scaling Build system to do content-aware scaling adaptively Evaluate the practical impact of the full system

24. Full System Architecture MPEG Server Input Motion Measurement High Low Temporal Filter Quality Filter Internet Feedback Generator Client

25. System Functionality Server is capable of quantifying motion as the movie plays The filtering system has five scale levels for finer granularity The system is adaptive and scales movies in real-time depending on the loss pattern as received from the feedback module

26. User Study Four clips (2 or more scene) ~30 seconds Four versions of each  Perfect Quality  Temporal scaling  Quality scaling  Adaptive scaling Bandwidth distribution functions: how often the rate changes Every 3 seconds Every 200ms Fit the scale values(1 through 5) on a normal curve [Floyd ‘00]

27. Future Work Accurately determine the threshold below which temporal scaling is unacceptable More accurate bandwidth distribution function Hybrid scaling methods (Quality + Temporal) Audio Scaling

28. Conclusions Application level solution to the problem of congestion due to unresponsive video streams Developed a mechanism to quantify the amount of change in a video stream Shown that content aware scaling can improve user perceived quality by as much as 50% Developed a system to do adaptive content- aware scaling and are in the process of determining it impact on user perceived quality