1. On Combining Temporal Scaling and Quality Scaling for Streaming MPEG Huahui Wu, Mark Claypool, Robert Kinicki Computer Science, Worcester Polytechnic Institute

2. NOSSDAV0605/22/2006 Introduction  Streaming video –Bitrate > Network capacity Media Scaling –Temporal Scaling (TS) or Quality Scaling (QS) –Packet loss Forward Error Correction (FEC)  Operations Research algorithm –To satisfy capacity constraint –To maximize perceived quality  Previous research –Temporal Scaling [TOMCCAP 05] –Quality Scaling [NOSSDAV 05]  This work –Combines Temporal Scaling and Quality Scaling

3. NOSSDAV0605/22/2006 Outline  Introduction  Model  –Streaming Bitrate (cost) –Video Quality (benefit)  Algorithm  Experiments  Conclusions

4. NOSSDAV0605/22/2006 System LayersParameters & Variables MPEG Scaling and FEC Network System Layers and Parameters

5. NOSSDAV0605/22/2006 Streaming Bitrate  Total streaming bitrate, including video packets and FEC packets: where G is the constant GOP rate N PD and N BD are the numbers of transmitting P and B frames depending on Temporal Scaling level l TS

6. NOSSDAV0605/22/2006  Two distortion factors –Frame Loss Caused by Temporal Scaling and network packet loss Appears jerky in the video playout Measured by Playable Frame Rate –Quantization Distortion Caused by a high quantization value with Quality Scaling Appears visually as coarse granularity in every frame Measured by ITS-VQM  Overall Quality –Distorted Playable Frame Rate Video Quality - Overview [Wu+ 05 TOMCCAP] [Pinson+ 04]

7. NOSSDAV0605/22/2006 Playable Frame Rate (R)  Frame Successful Transmission Probability –Where Frame Size  Frame Dependencies  Total Playable Frame Rate

8. NOSSDAV0605/22/2006  Quality scaling distortion varies exponentially with the quantization level  Distorted Playable Frame Rate Distorted Playable Frame Rate (R D ) [Frossard+ 01]

9. NOSSDAV0605/22/2006 Algorithm  For each Repair and Scaling combination Estimate video frame sizes (S I, S P, S B ) –Compute streaming bitrate B and make sure it’s under capacity constraint T –Use frame sizes and FEC amount to get successfully frame transmission rate (q I, q P, q B ) Compute playable frame rate (R) Estimate quality scaling distortion (D) –Compute distorted playable frame rate (R D )  Exhaustively search all FEC and Scaling combination and look for the optimal quality

10. NOSSDAV0605/22/2006 Outline  Introduction  Model  Algorithm  Experiments   Conclusions

11. NOSSDAV0605/22/2006 Methodology  Built a function RD() –Returns the distorted playable frame rate  Built an optimization program –TCP-Friendly Bitrate Constraint (T) –Searches repair and scaling levels for the highest RD  Compared three scaling choices with Adjusted FEC –Temporal Scaling –Quality Scaling –Temporal Scaling + Quality Scaling  Compared four FEC choices with the combination of Temporal Scaling and Quality Scaling –Non-FEC –Small Fixed FEC –Large Fixed FEC –Adjusted FEC

12. NOSSDAV0605/22/2006 System Setting  GOP: IBBPBBPBBPBBPBB Network LayerMPEG Layer t RTT 50 msNPNP 4 frames per GOP s1 KbyteNBNB 10 frames per GOP p0.005 to 0.08RFRF 30 frames per sec

13. NOSSDAV0605/22/2006 Low Motion High Motion Scaling Comparison with Adjusted FEC

14. NOSSDAV0605/22/2006 FEC Comparison with TS+QS

15. NOSSDAV0605/22/2006 Conclusions  Summary –Derives analytical models for streaming video with Temporal plus Quality Scaling and FEC –Uses OR algorithm to optimize the quality –Compares scaling methods and FEC methods  Conclusions –Quality Scaling is more effective than Temporal Scaling But when bandwidth is low and network loss is high, Quality Scaling should be used with Temporal Scaling Motion matters –Adjusting FEC improves video streaming quality significantly Better than fixed FEC and non-FEC

16. On Combining Temporal Scaling and Quality Scaling for Streaming MPEG Huahui Wu, Mark Claypool, Robert Kinicki Computer Science, Worcester Polytechnic Institute