Using Interleaving to Ameliorate the Effects of Packet Loss in a Video Stream Mark Claypool and Yali Zhu Computer Science Department Worcester Polytechnic Institute Worcester, MA, USA

May 2003 MNSA'03, Providence, RI, USA2 Multimedia over the Internet Often cannot use TCP –Interactive – retransmissions add delay –Multicast – retransmissions reduce scalability Use UDP –Packet loss deteriorates quality –Forward Error Correction (FEC) not effective under bursty loss –Single loss propagates with video

May 2003 MNSA'03, Providence, RI, USA3 Video Encoding - MPEG Frame types –I-frame (Intra-coded frame) –P-frame (Predictive-coded frame) –B-frame (Bi-directionally predictive frames) GOP is I-frame to next I-Frame How does loss of I- or P-frame effect GOP?

May 2003 MNSA'03, Providence, RI, USA4 Loss Propagation in MPEG Loss of single frame can result in multiple perceived losses –Loss of P-frame –Loss of I-frame –B-frame loss has no propagation

May 2003 MNSA'03, Providence, RI, USA5 Outline Introduction  Interleaving  Evaluation Conclusions

May 2003 MNSA'03, Providence, RI, USA6 Video Interleaving

May 2003 MNSA'03, Providence, RI, USA7 Benefits of Video Interleaving

May 2003 MNSA'03, Providence, RI, USA8 Bandwidth Costs of Interleaving Inter-frame encoding based on similarity among frames Interleaving decreases similarities among consecutive frames –Results in bigger B- and P- frames Our studies  about 15% bandwidth overhead

May 2003 MNSA'03, Providence, RI, USA9 Delay Costs of Interleaving Must delay for size of GOP frames Example: –Typical GOP 9 frames –At full-motion, about 3 GOPs / second 1 GOP every 300 ms –Interleaving would add 300 ms of delay

May 2003 MNSA'03, Providence, RI, USA10 Outline Introduction  Interleaving  Evaluation  Conclusions

May 2003 MNSA'03, Providence, RI, USA11 User Study on Perceptual Quality Simulate loss results –Loss Rates: no loss, 2%, 5%, 10%, 20% Encode videos: –24 video clips total –Variety of television programming Ex: sports, sit-com, news Variety of scene motion Watch videos: –About 40 users –Varied order of lossy clips evaluation Evaluate: –Perceived quality on unlabeled scale, 1-100

May 2003 MNSA'03, Providence, RI, USA12 Perceptual Quality versus Loss Rate

May 2003 MNSA'03, Providence, RI, USA13 Comparison with Other Repair Methods

May 2003 MNSA'03, Providence, RI, USA14 Conclusions on Video Interleaving Re-ordering of original frames can reduce perceived effects of loss –Loss propagation inherent in video compression –Bursty loss Significantly improves perceptual quality –Better under bursty loss than FEC Costs are bandwidth and delay –Bandwidth comparable to other approaches –Delay larger, but may be tolerable

May 2003 MNSA'03, Providence, RI, USA15 Future Work Combine repair techniques –FEC + Interleaving + Retransmissions Adaptive system that chooses best repair –Depends upon task –Depends upon network conditions Combine with audio –Video-Audio Interleaving