1. Impact of Reference Distance for Motion Compensation Prediction on Video Quality ACM/SPIE Multimedia Computing and Networking (MMCN) San Jose, California, January 31, 2007 Yubing Wang, Mark Claypool and Robert Kinicki WPI Computer Science Department Worcester, MA 01609 Wang_yubing@emc.com

2. MMCN 07, Jan. 31, 2007 2 Introduction Packet loss degrades video quality. Reference Picture Selection (RPS) selects one of several previous frames as a reference frame. Distance from selected frame is reference distance Higher reference distance, lower quality and vice versa Two modes: NACK and ACK A systematic study of the effects of reference distance on video quality. A set of videos are selected Motion and scene complexity H.264 encoded (supports RPS) Two objective measures: PSNR VQM

3. MMCN 07, Jan. 31, 2007 3 Outline Introduction Background Hypothesis Methodology Results and Analysis Conclusions

4. MMCN 07, Jan. 31, 2007 4 Reference Picture Selection (ACK) The decoder acknowledges all correctly received frames. Only the acknowledged frames are used as a reference. Error propagation is avoided entirely. Reference distance increases with round-trip delay. Coding efficiency decreases as reference distance increases. Video quality degrades due to limited bit-rate. 1 2 345 67 ACK(1) ACK(2)ACK(3)

5. MMCN 07, Jan. 31, 2007 5 Reference Picture Selection (NACK) The previous frame is used as a reference during the error-free transmission. Reference distance is always 1 regardless of RTT The decoder sends a NACK for the erroneous frame along with a reference frame Error propagation Impact of loss increases with RTT NACK(3) 1 2 34 5678

6. MMCN 07, Jan. 31, 2007 6 RPS ACK or RPS NACK? Which to use? Depends upon loss and loss pattern Depends upon round-trip delay Depends upon the effects of reference distance Can quantify effects of loss and delay Need to quantify the effects of reference distance on video Quality Will help determine choice between NACK and ACK

7. MMCN 07, Jan. 31, 2007 7 Video Quality Measurement Video Quality Metrics: SSIM (Z. Wang et al), MPQM (C. Branden et al), … PSNR VQM (ITS) Video Quality Metric (VQM) Developed by the Institute for Telecommunication Science (ITS). Provides an objective measurement for perceived video quality based on user studies. Measures the perceptual effects of video impairments and combines them into a single metric. VQM models include: Television, Videoconferencing, General, Developer, PSNR.

8. MMCN 07, Jan. 31, 2007 8 Outline Introduction Background Hypothesis Methodology Results and Analysis Conclusions

9. MMCN 07, Jan. 31, 2007 9 Hypothesis The y-intersect is determined by motion and scene complexity. High-motion video sequences starts with low quality, degrade slower. Low-motion video sequence starts with high quality, degrade faster. Low Motion: The similarities among frames are high; More macro-blocks are inter-coded; High motion: The similarities among frames are low; More macro-blocks are intra-coded;

10. MMCN 07, Jan. 31, 2007 10 Outline Introduction Background Hypothesis Methodology Results and Analysis Conclusions

11. MMCN 07, Jan. 31, 2007 11 Methodology Select a set of non-compressed video clips with a variety of motion content. All in YUV 4:2:0, CIF (352x288) Each video sequence contains 300 video frames with a frame rate of 30 fps. Change reference distances for each selected video sequence. Encode the video clips using H.264. Only P-slices or I-slices are used. Same bit-rate constraint is imposed for all experiments. Only one single reference frame is used Measure video quality using PSNR and VQM. Analyze the results.

12. MMCN 07, Jan. 31, 2007 12 Video Clips Used in the Experiments Video ClipMotionFraction of P-Blocks Description Akyio Low0.9666A news reporter talking Container Low0.9246A container ship moving slowly News Low0.8746Two news reporters talking Silent Medium0.8637A person demonstrating sign language Mom & Daughter Medium0.8423A mother and daughter talking Foreman High0.5947A foreman talking Mobile High0.5722Panning of toy train moving Coastguard High0.5225Panning of a coastguard ship moving

13. MMCN 07, Jan. 31, 2007 13 Outline Introduction Background Hypothesis Methodology Results and Analysis Conclusions

14. MMCN 07, Jan. 31, 2007 14 PSNR vs. Reference Distance (1) The video quality for videos with high motions tends to degrade slower than that for those videos with low motion.

15. MMCN 07, Jan. 31, 2007 15 PSNR vs. Reference Distance (2) Video ClipsabR- Squared Akiyo-2.011647.9650.9953 Container-1.902344.8380.9948 News-1.855643.2950.9984 Silent-1.528341.410.9929 Mom & Daughter -1.458141.4420.9904 Foreman-1.168138.5110.9265 Mobile-1.155326.6630.9754 Coastguard-0.862635.5820.9975 The relationship between PSNR and reference distance can be characterized using a logarithmic function:

16. MMCN 07, Jan. 31, 2007 16 VQM vs. Reference Distance (1) We use (1-VQM) as the quality metric. The same trend as the ones with PSNR.

17. MMCN 07, Jan. 31, 2007 17 VQM vs. Reference Distance (2) Video ClipsabR- Squared Akiyo-0.01130.98470.9869 Container-0.01140.97660.9848 News-0.01150.97320.9931 Silent-0.01240.96060.9937 Mom & Daughter -0.00850.92170.9821 Foreman-0.00680.90590.9779 Mobile-0.00220.80550.9076 Coastguard-0.00140.84230.9671 The relationship between VQM and reference distance can be characterized using a linear function:

18. MMCN 07, Jan. 31, 2007 18 Conclusion Both PSNR and VQM video quality degrade as reference distance increases. The degree of the video quality degradation is affected by the video content. High-motion video sequences starts with lower quality, degrade slower. Low-motion video sequences starts with higher quality, degrade more rapidly. Mathematical Characterization of the relationship between video quality and reference distance: PSNR: VQM:

19. MMCN 07, Jan. 31, 2007 19 Future Work Include a broader set of video clips. Develop an analytical model to compare RPS NACK and ACK mode: Use reference distance results! Packet loss rate Round-trip delay Bit-rate constraints Insight to other repair techniques.