1 Dude, Where is My Packet?

NUS.SOC.CS5248 OOI WEI TSANG 2 Overview Characteristics of the Internet General techniques Error recovery for audio Effect of loss on MPEG Error recovery for MPEG

3 Loss Characteristics of The Internet

NUS.SOC.CS5248 OOI WEI TSANG 4 Characteristics of Internet 60-70% of paths do not show any loss Those with loss have an average of 4.5 – 6% packet loss [Paxson97] End-to-end Internet packet dynamics

NUS.SOC.CS5248 OOI WEI TSANG 5 Packet Loss Pattern

NUS.SOC.CS5248 OOI WEI TSANG 6 Characteristic of Internet Bursts of loss are typically short (2-3 consecutively loss packets) Long burst do occur Burst may occur periodically

NUS.SOC.CS5248 OOI WEI TSANG 7 Wireless Link Loss rate measured in my office 10 ~ 20% up to 50% reported!

8 General Error Recovery Techniques Perkins, Hodson and Hardman IEEE Network Magazine 1998

NUS.SOC.CS5248 OOI WEI TSANG 9 Retransmission

NUS.SOC.CS5248 OOI WEI TSANG 10 Redundant Data

NUS.SOC.CS5248 OOI WEI TSANG Error Concealment

12 Retransmission for audio

NUS.SOC.CS5248 OOI WEI TSANG 13 Retransmission is Bad Need scalable feedback NACK consumes bandwidth May need to retransmit frequently Retransmission maybe useless

NUS.SOC.CS5248 OOI WEI TSANG 14 Retransmission is Good Only retransmit when needed Exact recovery

NUS.SOC.CS5248 OOI WEI TSANG 15 Scalable Retransmission On packet loss T = random(0, RTT) wait for T multicast NACK On receiving NACK from others suppress own NACK

NUS.SOC.CS5248 OOI WEI TSANG 16 Retransmit when group size is small loss rate is low large latency acceptable

17 Redundant Data for audio

NUS.SOC.CS5248 OOI WEI TSANG 18 Parity FEC XOR

NUS.SOC.CS5248 OOI WEI TSANG 19 Parity FEC

NUS.SOC.CS5248 OOI WEI TSANG 20 Parity FEC Ordering

NUS.SOC.CS5248 OOI WEI TSANG 21 Parity FEC Ordering

NUS.SOC.CS5248 OOI WEI TSANG 22 Parity FEC Ordering

NUS.SOC.CS5248 OOI WEI TSANG 23 Parity FEC Ordering

NUS.SOC.CS5248 OOI WEI TSANG 24 Parity FEC Ordering

NUS.SOC.CS5248 OOI WEI TSANG 25 Reed-Solomon Code RS(n,k) RS

NUS.SOC.CS5248 OOI WEI TSANG 26 Reed-Solomon Code

NUS.SOC.CS5248 OOI WEI TSANG 27 Media Specific FEC

NUS.SOC.CS5248 OOI WEI TSANG 28 Interleaving

NUS.SOC.CS5248 OOI WEI TSANG 29 Pro and Cons Media Independent FEC Overhead for redundant data Exact recovery Could be computationally costly Media Specific FEC Approximate recovery only

30 Error Concealment

NUS.SOC.CS5248 OOI WEI TSANG 31 Effect of Loss on Audio Speech Human ears can interpolate Loss up to length of phoneme can still be tolerable

NUS.SOC.CS5248 OOI WEI TSANG 32 Insertion-based Repair Splice Silence Substitution Noise Substitution Repetition

NUS.SOC.CS5248 OOI WEI TSANG 33 Other Repair Methods Interpolation Regeneration

NUS.SOC.CS5248 OOI WEI TSANG 34 Pro and Cons Could be computationally costly Approximate recovery only Does not work well for long burst of packet loss

35 Colin’s Recommendations

NUS.SOC.CS5248 OOI WEI TSANG 36 Non-Interactive Apps Interleaving FEC Retransmission for unicast only

NUS.SOC.CS5248 OOI WEI TSANG 37 Interactive Applications Media Specific FEC

NUS.SOC.CS5248 OOI WEI TSANG 38 Error Concealment Repeat

39 Effects of Packet Loss on MPEG Jill Boyce and Robert Gaglianello ACM Multimedia 1998

NUS.SOC.CS5248 OOI WEI TSANG 40 Measurement-based Study Need to understand the problem before proposing solution

NUS.SOC.CS5248 OOI WEI TSANG 41 Data Gathering Method From: NYC 13, Austin 21, London 18 To: Holmdel, NJ

NUS.SOC.CS5248 OOI WEI TSANG 42 Data Gathering Method Video: Two 5-mins MPEG 30 fps 384 kbps and 1 Mbps IBBPBBPBBPBBPBB QSIF 176x112 and SIF 352x240 One row per slice

NUS.SOC.CS5248 OOI WEI TSANG 43 Average Packet Loss

NUS.SOC.CS5248 OOI WEI TSANG 44 Packet Loss vs Time

NUS.SOC.CS5248 OOI WEI TSANG 45 How to Fill Packets B 1 frame 1 packet P 1 slice 1 packet I 1 slice 1 packet

NUS.SOC.CS5248 OOI WEI TSANG 46 Packet Size Distributions

NUS.SOC.CS5248 OOI WEI TSANG 47 Packet Size vs Loss Rate

NUS.SOC.CS5248 OOI WEI TSANG 48 Lesson 1 slice per packet even if larger than MTU If smaller packets means higher loss rate: fill packet until MTU Else one slice per MTU Alt: Change size of slice

NUS.SOC.CS5248 OOI WEI TSANG 49 Frame Effected by Errors

NUS.SOC.CS5248 OOI WEI TSANG 50 Correlation of Packet Loss

51 Overview of Error Recovery for Video

NUS.SOC.CS5248 OOI WEI TSANG 52 Methods Retransmission Error Concealment FEC Limiting Error Propagation Reference Frame Selection

NUS.SOC.CS5248 OOI WEI TSANG 53 Reference Frame Selection I P B B P X I

NUS.SOC.CS5248 OOI WEI TSANG 54 Methods Retransmission Error Concealment FEC Limiting Error Propagation Reference Frame Selection Changing Temporal Pattern

55 Changing Temporal Patterns Injong Rhee SIGCOMM ‘98

NUS.SOC.CS5248 OOI WEI TSANG 56 Basic Idea “Better Late Than Never!”, or Late packet is still useful

NUS.SOC.CS5248 OOI WEI TSANG 57 MPEG Frame Pattern IBBPBBP IPBBPBB

NUS.SOC.CS5248 OOI WEI TSANG 58 H.261 Error Propagation IPPPPPP XXXXXX IPPPPPP XXXX retransmission loss

NUS.SOC.CS5248 OOI WEI TSANG 59 H.261 Frame Pattern IPPPPPP X loss

NUS.SOC.CS5248 OOI WEI TSANG 60 IPPPPPP H.261 Frame Pattern X loss X X retransmission

NUS.SOC.CS5248 OOI WEI TSANG 61 PTDD Periodic Temporal Dependency Distance Large PTDD Later Deadlines Longer Error Propagations Less Temporal Dependencies

NUS.SOC.CS5248 OOI WEI TSANG 62 QAL IPPP IPPP Base Layer Enhancement Layer

NUS.SOC.CS5248 OOI WEI TSANG 63 QAL IPPP IPPP Base Layer + FEC Enhancement Layer X

NUS.SOC.CS5248 OOI WEI TSANG 64 QAL + PTDD IPPP IPPP Base Layer Enhancement Layer

65 SR-RTP Feamster and Balakrishnan PV’02

NUS.SOC.CS5248 OOI WEI TSANG 66 Overview An analytical model for packet loss SR-RTP Post Processing

NUS.SOC.CS5248 OOI WEI TSANG 67 Analytic Model p: prob. packet loss f : observed frame rate : frame drop rate S i : mean num. of packets in frames of type i P(i) : prob. frame type is i P(F) : prob. frame is useless

NUS.SOC.CS5248 OOI WEI TSANG 68 p: prob. packet loss f : observed frame rate : frame drop rate S i : mean num. of packets in frames of type i P(i) : prob. frame type is i P(F) : prob. frame is useless

NUS.SOC.CS5248 OOI WEI TSANG 69 Is it accurate?

NUS.SOC.CS5248 OOI WEI TSANG 70 SR-RTP Extensions to RTP for selective reliability

NUS.SOC.CS5248 OOI WEI TSANG 71 SR-RTP 0Length ADU Sequence Number ADU Length ADU Offset PriorityLayer Number

NUS.SOC.CS5248 OOI WEI TSANG 72 Feamster & Balakrishnan’s ADU 1 ADU = 1 Frame

NUS.SOC.CS5248 OOI WEI TSANG 73 Loss Detection LEN: 50 ADU: 0 ADU Size: 100 ADU Off: 0 LEN: 20 ADU: 0 ADU Size: 100 ADU Off: 80 LEN: 50 ADU: 1 ADU Size: 150 ADU Off: 40 LEN: 50 ADU: 3 ADU Size: 50 ADU Off: 0

NUS.SOC.CS5248 OOI WEI TSANG 74 Retransmission Request Based on priority Example: I-Frame: highest priority P-Frame: higher priority if closer to prev I B-Frame: no retransmition

NUS.SOC.CS5248 OOI WEI TSANG 75 Error Concealment for Video Repeat pixels from previous frame Interpolate pixels from neighbouring region Interpolate motion vectors from previous frame

NUS.SOC.CS5248 OOI WEI TSANG 76 Inferred MV for P Frame IP

NUS.SOC.CS5248 OOI WEI TSANG 77 Inferred MV for I Frame PPI

NUS.SOC.CS5248 OOI WEI TSANG 78 Evaluation Peak Signal to Noise Ratio (PSNR) over average square of differences in pixel values Calculated in dB (10 log 10 )

NUS.SOC.CS5248 OOI WEI TSANG 79 Performance of SR-RTP

80 Today’s Summary

NUS.SOC.CS5248 OOI WEI TSANG 81 How to recover packet loss Retransmission FEC Error Concealment

NUS.SOC.CS5248 OOI WEI TSANG 82 How to limit the damage of error Interleaving Key frame selection Change reference frame