Rate-Distortion Optimized Layered Coding with Unequal Error Protection for Robust Internet Video Michael Gallant, Member, IEEE, and Faouzi Kossentini, Senior Member, IEEE IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, MARCH 2001

Outline Introduction Background Layered coding Packetization Scheme Error-Concealment Method Prioritization Approach Proposed Method Statistical Distortion Measure Rate-Distortion Optimized Mode-Selection algorithm Experimental Results Conclusion

Introduction(1/2) How to facilitate video communications for different networks Layered coding Network congestion and buffer overflow lead to packets being delayed and discarded Retransmission protocols Forward-error correction(FEC) techniques Performing error concealment through post- processing

Introduction(2/2) This paper present an effective framework for video communication based on the layered coding, packetization, error concealment, and packet prioritization Providing a rate-distortion optimized mode- selection algorithm Want to select the coding mode for each block

Background Layered Coding Layered coding, as supported in H.263+ allows the source for prediction to be selected at the macroblock level The corresponding reconstructed base-layer macroblock A motion compensated macroblock from the previous enhancement-layer reconstruction The linear interpolation of the two This paper employ a fully standard-compliant H.263+ layered video-encoding algorithm

Background Packetization Scheme(1/3) A typical packet consists of header information for IP, UDP, RTP, and RTP payload Header requires about 40 bytes/packet Large packet sizes to reduce packetization overhead The maximum size the packet on the internet should be 1500 bytes

Background Packetization Scheme(2/3) Pay-load data are from one row of macroblocks(GOB) per packet to one entire coded frame per packet Good decoder error-concealment Reduce over head We can interleaving even and odd GOBs into separate packets

Background Packetization Scheme(3/3)

Background Error-Concealment Method(1/4) Packet loss must be de detected Using the sequence number Resynchronization markers can provide spatial error-resilience. For example ： H.263+ GOB header Error concealment in video communications — spatial and temporal domain This paper uses the median estimate for motion compensation

Background Error-Concealment Method(2/4) This paper generating two packets per coded frame for QCIF resolution and four packets per coded frame for CIF resolution

Background Error-Concealment Method(3/4) For layered scenarios, we can use base-layer information to estimate enhancement layer information Previous enhancement layer can also used for error concealment Base layer is inter-coded -> employ median estimator from enhancement layer Base layer is intra-coded -> using the available base- layer reconstruction

Background Error-Concealment Method(4/4)

Background Prioritization Approach(1/2) For layered coding, base layer has higher priority than enhancement layer Using unequal protection to achieve transport priortization (n,k) code, k data packet, n-k parity packet

Background Prioritization Approach(2/2)

Proposed method Want to select the coding mode for each block Optimal allocation of bit rate must consider Source coding elements Channel coding elements Also introduce error-resilience ability

Proposed method Statistical Distortion Measure Distortion occur from Packet loss and error propagated via motion compensation Residual packet loss probabilities (n,k) A macroblock lost probability Prediction layer Statistical distortion Macro block Current layer Given frame Relative weights

Proposed method Rate-Distortion Optimized Mode-Selection algorithm For base layer, four coding modes ： skipped, inter, intra and inter4v For enhancement layer, five coding modes ： skipped, inter-forward, inter-upward, inter-bidirectional, and intra mode Quantization distortion Distortion for prediction from a corrupted Coding mode Quantization level used to control the bit rate Source coding rate Channel coding rate

Experimental results(1/4)

Experimental results(2/4)

Experimental results(3/4)

Experimental results(4/4)

Conclusion Providing an effective framework for robust internet video communications based on the principle of layered coding with transport prioritization A rate-distortion optimized mode- selection algorithm