Investigation of Motion-Compensated Lifted Wavelet Transforms Information Systems Laboratory Department of Electrical Engineering Stanford University Markus Flierl and Bernd Girod

Flierl and Girod: Investigation of MC Lifted Wavelet Transforms April 23, Outline Motion-compensated wavelet coding scheme Experimental results for temporal Haar and 5/3 wavelets Mathematical model and performance bounds Comparison to predictive coding Can motion-compensated wavelet coding really do better than motion-compensated predictive coding? Why? Can motion-compensated wavelet coding really do better than motion-compensated predictive coding? Why?

Flierl and Girod: Investigation of MC Lifted Wavelet Transforms April 23, Motion-Compensated Wavelet Coder Temporal Decomposition Intraframe Coder DWT/ MC-Lifting Haar and 5/3 wavelet 16x16 block motion compensation ½ pixel accuracy 8x8 DCT coder Run-level entropy coding Same quantizer step-size in all frames H H H H LLL LLH LH Original frames

Flierl and Girod: Investigation of MC Lifted Wavelet Transforms April 23, Motion-Compensated Haar Wavelet Update step uses negative motion vector of corresponding prediction step Even frames Odd frames Low High

Flierl and Girod: Investigation of MC Lifted Wavelet Transforms April 23, Motion-Compensated 5/3 Wavelet Update steps uses negative motion vectors of corresponding prediction steps Frame 0 Frame 1 Frame 2 Low High Low

Flierl and Girod: Investigation of MC Lifted Wavelet Transforms April 23, R-D Performance of M.C.Wavelet Coder R [kbit/s] PSNR Y [dB] 5/3, K=32 Haar, K=32 Haar, K=16 Haar, K=8 Haar, K=2 Mother & Daughter, QCIF, 30 fps +

Flierl and Girod: Investigation of MC Lifted Wavelet Transforms April 23, R-D Performance of M.C. Wavelet Coder R [kbit/s] PSNR Y [dB] 5/3, K=32 Haar, K=32 Haar, K=16 Haar, K=8 Haar, K=2 Mobile & Calendar, QCIF, 30 fps

Flierl and Girod: Investigation of MC Lifted Wavelet Transforms April 23, Mathematical Model Can we explain the experimental findings by a mathematical model? Yes! Extend rate-distortion analysis of motion-compensated hybrid coding to motion-compensated subband coding B. Girod, "Efficiency Analysis of Multi-Hypothesis Motion-Compensated Prediction for Video Coding," IEEE Trans. Image Processing, vol. 9, no. 2, pp , February B. Girod, "Motion-Compensating Prediction with Fractional-Pel Accuracy," IEEE Transactions on Communications, vol. 41, no. 4, pp , April B. Girod, "The Efficiency of Motion-compensating Prediction for Hybrid Coding of Video Sequences," IEEE Journal on Selected Areas in Communications, vol. SAC-5, no. 7, pp , August 1987.

Flierl and Girod: Investigation of MC Lifted Wavelet Transforms April 23, Equivalent Motion-Compensated Haar Transform Assume invertible motion compensation operations Even frames Odd frames Low High

Flierl and Girod: Investigation of MC Lifted Wavelet Transforms April 23, Equivalent Motion-Compensated Haar Transform

Flierl and Girod: Investigation of MC Lifted Wavelet Transforms April 23, Mathematical Model of Motion-Compensated Transform vnoise-free picture kk k-th displacement error nknk k-th noise signal ckck k-th motion-compen- sated signal ykyk k-th transform signal Any input picture can be reference picture

Flierl and Girod: Investigation of MC Lifted Wavelet Transforms April 23, Coding Gain of Motion-Compensated Transform Rate difference for each picture k Difference of rate-distortion functions at high rates Compares m.c. transform coding to independent coding of frames... for the same mean squared reconstruction error... for Gaussian signals Total rate difference

Flierl and Girod: Investigation of MC Lifted Wavelet Transforms April 23, Rate Difference with Negligible Noise Calibration:  = 0.5 log 2 (12  2  ) Integer-pel  =0 Half-pel  =-1 Quarter-pel  =-2

Flierl and Girod: Investigation of MC Lifted Wavelet Transforms April 23, Rate Difference with White Noise Calibration:  = 0.5 log 2 (12  2  ) Integer-pel  =0 Half-pel  =-1 Quarter-pel  =-2 White noise at -30 dB

Flierl and Girod: Investigation of MC Lifted Wavelet Transforms April 23, Comparison to Predictive Coding Predictive coding scheme: Motion-compensated hybrid coder (like MPEG, H.263,... ) 16x16 block motion compensation with half-pel accuracy Previous reference frame only Intra-frame coding with 8x8 DCT and run-length coding Only one I-frame in the beginning of the sequence Same quantizer step-size for all P-frames Same components as motion-compensated wavelet coding scheme

Flierl and Girod: Investigation of MC Lifted Wavelet Transforms April 23, Comparison to Predictive Coding Mother & Daughter, QCIF, 30 fps

Flierl and Girod: Investigation of MC Lifted Wavelet Transforms April 23, Comparison to Predictive Coding Mobile & Calendar, QCIF, 30 fps

Flierl and Girod: Investigation of MC Lifted Wavelet Transforms April 23, Comparison to Predictive Coding Calibration:  = 0.5 log 2 (12  2  ) Integer-pel  =0 Half-pel  =-1 Quarter-pel  = bits

Flierl and Girod: Investigation of MC Lifted Wavelet Transforms April 23, Conclusions Investigated motion-compensated wavelet transform followed by intra-frame coder both experimentally and theoretically Biorthogonal 5/3 wavelet outperforms Haar wavelet Wavelet transform can outperform predictive coding with single reference frame Theory offers insights and some possible explanations Rate can decrease up to 1 bpp per displacement accuracy step Gain by accurate motion compensation is limited by residual noise Motion-compensated transform can outperform predictive coding by up to 0.5 bpp, due to better noise suppression Long GOPs needed for wavelet subband decomposition