Scalable Wavelet Video Coding Using Aliasing- Reduced Hierarchical Motion Compensation Xuguang Yang, Member, IEEE, and Kannan Ramchandran, Member, IEEE IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 9, NO. 5, MAY 2000

Outline Introduction Basic derivation Basic system structure Backward/forward hybrid motion compensation Computational complexity Coding results Conclusion and future research

Introduction TQ Entropy coding Image block Transform Coefficients Zigzag Scan (2D->1D) Bitstream Encoder For Video Sequence Q -1 T -1 Reconstructed Transform Coefficients Reconstructed Image block MC - Aliasing-Reduced motion estimation Backward/forward motion estimation DWT DWT

Features of wavelet base video coding Support scalability Free from blocky artifacts

Basic Derivation Aliasing come from downsampling. t π2π-2ππ t π2π-2ππ t π2π-2ππ cos wt f s =2πf s =1/2π f s =3/2π f s =3π --- Aliasing

Basic Derivation What’s Aliasing? … f f2f3f f2f3f f

Basic Derivation

Aliasing Problem

Aliasing Reduction Using an Interpolation Filter Signal preservation Aliasing reduction

Aliasing Reduction Using an Interpolation Filter Input power spectral density Expectation

Optimal Solution Time domain

Optimal Solution

Basic System Structure Three Level wavelet transform Use frame difference coding

Basic System Structure Q & E 4x4 OBMC 5 Repeat to next stage

Basic Operations

Backward/forward hybrid motion compensation Reason Experiments have revealed a degraded performance at low bit rates and very complicated motion. Accuracy is dependent on the reconstruction quality of coarser frames.

Backward/forward hybrid motion compensation Zerotrees of Mode Selections Mode Optimization Dynamic Programming Algorithm Choose of λ m

Zerotrees of Mode Selections forward

Mode Optimization Initializing all the tree nodes to backward mode. Mode selection is performed as comparing the R-D Lagrangian Bottom-up dynamic programming strategy Note that the distortion here is the motion compensated error energy, not the final coded distortion.

Dynamic Programming Algorithm Backward cost Forward cost D f < D b Forward Backward Aggregated Lagrangian gain 1 2

Dynamic Programming Algorithm Backward cost Forward cost D f < D b Forward Backward 3

A Toy Example

Choice of λ m “Lagrangian compression ratio”

Choice of λ m is almost solely a function of Given a certain Find C l (λ) by training, and send it as side information

Dynamic Programming Algorithm 1 2 3

Computational Complexity Great computational savings can be achieved by taking advantage of the striking similarities between motion vectors in successive resolution levels, and between the backward and forward motion vectors. The increment is proportional to the square of the ratio between forward search range and backward search range, which is typically 20%–30%.

Computational Complexity The quadtree optimization algorithm. The total computation for the optimization is therefore of O(N) complexity ( N is the total number of pixels), which is negligible compared to the O(N 2 ) complexity of motion estimation. While the proposed coder saves complexity at the encoder, it requires an increase in decoder complexity.

Coding Results Direct estimation Interpolated estimation using the synthesis lowpass filter G 0 (w) Interpolated estimation using the L(w)

The backward motion compensated error energy on 100 frames of the football sequence at three resolution levels

Final coded PSNR for luminance versus frame number at 15 frames/s Use L(w) Use G 0 (w) H.263 with full option MaD 48kb/s, 15fs Missa 24kb/s, 15fs dB over H.263 Average 0.87dB

Final coded PSNR for luminance versus frame number at 15 frames/s

Final coded PSNR for luminance versus frame number at 30 frames/s MPEG-2 Propose method

Final coded PSNR for luminance versus frame number at 30 frames/s

Scalable decoding 0.5Mb/s

Comparison of final coded subjective quality H.263 at 48 Kb/sProposed Coder “mosquito” noise ?

Comparison of final coded subjective quality H.263 at 24 Kb/sProposed Coder “mosquito” noise ?

Comparison of final coded subjective quality MPEG-2 at 2Mb/sProposed Coder

Conclusions and future research Proposed coder alleviates the aliasing problem in motion estimation. Backward/forward hybrid motion compensation attack the instability problem caused by quantization noise. (2dB) Spatially scalable. Ringing effects as a result of wavelet transform coding.