1. Communication & Multimedia C. -Y. Tsai 2005/12/15 1 Vidwav Wavelet Video Coding Specifications Student: Chia-Yang Tsai Advisor: Prof. Hsueh-Ming Hang Institute of Electronics, NCTU

2. Communication & Multimedia C. -Y. Tsai 2005/12/15 2 Outline Framework Framework Main modules Main modules Motion Motion Temporal transform Temporal transform Spatial transform Spatial transform Entropy coding Entropy coding Bitstream formation Bitstream formation Additional modules Additional modules Base layer Base layer In-band coding In-band coding Wavelet ringing reduction Wavelet ringing reduction Conclusions Conclusions

3. Communication & Multimedia C. -Y. Tsai 2005/12/15 3 References Microsoft Research Asia + ENST + INRIA, “ Vidwav Wavelet Video Coding Specifications ”, ISO/IEC JTC1/SC29/WG11, M12339, Poznan, July. 2005.

4. Communication & Multimedia C. -Y. Tsai 2005/12/15 4 Framework

5. Communication & Multimedia C. -Y. Tsai 2005/12/15 5 T+2D (Pre-spatial decomposition is void)

6. Communication & Multimedia C. -Y. Tsai 2005/12/15 6 2D+t

7. Communication & Multimedia C. -Y. Tsai 2005/12/15 7 Motion ME/MC Modes/MV coding

8. Communication & Multimedia C. -Y. Tsai 2005/12/15 8 ME/MC Adopted from H.264/AVC. Adopted from H.264/AVC. Performed by minimizing the Lagrangian cost function Performed by minimizing the Lagrangian cost function J= D + λR J= D + λR

9. Communication & Multimedia C. -Y. Tsai 2005/12/15 9 MV Coding Median prediction Median prediction For squre MC block size For squre MC block size 16x16, 8x8, 4x4 16x16, 8x8, 4x4 (ABC) or (ABD) (ABC) or (ABD) Directional segmentation prediction Directional segmentation prediction If predictor is not available, use median prediction If predictor is not available, use median prediction 16x8 8x16 8x44x8

10. Communication & Multimedia C. -Y. Tsai 2005/12/15 10 Temporal Transform Motion aligned temporal filtering Prediction step Update step

11. Communication & Multimedia C. -Y. Tsai 2005/12/15 11 Motion Aligned Temporal Filtering Lifting structure Lifting structure MAP (motion aligned prediction) MAP (motion aligned prediction) MAU (motion aligned update) MAU (motion aligned update) SplitMAPMAUMAPMAUMerge

12. Communication & Multimedia C. -Y. Tsai 2005/12/15 12 Motion Aligned Temporal Filtering (Cont.) Temporal lifting steps of 5/3 biorthogonal wavelet Temporal lifting steps of 5/3 biorthogonal wavelet

13. Communication & Multimedia C. -Y. Tsai 2005/12/15 13 Prediction Bidirectional connection Bidirectional connection Uni-directional connection Uni-directional connection When left-unidirectional connected When right-unidirectional connected

14. Communication & Multimedia C. -Y. Tsai 2005/12/15 14 Prediction (Cont.) OBMC OBMC

15. Communication & Multimedia C. -Y. Tsai 2005/12/15 15 Update Generated from H-frames through MC Generated from H-frames through MC Clipping Clipping

16. Communication & Multimedia C. -Y. Tsai 2005/12/15 16 Spatial Transform

17. Communication & Multimedia C. -Y. Tsai 2005/12/15 17 Structure Decomposition structure Decomposition structure Description Description S<>S<>E<>E<>E<>E<>E<>E<>E<> S<> S<> S<> E<> E<> E<> E<>E<> E<> E<> S<> E<> E<> E<> E<>

18. Communication & Multimedia C. -Y. Tsai 2005/12/15 18 Entropy Coding 3D-EBCOT Bitstream

19. Communication & Multimedia C. -Y. Tsai 2005/12/15 19 EBCOT Embedded Block Coding Embedded Block Coding Code block Fractional bitplanes Bitplanes   coding pass

20. Communication & Multimedia C. -Y. Tsai 2005/12/15 20 Coding Pass Operation Previous bit-plane Current bit-plane Insignificant sample Significant sample In previous bit-plane Significant sample In current bit-plane Significant Propagation Pass Magnitude Refinement Pass Normalization Pass ZC ZC ZC SC MR ZC SC

21. Communication & Multimedia C. -Y. Tsai 2005/12/15 21 Bitstream N of total bitplanes in the block N of total bitplanes in the block The coded bitstream of the block, which consists of 3N-2 segments, each segment corresponding to the output bitstream of one coding pass. The coded bitstream of the block, which consists of 3N-2 segments, each segment corresponding to the output bitstream of one coding pass. The length of each bitstream segment and/or the position of end-point of each coding pass. The length of each bitstream segment and/or the position of end-point of each coding pass. The R-D slope information at the end of each coding pass. The R-D slope information at the end of each coding pass.

22. Communication & Multimedia C. -Y. Tsai 2005/12/15 22 Bitstream Formation Syntax Bitstream selection

23. Communication & Multimedia C. -Y. Tsai 2005/12/15 23 Syntax Global Global GOP GOP Group of picture for MCTF Group of picture for MCTF Layer Layer For quick bitstream truncation For quick bitstream truncation Packet Packet One component (Y, U, or V) of one temporal sunbbad One component (Y, U, or V) of one temporal sunbbad Subband Subband 3D subband 3D subband Block Block Block of 3D EBCOT Block of 3D EBCOT Pass Pass Coding pass Coding pass

24. Communication & Multimedia C. -Y. Tsai 2005/12/15 24 Bitstream Selection Optimized Truncation Optimized Truncation RD slope Block n Block n+1 iteration m iteration m-1 RnRn+1 ΣRi <= bitrate constrain Coding pass index

25. Communication & Multimedia C. -Y. Tsai 2005/12/15 25 Base Layer

26. Communication & Multimedia C. -Y. Tsai 2005/12/15 26 Embedded Base Layer Codec

27. Communication & Multimedia C. -Y. Tsai 2005/12/15 27 Motion Information Prediction BASESKIP mode BASESKIP mode Use base-layer motion vectors as motion predictors Use base-layer motion vectors as motion predictors Macroblock partitions mode map rule to generate the predictors for higher spatial resolution

28. Communication & Multimedia C. -Y. Tsai 2005/12/15 28 Hierarchical B-Pictures Provide extra- temporal resolutions Provide extra- temporal resolutions

29. Communication & Multimedia C. -Y. Tsai 2005/12/15 29 In-Band Coding Leaky motion compensation Mode-based temporal filtering

30. Communication & Multimedia C. -Y. Tsai 2005/12/15 30 In-Band MCTF The forming of different quality reference of LL The forming of different quality reference of LL Low quality reference as IP_DIR Low quality reference as IP_DIR High quality reference as IP_LBS High quality reference as IP_LBS

31. Communication & Multimedia C. -Y. Tsai 2005/12/15 31 In-Band MCTF Leaky motion compensation Leaky motion compensation leaky factor leaky factor Attenuate the prediction based on the unknown information at the decoder Attenuate the prediction based on the unknown information at the decoder make a good trade-off between drifting errors and coding efficiency make a good trade-off between drifting errors and coding efficiency

32. Communication & Multimedia C. -Y. Tsai 2005/12/15 32 In-Band MCTF Mode-based temporal filtering Mode-based temporal filtering Mode I: Low quality reference Mode I: Low quality reference Mode 2: High quality reference Mode 2: High quality reference Mode is selected by RD cost Mode is selected by RD cost

33. Communication & Multimedia C. -Y. Tsai 2005/12/15 33 Wavelet Ringing Reduction

34. Communication & Multimedia C. -Y. Tsai 2005/12/15 34 Description Purpose Purpose Reduce artifact after EDWT Reduce artifact after EDWT Similar like the de-blocking filter in DCT based coding Similar like the de-blocking filter in DCT based coding After experiments, After experiments,

35. Communication & Multimedia C. -Y. Tsai 2005/12/15 35 Conclusions

36. Communication & Multimedia C. -Y. Tsai 2005/12/15 36 Wavelet Based SVC Advantages Advantages Nature for multi-resolution scalability Nature for multi-resolution scalability Open-loop prediction structure Open-loop prediction structure Provides elegant SNR scalability without impairing full exploitation of spatial-temporal correlation Provides elegant SNR scalability without impairing full exploitation of spatial-temporal correlation Simplifies the R-D model of the bitstreams. Simplifies the R-D model of the bitstreams. Facilitates the bitstream truncation Facilitates the bitstream truncation each subband is independent with other subbands each subband is independent with other subbands

37. Communication & Multimedia C. -Y. Tsai 2005/12/15 37 Wavelet Based SVC Disadvantages Disadvantages Decomposition modes (coding modes) selection Decomposition modes (coding modes) selection Texture & side information trade off Texture & side information trade off Intra-prediction Intra-prediction Badly-matched blocks Badly-matched blocks Downsampling filter problems Downsampling filter problems

38. Communication & Multimedia C. -Y. Tsai 2005/12/15 38 Thanks for your attention! Any questions?