Kai-Chao Yang Hierarchical Prediction Structures in H.264/AVC

Outline Analysis of Hierarchical B Pictures and MCTF ICME 2006 Multiple Description Video Coding using Hierarchical B Pictures ICME 2007 Rate-Distortion Optimization for Fast Hierarchical Picture Transcoding ISCAS 2006 All Related Researches

Heiko Schwarz, Detlev Marpe, and Thomas Wiegand ICME 2006 Analysis of Hierarchical B Pictures and MCTF

Hierarchical B-Pictures (1/2) Key pictures Hierarchical prediction structures Dyadic structure Non-dyadic structure IDR I/P GOP … Hierarchical prediction ……………………

Hierarchical B-Pictures (2/2) Coding delay Minimum coding delay = hierarchy levels – 1 Memory requirement Maximum decoded picture buffer (DPB): 16 Reference picture buffering type Sliding window Adaptive memory control Memory management control operation (MMCO) o0: End MMCO loop o1: mark a Short-term frame as “Unused” o2: mark a Long-term frame as “Unused” o3: assign a Long-term index to a frame o4: specify the maximum Long-term frame index o5: reset Minimum DPB size = hierarchy levels Coding order … … N-2 N-1 N Short-term frames Long-term frames NewNew OldOld replace Thomas Wiegand, “Joint Committee Draft (CD),” Joint Video Team, JVT-C167, 6-10 May, 2002 Frame buffer

Coding Efficiency of Hierarchical B- Pictures QP k = QP k-1 + ( k=1 ? 4:1 ) Problem : PSNR fluctuations High spatial detail and slow regular motionFast and complex motion

Visual Quality Comparison of visual quality Finer detailed regions of the background using larger GOP sizes. IBBPGOP 16

MCTF Versus Hierarchical B-Pictures Drawbacks of MCTF Open-loop encoder control Significant cost in update stage

Minglei Liu and Ce Zhu ICME 2007 Multiple Description Video Coding using Hierarchical B Pictures

Concept of Multiple Description Coding Multiple bit-streams are generated from one source signal and transmitted over separate channels MDC encoder Decoder 1 Decoder 2 Decoder 3 Channel 1 Channel 2 MDC decoder S1 S2 Source signal Decoded signal from S1 Decoded signal from S1 and S2 Decoded signal from S2

The proposed architecture for MDC GOP size = 8 Two output streams (S1, S2) are generated GOP S1 S2 Combination …… ii+8 i+1 i+9 ii+8 i+1 i+9i+3i+5i+7i+6i+4i+2

Coding Efficiency (1/2) Improvement of coding efficiency Increasing QP values for higher layers Transmitting MVs only for higher layers Skipping frames at higher layers

Coding Efficiency (2/2) Central distortion Side distortion Max. QP = 51 for highest level

Huifeng Shen, Xiaoyan Sun, Feng Wu, and Shipeng Li ISCAS 2006 Rate-Distortion Optimization for Fast Hierarchical Picture Transcoding

Rate Reduction Transcoding (1/3) Cascaded pixel-domain transcoding structure Fully decoding the original signal, and then re-encoding it A. Vetro, C. Christopoulos, and H. Sun, "Video transcoding architectures and techniques: an overview", IEEE Signal processing magazine, March 2003.

Rate Reduction Transcoding (2/3) Open-loop transcoding in coded domain Partially decoding the original signal and re-quantizing DCT coefficients drift A. Vetro, C. Christopoulos, and H. Sun, "Video transcoding architectures and techniques: an overview", IEEE Signal processing magazine, March 2003.

Rate Reduction Transcoding (3/3) Closed-loop transcoding with drift compensation Partially decoding the original signal, and then compensating the re-quantized drift data A. Vetro, C. Christopoulos, and H. Sun, "Video transcoding architectures and techniques: an overview", IEEE Signal processing magazine, March 2003.

Hierarchical B Pictures Transcoding Open-loop transcoding method can be used Motion information is unchanged; DCT coefficients are truncated, re-quantized, or partially discarded Drift inside a GOP will not propagate to other GOPs However, motions are more important in hierarchical B- pictures structure At low bit-rate, most bits are spent on motion information Proposed RDO model – combination of texture RDO and motion RDO

Traditional Rate-Distortion Model RD model S = ( S 1, …, S k ) denotes k MBs I = ( I 1, …, I k ) denotes k coding parameters of S Fully decoding and re-encoding is needed!

Proposed Rate-Distortion Model (1/4) Proposed RD model Claim R texture : rate spent in coding quantized DCT coefficients R motion : rate spent in coding MB modes, block modes, and MVs D texture : distortion caused by downscaled texture with unchanged MVs D motion : distortion caused by motion adjustment relative to the unchanged motion case

Proposed Rate-Distortion Model (2/4) Texture RDO model To minimize the RD function,  Let  N.Kamaci, Y. Altunbasak, and R.M. Mersereau, "Frame bit allocation for the H.264/AVC video coder via Cauchy-density-based rate and distortion models", IEEE Trans. on CSVT, Vol 15, No. 8, Aug

Proposed Rate-Distortion Model (3/4) Motion RDO model R motion can be easily computed, but D motion is unknow D motion can be approximated by mv mean-square error A. Secker and D. Taubman, "Highly scalable video compression with scalable motion coding", IEEE Trans. on Image Processing, Vol. 13, No.8, August 2004.

Proposed Rate-Distortion Model (4/4) Motion adjustment Original Adjustment … …

Simulation results

All related researches Rate control optimization Bit allocation Trade-off between coding efficiency and delay Multi-view Temporal scalable coding in SVC Elimination of PSNR fluctuation? More efficient hierarchical structures?