Optimum Bit Allocation and Rate Control for H.264/AVC Wu Yuan, Shouxun Lin, Yongdong Zhang, Wen Yuan, and Haiyong Luo CSVT 2006

Outline Introduction Rate-Distortion Modeling Rate Distortion Optimization Macroblock-Layer Rate Control Experimental Results –Comparisons with JVT-G012

Introduction (1) H.264/AVC Rate Controller Intra/Inter prediction Residual

Introduction (2) The Chicken and Egg Dilemma Rate Control QP RDO MAD Coding (Intra/Inter mode selection) !? (Residual calculation) Solution: Guess! MAD =  MAD prev +  MAD prev MAD

Rate-Distortion Modeling (1) Overhead Bit-Rate Prediction –Overhead bits: QP, MV, MB mode, … –Using history Coding Complexity Prediction –Complexity: Residual (MAD) –Using history R-D Behavior Prediction –Using overhead bits, residual, and initial QP Distortion Prediction –Using history Residual (org - pred) Overhead (MV, QP, … ) Encoded frame Entropy coding

Rate-Distortion Modeling (2) Overhead Bit-Rate Prediction –JVT-G012: H i = H i ave – Proposed: H i = H i prev n -1 n

Rate-Distortion Modeling (3) Coding Complexity Prediction –MAD =  MAD prev +  Data points are first selected by the spatial and temporal distance, and then removing outliers. JVT-G012 n -1 n RPE < 3% QPJVT-G012Proposed %23.05% %58.12%

Rate-Distortion Modeling (4) R-D Behavior Prediction –Assumption: DCT coefficients of residual can be approximated by –Distortion   (Laplace distribution) * A. Viterbi and J. Omura, Principles of Digital Comuunicatin and Coding. New York: McGraw-Hill Electrical Engineering Series,1979 * Taylor expansion Residual rate Let  /D = 1+x

Rate-Distortion Modeling (5) R-D Behavior Prediction –Taylor expansion of R(Qstep) at Qstep ave  – Rate of the i th MB: QP max QP ave QP min (Rate of residual) Rate of residualRate of overhead

Rate-Distortion Modeling (6) Distortion Prediction –Assumption 1: Distortion of DCT coefficients is uniformly distributed D =  2 = |x-y| 2 /12  (Qstep 2 /12) –Assumption 2: Qstep i  Qstep i prev Qstep i  (D i prev /Qstep i prev ) D i  Qstep i 2  (Qstep i D i prev /Qstep i prev )  D i =  i  Qstep i, where  i = D i prev /Qstep i prev Set as 1 (For scalability)

Rate Distortion Optimization Consider D i =  i  Qstep i for the i th MB – subject to  (Lagrange theory) Rate of the i th MB

Macroblock-Layer Rate Control (1)

Macroblock-Layer Rate Control (2)

Macroblock-Layer Rate Control (3) Rate Controller 1.Initialization: QP ave = QP start, QP min = max(QP ave -3, 0), and QP max = min(QP ave +3, 51). Let i = 0 2.Optimum Bit Allocation for i th MB: 1) k = i. 2)R-D modeling: 3)Optimum Computations: 4) k = k +1. If k  N, jump to 2). 5)Compute optimal QP i * : (overhead bits) (check if > T when adding with H k ) (by linear regression)

Macroblock-Layer Rate Control (4) Rate Controller 3.Adjust QP i * : QP i * = max{QP i-1 * -1, min{QP i *,QP i-1 * +1}}. Then QP i * = max{1,QP ave -3, min{51,QP ave +3,QP i * }} 4.Encoding 5.Update (Reducing blocking effect) (For smoothness)

Experimental Results (1) CIF: Mobile, Paris Rate Prediction Error Ratio:

Experimental Results (2)

Experimental Results (3)