1. A Sequence-Based Rate Control Framework for Consistent Quality Real-Time Video Bo Xie and Wenjun Zeng CSVT 2006

2. Outline Introduction Global Bit-allocation Target Bit Calculation Quantization Parameter Determination Simulation Results

3. Introduction (1) Rate control  Bit-allocation  Bit-allocation achievement (QP determination) MPEG-2 TM5, MPEG-4 Annex L, and H.263+ TMN8  GOP-based bit-allocation  Fixed size GOP  Constant bit-allocation among GOPs and among the same type frames  Some assumptions are necessary Stationary video sequence Similar characteristics of all GOPs  Fixed-size GOPs do not match the scene structure

4. Introduction QP determination  Different R-Q models are developed  There is no guarantee that a model is always accurate (model mismatch) Resulting in buffer overflow or underflow

5. Introduction (3) Problem Formulation Subject to # of frames Distortion for the i th frame with QP q i Average distortion Actual bits for the i th frame with QP q i Bit rate/Frame rate Buffer sizeBuffer fullness

6. introduction (4) Goal: Constant quality  Allocating more bits to high complexity scenes/frames, and less bits to low complexity scenes/frames Coding complexity: The number of bits required to encode a frame Three stages  Global bit-allocation model  Target bit calculation  QP determination

7. Global Bit-allocation (1) Relation between rate and complexity   R-J relation The variance of the i th MB in the j th frame Optimal target bit rate for the j th frame Bit budget for that GOP # of pixels for each MB# of MBs for each frame Average rate for the m th frame (bit/pixel) The energy of the j th frame Intensity of the j th residue pixel of the i th MB Average residue intensity of the i th MB Mean deviation Average bits for motion vectors of a frame

8. Global Bit-Allocation (2) Problems of traditional models  They cannot differentiate Intra and Inter frames (Too less bits for Intra frames) Proposed R-MAD model  Value of original pels for intra MBs Value of residue pels for inter MBs Constant Shift factor  Is chosen as ½  R = K*(MAD) ½ If a frame has MAD > average MAD, and calculated QP < average QP, set QP = average QP

9. Target Bit Calculation (1) Calculation of K    represents the complexity of a scene/frame Scene change detection Stationary assumption is no more necessary Most existing GOP-based bit-allocation schemes use only past source data Worst case: the scenes get more and more complex or simpler Buffer output rate (bit rate/frame rate) MAD of the current frame Average MAD of all previous encoded frames Target bit count of the current frame

10. Target Bit Calculation (2) Adjustment by actual bit account   Achievement of constant quality Buffer constraint  Target bit count for the i th frameActual bit count for the i th frame BsBs VBV_fullness

11. Quantization Parameter Determination (1) Bit-allocation guarantee  A R-Q model is used to determine QP for the target bits  A traditional model-based QP determination has no bit-allocation guarantee Resulting in “error propagation” e.g. MPEG-4 Annex L  QP re-adjustment If |Actualbit - Targetbit|/Targetbit > Threshold, re- quantization is performed to achieve bit-allocation guarantee

12. Quantization Parameter Determination (2) Proposed R-Q model  Initial QP determination (similar to )  … n Window size n-Mn-M # of re-quantization timesActual # of bits used for the j th re-quantization of the p* th frame in the past Actual QP used for the q* th re-quantization of the p* th frame in the past R-Q model Most similar MAD Most similar QP

13. Quantization Parameter Determination (3) Proposed R-Q model  QP re-adjustment  Check convergence If re-adjusted QP cannot converge to the target bit count, QP_leftQP_rightQP_prev QP_leftQP_right QP_left Actualbit < Targetbit: Actualbit > Targetbit: QP_leftQP_rightQP_prev QP_new

14. Quantization Parameter Determination (4) Proposed R-Q model  Re-quantization algorithm  QP final sanity check Ifthen

15. Quantization Parameter Determination (5) Proposed R-Q model  Buffer overflow checking

16. Simulation Results (1) Sequences (QCIF)  Foreman (medium motion, one scene change)  Sea World (high motion, two scene changes)  Glasgow (lots of scene cuts)  Charlie’s Angels (high motion, lots of scene changes)

17. Simulation Results (2) Model failure rate and re-quantization times |Actualbit-Targetbit| / Targetbit > 30% MPEG-4 Annex L

18. Simulation Results (3) Frame dropping and PSNR MPEG-4 Annex L Proposed Proposed without re-quantization R-  model* *Z. He and S. K. Mitra, “A unified rate-distortion analysis framework for transform coding,” CSVT 2001.

19. Simulation Results (4) Quality smooth  Smaller buffer (0.5s)

20. Simulation Results (5) Buffer fullness (0.5s)

21. Simulation Results (6) Quality smooth  Larger buffer (2s)