1. Improved input-state linearization in video bitrate controllers Noam Korem

2. Presentation outline Video encoding and rate control Classic models Suggested improvement Simulation results Summary and conclusions Video encoding and rate control Classic R-Q models Suggested improvement Simulation results Summary / Conclusions

3. Generic video encoder* IP i-2i-1 P i P i+1 DCT VLC..01001.. q i-2 + - Iq i-2 IDCT DCT VLC..01001.. q i-1 Iq i-1 IDCT + + 0 + + Iq i IDCT + + + - DCT qiqi VLC..01001.. - + DCT q i+1 VLC..01001.. IntraPredicted Discrete Cosine Transform Video encoding and rate control Classic R-Q models Suggested improvement Simulation results Summary / Conclusions

4. Discrete Cosine Transform (DCT) Resembles Discrete Fourier Transform Purely real (real transform to real) Allows representation in the frequency domain, usually more compact Spatial domainFrequency domain

5. Quantization Quantization scale = 1Quantization scale = 31

6. Generic video encoder with rate control mechanism Frame encoder bits target bit rate Non-linear Bit rate controller qiqi SiSi Linear controller q i =Q(T i,..) TiTi Non linear bitrate controller (Input-state linearization)

7. The problem P i-1 P i Iq i-1 IDCT + + Iq i IDCT + + + - DCT qiqi VLC..01001.. - + DCT q i+1 VLC..01001.. Energy of difference frame i is dependent on the q(i-1) Coded information of frame i is dependent on q(i), q(i-1) Video encoding and rate control√ Classic R-Q models Suggested improvement Simulation results Summary / Conclusions

8. The problem, cont. Classic Rate-Quantization models do not depend on reference frame quantization scale

9. Improved R-Q model i-1 i-1(q) i Q i-1 Δ Qi-1 Δ RiRi QiQi Q i-1 Video encoding and rate control√ Classic R-Q models√ Suggested improvement Simulation results Summary / Conclusions

10. Previous frame quantizer dependency R i (Q i-1 ), Q i =8R i (Q i )

11. Improved R-Q model, cont.

12. Simulations Encode real videos, compare accuracy of traditional model (TM5) against improved model ffmpeg open-source video encoder with enhanced quantization and bit rate control used as encoding model (MPEG4). All test videos are YUV 4:2:0, QVGA (320x240), encoded at 370kbps (74CR) Video encoding and rate control√ Classic R-Q models√ Suggested improvement√ Simulation results Summary / Conclusions TM5 model:

13. Simulations Absolute prediction error is calculated for each frame, for both models Delta of absolute prediction errors is used as performance measurement

14. Simulation results Mean Abs Err* TM5: 7.18% TM5i: 4.8% (33% improve) Mean Abs Err* TM5: 17.07% TM5i: 15.03% (12% improve)

15. Simulation results, cont. Mean Abs Err* TM5: 22.3% TM5i: 19% (14.7% improve) Mean Abs Err* TM5: 16.7% TM5i: 14.1% (15.6% improve)

16. Simulation results, cont. Mean Abs Err* TM5: 34.2% TM5i: 32.2% (6% improve)

17. Simulation results, degradation Mean Abs Err* TM5: 13.1% TM5i: 13.2% (1% degrade)

18. Summary & Conclusions Video encoding with rate controller scheme presented (camcorders, live streaming) Video rate and distortion depend on quantization of reference frame, classic models ignore. Improved R-Q model allows more accurate input-state linearization, demands more calculations Improved R-Q model does not deliver when scene complexity changes rapidly Video encoding and rate control√ Classic R-Q models√ Suggested improvement√ Simulation results√ Summary / Conclusions

19. Future work Prediction quality degrades due to input noise – changes in scene complexity More accurate complexity estimation (more accurate prediction for X i ) based on:  frame content  motion estimation would improve R-Q model accuracy Try the model improvement on other R-Q models A complete model will include past quantization parameters R i =R i (Q i,Q i-1,Q i-2,…)

20. Related work Video Group, "Test Model 5" JTC1/SC29/WG11 Coding of Moving Pictures and associated Audio MPEG 1994, section 10 Tihao Chiang and Ya-Qin Zhang "A New Rate Control Scheme Using Quadratic Rate Distortion Model", IEEE International Conference on Image Processing, 1996. 10.1109/ICIP.1996.560604 Liang-Jin Lin; Ortega, A. “Bit-rate control using piecewise approximated rate-distortion characteristics” Circuits and Systems for Video Technology, IEEE Transactions on, Volume 8, Issue 4, Aug 1998 Page(s):446 - 459 10.1109/76.709411 Saw, Y.-S.; Grant, P.M.; Hannah, J.M., "Rate-distortion analysis of nonlinear quantisers for MPEG videocoders: sigmoidal and unimodal quantiser control functions" Vision, Image and Signal Processing, IEE Proceedings- Volume 145, Issue 4, Aug 1998 Page(s):249 – 256 Ma, S.; Wen Gao; Yan Lu; "Rate-distortion analysis for H.264/AVC video coding and its application to rate control“ Circuits and Systems for Video Technology, IEEE Transactions on Volume 15, Issue 12, Dec. 2005 Page(s):1533 - 1544 10.1109/TCSVT.2005.857300

21. Many thanks: Shai Mazor Orly Wigderson Kobi Kohai For the guidance, patience and assistance on all aspects of the project