1. Region-Of-Interest Based Rate Control Scheme For High Efficiency Video Coding
Marwa Meddeb, Marco Cagnazzo, and B’eatrice Pesquet-Popescu
Institut Mines-T’el’ecom; T’el’ecom ParisTech; CNRS LTCI
2014 IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP)

2. Outline Introduction Related Works Proposed Approach
Experimental Results
Conclusion

3. Introduction
Rate control (RC) is an important tool that helps to deal with bit rate and compressed media quality fluctuations.
New HEVC standard has been recently finalized by ITUT and ISO/IEC [6] and many works have focused on rate control and developed new R-Q schemes for it.
[6] G. J. Sullivan, J.-R. Ohm, W.-J. Han, and T. Wiegand, “Overview of the high efficiency video coding (hevc) standard,” IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 12, pp. 1649–1668, 2012.

4. Introduction(Cont.)
In the reference software two different schemes have been proposed.
Based on a quadratic model and the mean absolute difference (MAD) between the original and the reconstructed signal [7] [8].
An R-λ model that takes into account the hierarchical coding structure has been adopted [9].
[7] H. Choi, J. Nam, J. Yoo, and D. Sim, “Rate control based on unified RQ model for HEVC,” in JCT-VC H0213, 8th meeting of Joint Collaborative Team on Video Coding of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, San Jos’e, CA, USA, 2012.
[8] ——, “Improvement of the rate control based on pixel-based URQ model for HEVC,” in JCT-VC I0094, 9th meeting of Joint Collaborative Team on Video Coding of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Geneva, CH, 2012.

5. Introduction(Cont.)
All the above-mentioned RC algorithms developed for HEVC do not take into account the importance of some regions of the frame.
We propose a new rate control scheme for videoconferencing systems at low bit rates which processes the faces and the background separately.

6. Related Works
In video coding, RC usually incorporates rate distortion optimization (RDO). Knowing the QP given by the rate control, RDO consists in minimizing the cost
D(QP) : distortion
λMODE : Lagrange multiplier
R(QP) : number of bits

7. Related Works(Cont.)

8. Related Works(Cont.)
In [14], the traditional linear model that was employed in TM5 for high bit rate video coding is studied for HEVC
C1 and C2 are model parameters.

9. Related Works(Cont.)
The most recent rate-distortion model in the HEVC test software is the R-λ model expressed as follows[9]:
α and β are model parameters.
[9] B. Li, H. Li, L. Li, and J. Zhang, “Rate control by R-lambda model for HEVC,” in JCT-VC K0103, 11th meeting of Joint Collaborative Team on Video Coding of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Shanghai, CN, 2012.

10. Related Works(Cont.)
Fig. 1. Rate control scheme

11. Proposed Approach
The proposed approach is based on the R-λ model for HEVC.
The relationship between R and λ is used to compute QP of the frame and each CU of the image.

12. Proposed Approach(Cont.)
Fig. 2. ROI-based rate control scheme

13. Proposed Approach(Cont.)
Region bit allocation
Frame level :
K : desired ratio between the ROI and non-ROI bit rates.
T : number of bits.
M : total number of pixels of the frame.
Pn : area of non-ROI.

14. Proposed Approach(Cont.)
CU level :
Tr’: effective number of bits of already encoded CUs of the ROI.
Ir : set of indexes of ROI CU that have not yet been coded.
ωCUr : weight of the current CU of the ROI.

15. Proposed Approach(Cont.)
Region independent rate control models
Once the number of allocated bits for each CU is initialized, the QP is computed using the R-λ model[9].
α’, β’ and γ’ : old values of α, β and γ.

16. Proposed Approach(Cont.)
QP and λ variation
The last modification consists in considering new clipping ranges for λ and QP, at CU level.
We can make different clipping for CUs of ROI and the other CUs.

17. Experimental Results
We implemented the proposed rate control scheme in the HM.10 of HEVC.
To compute the ROI map, we used the Viola and Jones object detection method [11].
A reference test is performed using the rate control algorithm described in [9].
[9] B. Li, H. Li, L. Li, and J. Zhang, “Rate control by R-lambda model for HEVC,” in JCT-VC K0103, 11th meeting of Joint Collaborative Team on Video Coding of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Shanghai, CN, 2012.
[11] P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR, vol. 1, 2001, pp. I–511–I–518.

18. Experimental Results(Cont.)
“Johnny” and “Vidyo4” (1280x720 resolution with frame rate 60fps).
Tested clipping ranges are (ΔQPPic, Δ QPCU) ∈ {(2, 1); (3, 2)}
Target bit rate evaluated is 128 kbps.
The tests are performed using a low delay B configuration with a GOP size equal to 4.

21. Conclusion
The scheme achieves better visual quality in ROIs thanks to an independent processing at CU level of the two regions and a larger QP clipping range.
The proposed algorithm shows better quality in ROI, while respecting the global bit rate constraint.

22. Xin Deng, Mai Xu, and Zulin Wang
A ROI-based Bit Allocation Scheme for HEVC towards Perceptual Conversational Video Coding
Xin Deng, Mai Xu, and Zulin Wang
2013 Sixth International Conference on Advanced Computational Intelligence
October 19-21, 2013, Hangzhou, China

23. Outline Introduction Related Works Proposed Approach
Experimental Results
Conclusion

24. Introduction
The bit allocation scheme in HEVC is not adaptive to subjective video quality, which is of vital importance to improve the quality of human experience (QoE).
We propose in this paper a simple yet effective bit allocation scheme (namely PBAS) to improve the subjective quality of conversational video coding of HEVC.

25. Introduction(Cont.)
For H.263, a perceptual rate control scheme [4] was proposed to vary QP, which is dependent on perceptual sensitive weight map.
For H.264, a ROI-based bit allocation method [6] was proposed to optimize the QPs in conversational video coding.
[4] X. Yang, W. Lin, Z. Lu, X. Lin, S. Rahardja, E. Ong, and S. Yao, "Rate control for videophone using local perceptual cues", IEEE Transations on Circuits and Systems for Video Technology, vol. 15, no. 4, Apr
[6] Y. Liu, Z. G. Li, and Y. C. Soh, "Region-of-interest based resource allocation for conversational video communication of H.264/AVC", IEEE Transations on Circuits and Systems for Video Technology, vol. 18, no. 1, Jan

26. Proposed Approach
Face detection for extracting perceptual ROI regions.
In this paper, we apply a robust speed up robust features (SURF) cascade face detector [10] to extract the ROI regions for our conversational video coding.
Evaluation on perceptual performance of conversational video coding of HEVC.
HEVC adopts a similar coding structure as H.264/ AVC.
The ill-suited bit allocation problem which may cause an unsatisfactory visual perception performance.
[10] H. Baya, A. Essa, T. Tuytelaarsb, and L. V. Goola, "Speeded-Up Robust Features (SURF)", Computer Vision and Image Understanding, vol. 110, no. 3, Jun

27. Proposed Approach(Cont.)

28. Proposed Approach(Cont.)
Bit allocation scheme for conversational video coding of HEVC
We utilize the simple rate-distortion model [12], but with different optimization objectives of distortion over ROI and non-ROI regions, for improving the subjective perceptual performance in ROI regions.
[12] J. Ribas and S. Lei, "Rate control in dct video coding for low-delay communications", IEEE Transations on Circuits and Systems for Video Technology, vol. 9, no. 1, pp , Feb

29. Proposed Approach(Cont.)
σ : standard deviations of pixel values.
A : number of pixels.
D : perceptual distortion.
w : relative importance of ROI regions over non-ROI regions.
B : target bit-rate we have for the conversational video compressed by HEVC.
N = AI + AO.

30. Experimental Results
The proposed PBAS scheme has been implemented in three test sequences:
“Akiyo”, “Foreman” and “Carphone”.

31. Fig, 3. The rate-distortion performance comparison between conventional and the proposed PBAS schemes in HEVC for ROI, non-ROI and whole regions in three sequences.

32. Fig. 4. The 47th decoded frame for Foreman at various bit rates:
(a) (PSNRF, PSNRB, PSNRw )=(32.48,32.72,32.64)dB (b) (PSNRF, PSNRB, PSNRw)=(35.13,29.43,30.36)dB
Fig. 4. The 47th decoded frame for Foreman at various bit rates:
(a) Conventional scheme at 60kbps, (b) PBAS at 60kbps,
(e) Conventional scheme at 100kbps,
(f) PBAS at 100kbps.
(e) (PSNRF, PSNRB, PSNRw )=(34.70,34.42,34.43)dB (f) (PSNRF, PSNRB, PSNRw)=(37.48,31.13,32.16)dB

33. Fig. 5. The comparison of rate-distortion performance using PBAS scheme when w is set to 8 and 6, for ROI, non-ROI and whole regions in three sequences.

34. Conclusion
In this paper, we have proposed a perceptual bit allocation scheme, for improving the subjective visual quality of conversational video on the HEVC platform.
Our experimental results on compressing the standard conversational videos with HEVC have verified that the PBAS scheme performs better in terms of the PSNR in ROI regions, against the conventional rate control scheme of HEVC.