1. PERFORMANCE COMPARISON OF DAALA AND HEVC By Rohith Reddy Etikala rohith.Etikala@mavs.uta.edu 1001049552

2. TABLE OF CONTENTS Acronyms and Abbreviations Goal of the Project Overview of High Efficiency Video Coding (HEVC) – HEVC Encoder and Decoder – Features of HEVC Partitioning Prediction Transform and Quantization Entropy Coding In-loop Filters Overview of DAALA Metrics used – Peak Signal to Noise Ratio (PSNR) – Bjontegaard Delta Bitrate (BD-BR) and Bjontegaard Delta PSNR (BD-PSNR) Profiles used

3. Acronyms AVC: Advanced Video Coding BD-BR: Bjontegaard Delta Bitrate. BD-PSNR: Bjontegaard Delta Peak Signal to Noise Ratio. CABAC: Context Adaptive Binary Arithmetic Coding. CTB: Coding Tree Block. CTU: Coding Tree Unit. CU: Coding Unit. DCT: Discrete Cosine Transform. fps: frames per second. HEVC: High Efficiency Video Coding. HM: HEVC Test Model. IEC: International Electro-technical Commission. ISO: International Organization for Standardization. ITU-T: International Telecommunication Union- Telecommunication Standardization Sector. JCT: Joint Collaborative Team. JCT-VC: Joint Collaborative Team on Video Coding. MAD: Mean Absolute Difference. MC: Motion Compensation. MPEG: Moving Picture Experts Group. MSE: Mean Square Error. PB: Prediction Block. PSNR: Peak Signal to Noise Ratio. QP: Quantization Parameter SAO: Sample Adaptive Offset. TB: Transform Block. TU: Transform Unit. VCEG: Video Coding Experts Group.

4. Goal of the Project The main goal of this project is to implement/build the reference software for both DAALA and HEVC, use various profiles and test sequences [31] with varying resolutions and compare the performance with different metrics such as PSNR, BD-PSNR [9] and SSIM[39]. Also, computational complexity of both the systems will be compared.

5. Overview of HEVC The High Efficiency Video Coding (HEVC) [12] is the latest video standard developed by Joint Collaborative Team on Video Coding (JCT-VC), a group of video coding experts from ITU-T Video Coding Experts Group and SO/IEC Moving Picture Experts Group (MPEG). As the demand for HD video (4K and 8K) increased, there is a need for stronger coding efficiency than H.264/AVC. Also, there is increased use of parallel processors. So, HEVC [25] has been introduced to support increased video resolution and parallel processing. HEVC obtains about 50% reduction in bit rate when compared to its predecessor H.264/AVC [1] at the same visual quality.

6. HEVC Codec Block Diagram of HEVC CODEC [11].

7. HEVC Encoder HEVC Encoder Diagram In-loop filters [32]

8. HEVC Decoder HEVC Decoder Block Diagram[21]

9. HEVC Tiles and Slices HEVC Slice and Tile Structure [3]. [21]

10. HEVC Block structure HEVC divides a picture into coding tree units(CTU) of 16x16,32x32, or 64x64 samples. The CTU can further be divided into smaller blocks using a quad tree structure, called a coding unit(CU), can further be split into prediction units(PU) and is also a root of transform quad tree.Each of the child nodes of the transform quad tree defines a transform unit(TU).The size of transforms used in the prediction error coding can vary from 4x4 to 32x32 samples. [25]

11. HEVC CTU Partitioning and Code Tree Structure CTU partitioning and code tree structure [18].

12. HEVC Intra Prediction modes Modes and directional orientations for intra prediction in HEVC [4].

13. HEVC Motion Estimation Process Illustration of Motion Estimation Process [27].

14. Other HEVC Features Transform and Quantization Residual CU is transformed by using block transforms such as DCT of sizes 32x32, 16x16 and 4x4. Then the transformed data is quantized [25]. Entropy Coding Context Adaptive Binary Arithmetic Coding (CABAC) is used to encode quantized transform coefficients, motion vector data and header data [25]. In-loop Filters HEVC employs in-loop filters such as deblocking and Sample Adaptive Offset (SAO) [3] to smoothen block transitions and remove ringing artifacts.

15. Overview of DAALA Daala is a video codec developed by Mozilla foundation, Xiph.Org foundation and other contributors [36]. The goal of this project is to provide a free implementation and performance superior to HEVC. Daala uses Lapped transforms rather than DCT. Perceptual vector quantization. Overlapping-block motion compensation. Chroma plane prediction from luma planes Multisymbol arithmetic encoding Time-frequency resolution switching.

16. Metrics Used Peak Signal to Noise Ratio (PSNR). Given a noise-free m x n monochrome image I and its noisy approximation K, MSE is defined as: PSNR is defined via the Mean Square Error (MSE). Here, MAX I is the maximum possible pixel value of the image. The PSNR is defined as:

17. Contd… Bjontegaard Delta Bitrate (BD-BR) and Bjontegaard Delta PSNR (BD-PSNR): To objectively evaluate the coding efficiency of video codecs, Bjontegaard Delta PSNR (BD-PSNR) was proposed. Based on the rate-distortion (R-D) curve fitting [8][9], BD-PSNR provides a good evaluation of the R-D performance SSIM The structure similarity (SSIM) index is a method for measuring the similarity between two images. The SSIM index is a full reference metric; in other words, the measurement of image quality based on an initial uncompressed or distortion-free image as reference. SSIM is designed to improve on traditional methods like peak signal-to-noise ratio (PSNR) and mean square error (MSE), which have proven to be inconsistent with human eye perception. Complexity The complexity in terms of encoding time for both HEVC and DAALA will be compared in addition to PSNR, SSIM and BD-PSNR.

18. Profiles For HEVC, HM 16.7 [17] main profile will be used for the project. Official Git repository code will be used for daala.

19. References [1] I.E.G. Richardson, “Video Codec Design: Developing Image and Video Compression Systems”, Wiley, 2002. [2] K.R. Rao, D.N. Kim and J.J. Hwang, “Video Coding Standards: AVS China, H.264/MPEG-4 Part 10, HEVC, VP6, DIRAC and VC-1”, Springer, 2014. [3] G. J. Sullivan et al, “Overview of the High Efficiency Video Coding (HEVC) Standard”, IEEE Trans. on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1649-1668, Dec. 2012. [4] F. Bossen et al, “HEVC Complexity and Implementation Analysis”, IEEE Trans. on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1685-1696, Dec. 2012. [5] J. R. Ohm et al, “Comparison of the Coding Efficiency of Video Coding Standards—Including High Efficiency Video Coding (HEVC)”, IEEE Trans. on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1669-1684, Dec. 2012. [6] X. Li et al, “Rate-complexity-distortion evaluation for hybrid video coding”, IEEE International Conference on Multimedia and Expo (ICME), pp. 685-690, July 2010. [7] B. Bross et al, “High Efficiency Video Coding (HEVC) Text Specification Draft 10”, Document JCTVC- L1003, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC), Mar. 2013 available on http://phenix.it- sudparis.eu/jct/doc_end_user/current_document.php?id=7243 [8] J. Vanne et al, “Comparative Rate-Distortion-Complexity Analysis of HEVC and AVC Video Codecs”, IEEE Trans. on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1885-1898, Dec. 2012. [9] G. Bjontegaard, “Calculation of Average PSNR Differences between RD Curves”, document VCEG- M33, ITU-T SG 16/Q 6, Austin, TX, Apr. 2001. [10] D. Grois et al, “Performance Comparison of H.265/ MPEG-HEVC, VP9, and H.264/ MPEG-AVC Encoders”, available on: http://iphome.hhi.de/marpe/download/Performance_HEVC_VP9_X264_PCS_2013_preprint.pdf [11] HEVC tutorial by I.E.G. Richardson: http://www.vcodex.com/h265.html [12] H.264 tutorial by I.E.G. Richardson: https://www.vcodex.com/h264.html [13] HEVC white paper-Ittiam Systems: http://www.ittiam.com/Downloads/en/documentation.aspx [14] HEVC white paper-Ateme: http://www.ateme.com/an-introduction-to-uhdtv-and-hevc

20. Contd… [15] HEVC white paper-Elemental Technologies: http://www.elementaltechnologies.com/lp/hevc- h265-demystified-white-paper [16] Test Sequences: ftp://ftp.kw.bbc.co.uk/hevc/hm-11.0-anchors/bitstreams/ [17] Access to HM 16.0 Reference Software: http://hevc.hhi.fraunhofer.de/http://hevc.hhi.fraunhofer.de/ [18] Detailed Overview of HEVC/H.265 by Shevach Riabtsev https://app.box.com/s/rxxxzr1a1lnh7709yvih [19] Access the website http://www.uta.edu/faculty/krrao/dip/Courses/EE5359/ [20] Access to HM 16.0 Software Manual: http://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/tags/HM-16.0/doc/software-manual.pdf [21] C. Fogg, “Suggested figures for the HEVC specification”, ITU-T / ISO-IEC Document: JCTVC J0292r1, July 2012. [22] Z. Wang et al, “Image Quality Assessment: From Error Visibility to Structural Similarity”, IEEE Trans. on Image Processing, Vol. 13, No. 4, pp. 600-612, Apr. 2004. [23] G. Correa et al, “Performance and computational complexity assessment of high efficiency video encoders”, IEEE Trans. on Circuits and Systems for Video Technology, Vol.22, pp.1899-1909, Dec.2012. [24] L.-K.Liu and E.Feig, “A block based gradient descent search algorithm for block motion estimation in video coding,” IEEE Trans. on Circuits and Systems for Video Technology,Vol. 6, No. 4, pp. 419-422, Aug. 1996. [25] V. Sze and M. Budagavi, “Design and Implementation of Next Generation Video Coding Systems (H.265/HEVC Tutorial)”, IEEE International Symposium on Circuits and Systems (ISCAS), Melbourne, Australia, June 2014, available on http://www.rle.mit.edu/eems/publications/tutorials/ [26] D. Grois, B. Bross and D. Marpe, “HEVC/H.265 Video Coding Standard (Version 2) including the Range Extensions, Scalable Extensions, and Multiview Extensions,” (Tutorial) Sunday 27 Sept 2015, 9:00 am to 12:30 pm), IEEE ICIP, Quebec City, Canada, 27 – 30 Sept. 2015. The tutorial below is for personal use only [Password: a2FazmgNK ] https://datacloud.hhi.fraunhofer.de/owncloud/public.php?service=files&t=8edc97d26d46d4458a9c1a17964bf881 https://datacloud.hhi.fraunhofer.de/owncloud/public.php?service=files&t=8edc97d26d46d4458a9c1a17964bf881 [27] M. J. Jakubowski and G. Pastuszak, “Block-based motion estimation algorithms – a survey,” Opto-Electronic Review, Vol. 21, pp 86-102, March2013. [28] W. Hong, “Coherent Block-Based Motion Estimation for Motion-Compensated Frame Rate Up-Conversion", IEEE International Conference on Consumer Electronics, pp. 165-166, Jan.2010.

21. Contd… [29] L.N.A. Alves and A. Navarro, " Fast Motion Estimation Algorithm for HEVC ", Proc. IEEE International Conf. on Consumer Electronics -ICCE Berlin, Germany, vol.11, pp. 11 - 14, Sep., 2012. [30] A. Abdelazim, W. Masri and B. Noaman "Motion estimation optimization tools for the emerging high efficiency video coding (HEVC)", SPIE vol. 9029, Visual Information Processing and Communication V, 902905, Feb. 17, 2014. [31] Video test sequences - http://forum.doom9.org/archive/index.php/t-135034.html or http://media.xiph.org/video/derf/ [32] G. J. Sullivan et al “Standardized Extensions of High Efficiency Video Coding (HEVC)”,IEEE Journal of selected topics in Signal Processing, vol. 7, pp.1001-1016, Dec. 2013. [33] M. Wien, “High efficiency video coding: Tools and specification”, Springer, 2015. [34] I.E. Richardson, “Coding video: A practical guide to HEVC and beyond”, Wiley, 11 May 2015. [35] V.Sze, M.Budagavi and G.J.Sullivan “High Efficiency Video Coding (HEVC) –Algorithms and Architectures”, Springer, 2014. [37] Daala info: https://xiph.org/daala/https://xiph.org/daala/ [38] Daala codec Git repository: https://git.xiph.org/?p=daala.githttps://git.xiph.org/?p=daala.git [39] Z.Wang et al, “Image quality assessment: From error visibility to structural similarity”, IEEE Transactions on Image Processing, vol.13, pp. 600- 612, April 2004.

22. Thank you