1. INTERIM Presentation on Topic: Advanced Video Coding (Comparison of HEVC with H.264 and H.264 with MPEG-2) A PROJECT UNDER THE GUIDANCE OF DR. K. R. RAO COURSE: EE5359 - MULTIMEDIA PROCESSING, SPRING 2015 SUBMITTED BY JAGRITI DHINGRA UT ARLINGTON ID: 1001103750 EMAIL ID: jagriti.dhingra@mavs.uta.edu DEPARTMENT OF ELECTRICAL ENGINEERING UNIVERSITY OF TEXAS, ARLINGTON 1

2. List of Acronyms and Abbreviations AVC: Advanced Video Coding. BD-BR: Bjontegaard Delta Bitrate. BD-PSNR: Bjontegaard Delta Peak Signal to Noise Ratio. CABAC: Context Adaptive Binary Arithmetic Coding. CTB: Coding Tree Block. CTU: Coding Tree Unit. CU: Coding Unit. DBF: De-blocking Filter. DCT: Discrete Cosine Transform. HEVC: High Efficiency Video Coding. HM: HEVC Test Model. IEC: International Electro-technical Commission. ISO: International Organization for Standardization. ITU-T: International Telecommunication Union- Telecommunication 2

3. Standardization Sector. JCT: Joint Collaborative Team. JCT-VC: Joint Collaborative Team on Video Coding. JM: H.264 Test Model. JPEG: Joint Photographic Experts Group. MC: Motion Compensation. ME: Motion Estimation. MPEG: Moving Picture Experts Group. MSE: Mean Square Error. PB: Prediction Block. PSNR: Peak Signal to Noise Ratio. QP: Quantization Parameter SAO: Sample Adaptive Offset. SSIM: Structural Similarity Index. TB: Transform Block. TU: Transform Unit. VCEG: Video Coding Experts Group. 3

4. Scope of Project This project aims at studying the state-of-the-art High Efficiency Video Coding (HEVC) [12],H.264/Advanced Video Coding (AVC) [13]and MPEG-2 video codecs and gaining an understanding of various techniques in video coding such as prediction, transform, quantization and coding. A performance comparison of these video codecs based on various metrics such as computational time, PSNR, BDBitrate and BD-PSNR [11] [15] are carried out. The HM 13.0 [24] and JM 18.6 [25] test models for HEVC and H.264 respectively are used for this purpose. 4

5. HEVC High Efficiency Video Coding (HEVC) [12] is an international standard for video compression developed by a working group of ISO/IEC MPEG (Moving Picture Experts Group) and ITU-T VCEG (Video Coding Experts Group). The main goal of HEVC standard is to significantly improve compression performance compared to existing standards (such as H.264/Advanced Video Coding [13]) in the range of 50% bit rate reduction at similar visual quality [6]. HEVC is designed to address existing applications of H.264/MPEG-4 AVC and to focus on two key issues: increased video resolution and increased use of parallel processing architectures [6]. It primarily targets consumer applications as pixel formats are limited to 4:2:0 8-bit and 4:2:0 10-bit. The next revision of the standard, planned for 2014, will enable new use-cases with the support of additional pixel formats such as 4:2:2 and 4:4:4 and bit depth higher than 10- bit [20], embedded bit-stream scalability and 3D video [33]. 5

6. HEVC Codec Figure 1: Block Diagram of HEVC CODEC [17] 6

7. HEVC Decoder Figure 2: Block Diagram of HEVC Decoder [31] 7

8. HEVC Encoder Figure 3: Block Diagram of HEVC Encoder [9] 8

9. H.264 H.264 or MPEG-4 Part 10, Advanced Video Coding (MPEG-4 AVC) is a video compression format that is currently one of the most commonly used formats for the recording, compression, and distribution of video content. H.264 technology aims to provide good video quality at considerably low bit rates, at reasonable level of complexity while providing flexibility to wide range of applications. [1] H.264 is perhaps best known as being one of the video encoding standards for Blu-ray Discs; all Blu-ray Disc players must be able to decode H.264. It is also widely used by streaming internet sources, such as videos from Vimeo, YouTube, and the iTunes Store, web software such as the Adobe Flash Player and Microsoft Silverlight, and also various HDTV broadcasts over terrestrial (ATSC, ISDB-T, DVB T or DVB-T2), cable (DVB-C), and satellite (DVB-S and DVB- S2).[4] 9

10. H.264 Codec Figure 4: Block Diagram of H.264 CODEC [18] 10

11. H.264 Encoder Figure 5: Block Diagram of H.264 Encoder [10] 11

12. H.264 Decoder Figure 6: Block Diagram of H.264 Decoder [10] 12

13. Comparison Metrics Peak Signal to Noise Ratio Peak signal-to-noise ratio (PSNR) [22] [26] is an expression for the ratio between the maximum possible value (power) of a signal and the power of distorting noise that affects the quality of its representation. PSNR is most commonly used to measure the quality of reconstruction of lossy compression codecs. The signal in this case is the original data, and the noise is the error introduced by compression. When comparing compression codecs, PSNR is an approximation to human perception of reconstruction quality. Although a higher PSNR generally indicates that the reconstruction is of higher quality, in some cases it may not. One has to be extremely careful with the range of validity of this metric; it is only conclusively valid when it is used to compare results from the same codec (or codec type) and same content. PSNR is defined via the mean squared error (MSE). Given a noise-free m x n monochrome image I and its noisy approximation K, MSE is defined as: 13

14. 14

15. Profiles used for Comparison The HM 16.0 main profile [6] and JM 18.6 high profile [8] will be used for comparison in this project. Main profile in HEVC This profile allows for a bit depth of 8-bits per sample with 4:2:0 chroma sampling, which is the most common type of video used with consumer devices. High Profile in H.264 This profile allows for a bit depth of 8-bits per sample with 4:2:0 Chroma sampling. It is used for broadcast and disc storage applications, particularly for high-definition television applications (for example, this is the profile adopted by the Blu-ray Disc storage format and the DVB HDTV broadcast service). 15

16. Test Sequences 16

17. Test Sequences [23] 17

18. Results 18

19. Graphs 19

20. 20

21. 21

22. References [1] “Draft ITU-T recommendation and final draft international standard of joint video specification (ITU-T Rec. H.264/ISO/IEC 14 496-10 AVC,” in Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG, JVTG050, 2003. [2] “Generic Coding of Moving Pictures and Associated Audio Information - Part 2: Video,” ITU-T and ISO/IEC JTC 1, ITU-T Recommendation H.262 and ISO/IEC 13 818-2 (MPEG-2), 1994. [3] IEEE SIGNAL PROCESSING MAGAZINE [148] MARCH 2007. https://www.ic.tberlin.de/fileadmin/fg121/publications/2007_42_h264avc_nutshell.pdf [4] http://en.wikipedia.org/wiki/H.264/MPEG-4_AVChttp://en.wikipedia.org/wiki/H.264/MPEG-4_AVC [5] I.E.G. Richardson, “Video Codec Design: Developing Image and Video Compression Systems”, Wiley, 2002. [6] G. J. Sullivan et al, “Overview of the High Efficiency Video Coding (HEVC) Standard”, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1649-1668, Dec. 2012 [7] IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 13, NO. 7, JULY 2003. [8] T. Wiegand et al, “Overview of the H.264/AVC Video Coding Standard”, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 13, No. 7, pp. 560-576, July 2003. [9] D. Marpe et al, “The H.264/MPEG4 advanced video coding standard and its applications”, IEEE Communications Magazine, Vol. 44, pp. 134-143, Aug. 2006. [10] A. Puri et al, “Video coding using the H.264/MPEG-4 AVC compression standard”, Signal Processing: Image Communication, vol. 19, pp. 793-849, Oct. 2004. 22

23. [11] X. Li et al, “Rate-complexity-distortion evaluation for hybrid video coding”, IEEE InternationalConference on Multimedia and Expo (ICME), pp. 685-690, July 2010. [12] B. Bross et al, “High Efficiency Video Coding (HEVC) Text Specification Draft 10”, Document JCTVC- L1003, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC), Mar. 2013 available on http://phenix.it- sudparis.eu/jct/doc_end_user/current_document.php?id=7243 [13] JVT Draft ITU-T recommendation and final draft international standard of joint video specification (ITU-T Rec. H.264-ISO/IEC 14496-10 AVC), March 2003, JVT-G050 available on http://ip.hhi.de/imagecom_G1/assets/pdfs/JVT- G050.pdf [14] J. Vanne et al, “Comparative Rate-Distortion-Complexity Analysis of HEVC and AVC Video Codecs”, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1885-1898, Dec. 2012. [15] G. Bjontegaard, “Calculation of Average PSNR Differences between RD Curves”, document VCEG-M33, ITU-T SG 16/Q 6, Austin, TX, Apr. 2001. [16] D. Grois et al, “Performance Comparison of H.265/ MPEG-HEVC, VP9, and H.264/ MPEG-AVC Encoders”, available on: http://iphome.hhi.de/marpe/download/Performance_HEVC_VP9_X264_PCS_2013_preprint.pdf [17] HEVC tutorial by I.E.G. Richardson: http://www.vcodex.com/h265.html [18] H.264 tutorial by I.E.G. Richardson: https://www.vcodex.com/h264.html [19] HEVC white paper-Ittiam Systems: http://www.ittiam.com/Downloads/en/documentation.aspx [20] HEVC white paper-Ateme: http://www.ateme.com/an-introduction-to-uhdtv-and-hevc 23

24. [21] HEVC white paper-Elemental Technologies: http://www.elementaltechnologies.com/lp/hevc-h265-demystified- white-paper [22] White paper on PSNR-NI: http://www.ni.com/white-paper/13306/en/ [23] Test Sequences: ftp://ftp.kw.bbc.co.uk/hevc/hm-11.0-anchors/bitstreams/ [24] Access to HM 16.0 Reference Software: http://hevc.hhi.fraunhofer.de/ [25] Access to JM 18.6 Reference Software: http://iphome.hhi.de/suehring/tml/ [26] Website on PSNR: http://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio [27] Website on SSIM: http://en.wikipedia.org/wiki/Structural_similarity [28] Access the website http://www-ee.uta.edu/Dip/Courses/EE5359/ and refer to the project by S. Kulkarni on “Transcoding from H.264/AVC to High Efficiency Video Coding (HEVC)”, University of Texas, Arlington, Spring 2013. [29] Access the website http://www-ee.uta.edu/Dip/Courses/EE5359/ and refer to the project by H. B. Jain on “Comparative and performance analysis of HEVC and H.264 intra frame coding and JPEG 2000”, University of Texas, Arlington, Spring 2013. [30] Access the website http://www-ee.uta.edu/Dip/Courses/EE5359/ and refer to the thesis by S. Vasudevan on “Fast intra prediction and fast residual quad-tree encoding implementation in HEVC”, University of Texas, Arlington, 2013. 24

25. [31] C. Fogg, “Suggested figures for the HEVC specification”, ITU-T / ISO-IEC Document: JCTVC J0292r1, July 2012. [32] Z. Wang et al, “Image Quality Assessment: From Error Visibility to Structural Similarity”, IEEE Transactions on Image Processing, Vol. 13, No. 4, pp. 600-612, Apr. 2004. [33] G. Sullivan et al, “Standardized Extensions of High Efficiency Video Coding (HEVC)”, IEEE Journal of selected topics in Signal Processing, Vol. 7, No. 6, pp. 1001-1016, Dec. 2013 25

26. Future Work Comparison of H.264 with MPEG-2 Using comparison metrics and JM software. 26

27. Thank You 27