MULTIMEDIA PROCESSING STUDY AND IMPLEMENTATION OF POPULAR PARALLELING TECHNIQUES APPLIED TO HEVC By: Karthik Suresh (1000880819) Under the guidance of Dr. K. R. Rao

Overview HEVC Improvements Need for parallel processing Parallelization approaches Proposed work References

HEVC (High Efficiency Video Coding) Also known as H.265, it is the newest video coding standard of the ITU-T VCEG and ISO/IEC MPEG [1]. The best performance improvement of HEVC over H.264 is ~50% bit rate reduction for equal perceptual video quality. Coding efficiency, data loss resilience, enabling parallel processing architectures are the other significant upgrades that were incorporated.

HEVC Encoder Figure 1: Block diagram of HEVC Encoder

Improvements in Encoder Coding Tree Block (CTB) [1] size can be up to 64×64 spatial dimension. Figure 2: various sizes of the CTU Coding Tree Unit (CTU) size can be selected by the encoder. Coding Tree Block (CTB) is the largest supported size for a luma CB. One luma CB and two chroma CBs along with the syntax form a Coding Unit (CU).

Improvements in Encoder (contd). The decision whether to code a picture using inter or intra prediction is done at the CU level. Motion Vector Signaling: Advanced Motion Vector Prediction (AMVP) [1] or Merge Mode is used. Improvements in skipped and direct motion inference. Motion Compensation: Quarter sample precision is used for the MVs and 7-tap or 8-tap filters are used for interpolation of fractional-sample positions.

Need for Parallel Processing HEVC is more complex compared to H.264 Thus it takes ~40% more time for computation making it power intensive. Parallel processing helps reduce the computational time without significantly affecting the quality of the output.

Parallelization approaches (internal) Slices Parallel processing with slices has several advantages like coarse-grain parallel processing [3], data locality, low delay and low memory bandwidth. They have the largest coding penalty as they break entropy decoding and prediction dependencies.

Wavefront Parallel Processing (WPP): There is one picture partition per row and both entropy decoding and prediction are allowed to cross partitions. Coding losses are minimized while at the same time wavefront parallelism can be exploited. They define horizontal and vertical boundaries that partition a picture into tile columns and rows. Similar to slices, tiles break entropy decoding and prediction dependencies, but does not require a slice header for each tile. Tiles

Entropy slices Figure 3: Graph showing the advantage of Entropy slices They are proposed for parallelism not for error resilience. Like slices, they break entropy decoding dependencies but allow prediction (and filtering) to cross slice boundaries. It allows to perform entropy decoding in parallel without data dependencies.

Paralleling approaches (hardware) Multi-core processing Figure 4: Multicore architecture Intel/OpenMP [4] /CUDA [6] : Using multicore [5] processors to run the code in parallel will decrease the time taken. We run the code on multiple threads on multiple cores. When there are multiple cores, the task is passed on to a core which is idle.

Paralleling approaches (hardware) GPU assisted video coding Graphic Processing Units (GPUs) [5] are specialized hardware for 3D graphic rendering. They accelerate arithmetic intensive application in computationally intensive equipments. Using GPUs along with the CPU will decrease the computation time significantly.

Proposed Work Implement paralleling approaches with optimizing algorithms which have a greater impact and try to obtain performance enhancement. Based on various test sequences, compare these results with those obtained without paralleling approaches.

References [1] G.J. Sullivan et al, “Overview of the high efficiency video coding (HEVC) standard”, IEEE Trans. CSVT, vol. 22,pp.1649- 1668, Dec.2012. [2] C.C.Chi et al, “Parallel scalability and efficiency of HEVC parallelization approaches”, IEEE Trans. CSVT, vol. 22, pp.1827-1838, Dec.2012. [3] M.A.Mesa, et al., "Parallel video decoding in the emerging HEVC standard“, ICASSP 2012, pp. 1545 - 1548, March 2012. [4] Intel tutorial on OpenMP https://www.youtube.com/watch?v=FQ1k_YpyG_A&list=SPL X-Q6B8xqZ8n8bwjGdzBJ25X2utwnoEG.

References (contd) [5] Ngai-Man Cheung, et al., "Video coding on multicore graphics processors", Signal Processing Magazine IEEE, Vol 27 Issue 2, pp. 79 - 89, March 2010. [6] Thesis by Sudeep Gangavati on Complexity reduction of H.264 using parallel programming. http://www- ee.uta.edu/Dip/Courses/EE5359/index.html [7] Project by Valay Shah on Study and optimization of Deblocking filter in H.265 and its advantages over H.246/AVC. http://www- ee.uta.edu/Dip/Courses/EE5359/index.html [8] N.M. Cheung, et al, "Video coding on multicore graphics processors", IEEE Signal Processing Magazine, vol 27, Issue 2, pp. 79 - 89, March 2010.

References (contd) [9] E. Kalali, et al, "A High Performance And Low Energy Intra Prediction Hardware For HEVC Video Decoding", DASIP 2012, pp. 1 - 8, Karslruhe, Germany, Oct. 2012. [10] K. Miyazawa, et al, "Real-Time Hardware Implementation of HEVC Encoder for 1080p HD Video", IEEE PCS 2013, pp. 225 - 228, San Jose, California, USA, Dec 2013. [11] S. Kim, et al, "A Novel Fast and Low-complexity Motion Estimation for UHD HEVC", IEEE PCS 2013, pp. 105 - 108, San Jose, California, USA, Dec 2013. [12] F. Bossen, et al, ” HEVC Complexity and Implementation Analysis”, IEEE Trans. on CSVT, vol.22, no.12, pp.1685-1696, Dec. 2012. [13] K.R. Rao, D.N. Kim and J.J. Hwang, "Video Coding Standards: AVS China, H.264/MPEG-4 Part10, HEVC, VP6, DIRAC and VC-1", Springer, 2014.

References (contd.) [14] G.J. Sullivan, et al, "Standardized Extensions of High Efficiency Video Coding (HEVC)", IEEE Journal of Selected Topics in Signal Processing, vol. 7, Issue 6, pp. 1001 - 1016, Dec. 2013. [15] G.J. Sullivan, et al, "HEVC Range Extensions Draft 5", JCT- VC, version 1, Geneva, Nov. 2013. [16] M. Jakubowski and G. Pastuszak, “Block-based motion estimation algorithms – a survey”, Opto-Electronics Review, vol 21, Issue 1, pp. 86 – 102, March 2013. [17] Access to HM 13.0 Reference Software: https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/branche s/HM-13.0-dev/

References (contd.) [18] Access to HM Software Manual: https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/bran ches/HM-13.0-dev/doc/ [19] B. Bross et al, “High Efficiency Video Coding (HEVC) Text Specification Draft 10”, Document JCTVC-L1003, ITU- T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC), Mar. 2013 available on http://phenix.it- sudparis.eu/jct/doc_end_user/current_document.php?id=7 243