1. E ARLY TERMINATION FOR TZ SEARCH IN HEVC MOTION ESTIMATION PRESENTED BY: Rajath Shivananda (1001096626) 1 EE 5359 Multimedia Processing Individual Project

2. ACRONYMS AMVP- Advanced Motion Vector Prediction AVC – Advanced Video Coding BMA – Block Matching Algorithm CABAC – Context Adaptive Binary Arithmetic Coding CB – Coding Block CTU – Coding Tree Unit CTB – Coding Tree Block CU – Coding Unit HEVC – High Efficiency Video Coding HM – HEVC Test Model ICASSP - International Conference on Acoustics, Speech and Signal Processing ICIP – International Conference on Image Processing JCT-VC - Joint Collaborative Team on Video Coding LCU – Large coding unit 2

3. ACRONYMS CONTD.. MC – Motion Compensation ME – Motion Estimation MV – Motion Vector MP – Median Predictor MPEG – Moving Picture Experts Group PB – Prediction Block PU – Prediction Unit SAD- Sum of absolute Difference SCU- Small coding unit SSD -Sum Of Squared Differences TSS – Three Step Search TU – Transform Unit TZS- Test Zone Search WP – Weighted Prediction 3

4. HEVC Encoder block diagram FIGURE 1. HEVC ENCODER [6] 4

5. Motivation for Early Termination [1] Computational complexity of multiple initial search point decision and hybrid block matching search is very high[1]. Large probability of selecting MP as final best point in ME process. CU based on quad-tree structure such as LCU and SCU. The 2 CUs may have different characteristics. Figure 2 shows the depth of the CU quad-tree structure in HEVC. 5

6. Motivation for Early Termination (continued) 6 F IGURE 2. CU QUAD - TREE STRUCTURE IN HEVC [1]

7. Motivation for Early Termination (continued) [1] The video sequences used for the present experiment are BQMall (480p) Johnny (720p) Mobisode2 (240p) Parkscene (1080p) Table 1 shows the test conditions [1]. 7

8. Video Sequences[1] BQMall (480p) - Figure 3 shows the 1 st frame of BQMall video sequence. Johnny (720p) - Figure 4 shows the 1 st frame of Johnny video sequence. Figure 3. BQMall Video Sequence [1] Figure 4. Johnny Video Sequence [1] 8

9. Video Sequences (continued)[1] Mobisode2 (240p) - Figure 5 shows the 1 st frame of Mobisode2 video sequence. ParkScene (1080p) - Figure 6 shows the 1 st frame of ParkScene video sequence. Figure 5. Mobisode2 Video Sequence [1] Figure 6. ParkScene Video Sequence [1] 9

10. Motivation for Early Termination (continued)[4] [5] It is expected that there is 45% to 67% MPs selected as best point in CUs with depth L (LCU) and 56% to 89% with depth S (SCU). Hence, if these MPs who are selected as final best points can be early terminated then much more encoding time can be saved. 10

11. Proposed Early termination Algorithm [1] 11

12. Diamond search to find the best match [3] 12 FIGURE 2 [12]

13. SAD (SUM OF ABSOLUTE DIFFERENCE)[2] SAD architecture is used to calculate the minimum cost between the current block and reference block. Since Motion Estimation(ME) is the most time consuming portion in HEVC encoder it is required to reduce the residual information that is sent to the transform block [2]. The rate-distortion cost J is shown in equation(1), where λ-> Lagrange multiplier R-> # of Bits required to encode the motion vector difference D-> Distortion function. One of the most widely used distortion function for motion estimation is SAD that can be defined as shown in equation (2), where CB-> Current block pixels RB-> Reference block pixels MxN is the size of the current PU block. Minimum of J is considered. 13

14. To Confirm The Results Reported n [1] Total encoding time can be saved on an average 38%. Average PSNR (Peak signal to noise ratio) would degrade by only 0.008dB. Average Bit Rate (BR) will increase by only 0.28%. 14

15. REFERENCES 15 [1] Z Pan, Y Zhang, S Kwong, X Wang and L Xu, “Early Termination for TZSearch in HEVC Motion Estimation”, IEEE International Conference on Acoustics, Speech and Signal Processing. Page(s) : 1389 – 1393, May 2013. [2] P. Nalluri et al, “High Speed SAD Architectures for variable block size estimation in HEVC video coding”, IEEE International Conference on Image Processing (ICIP). Page(s): 1233 – 1237, Oct 2014. [3] M. A. B. Ayed, et al, “TZ Search pattern search improvement for HEVC motion estimation modules,” Advanced Technologies for Signal and Image Processing (ATSIP). Page(s): 95 – 99, March 2014. [4] Z. Pan, S. Kwong, L. Xu, Y. Zhang, T. Zhao, “Predictive and distribution-oriented fast motion estimation for H.264/AVC” Journal of Real-Time Image Processing, vol. 9, page(s): 597 – 607, December 2014. [5] R. Li, B. Zeng, and M.L. Lio, “A new three-step search algorithm for block motion estimation” IEEE Trans. Circuits and Systems for Video Technology, vol. 4, no. 4, pp. 438-443,August 1994. [6] G.J. Sullivan et al, “Overview of the high efficiency video coding (HEVC) standard”, IEEE Trans. Circuits and Systems for Video Technology, vol. 22,no.12, pp. 1649 – 1668, Dec 2012. [7] Introduction to Motion estimation and Motion compensation---> http://www.cmlab.csie.ntu.edu.tw/cml/dsp/training/coding/motion/me1.html

16. REFERENCES (continued) [8] HM Software Manual - https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/ [9] Visual studio: http://www.dreamspark.comhttp://www.dreamspark.com [10] Tortoise SVN: http://tortoisesvn.net/downloads.htmlhttp://tortoisesvn.net/downloads.html [11] Tutorials---> N. Ling, “High efficiency video coding and its 3D extension: A research perspective,” Keynote Speech, IEEE Conference on Industrial Electronics and Applications, Singapore, July 2012. [12] Tutorials---> X. Wang et al, “Paralleling variable block size motion estimation of HEVC on CPU plus GPU platform”, IEEE International Conference on Multimedia and Expo workshop, 2013. [13] Tutorials---> H.R. Tohidpour, et al, “Content adaptive complexity reduction scheme for quality/fidelity scalable HEVC”, IEEE International Conference on Image Processing, pp. 1744-1748, June 2013. [14] X. Zhang, S. Wang, S. Ma,“ Early termination of coding unit splitting for HEVC”, Signal & Information Processing Association Annual Summit and Conference. Page(s):1-4, December 2012. [15] HEVC tutorial 2014 ISCAS ---> http://www.rle.mit.edu/eems/wp-content/uploads/2014/06/H.265-HEVC-Tutorial-2014-ISCAS.pdf 16

17. REFERENCES (continued) [16] Video Lecture on Digital Voice and Picture Communication by Prof.S. Sengupta, Department of Electronics and Electrical Communication Engineering,IIT Kharagpur -> https://www.youtube.com/watch?v=Tm4C2ZFd3zE. https://www.youtube.com/watch?v=Tm4C2ZFd3zE [17] Lecture on video coding standards -> http://nptel.iitm.ac.in http://nptel.iitm.ac.in 17