Liquan Shen Zhi Liu Xinpeng Zhang Wenqiang Zhao Zhaoyang Zhang An Effective CU Size Decision Method for HEVC Encoders IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 15, NO. 2, FEBRUARY

Outline Introduction Related Works Proposed CU Size Decision Method Experimental Results Conclusion 2

Introduction HEVC encoders 2.0 enable 7 different modes including for inter slice. 3

Introduction At each depth level (CU size), ME was performed by using different sizes 2N×2N, 2N×N, N×2N, and N×N. Mode decision process is performed using all the possible depth levels and prediction modes find the least RD cost using Lagrange multiplier [5].. [5] High Efficiency Video Coding (HEVC) Test Model 2 (HM 2) Encoder Description, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, document JCTVC- D502, 4th Meeting. Daegu, Korea, Jan. 2011, pp. 20–28. 4

Introduction This “try all and select the best” method achieves the highest coding efficiency, but will result in high computational complexity. limited in real-time applications. A fast CU size decision algorithm is proposed in this paper for HM. reduce the complexity of CU size decision. without compromising coding efficiency. 5

Introduction The optimal depth level is highly content-dependent, it is not efficient to use all levels. The proposed algorithm determine CU depth range skip some specific depth levels CU depth = 0 CU depth = 1 CU depth = 2 CU depth = 3 6

Introduction The proposed algorithm introduces Skip ME on unnecessary CU sizes. Early Termination methods Motion Homogeneity checking RD cost checking SKIP mode checking 7

Outline Introduction Related Works Proposed CU Size Decision Method Experimental Results Conclusion 8

Related Works HM adopted a fast mode decision based on RD cost from SKIP mode to terminate procedures of CU splitting and mod decision [5]. An adaptive CU depth range algorithm is proposed in [13] to reduce encoding complexity of HM, and a fast CU size decision method is proposed based on coding tree pruning in [14]. A fast CU decision algorithm at either frame level or CU level is proposed in [15] to accelerate the encoding process. 9

Reference [5] High Efficiency Video Coding (HEVC) Test Model 2 (HM 2) Encoder Description, Joint Collaborative Team on Video Coding (JCT- VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, document JCTVC-D502, 4th Meeting. Daegu, Korea, Jan. 2011, pp. 20–28. [13] Adaptive CU Depth Range, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16WP3 and ISO/IEC JTC1/SC29/WG11, document JCTVC-E090, 5th Meeting. Geneva, Switzerland, Apr. 2011, pp. 16–23. [14] Coding Tree Pruning Based CU Early Termination, Joint Collaborative Team on Video Coding of ITU-T SG16 WP3 and ISO/IEC JTC1/ SC29/WG11, document: JCTVC-F092, 6th Meeting. Torino, Italy, Jul. 2011, pp. 14–22. [15] J. Leng, L. Sun, T. Ikenkga, and S. Sakaida, “Content based fast coding unit decision algorithm for HEVC,” in Proc. IEEE Int. Conf. Multimedia and Signal Proc., 2011, pp. 56–59. 10

Related Works Most of these methods only utilize the spatial or temporal correlation. Coding information correlations among different depth levels are not fully studied. 11

Outline Introduction Related Works Proposed CU Size Decision Method Adaptive CU Depth Range Determination Early Termination of ME on Unnecessary CU Sizes Experimental Results Conclusion 12

Adaptive CU Depth Range Determination 13

CU Size Decision Method HM usually allows the maximum CU size equal to 64, and the depth level range is from 0 to 3. Depth level distribution 14

CU Size Decision Method Small depth levels are selected at treeblocks in the homogeneous region. a large area with low activities Large depth levels are selected at treeblocks with active motion or rich texture. a large area with high activities ME on small CU sizes could be skipped in most cases without loss of coding performance. 14% treeblocks choose the depth level 3 15

Adaptive CU Depth Range Determination Natural video sequences have strong spatial and temporal correlations. especially in the homogeneous regions. The optimal depth level of a certain treeblock is very similar to its spatially adjacent blocks. not wide variation from the co-located block in the previous frame. 16

Adaptive CU Depth Range Determination Consider spatial and temporal correlations to analyze region properties. skip ME on unnecessary CU sizes. Depth prediction

Adaptive CU Depth Range Determination According to the predicted value of the optimal depth level, each treeblock is divided into G 0,G 1,G 2,G 3,G 4. Depth pred = 0 Depth pred < < Depth pred < < Depth pred < 2.5 Otherwise 18

Adaptive CU Depth Range Determination Depth level distribution for these 5 types of treeblocks 19

Adaptive CU Depth Range Determination RDO is performed only on the candidate depth levels according to the treeblock type. Coding efficiency is very similar to the exhaustive CU size decision. 20

Early Termination of ME 21

Early Termination of ME on Unnecessary CU Sizes MVs of the current CU and its spatial neighboring CUs are homogeneous ME is usually unnecessary on the next depth level. Apply the coding information correlation among depth levels. Apply the spatio-temporal correlation among neighboring blocks. 22

Early Termination of ME on Unnecessary CU Sizes The main idea of proposed ET methods is to check whether the prediction mode and MVs in the current CU size can represent motion efficiently? whether necessary to perform ME on the next depth level? 23

Method 1: motion homogeneity checking based ET The best inter CU size is chosen by motion homogeneity. skip ME on unnecessary CU size. speed the procedure of ME. MVs from the current CU and its neighboring can be used to evaluate the motion homogeneity. 24

Method 1 Continue The threshold(T) is set to 0.1 T >= 0.1, considered with complex motion. T < 0.1, considered with homogeneous motion. Homogeneous motion means not necessary to split into four sub-CUs and perform ME on sub-CUs. 25

Method 2: RD cost checking based ET Spatial and temporal neighboring treeblocks or CUs usually show a similar RD cost distribution. RD cost of the current CU size < determined threshold skip ME on the next depth level

Method 3: SKIP mode checking based ET SKIP mode is selected as the best prediction mode current CU is homogeneous motion or static region lower energy residual than other prediction modes Utilize the prediction mode information in the upper depth level and the current depth level. Skip checking unnecessary ME on next depth level. 27

Accuracy of three ET methods Compare to HM with exhaustive CU size. 28

Overall Algorithm 29

30

Outline Introduction Related Works Proposed CU Size Decision Method Experimental Results Conclusion 31

Experimental Results Random Access Case Broadcast scenario Max GOP = 8 Intra frame period = 1.1 sec Low Delay Case Low delay scenario No picture reordering 32

Experimental Results [13] Adaptive CU Depth Range, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16WP3 and ISO/IEC JTC1/SC29/WG11, document JCTVC-E090, 5th Meeting. Geneva, Switzerland, Apr. 2011, pp. 16–23. 33

Experimental Results (a) RD curves (b) Time saving curves 34

Outline Introduction Related Works Proposed CU Size Decision Method Experimental Results Conclusion 35

Conclusion This paper presents a fast CU size decision algorithm by exploiting two fast approaches Adaptive depth range determination Early Termination of unnecessary ME Reduce the computational complexity. Maintaining almost the same RD performances as the original encoder. Achieves a higher gain time saving. 36