/ Fast block partitioning method in HEVC Intra coding for UHD video / Hello everybody, my name is Nicolas Dhollande, from b<>com and today I’m going to present you a “Fast block partitioning method in HEVC Intra coding for UHD video”, a work achieved by Xavier Ducloux, Olivier Le Meur, Christine Guillemot, and myself. Authors: Nicolas Dhollande, Xavier Ducloux, Olivier Le Meur, Christine Guillemot 11/28/2018

High Efficiency Video Coding (HEVC) Problem addressed High Efficiency Video Coding (HEVC) Latest video standard (2013) Joint Coding Team on Video Coding (JCT-VC) HEVC bitrate savings over MPEG.4/AVC with the same perceived quality: 64% in UHD, 62% in HD [Tan and al, JCTVC-Q0204] HEVC will be key for UHD deployment Goal: limit as far as possible the encoding complexity with a limited quality loss HEVC: 2-4x more encoding power than MPEG.4/AVC UHD: 4-5x more encoding power than HD We’ll speak about the latest video standard … At the beginning, the goal of the standardization was to halve the bitrate required… it turns out that HEVC achieves … HEVC necessary for the UHD deployement, given that UHD contains 8 times more information to encode, it allows to transmit UHD with 10-20Mbits .. but also to alleviate the recent video traffic growth over Internet. the encoding power is … and up to …

HEVC vs previous standard MPEG.4/AVC: Problem addressed HEVC vs previous standard MPEG.4/AVC: Coding Tree Unit (composed of 64x64 to 8x8 sizes CU’s) more accurate Intra coding (35 luma predictions) within a PU extended transforms (32x32 to 4x4) with a QT structure The codec brings new features compared to MPEG.4/AVC… A Slice is divided in 64x64 size Coding Tree Units, which are divided in a quad-tree of Coding Units, ranged from the 64x64 size to the 8x8 size. In Intra, CU contains 1 PU, or 4 sub –PU for the minimum CU size… In a PU, 35 prediction patterns ... Instead of using either 4x4 or 8x8 transforms, HEVC enables to divide a CU in a quadtree of TU, with sizes 4x4 to 32x32.

Highest speed-up potential with UHD ! Problem addressed The quad-tree structure leads to intensive computation when the encoder builds the CTU Depth first partitioning process: 85 CU / 341 Intra PU to be evaluated Just a few CU coded in the syntax What if the encoder knows the optimal PU quad-tree in advance … The determination of the optimal quad-tree structure in a CTU is heavy in term of encoding power. In the HEVC reference software, a depth-first partitioning strategy is employed … consists in evaluating 85 CU for a CTU Imagine that the encoder knows in advance the optimal PU quad-tree, the potential of encoding time speed-up is more than 6x in UHD, and up to 4x in HD format This motivates Video sequences Intra only (AI) Speed-up ratio EBU set UHD 6.33 Class B 1080p 3.82 Highest speed-up potential with UHD ! Encoding of the optimal PU quad-tree vs reference

Typical algorithms for fast Intra partitioning: Problem addressed Typical algorithms for fast Intra partitioning: Intuition that flat regions are divided in large blocks, and sharp/complex regions are divided in small blocks: Judgement process based on the variance complexity on the downscaled content [Tian & Goto, 2012] Judgement process based on the entropy [Zhang, 2013] Modes inference from a lower resolution pre-coding pass. [Dhollande, 2014] Typically, papers propose to use pre-analysis of the content in order to skip the less probable blocks from the depth first search Another paper, propose to first encode a downscaled version of the video content, and then infers the modes in the target encoder. In this paper, we focus on the first category which uses pre-analysis of the content, and we simply use the variance for the judgement process.

Our approach: a fast Intra partitioning scheme for the Intra coding Off-line training Low complexity encoder A fast Intra partitioning scheme consists of an off-line learning of thresholds in the one hand, and a low complexity encoder which uses these thresholds on the other hand …

Let S be a Bernoulli random variable with parameter P: Motivations Let S be a Bernoulli random variable with parameter P: P: proba. that a PU is further split by the ref. encoder σ: variance of the luminance pixels into the current PU d: PU depth into the CTU {0,1,2,3} qp: quantization parameter {22,27,32,37} Let define S, a Bernoulli random variable which can take 2 values: split or not split… P is the probability that the current block is further split given the variance, the PU depth and the QP. Below is an example of this conditional probability with QP equal to 32 and for the PU size 32x32 (d=1) sigma is represented with a logarithm scale The probability to split in red increases with the sigma value… Block partitioning probability in 32x32-size PU given σ within a qp=32 reference UHD encoding

Two thresholds on the variance for the splitting decision: Off-line training σ well associates the local signal characteristics with the HEVC Intra block sizes Two thresholds on the variance for the splitting decision: ThL such that: ThH such that: is the probability level used to select ThL and ThH from the conditional probabilities. It can be stated that sigma well associates… (PL) ? (1-PL) High proba of non splitting High proba of splitting ThL ThH

Off-line training PL=90% ThL ThH ThL ThH d=1 qp=22 d=1 qp=37 ThL ThH Accuracy++ Speed-up++ ThL ThH ThL ThH d=1 qp=22 d=1 qp=37 ThL ThH ThL ThH d=3 qp=22 d=0 qp=37

Off-line training PL=60% ThL ThH ThL ThH d=1 qp=22 d=1 qp=37 ThL ThH Accuracy++ Speed-up++ ThL ThH ThL ThH d=1 qp=22 d=1 qp=37 We’ll show with experimental results that an high PL value favours the accuracy whereas a low PL value favour the encoding time speed-up ThL ThH ThL ThH d=3 qp=22 d=0 qp=37

A reference encoding is performed for each qp. Off-line training A reference encoding is performed for each qp. Conditional probabilities are calculated for each d and qp. Depending on the PL parameter, a couple (ThL, ThH) is chosen for each d and qp. To summarize,

Proposed scheme for the low complexity encoder PU classification: Our fast partitioning scheme: Variant 1: unconstrained CMP class (reference partitioning) Variant 2: constrained CMP class (1 split only) Next PU level

Our approach: a fast Intra partitioning scheme for the Intra coding Summary Problem addressed Our approach: a fast Intra partitioning scheme for the Intra coding Motivations Off-line training Proposed scheme for the low complexity encoder Results Strengths of our algorithm / conclusion

Settings Performance measure Results HM test model v11.0 All Intra (AI) – Main profile 4 quantization parameters: 22,27,32,37 10 EBU UHD-1 sequences for the off-line training 750 frames each Performance measure Encoding time speed-up ratio = TREF / TALGO Vs Average BD-rate

Performance of the low complexity encoder in UHD: Results Performance of the low complexity encoder in UHD: In function of the application we want to address, PL value allow us to obtain different operating points.

Performance with others sequences and resolutions Results Performance with others sequences and resolutions

Our approach: a fast Intra partitioning scheme for the Intra coding Summary Problem addressed Our approach: a fast Intra partitioning scheme for the Intra coding Motivations Off-line training Proposed scheme for the low complexity encoder Results Strengths of our algorithm / conclusion

Strengths of our algorithm / conclusion Simple fast Intra partitioning scheme which skips the less probable PU from the decision process. High accuracy of our early terminations Speed-up of 2x with no loss Strong complexity reduction in UHD About 6x against a limited loss of 5% No subjective quality degradation Possible improvements: On-line learning of thresholds (each N frames) Improved Intra modes pre-selection using the texture orientations Fast RQT construction We have shown that the threshold database we have generated is consistent enough for most of the UHD sequences. However, the fast partitioning can be combined with others kinds of fast algorithms …

/ nicolas.dhollande@b-com.com /