Low-complexity mode decision for MVC Liquan Shen, Zhi Liu, Ping An, Ran Ma and Zhaoyang Zhang CSVT

Outline Introduction Early SKIP mode decision Adaptively early termination Fast mode size decision Selectively intra coding in inter frame Overall algorithm Experiment results 2

Introduction Two frames of the same time instant from different views correspond to the same content with similar video characteristic.(prediction mode, RDcost, motion vector) SKIP mode and RDcost correlation among inter-views are not fully studied Early SKIP mode decision Adaptively early termination Fast mode size decision Selectively intra coding in inter frame 3 View 0 View 1View 2

Early SKIP mode decision Observation – Many MBs finally end up with being decided as SKIP mode because they belong to background or motionless object. Previous method – If SKIP mode has an RDcost less than a predefined threshold, the variable size ME and DE process are skipped. – How to find a reasonable threshold is a serious problem This approach method – Coding mode among inter-views are similar. – Utilizing DE to find corresponding MBs among different views, coding information can be effectively shared and reused from the coded view channel. 4

Early SKIP mode decision Idea – When the corresponding MB in the frame of the same time instant from the neighbor view is with SKIP mode, the MB tends to have SKIP mode as well. 5 GDV

Early SKIP mode decision In this work, coding information is derived from the forward inter-view reference frame. GDV is not exactly disparity between the current MB and the corresponding one, the mode of the corresponding MB and its eight neighboring MBs are used to estimate mode characteristics of a MB. A MB after quantization with all zero coefficient (All-zero block) is preferred to be coded with SKIP mode. – Once the SAD between a block and its matching block is smaller than a threshold, the current MB is early decided as SKIP mode. 6

Early SKIP mode decision “I”  the MBs whose corresponding MB and its eight neighboring MBs in the previously coded view are all coded with SKIP mode. “II”  remaining MBs in the current frame 7 36% 13.6%

Adaptive early termination Idea – The majority of best prediction modes after mode decision are with large sizes such as 16x16, 16x8, 8x16. Threshold-based early termination – N: 9 – When the RDcost of current MB is smaller than the threshold  early termination 8

Adaptive early termination 9 < 0.2% 96%

Fast inter mode size decision 10 [9] L. Shen, T. Yan, Z. Liu; Z. Zhang; P. An; Lei Yang, Fast mode decision for multiview video coding, in Proc. IEEE ICIP, Nov. 2009, pp

Fast inter mode size decision 11

Fast inter mode size decision Fast sub-mode decision – Based on the information from the corresponding 8x8 block and its eight neighboring blocks in the previously coded view. One of them with 4x4  complex sub-mode – perform all sub-mode One of them with 8x4 or 4x8  normal sub-mode – perform 8x8, 8x4, 4x8 Otherwise – perform only 8x8 12

Selective intra prediction in inter frame The intra mode are enabled when a MB satisfies one of the following two conditions: – One MB among the corresponding MB and its eight neighboring MBs in the previously coded view is coded with intra modes. – RDcost value of inter prediction for current MB is larger than the maximum of RDcost from these nine corresponding MBs. 13 Enableskip

Overall algorithm Step1: start mode decision for a MB Step2: locate the corresponding MB and its eight neighboring MBs from the previously coded view by GDV Step3: derive the prediction mode and RDcost of these MBs 14

Overall algorithm Step4: if the corresponding MB and its eight neighboring MBs are all coded with SKIP mode or the SAD value of SKIP mode for the current MB satisfied the AZB condition, perform early SKIP mode decision Step5: compute Thr for RDcost based on Eq.1 and MC based on Eq.2 Step6: test inter16x16 and get the minimal RDcost. If RDcost is smaller than Thr, go to Step12; else if the current MB is simple mode, go to Step10; else go to Step8 (Early termination&Fast inter mode size decision) 15

Overall algorithm Step7: test inter16x8 and get the minimal RDcost. If the minimal RDcost is smaller than Thr, goto Step12; else if the current MB is normal mode, go to Step10; else go to Step9(Early termination&Fast inter mode size decision) Step8: test inter8x16 and get the minimal RDcost. If the minimal RDcost is smaller than Thr, goto Step12; else if the current MB is normal mode, go to Step10; else go to Step9(Early termination&Fast inter mode size decision) Step9: for each 8x8, when one of the corresponding block and its eight neighboring blocks – With 4x4, test all sub-mode – With 4x8 or 8x4, test inter8x8, 8x4, 4x8 – Otherwise, test inter8x8 if RDcost is smaller than Thr, go to Step12; else go to Step10 (Early termination) 16

Overall algorithm Step10: check the conditions to skip intra prediciton – If RDcost of inter prediction for current MB is smaller than the maximum of RDcosts from the corresponding MB and its eight neighboring MBs and these MBs are all coded with inter modes, skip intra prediction and go to Step 12; else go to Step 11. Step11: perform intra prediction Step12: determine the best prediction mode. Go to Step1 and proceed with next MB 17

Experiment results JMVM 6.0 GOP 16 Search range 64 QP varies from 24 to 36 Enable motion skip mode Compare to JMVM & the fast inter frame prediction algorithm for MVC (FIFPMVC) [7] & the selective DE and variable size ME algorithm (SDEME) [8] & fast mode decision algorithm (FMD) [9] Three views of test sequences are chosen for simulation. The second view is used for the implementation of the proposed algorithm [7] X. M. Li, D. B. Zhao, X. Y. Ji, Q. Wang, W. Gao, A fast inter frame prediction algorithm for multi-view video coding, in Proc. IEEE ICIP, Sep. 2007, pp ,. [8] L. Shen, Z. Liu, S. Liu, Z. Zhang, and P. An, Selective disparity estimation and variable size motion estimation based on motion homogeneity for multi-view coding, IEEE Trans. Broadcast., vol. 55, no. 4, pp , Dec [9] L. Shen, T. Yan, Z. Liu; Z. Zhang; P. An; Lei Yang, Fast mode decision for multiview video coding, in Proc. IEEE ICIP, Nov. 2009, pp

Individual performance results 19

Individual performance results 20

Individual performance results 21

Combine results 22

Combine results 23