Feature-Based Intra-/InterCoding Mode Selection for H.264/AVC C. Kim and C.-C. Jay Kuo CSVT, April 2007

Outline Introduction Overview of Proposed Algorithm Feature Selection Feature Space Partitioning Coding Mode Prediction Experimental Results

Introduction Inter/Intra Mode Decision in H.264  Skip mode, direct mode, intra modes, and inter modes  Full mode decision Testing all possible modes and then choosing the best mode with smallest cost  Fast algorithms Selection of optimal inter-prediction mode Selection of optimal intra-prediction mode Binary decision of intra/inter mode

Overview of Proposed Algorithm Motion activity f 1, Residual of intra prediction f 0, Residual of inter prediction MB Risk-Free Risk-Tolerable Risk-Intolerable Choose min(f 0,f 1 ) Compute risk- minimizing mode Full mode decision

Feature Selection (1/4) Intra mode feature  Calculate SATD for 5 modes DC, vertical, horizontal, diagonal down-left, and diagonal down-right  Let f 1 or f Intra be the SATD of the MB of the chosen modes

Feature Selection (2/4) Inter mode feature  MV is obtained by MVFAST + Two more candidates Residual of every visited point is remembered in the memory Search points of a MB < 512  Let f 0 or f Inter be SATD of MB residual of the chosen MV (i,j)(i,j) (i-1,j-1) nn-1

Feature Selection (3/4) Motion activity classification  Motion activity, decision error, and skipped frames  Decision metric d f = f 1 – f 0 Intra (Inter): d f 0)  Decision error probability P( d f 0  intra)

Feature Selection (4/4) Motion activity, RD cost difference d c, and feature difference d f  d c = (D  R 1 ) - (D  R 0 )  Positive (Negative) if inter (intra) is better Low motionHigh motionmedium motion Best intra modeBest inter mode

Feature Space Partitioning The 3-D feature space is partitioned into three regions (off-line)   L p : normalized RD cost between the best mode and the wrongly selected mode Inter mode feature Intra mode feature Motion activity Threshold

Feature Space Partitioning Let every cell has about equal training data |MV| f0f0 f1f1

Feature Space Partitioning Getting training data from Akiyo, Hall Monitor, Foreman, Coastguard, Stefan, Table Tennis, and Mobile.

Coding Mode Prediction (1/4) Risk-Free region  Distribution of f 0 and f 1 in a given motion class based on feature difference Risk free

Coding Mode Prediction (2/4) Risk-tolerable/-intolerable region Risk-tolerable and Risk-intolerable

Coding Mode Prediction (3/4) Risk-tolerable region  Risk function  For simplicity, let  stands for cost instead of R m j is the best mode m 0 : intra m 1 : inter The chosen mode Cost of deciding ~m i under m j

Coding Mode Prediction (4/4) Risk-minimizing mode selection  Mode selection rule 00 Likelihood ratio 1.Parametric 2.Semi-parametric 3.Nonparametric

Experimental Results (1/6) Environments  JM7.3a  32 x 32 motion search range  Fast full search with 5 reference frames  No B-frame  QP= {10, 16, 22, 28, 34}  5 skipped frames

Experimental Results (2/6) QCIF Table Tennis

Experimental Results (3/6) QCIF Table Tennis Computation complexity Saving time

Experimental Results (4/6) QCIF Foreman  5 skipped frames 0 skipped frames

Experimental Results (5/6) QCIF Stefan

Experimental Results (6/6)