1. Feature-Based Intra-/InterCoding Mode Selection for H.264/AVC C. Kim and C.-C. Jay Kuo CSVT, April 2007

2. Outline Introduction Overview of Proposed Algorithm Feature Selection Feature Space Partitioning Coding Mode Prediction Experimental Results

3. Introduction Inter/Intra Mode Decision in H.264  Skip mode, direct mode, intra modes, and inter modes  Full mode decision Testing all possible modes and then choosing the best mode with smallest cost  Fast algorithms Selection of optimal inter-prediction mode Selection of optimal intra-prediction mode Binary decision of intra/inter mode

4. Overview of Proposed Algorithm Motion activity f 1, Residual of intra prediction f 0, Residual of inter prediction MB Risk-Free Risk-Tolerable Risk-Intolerable Choose min(f 0,f 1 ) Compute risk- minimizing mode Full mode decision

5. Feature Selection (1/4) Intra mode feature  Calculate SATD for 5 modes DC, vertical, horizontal, diagonal down-left, and diagonal down-right  Let f 1 or f Intra be the SATD of the MB of the chosen modes

6. Feature Selection (2/4) Inter mode feature  MV is obtained by MVFAST + Two more candidates Residual of every visited point is remembered in the memory Search points of a MB < 512  Let f 0 or f Inter be SATD of MB residual of the chosen MV (i,j)(i,j) (i-1,j-1) nn-1

7. Feature Selection (3/4) Motion activity classification  Motion activity, decision error, and skipped frames  Decision metric d f = f 1 – f 0 Intra (Inter): d f 0)  Decision error probability P( d f 0  intra)

8. Feature Selection (4/4) Motion activity, RD cost difference d c, and feature difference d f  d c = (D 1 + 1  R 1 ) - (D 0 + 0  R 0 )  Positive (Negative) if inter (intra) is better Low motionHigh motionmedium motion Best intra modeBest inter mode

9. Feature Space Partitioning The 3-D feature space is partitioned into three regions (off-line)   L p : normalized RD cost between the best mode and the wrongly selected mode Inter mode feature Intra mode feature Motion activity Threshold

10. Feature Space Partitioning Let every cell has about equal training data |MV| f0f0 f1f1

11. Feature Space Partitioning Getting training data from Akiyo, Hall Monitor, Foreman, Coastguard, Stefan, Table Tennis, and Mobile.

12. Coding Mode Prediction (1/4) Risk-Free region  Distribution of f 0 and f 1 in a given motion class based on feature difference Risk free

13. Coding Mode Prediction (2/4) Risk-tolerable/-intolerable region Risk-tolerable and Risk-intolerable

14. Coding Mode Prediction (3/4) Risk-tolerable region  Risk function  For simplicity, let  stands for cost instead of R m j is the best mode m 0 : intra m 1 : inter The chosen mode Cost of deciding ~m i under m j

15. Coding Mode Prediction (4/4) Risk-minimizing mode selection  Mode selection rule 00 Likelihood ratio 1.Parametric 2.Semi-parametric 3.Nonparametric

16. Experimental Results (1/6) Environments  JM7.3a  32 x 32 motion search range  Fast full search with 5 reference frames  No B-frame  QP= {10, 16, 22, 28, 34}  5 skipped frames

17. Experimental Results (2/6) QCIF Table Tennis

18. Experimental Results (3/6) QCIF Table Tennis Computation complexity Saving time

19. Experimental Results (4/6) QCIF Foreman  5 skipped frames 0 skipped frames

20. Experimental Results (5/6) QCIF Stefan

21. Experimental Results (6/6)