Mode Decision and Fast Motion Estimation in H.264 K.-C. Yang Qionghai Dai, Dongdong Zhu and Rong Ding,”FAST MODE DECISION FOR INTER PREDICTION IN H.264,” ICIP 2004 Qionghai Dai, Dongdong Zhu and Rong Ding,” FAST MODE DECISION FOR INTER PREDICTION IN H.264,” ICIP 2004 Zhi Zhou and Ming-Ting Sun,”FAST MACROBLOCK INTER MODE DECISION AND MOTION ESTIMATION FOR H.264/MPEG-4 AVC,” ICIP 2004 Zhi Zhou and Ming-Ting Sun,” FAST MACROBLOCK INTER MODE DECISION AND MOTION ESTIMATION FOR H.264/MPEG-4 AVC,” ICIP 2004

Introduction Mode decision Motion estimation strategies –Full Search (FS) –Three Step Search (TSS) –Diamond Search (DS) –? 16*1616*88*16 8*88*44*8 4*4

FAST MODE DECISION FOR INTER PREDICTION IN H.264 Qionghai Dai, Dongdong Zhu and Rong Ding Tsinghua University, China ICIP 2004

Outline Framework Down Sampling Complex Motion Edge Detector Motion Estimation Strategy Simulation Results

Framework Down sampling Pre- encode Encode with original image Reference image Original image Current image Reference image Mode Motion vector Output

Down Sampling(1) 2:1 down sampling –16x16  8x8, 16x8  8x4, 8x16  4x8, 8x8  4x4 –8x4  ?, 4x8  ?, 4x4  ? (Complex motion inside)

Down Sampling(2) ME and mode decision on down sampled image –DIRECT, 8x8, 8x4, 4x8, 4x4 in the down sampled image Remember the top 2 best mode – Down sampled imageCandidate modes in the Real image DIRECTDIRECT, 16x16 8x88x8, 16x16 8x48x4, 16x8 4x84x8, 8x16 4x4Complex motion

Complex Motion in Smaller Blocks Complex motion –Edge detection on the original image Use Sobel operator Horizontal edges  Vertical edges  Others ,

Edge Detector(1) Edge vector – –Amp ij = |dx ij |+|dy ij | –Ang ij = (180°/π)  arctan(dy ij /dx ij ) –Histogram(k) = ∑ (i,j)  SET(k) Amp ij SET(k) = all pels s.t. Ang ij  a k –a 1 = [-22.25°, 22.25°] (horizontal) –a 2 = (-90°,-67.5°)  (67.5°,90°) (vertical) –a 3 = o.w.  dx i, j  dy i, j dx i, j dy i, j 0° a1a1 a2a2 a2a2 a3a3 a3a3 -90° 90°

1: Horizontal edges 2: Vertical edges 3: others Edge Detector(2) Histogram(1) > 2Histogram(2) and Histogram(1) > 2Histogram(3) –8x4 mode Histogram(2) > 2Histogram(1) and Histogram(2) > 2Histogram(3) –4x8 mode O.W. –4x4 and 8x8 mode

Motion Estimation Strategy MVFAST motion estimation 1.Initial motion vectors S1 : (0, 0), MVs of its neighbors S2 : scaled MVs obtained from pre-encoding process 2.If min SAD for a MV in S2 is smaller than T1, STOP T1 can be the number of pels of the examined block type 3.ELSE, let L be maximum amplitude of MVs in S1 L  TH1 ( small motion ) –Small diamond pattern TH1 < L  TH2 ( medium motion ) –Large diamond pattern L > TH2 ( large motion ) –Use MV with minimum SAD from initial MVs –Small diamond is then used

Simulation Results Compare with –JM61 with MVFAST algorithm Config 1. All 7 inter block modes Config 2. 16x16 inter block only SequenceMax PSNR Change(db) Avg Time Saving(%) Config 1Config 2Config 1Config 2 Stefan Foreman Paris

Simulation Results Stefan

FAST MACROBLOCK INTER MODE DECISION AND MOTION ESTIMATION FOR H.264/MPEG-4 AVC Zhi Zhou and Ming-Ting Sun Department of Electrical Engineering, University of Washington, Seattle ICIP 2004

Outline Diversity-Based Fast Block Motion Estimation Fast Variable Block-Size Motion Estimation Algorithms Based on Merge And Split Procedures for H.264/MPEG-4 AVC Framework of Fast Mode Decision And ME Simulation Results Conclusion

Relative Work DIVERSITY-BASED FAST BLOCK MOTION ESTIMATION ICME 2003 FAST VARIABLE BLOCK-SIZE MOTION ESTIMATION ALGORITHMS BASED ON MERGE AND SPLIT PROCEDURES FOR H.264/MPEG-4 AVC ISCAS 2004

Diversity-Based Fast Block Motion Estimation Adaptive diversity search strategy (ADSS) 1.Perform DS with {a} and {d}, and denote the two obtained MV v 1 and v 2. 2.If (v 1 == v 2 ) Return v 1. 3.If (|v 1 -v 2 |<TH) Use DS for {b} and {c}. Else use TSS for {b} and {c}. 4.Check v 1, v 2, v 3, and v 4. Return the MV with minimum SAD. ab cd ba ab cd ba cdcd... TH = 2 CIF_Foreman 68% v 1 and v 2 are the same 96% of them are final MVs ADSSDSFS MSE/ pel Search pels/MB MSE/ pel Search pels/MB MSE/ pel Search pels/MB Akiyo Foreman Tennis

Fast Variable Block-Size Motion Estimation Algorithms Based on Merge And Split Procedures for H.264/MPEG- 4 AVC Motion vector merging and splitting ADSS for initial blocks Small Diamond Pattern for refinement AB CD E F GHI MV E = (MV A +MV B )/2 MV F = (MV C +MV D )/2 MV G = (MV A +MV C )/2 MV H = (MV B +MV D )/2 MV I = (MV E +MV F + MV G +MV H )/2 A B C DE FG HI MV B = MV C = MV D = MV E = MV A MV F = (MV B +MV D )/2 MV G = (MV B +MV E )/2... Compare actual MV 16*16, 16*8, 8*168*4, 4*84*4 Error=0Error=3Error=0Error=3Error=0Error=3 QCIF_Foreman49.44%96.62%72.25%97.30%66.06%98.20% CIF_Coast Guard49.04%97.83%67.49%97.00%62.82%96.90% CIF_Mobile48.58%93.52%68.60%94.51%60.07%93.71%

Framework of Fast Mode Decision And ME Predicted MV ME 16x16 Done 16*16 ME Done 8*8 Merge+ME good Done 16*16, 16*8, 8*16 good Merge Splitting+ME good Splitting+ME Done 4*4, 4*8, 8*4 Done 8*8

Motion cost of 16*16 at (0, 0) or PMV < TH 16x16 ? 8*8 ME by ADSS Four motion cost of 8*8 < TH 8*8 ? 8*16 and 16*8 ME by MV merging Compare motion cost of 8*16, 16*8 with that of 8*8 8*8 sub-MB mode decision16*16 ME by MV merging Choose best on from 16*16, 16*8 and 8*16 Four 8*8 sub-MBs done? Best mode=16*16, MV=(0,0) or PMV Best mode=8*8, MV=MV by ADSS End No Yes No Yes No Both are larger than 8*8 Otherwise AB C Prediction MV 0 modes 1 modes4 modes

8*4 and 4*8 ME by MV splitting Compare motion cost of 8*4, 4*8 with that of 8*8 Best mode=8*8 4*4 ME by MV splitting Choose best on from 4*4, 8*4 and 4*8 Both are larger than 8*8 Otherwise End D E 5 modes6 modes

Threshold TH 16*16 = (min cost of previous 20 16*16 blocks) TH 8*8 = (min cost of previous 20 8*8 blocks) + 150

Simulation Results LCM-FFS: Low-Complexity Mode JM with Fast Full-Search LCM-MSS: Low-Complexity Mode JM with Merge-Split Search C D E

Conclusion A new method for mode decision and ME –Block splitting and merging –Diversity-based fast ME –Thresholds –Compare with JM with FFS Save about ½ computation Quality degradation for 0.1~0.4 dB