Fine-granular Motion Matching for Inter-view Motion Skip Mode in Multi-view Video Coding Haitao Yanh, Yilin Chang, Junyan Huo CSVT

Outline Motivation Introduction of motion skip mode Methodology Experimental results Conclusion

Motivation Global disparity can not well describe the inter-view corresponding relations in different image regions. Fine granularity is introduced to obtain mare accurate motion information. Akko&Kayo, 640*480, 30fps

Introduction – Motion Skip Mode Use global disparity vector to search for the corresponding macroblock. Motion information is derived from the corresponding MB in the picture of neighboring view.

Introduction – Motion Skip Mode (cont’d) Assume there is one inter-view reference picture and one temporal reference picture: – I v,t : the picture in view v at time t – B v,t : a 16×16 block in I v,t –,where V and V ref denote the coding view and the reference view –, where T and T ref denote the time instance of the coding picture and the reference picture

Introduction – Motion Skip Mode (cont’d) Use mean absolute difference(MAD) to evaluate the matching error: – h, w : height and width of coding picture – accuracy: 16-pel h w ( x, y ) h w reference framecoding frame ( x, y )

Fine-granular motion matching In H.264/AVC, 8×8 block is the basic unit to perform MC. To estimate 8-pel accuracy global disparity vector between the coding picture and the inter-view reference picture. – D G : global disparity vector – X G, Y G : x and y component of global disparity vector – S : search range with 8-pel accuracy where

Fine-granular motion matching (cont’d) After the estimation of D G, we need to find the optimal disparity of the coding macroblock B V,T. A search window of (4×8-pel) ×(4×8-pel) centers at ( x+x G, y+y G ) Each × sign indicates a search point. 16*16 MB 8*8 MB

Fine-granular motion matching (cont’d) The 16×16 block centers at each search point, ( x+x G +Δx i, y+y G +Δy i ) for the i th search point. Each 16×16 block is composed of four 8×8 blocks, {b i,j |j=1,2,3,4}. Each 8×8 block b i,j has its own motion information m i,j, M i = {m i,j |j=1,2,3,4} 16*16 MB 8*8 MB

Fine-granular motion matching (cont’d) Disparity vector at each search point is represented as: To find the optimal disparity D opt, Lagrangian cost function is employed: where – M i is the motion information of the block B V ref,T (i) at the i th search point. – D REC (M i ) is measured as the sum of the squared differences (SSD) between the original MB and the reconstructed MB. – R REC (ΔD i ) is the sum of bits to encode the whole MB and ΔD i.

Fine-granular motion matching (cont’d) To lower the complexity, the cost function, instead, is replaced for fast RD performance evaluation: where – M i (x,y) x and M i (x,y) y denote the MV components at x and y direction. – λ MOTION = λ MODE – The optimal motion information M opt can be obtained once ΔD opt is determined.

Fine-granular motion matching (cont’d) In case there are multiple inter-view reference pictures, the optimal incremental disparity ΔD opt and the index k opt of the selected reference picture can be obtained with: where ΔD i,k represent the incremental disparity at the ith search point in the k th inter-view reference picture.

Experimental environment JMVM V5.0 Test sequences: QP: 22,27,32,37 Search rage of disparity estimation: 96 Size of the search window for the proposed fine-granular motion matching algorithm: (10×8-pel) × (10×8-pel)

Experimental results Ration of motion skipped MBs:

Experimental results (cont’d)

Rate-distortion comparison With/without base view: With base view Without base view

conclusion 8-pel precision motion matching is applied to inter-view reference pictures. Results show that the proposed algorithm increase the number of motion skip MBs. Further improvement on overall RD performance for MVC can be achieved.