1. Fine-granular Motion Matching for Inter-view Motion Skip Mode in Multi-view Video Coding Haitao Yanh, Yilin Chang, Junyan Huo CSVT

2. Outline Motivation Introduction of motion skip mode Methodology Experimental results Conclusion

3. Motivation Global disparity can not well describe the inter-view corresponding relations in different image regions. Fine granularity is introduced to obtain mare accurate motion information. Akko&Kayo, 640*480, 30fps

4. Introduction – Motion Skip Mode Use global disparity vector to search for the corresponding macroblock. Motion information is derived from the corresponding MB in the picture of neighboring view.

5. Introduction – Motion Skip Mode (cont’d) Assume there is one inter-view reference picture and one temporal reference picture: – I v,t : the picture in view v at time t – B v,t : a 16×16 block in I v,t –,where V and V ref denote the coding view and the reference view –, where T and T ref denote the time instance of the coding picture and the reference picture

6. Introduction – Motion Skip Mode (cont’d) Use mean absolute difference(MAD) to evaluate the matching error: – h, w : height and width of coding picture – accuracy: 16-pel h w ( x, y ) h w reference framecoding frame ( x, y )

7. Fine-granular motion matching In H.264/AVC, 8×8 block is the basic unit to perform MC. To estimate 8-pel accuracy global disparity vector between the coding picture and the inter-view reference picture. – D G : global disparity vector – X G, Y G : x and y component of global disparity vector – S : search range with 8-pel accuracy where

8. Fine-granular motion matching (cont’d) After the estimation of D G, we need to find the optimal disparity of the coding macroblock B V,T. A search window of (4×8-pel) ×(4×8-pel) centers at ( x+x G, y+y G ) Each × sign indicates a search point. 16*16 MB 8*8 MB

9. Fine-granular motion matching (cont’d) The 16×16 block centers at each search point, ( x+x G +Δx i, y+y G +Δy i ) for the i th search point. Each 16×16 block is composed of four 8×8 blocks, {b i,j |j=1,2,3,4}. Each 8×8 block b i,j has its own motion information m i,j, M i = {m i,j |j=1,2,3,4} 16*16 MB 8*8 MB

10. Fine-granular motion matching (cont’d) Disparity vector at each search point is represented as: To find the optimal disparity D opt, Lagrangian cost function is employed: where – M i is the motion information of the block B V ref,T (i) at the i th search point. – D REC (M i ) is measured as the sum of the squared differences (SSD) between the original MB and the reconstructed MB. – R REC (ΔD i ) is the sum of bits to encode the whole MB and ΔD i.

11. Fine-granular motion matching (cont’d) To lower the complexity, the cost function, instead, is replaced for fast RD performance evaluation: where – M i (x,y) x and M i (x,y) y denote the MV components at x and y direction. – λ MOTION = λ MODE – The optimal motion information M opt can be obtained once ΔD opt is determined.

12. Fine-granular motion matching (cont’d) In case there are multiple inter-view reference pictures, the optimal incremental disparity ΔD opt and the index k opt of the selected reference picture can be obtained with: where ΔD i,k represent the incremental disparity at the ith search point in the k th inter-view reference picture.

13. Experimental environment JMVM V5.0 Test sequences: QP: 22,27,32,37 Search rage of disparity estimation: 96 Size of the search window for the proposed fine-granular motion matching algorithm: (10×8-pel) × (10×8-pel)

14. Experimental results Ration of motion skipped MBs:

15. Experimental results (cont’d)

16. Rate-distortion comparison With/without base view: With base view Without base view

17. conclusion 8-pel precision motion matching is applied to inter-view reference pictures. Results show that the proposed algorithm increase the number of motion skip MBs. Further improvement on overall RD performance for MVC can be achieved.