1. Guillaume Laroche, Joel Jung, Beatrice Pesquet-Popescu CSVT 2008 1

2. For the purpose of reducing the bitrate, the paper proposes two schemes: A competition-based spatial-temporal scheme for the prediction of motion vector Increasing the amount of skipped macroblocks via using a competition-based SKIP mode 2

3. Introduction MV prediction and selection MV and SKIP mode competition Competition-based MV coding Competition-based Skip mode Multiple reference frames MV competition for B-slice Experimental Results Conclusion 3

4. mv col mv 1 mv 7 mv 0 mv 6 mv 3 mv 2 mv 5 mv 4 mv mv c mv b mv a mv d Frame NFrame N-1 mv col is the collocation of macroblock “mv” Frame N Frame N-1 mvmv col 4

5. We choose ： Motion vector residual is given by: ε mv ： motion vector residual mv ： motion vector p ： motion vector predictor (MVp) E BC AE B C A E B C A 5

6. A skipped MB only has the mode itself needing to be transmitted Most used in static background 6

7. Two types: spatial and temporal Spatial direct mode uses neighboring MV to predict MV In temporal direct mode list0 and list1 predicted vectors are scaled Ref 1 Ref 0 Ref 2 Current B frame mv col L1 mv col L0 d L0L2 d L0L1 d L0 7

8. By minimizing the RD-criterion: D ： distortion LR ： weighted rate and the corresponding bitrate components:  R r ： the rate for block residue (luma+chroma)  R m ： the rate of the macroblock mode (SKIP or intra/inter prediction and macroblock partition type)  R mv ： the rate of the motion vector residue  R o ： the rate of the others components (header, CBP…) 8

9. For SKIP mode, the RD-criterion becomes: where no any R o, R r, or R mv is necessary to be transmitted in SKIP mode. In practice, the cost λ m R m is negligible compared with the distortion. 9

10. Predictor set: Spatial predictors: mv a, mv b, mv c, mv d,H.264 median predictor mv H.264, and extended spatial predictor mv spaEXT, where if 3 vectors are available. Otherwise equal to mv a,, otherwise equal to mv b, otherwise mv c, or 0 if none is available. 10 mv mv c mv b mv a mv d Frame N Ref: J. Jung and G. Laroche, “Competition-based scheme for motion vector selection and coding” ITU-T VCEG, Klagenfurt, Austria, 2006, Information VCEG-AC06

11. Predictor set: Temporal predictors: mv col, mv tf, mv tm5, mv tm9, where 11 mv col mv 1 mv 7 mv 0 mv 6 mv 3 mv 2 mv 5 mv 4 mv mv c mv b mv a mv d Frame NFrame N-1 Ref 1 Ref 0 Current frame Current block Collocated block mv H.264 mv tf mv col

12. Predictor set: Spatial-temporal predictors: It gives a higher importance to the mv col value 12

13. Choices of MV: Adaptive choices  Based on content or statistical criteria  No need to transmit index of the mode if decoder is able to determine the mode Exhaustive choices  All possible predictions are tested  A mode needs to be transmitted in the bit stream  An index i and a residual ε mv i are associated with each predictor : where n is the number of predictors in the defined predictor set P 13

14. For the selection of the MV, the bitrate of the motion vector residue R mv is replaced by R mv/mm to yield: where R mv/mm contains the cost of the residual ε mv i and the cost of the index information i 14

15. We change the equation to J SKIP i ： RD cost D SKIP i ： distortion related to p i where P s is the set of motion vectors for the SKIP mode If Skip mode is chosen, the index of the predictor is sent. 15

16. Assuming an object moves with constant speed, the predictor mv col R 0 is scaled according to the temporal distances of the reference pictures used to the current block and the temporal distance between Ref 0 and Ref j. 16 Current frame Ref i Ref j Ref 0 mv mv col R 0 djdj didi mv Scol R 0 ： Scaled predictor Ref 0 ： previous reference frame

17. Another predictor: the sum of temporally successive collocated vectors Considering the all MV in each reference frame only point to their first previous frame. In this configuration, mv Scol i is scaled MV collocated in Ref i pointing to Ref i+1 The sum of these successive temporal predictors mv Tsum j is defined by:  j ： the reference frame number of the current predictor block 17

18. We consider mv tfsum j, a sum of predictors derived from the predictor mv tf : mv Stf R i is the MV at the position given by mv Stf R i-1 in Ref i-1 pointing to Ref i,except mv Stf R 0 which is mv Scol 0 18 Ref 3 Ref 1 Ref 0 Ref 2 Current B frame mv Stf R 1 mv Stf R 2 mv Stf R 3 mv Stf R 0 =

19. No modification of the Direct mode is proposed The MV resulting from the spatial Direct mode is not considered in the set of predictors Considering the case of N successively coded B-frames 19 Ref 1 Ref 0 Ref 2 Current B frame mv col L1 mv col L0 d L0L2 d L0L1 d L0

20. Vector mv col B-1 L0 and mv col B-1 L1 are used for the scaling of predictors pair:, and, respectively. 20 Ref 1 Ref 0 Current B frame mv col B-1 L0 d L0B-1 d L0L1 d L0 B-1 mv col B-1 L1

21. Bitrate saving on the first and second B- frame for CIF sequences First predictor: mv H.264 mv col L1 : MV collocated in the future frame without scaling mv Bcol = (collocated block == intra mode ? mv a : mv Scol L1 ) mv Scol L0 and mv Scol L1 proves that MV field of a B-frame is more correlated with the future reference frame 21

22. Two profile: Baseline profile, High profile 32*32 search range 8*8 transform 4 reference frames Test set: 9 CIF, 4 SD(640*480), and 2 720p(1280*720) sequences QP=28, 32, 36, 40 22

23. Predictor sets: 11 predictors in the set P : Percentage of the selection of each proposed predictor for MV competition for the CIF test set in the Baseline profile: 23

24. Comparing P sets containing two predictors For all CIF sequences, mv H.264 is combined one by one with each predictor. The bitrate savings for different pairs of predictors: 24

25. Selecting the optimal number of predictors in the sets P sets of MV predictor are: P s sets of MV SKIP mode are: 25

26. Spatial and temporal predictor competition Temporal predictors are useful The temporal selection is correlated with the reference frame 26

27. The percentage of increase of the number of macroblocks encoded with the SKIP mode 27 For sequences with large objects and fluid motion  A spatial predictor as the second predictor is less efficient for sequences with static background

28. A compression gain is acquired for all test sequences 28 For simple or no motion sequences, SKIP mode is widely used, so the gains are lower. Fast or complex motion sequences take full advantage of the temporal prediction

29. RD curves for 4 of the test sets At low bitrate, motion information tends to become a significant part of the total bitstream The bitrate reduction is not related to the resolution, but related the frame rate 29

30. The problem is modified due to the presence of B pictures and multiple reference frames Is the P set used for the P-frames in the Baseline profile still adapted to the High profile, where the temporal distance between P-frames is increased? Which set is the most adapted to the B-frames, and is it the same for all the B-frames between two P-frames? 30

31. The same sets as the ones proposed for the Baseline profile gives the best results The temporal distance between two P-frames is larger, so the temporal correlation between motion vector fields is smaller 31 Distribution of the predictor selection in the High IBBP profile for the P- and B-frames Bitrate saving in the high IBBP profile (only computed for CIF sequences)

32. Bitrate saving on the first and second B- frame for CIF sequences First predictor: mv H.264 mv col L1 : MV collocated in the future frame without scaling mv Bcol = (collocated block == intra mode ? mv a : mv Scol L1 ) mv Scol L0 and mv Scol L1 proves that MV field of a B-frame is more correlated with the future reference frame 32

33. Bitrate reduction of each sequences The gain is lower than the Baseline profile is explained by the results obtained on P-frames 33

34. Average bitrate reduction of Baseline and High profile are 7.7% and 4.3% respectively. The MV predictions are selected via an RD-criterion that considers the cost of the residual and the index for the prediction. An adaptation of predictors set according to the statistical characteristics for the sequence should allow to increase even more bitrate saving. 34