1. 1/39 Motion Adaptive Search for Fast Motion Estimation 授課老師：王立洋老師 製作學生： M9535204 蔡鐘葳

2. 2/39 Outline ▓ Introduction ▓ Motion Activity ▓ Motion Adaptive Search Algorithm ▓ Experimental Results ▓ Reference

3. 3/39 Introduction This paper presents the motion adaptive search (MAS) that effectively utilizes the motion activity information to improve The speed The accuracy of motion search

4. 4/39 Motion Activity Different sets of frames in a video have different motion activities corresponding to various scenes We use the term Global motion activity (GMA) to characterize motion activity over an entire video frame Local motion activity (LMA) to characterize motion activity within a small region

5. 5/39 LMA The motion vectors of spatially neighboring macroblocks are highly correlated Therefore, We compute the LMA at a macroblock position based on The knowledge of the motion vectors of its neighboring macroblocks

6. 6/39 Region of Support (ROS) Let MB i, j be the (i, j) th macroblock in a frame i and j indicate the horizontal and vertical positions We define the region of support (ROS) of MB i, j, denoted by R i, j As the set of macroblocks to the left, above, and above right of MB i, j, respectively. That is

7. 7/39 The City-block Lengths We compute the LMA at MB i, j based on the information of city-block lengths of motion vectors of the macroblocks in R i, j Let V i. j ≡ (x i, j,y i, j ) be the motion vector of MB i, j The city city-block length of V i, j, denoted by LV i j, is given by

8. 8/39 The Grade of LMA The grade of LMA at MB i, j, denoted by G LMA (i, j), is defined as the maximum city-block length of the motion vectors of all macroblocks in R i, j. Mathematically,

9. 9/39 Example for LMA Let LMA(i, j) denote the LMA at MB i, j. Based on G LMA (i, j), we categorize LMA(i, j) into two classes as follows Fig. 1 depicts an example of distribution of G LMA The LMA=Low at most of the macroblocks belonging to the background object The LMA=High at most of the macroblocks belonging to the foreground object

10. 10/39 Fig. 1

11. 11/39 LMA and Motion Vector Bias When LMA(i, j) is Low, all the neighborhood motion vectors of MB i, j are close to (0,0) LMA(i, j)=High the motion vector of MB i, j is away from (0,0) The motion vector histogram commonly observed in most video sequences reveals the link between local motion activity and motion vector bias

12. 12/39 Fig. 2

13. 13/39 CMV and SAD Consider the candidate motion vector set, denoted by CMV given as follows We define the predicted motion vector, denoted V i, j As the vector in CMV that yields minimum sum of absolute difference (SAD). Mathematically,

14. 14/39 The Differential Motion Vector Consider the differential motion vector The differential motion vector d v is computed for each macroblock at which LMA=High The histogram of d v is shown in Fig. 3

15. 15/39 Fig. 3

16. 16/39 LMA and Camera Motion The camera motions can be divided into two categories Nontranslational (e.g., zoom and rotation) Translational (e.g., pan and tilt) Translational camera motion is the most often encountered camera motion in the real world

17. 17/39 Translational Camera Motion (1/3) The translational camera motion introduces High LMA over the entire frame Thus, the presence of translational camera motion is detected by first investigating Whether the LMA is High at a majority of the macroblocks Whether a high peak occurs at a point away from (0,0)

18. 18/39 Translational Camera Motion (2/3) Let H be the set of motion vectors of those macroblocks, at which LMA = High Consider a subset of H, denoted by H 1, whose city- block lengths are greater than one Let C be the motion vector at which the histogram of H 1 has a peak The cardinalities of H and H 1 are denoted by | H | and |H 1 |

19. 19/39 Translational Camera Motion (3/3) We declare that translational camera motion is present in the frame if the following two conditions are satisfied |H| is greater than or equal to 25% of the total number of macroblocks The peak value of H 1 is greater than or equal to 0.1× |H 1 | Then C is assigned as the candidate motion vector representing the camera motion

20. 20/39 Global Motion Activity (GMA) The GMA is estimated based on the motion vectors of all the macroblocks in a frame to characterize the overall motion activity in the entire frame We define the grade of GMA, denoted by G GMA

21. 21/39 Example for GMA (1/2) An example of variation of GGMA obtained with FS over the 300 frames of the Foreman video sequence is shown in Fig. 4

22. 22/39 Example for GMA (2/2) From 0 to 150 An unstable camera is focused (a moderate value) From 150 to 225 The camera pans to the right (the sudden increase due to the high motion activity caused by the camera motion) From 225 to 300 The camera focuses on a building under construction (low value)

23. 23/39 Motion Adaptive Search Algorithm (1/2) The current frame are estimated from the motion vector field of the immediately previous predicted frame (P-frame) The LMA and GMA information is used to adjust the search range for each macroblock The estimated camera motion vector is used as a point on one of the search patterns The search patterns are switched intelligently to suit the LMA

24. 24/39 Motion Adaptive Search Algorithm (2/2) Before the motion search begins, the current macroblock is first tested to determine if it is a stationary macroblock by comparing its SAD at (0,0) If yes, the search is terminated immediately and its motion vector is assigned to be (0,0) Otherwise, an initial motion vector is predicted based on the LMA and local search is performed to obtain the final motion vector A detailed technical description of MAS is provided in the following

25. 25/39 Estimation of Camera Motion Vector and Grade of GMA Let C and G GMA denote the estimated values and the grade of GMA in the current frame C ' and G' GMA be the true values and the grade of GMA If the current frame is the very first inter-frame We set C equal to (0,0) and G GMA equal to 16 times the temporal distance between the current frame and its reference frame Otherwise, G GMA = G' GMA and C = C'

26. 26/39 Adaptation of Search Range The maximum search range for each macroblock in the current frame sr max is given by We adapt the search range for a as follows sr i, j is the search range LMA(i, j) is the LMA at MB i, j

27. 27/39 Early Elimination of Search for Stationary Macroblocks We compare the SAD of a macroblock MB i, j at (0,0), denoted by SAD i, j (0,0), with a threshold value T If SAD i, j (0,0) ≤ T The motion search is terminated immediately Assigning its motion vector as (0,0) We call this process early elimination of search (EES)

28. 28/39 Search Patterns The two search patterns employed in the MAS are The diamond search pattern (DSP) The elastic search pattern (ESP) as shown in Fig. 5

29. 29/39 Prediction of Initial Motion Vector The initial motion vector at MB i, j, denoted by I i, j, is expressed as follows

30. 30/39 Intelligent Local Search (1/5) We categorize the moves of DSP into three types, Type-1, Type-2 and Type-3 We make use of the vector cross product between c k −1 c k and c k c k +1 to determine the type

31. 31/39 Fig. 6

32. 32/39 Intelligent Local Search (2/5) From vector algebra, we have Where ‘ ⊗ ’ represents the vector cross product u z is the unit vector perpendicular to both c k −1 c k and c k c k +1

33. 33/39 Intelligent Local Search (3/5) Another definition of cross product is The value of sinθ is equal to zero when θ= 0 o The value of sinθ is equal to 1 when θ= 90 o The value of sinθ is equal to -1 when θ= 270 o Therefore, the value of Z distinguishes the types of moves of the DSP Z=0 (Type-1), Z>0 (Type-2 ) and Z<0 (Type-3)

34. 34/39 Intelligent Local Search (4/5) If the move of D k +1 is of Type-1 The other non-overlapping points are the two points on either side of d n k+1 If the move of D k +1 is of Type-2 The point next to d n k+1 is the other non-overlapping point If the move of D k +1 is of Type-3 The other non-overlapping points is the point previous to d n k+1

35. 35/39 Intelligent Local Search (5/5) Thus, the sets of non-overlapping points of D k +1 are for Type-1 move for Type-2 move

36. 36/39 The intelligent local search algorithm (1/2)

37. 37/39 The intelligent local search algorithm (2/2)

38. 38/39 Experimental Results

39. 39/39 Reference [1] Prabhudev Irappa Hosur, Member, IEEE, “Motion Adaptive Search for Fast Motion Estimation,” IEEE Signal Processing Lett. Volume 49, No. 4, Nov. 2003 Page(s):1330 - 1340.