1. 1 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Motion Detection and Estimation

2. 2 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Definition Motion Detection: Whether image points are moving or not? Motion estimation: How image points move?

3. 计算机学院 图像与视频处理 Motion Detection

4. 4 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Motion Detection Goals of motion detection Identify moving objects Detection of unusual activity patterns Computing trajectories of moving objects Applications of motion detection Indoor/outdoor security Real time crime detection Traffic monitoring Many intelligent video analysis systems are based on motion detection.

5. 5 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Motion Detection Methods Hypothesis Testing with a Fixed threshold Hypothesis Testing with Adaptive Threshold MAP Detection

6. 6 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Hypothesis Testing with a Fixed threshold Let H M and H S be two hypotheses declaring an image point at n as moving (M) and stationary (S), respectively. Let We can write the hypothesis test as follows :

7. 7 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Hypothesis Testing with a Fixed threshold OriginalNoise Stationary but ρ>θ in many places.

8. 8 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Hypothesis Testing with a Fixed threshold How to handle noise: where W is a spatial windows.

9. 9 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Hypothesis Testing with a Fixed threshold Original Light Stationary but ρ>θ.

10. 10 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Hypothesis Testing with a Fixed threshold Use a linear transformation to make mean (μ) and variance (σ) of the normalized image equal to the mean and variance of the original image, respectively. How to solve a and b?

11. 11 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Hypothesis Testing with a Fixed threshold Compare intensity gradients to handle illumination change:

12. 12 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Hypothesis Testing with Adaptive Threshold Let E k be a MRF of all labels assigned at time t k, and let e k be its realization. Based on Bayes criterion, we can write:

13. 13 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Hypothesis Testing with Adaptive Threshold To increase the detection robustness to noise, the temporal differences should be pooled together, for example with in a spatial window W l centered at l. The hypothesis becomes: Where N is the number of pixels in W l.

14. 14 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 MAP Detection Let where q is zero-mean uncorrelated Gaussian noise and

15. 15 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 MAP Detection The overall energy function can be written as:

16. 16 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Experimental Comparison of Motion Detection Methods

17. 计算机学院 图像与视频处理 Motion Estimation

18. 18 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Motion Models Spatial Motion Models Temporal Motion Models Region of Support Observation Models

19. 19 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Spatial Motion Models The velocity at position x in the image plane is described by: When combined with 3-D affine motion of a planar surface, it leads to:

20. 20 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Temporal Motion Models When the trajectories are linear and the velocity v t (x) is constant between t=t k-1 and τ (τ>t), a linear trajectory can be expressed as: A natural extension of the linear model is a quadratic trajectory model:

21. 21 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Region of Support The set of points x to which a spatial and temporal motion model applies is called a region of support, denoted R.

22. 22 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 the whole image

23. 23 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 irregularly shaped region

24. 24 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Observation Models Image intensity remains constant along a motion trajectory => Using, Take noise into account:

25. 25 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Observation Models Let s be a variable along a motion trajectory. Then: where.

26. 26 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Observation Models Again, when illumination changes, we use the gradients:

27. 27 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Estimation Criteria The models discussed have to be incorporated into an estimation criterion that will be subsequently optimized. Pixel-Domain Criteria Frequency-Domain Criteria Regularization Bayesian Criteria

28. 28 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Pixel-Domain Criteria Minimize the following error: A common choice for the estimation criterion is the following sum: where Φ is a nonnegative real-valued function.

29. 29 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Pixel-Domain Criteria How to choose Φ?

30. 30 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Pixel-Domain Criteria

31. 31 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Regularization A motion field vt must be sought that satisfies the motion constraint as closely as possible and simultaneously is as smooth as possible. This may be achieved by minimizing the following criterion: where D is the domain of the image.

32. 32 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Bayesian Criteria If motion field d k is a realization of a vector random field D k with a given posteriori probability distribution, and image I k is a realization of a scalar random field I k, then the MAP estimate of d k can be computed as follows:

33. 33 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Search Strategies Once models have been identified and incorporated into an estimation criterion, the last step is to develop an efficient (complexity) and effective (solution quality) strategy for finding the estimates of motion parameters. –Minimizing a prediction error –Gradient-based techniques –Relaxation techniques

34. 计算机学院 图像与视频处理 Practical Motion Estimation Algorithms

35. 35 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Practical Motion Estimation Algorithms Global Motion Estimation Block Matching Phase Correlation Optical Flow by Means of Regularization MAP Estimation of Dense Motion

36. 36 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Global Motion Estimation

37. 37 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院

38. 38 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Block Matching Block matching uses a spatially constant and temporally linear motion over a rectangular region of support. We can describe the method by the following minimization: where P is the search area to which d m belongs, defined as follows: and B m is an M×N block of pixels with the top-left cornet coordinate at m=(m 1,m 2 ).

39. 39 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院

40. 40 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Block Matching Search Methods –An exhaustive search for d m ∈ P that gives the lowest error ε is computationally costly. –Logarithmic search: Assuming that P=2 k -1 and denoting P l =(P+1)/2 l, where k and l are integers, we establish the new reduced-size search area as follows:

41. 41 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院

42. 42 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Block Matching 2-D search vs 1-D search –One-at-a-time search –Parallel hierarchical one-dimensional search –……

43. 43 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Experimental Comparison of Motion Estimation Methods

44. 44 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Perspectives In the past two decades, motion detection and estimation have moved from research laboratories to specialized products. This has been made possible by two factors. –Enormous advances in VLSI have facilitated practical implementation of CPU hungry motion algorithms. –New models and estimation algorithms have lead to an improved reliability and accuracy of the estimated motion. With the continuing advances in VLSI, the complexity constraints plaguing motion algorithms will become less of an issue. This should allow practical implementation of more advanced motion models and estimation criteria, and, in turn, further improve the accuracy of the computed motion. One of the promising approaches studied today is the joint motion segmentation and estimation that effectively combines the detection and estimation discussed separately in this chapter.