DSP Based Motion Tracking System Dani Cherkassky Ronen Globinski Advisor: Mr Slapak Alon

Presentation contents Introduction Summery Results Implementation Objective The tracking task Utilization potential Design stages System diagram Data flow diagram Tracking flow chart Motion detection algorithm Tracking algorithm Conclusions Future work

Design and implement a DSP based vision system that can detect and track a single, rigid, moving object Objective Introduction ResultsImplementationSummery

The problem: Detecting and Tracking moving objects in a camera vision field often requires the constant attention of human operators, thus making the tracking task exposed to human errors and working hours problems The tracking task Introduction ResultsImplementationSummery The solution: We offer a DSP based machine vision application that can perform the motion detection and tracking tasks automatically. Our solution is based on an image processing algorithms that can be implemented on a digital signal processor platform and deliver a real-time performance.

Utilization potential Introduction ResultsImplementationSummery Motion Detection and tracking system Security Surveillance Military systems Traffic control

Design stages Introduction ResultsImplementationSummery Stage C Stage D Stage E Stage A Stage B Possible Algorithms Examination (literature survey) Implementing The selected Algorithms (Matlab real time) Implementing The selected Algorithms (DSP platform) Building the Camera motor system Tests and Benchmarks

System Diagram IntroductionResults Implementation Summery

Data Flow Diagram IntroductionResults Implementation Summery Video Data Flow on the slave DSP board

IntroductionResults Implementation Summery Tracking Flow Chart Motion tracking algorithms running on the master DSP board

IntroductionResults Implementation Summery Motion Detection Algorithm In this section a motion detection algorithm(∑-∆ filter) proposed by A. Manzanera J. C. Richefeu is introduced. Methods for motion detection : Background subtraction algorithms (the naive method) The background must stay constant Irrelevant motion is not discarded Last K frames analysis algorithms (histogram, entropy, linear prediction) Consumes memory (K have to be large) Not suitable for DSP applications ! Recursive methods that use fixed number of estimates (Kalman filter, ∑-∆ filter) OK! low complexity, low memory consumption, statistical measure on the temporal activity Not compatible with noisy real world situations! Motion detection, how ??? The principle of the motion detection method is to build a model of the static scene (i.e. without moving objects) called background, and then to compare every frame of the sequence to this background in order to discriminate the regions of unusual motion, called foreground (the moving objects).

IntroductionResults Implementation Summery Motion Detection Algorithm The ∑-∆ algorithm – a recursive approximation of the temporal statistics, allowing a simple and efficient pixel-level change detection Background estimation: Computation of the ∑-∆ mean (an approximation of the median) Motion likelihood measure: Computation of the difference between the image and the median Computation of the variance: Defined as the ∑-∆ mean of N times the non-zero differences Motion computation: Comparison between the difference and the variance

IntroductionResults Implementation Summery Motion Detection Algorithm Example: two frames from a sequence acquired by a stationary camera Still area Clutter area Motion area Algorithm results for 3 particular pixels from 3 different areas The algorithm can discard irrelevant (clutter) motion - tree moving with the wind, and also has low sensitivity to noise Moving object (foreground) The motion field corresponds to the Boolean indicator of the condition: “the green line is over the black line” – every time the difference is greater than the variance the pixel is considered to be moving.

IntroductionResults Implementation Summery Motion Detection Algorithm Example - cont’d The motion field of the moving car The sensitivity in this case is too high, thus the motion field is very noisy and unusable The effect of the parameters X, Y, N on the motion field: X=1, Y=1, N=4 X=1, Y=3, N=4 X=1, Y=3, N=4 Filtered Changing the parameter Y affects the variance calculation and in this case lowering the sensitivity which results in a cleaner motion field The motion field can be further cleaned by applying a 3x3 median filter to remove the “salt” noise. Salt noise

IntroductionResults Implementation Summery Motion Tracking Algorithm In this section a method proposed by J. Shi and C. Tomasi is introduced. It is an extension of a method proposed by B. D. Lucas and T. Kanade and will therefore, from now on, be called Extended Lucas-Kanade Tracking (ELKT). Image Motion Model ( affine motion model ) The ELKT is based on the assumption that most changes between two frames are caused by image motion and affine transformations. Moving template in Frame 1, Moving template in Frame 2, Affine Model Lets look at two sequential frames and express the transformation of the moving object in mathematical terms

IntroductionResults Implementation Summery Motion Tracking Algorithm Affine transform two samples of the image sequence Image Motion Estimation We cannot assume perfect correspondence between J(x) and K(Dx + d). This due to image noise and imperfections in the image motion model. There for, some dissimilarity measure should be minimized. In ELKT the sum of squared differences (SSD), is chosen as the dissimilarity measure. Motion estimation = minimizing the dissimilarity . The SSD method ensures that the dissimilarity function has only one minimum. The natural approach to minimize the dissimilarity would be to differentiate the dissimilarity function and set the results to zero. Methods for minimizing the dissimilarity: This equation would however be non-linear, and therefore difficult to solve. Not a good solution for our DSP Real-time system. The better solution is to linearize the function with a truncated Taylor expansion with respect to some good estimate and solve the new system by iterating the procedure.

IntroductionResults Implementation Summery Motion Tracking Algorithm The tracking algorithm Tracking = determining the six parameters motion vector µ. The final equation for estimating the six parameters of the Affine motion vector: The tracking is performed by updating µ using the iterative calculation of µ’ for every input frame. The number of iterations can be determined in advance according to the tracking performance. For each frame n: 1.Calculate µ’ (n+1) by iterating this equation. 2.Update the motion vector µ. 3.Use µ to update the position of the target object.

IntroductionResults Implementation Summery Motion Tracking Algorithm The EKLT algorithm is recursive, we need to determine the number of iterations We have discovered that if the algorithm doesn't converge after 4-5 iterations there will be practically no convergence at all. Instead of making the exhaustive iterative process we offer a different approach: We have found that method problematic for our system because : T have to be larger then some picture noise level (noise level must be measured). It may take too long for the algorithm to converge (not capable with RT systems). Run the EKLT algorithm for a fix number of iterations. Calculate the tracking error using the next equation : Check the tracking quality: If the previous inequality is satisfied continue tracking, else, go back to motion detection. It was suggested in [4] to continue the iterations till : The average error calculated using Alfa filter The Advantages of our method are : 1.No need to measure picture noise level because is an approximation of that level. 2.Tracking quality is estimated for each frame, it is easy to change the template when that estimate goes low.

Introduction Results ImplementationSummery Results Algorithm Complexity: CPU Utilization (for 10 x 20 Template size ) : a.Motion detection: 82,070,156 (Cycles) b.Feature extraction: 20,857 (Cycles) c.Tracking initialization: 6,550,668 (Cycles) d.Solving EKLT equation: 5,205,380 (Cycles) a.Motion detection: O{mxn}, mxn = Frame Size b.Feature extraction: O {N}, N = Template Size c.Tracking initialization – compute : O {N} d.Solving EKLT equation: O {N}

Introduction Results ImplementationSummery System limitation : Results - cont’d The system’s tracking speed is limited Tracking results : Calculation of the angular velocity of the target object:

IntroductionResultsImplementation Summery Conclusions The motion detection algorithm: The ∑-∆ filter, combined with 3x3 median filter, proved to be a robust and accurate method of detection of moving objects for a small cost in memory consumption and computational complexity. The tracking algorithm: The EKLT tracking algorithm performed well on our system. The algorithm is efficient – the on-line calculations are relatively small and the convergence time is fairly good. The performance can be greatly improved by better selection of the initial features for the tracking process. The good performance of the algorithms proved our concept that a real-time tracking system can be implemented successfully on DSP platforms

IntroductionResultsImplementation Summery Future work The DSP implementation must be optimized Consider using a DSP with a dual PPI core to improve the performance Improve the pan and tilt mechanism Improve the system’s transition from motion detection to tracking

IntroductionResultsImplementation Summery References [1] A.Manzanera, J.C.Richefeu "A robust and computationally efficient motion detection algorithm based on Background estimation" [2] G.Jing, C.Eng and D.Rajan. "Foreground motion detection by difference-based spatial temporal entropy" [3] J.Shi and C.Tomashi, "Good Features to Track" Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp IEEE CS Press [4] D.Hager and N.Belhumeur, "Efficient Region Tracking Whit Parametric Models of Geometry and Illumination" IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 20, No.10, October 1198.