V4 – Video Tracker for Extremely Hard Night Conditions December 2011 Omer Cohen Ophir Gozes Davidov Gavriel

Table Of Contents The challenge Demonstration videos Usual solutions Our solution Basic Assumptions for Tracking Block Diagram Performance examination Conclusion

The Challenge Pointing and tracking a maneuvering target. No prior knowledge of the target’s shape. Dealing with hiding objects – Position estimation Dealing with very low SNR videos. Display velocity and acceleration of the tracked target.

Demonstration Videos – FLIR Human FLIR video of a human crawling at night. Very low SNR. Watch video

Demonstration Videos – Maneuvering Super-Hornet Night video of a maneuvering Super-Hornet. Very low SNR. Watch video

Demonstration Videos – Severely Maneuvering Particle A Severely maneuvering particle. High speed. Accelerating. Hiding object Low SNR. Watch video

Combined Scenarios Combining all 3 Scenarios into one movie, one after another. The algorithms may cope with the scenario changes and identify the target automatically.

Usual Solutions Center of mass / Image Processing Finding the maximal value at the frame: Wrong, if working with a low SNR video is required. A pre-processing filtering is required. Correlation – with a prior known object Limited for dealing with certain known objects.

Our Solution Combines 2 methods of objects tracking. Each of them with his own advantages. Correlation of 2-4 following frames, based on (Quantitative Comparison of Algorithms for Tracking Single Fluorescent Particles) Cheezum,Walker,2001 Ideal for very low SNR videos. Limited for objects at medium velocity. Image Processing Peak Search Deals with high velocities at higher SNR.

Correlating Following Frames We’ve based our solution on a method of Correlating Following Frames Based on “name of an article” Frame x+1 Frame x X

Correlating Following Frames (2) We’ve implemented a 4-frame correlator Better sensitivity (ADD THE CALCULATION) Less fitted for tracking accelerating targets Frame x+3 Frame x+2 Frame x+1 X Frame x X X

4 Frame Correlator Demonstration The 2 frames correlation matrix before and after filtering. The peak cannot be detected. Example: a rectangle at a noisy frame. Very low SNR. The object in hardly noticed. The 4 frames correlation matrix before and after filtering. The peak is clearly detected.

The Correlation Matrix Out of the correlation matrix one can calculate the object’s velocity in each axis. However, the correlation matrix doesn’t give object’s location. Center Of Frame Frame x+1 Frame x X y Velocity X Velocity Max Correlation Point

Tracking Controller - Correlation In order to determine object’s position by the correlation method, we need to produce some spatial information about the object: Minimize the frame into 4 sub-frames. Correlate each 4 sub-frames with his following sub-frames. Choose “best” correlation matrix. Begin a new iteration. Frame x+1 Frame x Up Left Right X Down

Low-Pass: Convolution with a nXn square Image Processing Because of the noisy frames we perform a simple image processing: filtering with a Low-Pass filter. Then we can clearly find maximum values. Low-Pass: Convolution with a nXn square

Tracking Controller The purpose is to combine our tracking methods (correlation of previous frames & center of mass) into one efficient algorithm, bringing each method’s advantages. New frame arrived if( first frame in the movie || certainty (last frame) < threshold || velocity (last frame) > threshold Determine object’s position by IP Determine object’s position by correlation result Decide if object is present in the correlation frame Certainty > threshold -> Object estimated position Kalman Filter Correlate using new frame

Basic Assumptions for tracking IP Algorithm Thr. Correlation Algorithm Thr. Parameter --- 0.5 X Object’s width (pixels) / Frame Velocity 1 0.3 SNR = for Wobject ~ 0.05 X Wframe Fairly Symmetric wObject< wFrame Object’s Shape

Dealing With Hiding Targets In order to deal with hiding targets, we should be able to filter frames in which the object is absent. We use a 4 dimension Kalman filter (Xx,Xy,Vx,Vy) to filter the samples. The certainty generated by the correlation algorithm is used as the samples noise. Watch video

Performance Examination Algorithm result for combined videos: Watch video Algorithm Result for combined videos (2): Algorithm result - hiding objects:

Low Snr Scenario at Low speed Correlation is suprerior Image Processing False detections are evident

Image Processing Is superior SNR =1 at High Speed Image Processing Is superior Correlation Lock lost at Velocity>0.7square_width Image Processing

Block Diagram Main Function MainControl SearchParams mainusemaincontrolv4f Parameter initalization, Output Video Creation MainControl mainusemaincontrolv4f SearchParams Determines The next correlation window Multi Frame Correlator mfmulticorrelatorv4 Performs 5 window 4 frame correlation Embeds detection Markers on the Output Video . PlotResults

Algorithm’s Weaknesses The method of correlating previous frames is limited to deal with low-medium velocities (computational complexity). In case of higher velocities, IP algorithm is limited in SNR threshold. Because of the complexity of the correlation method, processing time may become critical.

Looking Forward Improving correlation of previous frames method to reduce calculation time. Implementation on hardware – achieve real time performance.

Bibliography Quantitative Comparison of Algorithms for Tracking Single Fluorescent Particles Michael K. Cheezum, William F. Walker, and William H. Guilford Department of Biomedical Engineering, University of Virginia, Charlottesville, Virginia 22908 USA 2001